Data Engineer / Senior Data Engineer
ArcadiaSalary not specified
Full Time
Senior (5 to 8 years)
Not SpecifiedCompensation
Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level
Full TimeJob Type
UnknownVisa
Fintech, Payments, TechnologyIndustries
Requirements
- Deep expertise in data engineering, including SQL Server data warehouses, Hadoop, Databricks, Spark, Kafka, Airflow, and Delta Lake
- Experience with ETL/ELT frameworks for processing transactional, behavioral, and unstructured data at scale
- Proficiency in optimizing SQL Server models (facts, dimensions, indexes, stored procedures), T-SQL development, schema design, and stored procedure optimization
- Knowledge of database performance tuning, replication, encryption, high-availability (HA) solutions, and troubleshooting in SQL Server and Hadoop clusters
- Familiarity with RAG (Retrieval-Augmented Generation) pipelines, vector database integrations, and LLM data provisioning
- Understanding of data governance, partitioning, retention policies, data quality, lineage, privacy, encryption, and access control
- Ability to lead proof-of-concept (POC) initiatives and evaluate new data engineering technologies
- Hands-on leadership skills, including mentoring junior engineers and collaborating cross-functionally with ML teams, business stakeholders, product, platform engineering, DBA, and InfoSec teams
Responsibilities
- Lead the transition from SQL Server data warehouse to Hadoop/Databricks platform, ensuring performance, reliability, and minimal disruption
- Architect hybrid data pipelines bridging legacy and modern ecosystems for near real-time data access in analytics and AI applications
- Optimize existing SQL Server models and design modern equivalents in Hadoop and Spark environments
- Define long-term migration strategy, data partitioning, and retention policies aligned with data governance standards
- Design and implement scalable, distributed data pipelines using Spark, Kafka, Airflow, and Delta Lake
- Build robust ETL/ELT frameworks to process data at scale and partner with Agentic AI team for RAG pipelines, vector databases, and LLM data provisioning
- Provide guidance to junior engineers on maintaining SQL Server data warehouses, including performance tuning, replication, encryption, and HA solutions
- Perform proactive performance analysis, troubleshooting, and resolution of production issues in SQL Server and Hadoop clusters
- Collaborate with DBA and application teams to ensure uptime within SLAs and compliance with data policies
- Define and enforce standards for data quality, lineage, and governance across platforms
- Ensure compliance with data privacy, encryption, and access control frameworks; partner with InfoSec on data security
- Mentor and coach data engineers, fostering technical excellence and innovation
- Collaborate cross-functionally on architecture decisions, roadmap execution, and serve as a bridge between legacy and next-gen AI platform teams
Skills
Key technologies and capabilities for this role
Questions & Answers
Common questions about this position
Is this Data Engineer role remote or on-site?
This is an on-site position.
What technical skills are required for this Senior Data Engineer role?
The role requires expertise in SQL Server (T-SQL, facts, dimensions, indexes, stored procedures), Hadoop, Databricks, Spark, Kafka, Airflow, and Delta Lake, along with experience in ETL/ELT pipelines and data modernization.
What is the salary or compensation for this position?
This information is not specified in the job description.
What does the team structure look like for this Data Engineer role?
You will collaborate closely with the Principal Agentic Engineer, ML teams, business stakeholders, and provide guidance to junior Engineers.
What makes a strong candidate for this Senior Data Engineer position?
A strong candidate has hands-on leadership experience in data platform modernization from SQL Server to Hadoop/Databricks, deep expertise in designing scalable data pipelines with Spark and Kafka, and the ability to bridge legacy and modern systems while guiding junior engineers.
Visa
Global digital payment network provider
About Visa
Visa operates a global digital payment network that facilitates electronic payments for millions of people daily. The company connects consumers, businesses, financial institutions, and governments, allowing them to make transactions using Visa cards. Each time a card is used, Visa earns money through transaction, service, and data processing fees. Unlike many competitors, Visa focuses on expanding access to financial services for underserved communities and supporting local economies. The company's goal is to promote financial inclusivity and drive sustainable commerce, ensuring that more people can participate in the global economy.
San Francisco, CaliforniaHeadquarters
1958Year Founded
$55.8MTotal Funding
ANGEL_INDIVIDUALCompany Stage
Fintech, Financial ServicesIndustries
10,001+Employees
Benefits
Health Insurance.
Life Insurance.
Dental Insurance.
Disability Insurance.
Accidental Death & Dismemberment Insurance.
Risks
CBDCs could reduce reliance on Visa's payment network.
'Buy Now, Pay Later' services may decrease traditional credit card transactions.
Fintech startups offering zero-fee transactions could pressure Visa's revenue model.
Differentiation
Visa operates a global digital payment network connecting millions daily.
The company focuses on financial inclusivity and sustainability in its operations.
Visa collaborates with central banks on Central Bank Digital Currencies (CBDCs).
Upsides
Visa's partnership with fintechs enhances cross-border payment solutions.
Adoption of blockchain technology could revolutionize digital payments.
AI-driven fraud detection systems are reducing fraudulent activities.
Related Jobs
RemoteRemoteLead, Anaplan & Data Architect
LimeSalary not specified
- Full Time
- Senior (5 to 8 years), Expert & Leadership (9+ years)
RemotePrincipal Software Engineer, Data Platforms
Philo$180,000 - $200,000/year
- Full Time
- Mid-level (3 to 4 years), Senior (5 to 8 years)
RemoteSolutions Architect, Data Engineering
Snowflake$128,000 - $178,500/year
- Full Time
- Junior (1 to 2 years)
RemoteStaff Data Engineer (Spark)
WizardSalary not specified
- Full Time
- Expert & Leadership (9+ years)
RemoteSenior Analytics Engineer II
AlmaSalary not specified
- Full Time
- Senior (5 to 8 years)
RemoteEngineering Manager, Data Engineering
TRM LabsSalary not specified
- Full Time
- Expert & Leadership (9+ years)
RemoteLead Data Engineer
LumosSalary not specified
- Full Time
- Expert & Leadership (9+ years)