Staff Data Engineer
Life360$166,500 - $245,000/year
Full Time
Expert & Leadership (9+ years)
Not SpecifiedCompensation
Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level
Full TimeJob Type
UnknownVisa
Payments, Fintech, TechnologyIndustries
Requirements
- Sound knowledge on managing large scale Hadoop platforms including monitoring the platform, debugging issues, and tuning the performance of the cluster
- In-depth knowledge of the Hadoop ecosystem, including Zookeeper, HDFS, Yarn, HIVE, SPARK, Trino and Kafka
- Proven experience in debugging issues on both Hadoop platform and applications
- Familiarity with security tools such as Kerberos, Ranger, and active directory integrations
- Experience on Cloud technologies preferably AWS EMR
- Knowledge on Kubernetes, AI, MLOPS (advantageous)
- Hands-on experience with automations using Ansible, Shell, python, or any programming languages
- Knowledge on observability tools like Grafana, opera, Prometheus and Splunk
- Understanding of Linux, networking, CPU, memory, and storage
- Knowledge of and ability to code or program in one of python, Java or a widely used coding language
- Excellent interpersonal skills, along with superior verbal and written communication abilities
- Master’s degree in Math, Science, Engineering, Computer Science, Information Systems, or a related field; OR Bachelor’s degree in Math, Science, Engineering, Computer Science, Information Systems, or a related field, AND a minimum of five years of relevant experience; OR a minimum of five years of experience working with Hadoop systems
- Experience in Big Data SRE and Engineering across open-source platforms such as Hadoop, Kafka, HBase, and Spark, with strong troubleshooting and debugging skills (preferred)
- Proven ability to conduct effective root cause analysis of major production incidents, document findings, and implement high-availability solutions for critical services (preferred)
- Expertise in capacity planning, system expansions, and timely upgrades to mitigate scaling challenges, while automating repetitive tasks to reduce manual effort and prevent errors (preferred)
- Ability to fine-tune alerting and set up observability tools to proactively identify and resolve performance issues (preferred)
Responsibilities
- Maintain and support Visa’s Data Platform, ensuring the reliability and performance of critical Big Data systems
- Drive innovation for partners and clients globally by working on open-source Big Data clusters, optimizing their availability, efficiency, and scalability
- Collaborate closely with L-3 teams to review new use cases and implement cluster hardening techniques, ensuring the development of robust and reliable platforms
- Foster cross-team collaboration, building and maintaining strong relationships with customer teams, user communities, architects, and engineering teams
- Work jointly on key deliverables to ensure production scalability and stability
- Collaborate with Level-3 teams on use case reviews and cluster hardening
Skills
Hadoop
HDFS
Yarn
Hive
Spark
Trino
Kafka
Zookeeper
Kerberos
Ranger
AWS EMR
Kubernetes
Ansible
Python
Shell
Grafana
Prometheus
Splunk
Linux
Networking
Visa
Global digital payment network provider
About Visa
Visa operates a global digital payment network that facilitates electronic payments for millions of people daily. The company connects consumers, businesses, financial institutions, and governments, allowing them to make transactions using Visa cards. Each time a card is used, Visa earns money through transaction, service, and data processing fees. Unlike many competitors, Visa focuses on expanding access to financial services for underserved communities and supporting local economies. The company's goal is to promote financial inclusivity and drive sustainable commerce, ensuring that more people can participate in the global economy.
San Francisco, CaliforniaHeadquarters
1958Year Founded
$55.8MTotal Funding
ANGEL_INDIVIDUALCompany Stage
Fintech, Financial ServicesIndustries
10,001+Employees
Benefits
Health Insurance.
Life Insurance.
Dental Insurance.
Disability Insurance.
Accidental Death & Dismemberment Insurance.
Risks
CBDCs could reduce reliance on Visa's payment network.
'Buy Now, Pay Later' services may decrease traditional credit card transactions.
Fintech startups offering zero-fee transactions could pressure Visa's revenue model.
Differentiation
Visa operates a global digital payment network connecting millions daily.
The company focuses on financial inclusivity and sustainability in its operations.
Visa collaborates with central banks on Central Bank Digital Currencies (CBDCs).
Upsides
Visa's partnership with fintechs enhances cross-border payment solutions.
Adoption of blockchain technology could revolutionize digital payments.
AI-driven fraud detection systems are reducing fraudulent activities.
Related Jobs
RemoteRemotePigment Systems Administrator
Huntress$140,000 - $150,000/year
- Full Time
- Senior (5 to 8 years), Expert & Leadership (9+ years)
RemoteSoftware Engineer - Data Infrastructure, Quora (Remote)
QuoraSalary not specified
- Full Time
- Junior (1 to 2 years)
RemoteCustomer Reliability Engineer, Infrastructure
AstronomerSalary not specified
- Full Time
- Mid-level (3 to 4 years)
RemoteStaff Engineer, Distributed Systems
LaunchDarklySalary not specified
- Full Time
- Senior (5 to 8 years)
RemotePlatform Architect
TetraScienceSalary not specified
- Full Time
- Expert & Leadership (9+ years)
RemoteDevOps Engineer
Swish AnalyticsSalary not specified
- Full Time
- Senior (5 to 8 years)
RemoteSite Reliability Engineer
Stitch FixSalary not specified
- Full Time
- Mid-level (3 to 4 years)