Senior Data Engineer

Remote

Apply with AI Apply

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Information Technology & ServicesIndustries

Requirements

Candidates should possess 5+ years of data engineering experience with a significant focus on on-premise systems such as Hadoop and HDFS, demonstrating consistent tenure at companies with an average of 2+ years per engagement. Advanced proficiency in Python and Java/Scala is required, along with expertise in tools like Airflow, Kafka, Spark, and Hive, and working knowledge of SQL across various database dialects. Experience in data governance, data lineage, and data quality initiatives is desirable, as is exposure to architectural/system design or technical leadership tasks.

Responsibilities

As a Senior Data Engineer, you will design and build scalable data pipelines using tools like Airflow, Spark, and Kafka, monitor and alert for data quality issues, support data governance and lineage solutions, contribute to the development of the shared data platform for use cases like product analytics and bot detection, and enhance operational excellence by identifying and implementing improvements in system reliability and performance. You will also interact effectively within and across teams, producing clear technical designs and articulating ideas to both technical and non-technical stakeholders.

Skills

Airflow

Kafka

Spark

Hive

Python

Java

Scala

SQL

Database querying

Data governance

Data lineage

Data quality

Wikimedia Foundation

Operates Wikipedia and free knowledge projects

About Wikimedia Foundation

The Wikimedia Foundation operates Wikipedia and other free knowledge projects, aiming to create a world where everyone can freely access and share knowledge. It provides a platform for users to read, contribute, and share content, while also supporting the volunteer communities that help maintain these projects. The foundation is funded by donations from individuals and institutions, emphasizing its nonprofit status. Unlike many other organizations, it focuses on making knowledge accessible to all without charge, advocating for policies that support free knowledge initiatives. Its goal is to empower individuals to contribute to and benefit from a collective pool of knowledge.

San Francisco, CaliforniaHeadquarters

2003Year Founded

$145.9MTotal Funding

GRANTCompany Stage

Social Impact, EducationIndustries

501-1,000Employees