Senior Data Engineer

Remote

Apply with AI Apply

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Information Technology & ServicesIndustries

Requirements

Candidates should possess 5+ years of data engineering experience with a significant focus on on-premise systems such as Hadoop and HDFS, demonstrating consistency with tenure at companies (e.g., average of 2+ years). Practical knowledge of engineering best practices, emphasizing system robustness and maintainability, is required, along with experience in troubleshooting systems and pipelines for performance and scaling. Desirable experience includes exposure to architectural/system design, data governance, data lineage, and data quality initiatives.

Responsibilities

As a Senior Data Engineer, you will design and build scalable data pipelines using tools like Airflow, Spark, and Kafka, monitor and alert for data quality issues, and support data governance and lineage solutions. You will contribute to the development of the shared data platform, enabling critical use cases such as product analytics, bot detection, and image classification, while also enhancing operational excellence by identifying and implementing improvements in system reliability and performance.

Skills

Airflow

Kafka

Spark

Hive

Python

Java

Scala

SQL

Database querying

Data governance

Data lineage

Data quality

Wikimedia Foundation

Operates Wikipedia and free knowledge projects

About Wikimedia Foundation

The Wikimedia Foundation operates Wikipedia and other free knowledge projects, aiming to create a world where everyone can freely access and share knowledge. It provides a platform for users to read, contribute, and share content, while also supporting the volunteer communities that help maintain these projects. The foundation is funded by donations from individuals and institutions, emphasizing its nonprofit status. Unlike many other organizations, it focuses on making knowledge accessible to all without charge, advocating for policies that support free knowledge initiatives. Its goal is to empower individuals to contribute to and benefit from a collective pool of knowledge.

San Francisco, CaliforniaHeadquarters

2003Year Founded

$145.9MTotal Funding

GRANTCompany Stage

Social Impact, EducationIndustries

501-1,000Employees