Senior Data Engineer
Rad AI$145,000 - $190,000/year
- Full Time
- Senior (5 to 8 years)
Not SpecifiedCompensation
Senior (5 to 8 years)Experience Level
Full TimeJob Type
UnknownVisa
Nonprofit, Data & AnalyticsIndustries
Requirements
Candidates should possess 5+ years of data engineering experience, with a significant focus on on-premise systems such as Hadoop and HDFS, and demonstrate consistency with tenure at companies, ideally averaging 2+ years per engagement. Practical knowledge of engineering best practices, particularly regarding system robustness and maintainability, is required, along with expertise in tools like Airflow, Kafka, Spark, and Hive. Advanced proficiency in Python and Java/Scala, with deep knowledge of one language, and advanced working knowledge of SQL with experience across various database dialects are also necessary. Exposure to architectural/system design or technical leadership tasks and experience in data governance, data lineage, and data quality initiatives are desirable.
Responsibilities
As a Senior Data Engineer, you will design and build scalable data pipelines using tools like Airflow, Spark, and Kafka, monitor and alert for data quality issues, and support data governance and lineage initiatives. You will contribute to the design and improvement of the shared data platform, enabling critical use cases such as product analytics, bot detection, and image classification. Furthermore, you will enhance operational excellence by identifying and implementing improvements in system reliability, maintainability, and performance, and effectively communicate technical designs to both technical and non-technical stakeholders.
Skills
Airflow
Kafka
Spark
Hive
Python
Java
Scala
SQL
Hadoop
HDFS
Data governance
Data lineage
Data quality
Wikimedia Foundation
Operates Wikipedia and free knowledge projects
About Wikimedia Foundation
The Wikimedia Foundation operates Wikipedia and other free knowledge projects, aiming to create a world where everyone can freely access and share knowledge. It provides a platform for users to read, contribute, and share content, while also supporting the volunteer communities that help maintain these projects. The foundation is funded by donations from individuals and institutions, emphasizing its nonprofit status. Unlike many other organizations, it focuses on making knowledge accessible to all without charge, advocating for policies that support free knowledge initiatives. Its goal is to empower individuals to contribute to and benefit from a collective pool of knowledge.
San Francisco, CaliforniaHeadquarters
2003Year Founded
$145.9MTotal Funding
GRANTCompany Stage
Social Impact, EducationIndustries
501-1,000Employees