Senior Site Reliability Engineer (SRE)

Employment Type:

Location Type:

Salary:

Position Overview

The Wikimedia Foundation is seeking a Senior Site Reliability Engineer (SRE) to join our Service Operations SRE team. This team is responsible for the infrastructure that powers wikipedia.org and other Wikimedia Foundation projects. We are a distributed, diverse team passionate about exploring, experimenting, and adopting new technologies. Our focus is on the application layer of our complex infrastructure and our developer-facing services.

Our technology stack includes MediaWiki instances on bare metal, microservices on our internal Kubernetes platform, a custom deployment pipeline, and open-source collaboration software like Gerrit, Gitlab, and Phabricator. We are strong advocates for automation, primarily using Python, Puppet, and Spicerack (https://github.com/wikimedia/operations-software-spicerack). We operate transparently, documenting our work on our internal task management system and our technical wiki (https://wikitech.wikimedia.org).

As a Senior SRE in the Service Operations team, you will be instrumental in designing, running, and maintaining the infrastructure and services that underpin the Wikimedia Foundation's projects. This includes Kubernetes clusters, application servers, related datastores, and developer-facing services. You will actively participate in 24/7 incident response and on-call rotations. Collaboration with fellow SREs and other engineering teams (Security, Release, Software Engineers) is crucial as we collectively advance our projects and technologies.

We utilize a broad range of technologies, and we encourage you to apply even if your experience doesn't perfectly align with every keyword listed. If our mission and work excite you, and you're interested in working with a globally distributed team managing a Top-15 website, we encourage you to explore this opportunity.

Responsibilities

• Design, implement, and maintain public-facing infrastructure and services.
• Utilize configuration management and deployment tools.
• Engage in architectural design and operation at scale.
• Monitor systems and services, optimizing performance and resource utilization.
• Proactively identify sources of instability in distributed systems and analyze complex system failures from a reliability and resilience perspective.
• Perform common operating system level tasks, including logging and backup/restore procedures.
• Implement cookbooks and runbooks for routine maintenance actions.
• Participate in a 24/7 on-call rotation and escalations for production issue resolution.
• Lead incident response efforts and post-incident reviews, contributing to failure analysis and implementing preventive measures.
• Automate and streamline tasks, identifying and addressing process gaps.
• Collaborate effectively with a global team communicating asynchronously (support will be provided for remote work integration).
• Mentor peers in areas of technical and operational expertise.
• Travel domestically or potentially internationally 2-3 times per year for team gatherings and conferences.

Skills and Experience

• 5+ years of experience in an SRE, Operations, or DevOps role.
• Proven experience operating highly available infrastructure.
• Experience running applications and services at scale.
• Experience implementing containerization solutions (e.g., Docker, Kubernetes).
• Proficiency in shell scripting and at least one programming language commonly used in SRE/Operations engineering (e.g., Python, Go, Ruby).
• Comfort with open-source configuration management and orchestration tools (e.g., Puppet, Ansible, Terraform).
• Strong command of technical English.

Additionally, we’d love it if you have:

• Experience with operating system package management (e.g., Debian).
• We are avid supporters (and users) of o