Senior Site Reliability Engineer (SRE)

Position Overview

The Wikimedia Foundation is seeking a Senior Site Reliability Engineer (SRE) to join our Service Operations SRE team. This team is responsible for the infrastructure that powers wikipedia.org and other Wikimedia Foundation projects. Our SRE team is a distributed, diverse group of engineers passionate about exploring, experimenting with, and adopting new technologies. The Service Operations team specifically focuses on the application layer of our complex infrastructure and our developer-facing services.

Our technology stack includes MediaWiki instances on bare metal, surrounding microservices on our in-house Kubernetes platform, a custom deployment pipeline, and open-source collaboration software like Gerrit, Gitlab, and Phabricator. We are strong advocates for automation, utilizing Python, Puppet, and Spicerack for this purpose. We operate transparently, documenting our work on our internal task management system and our technical wiki (https://wikitech.wikimedia.org).

As a senior engineer in this role, you will be instrumental in designing and operating the infrastructure and services that underpin the Wikimedia Foundation's projects. This includes, but is not limited to, Kubernetes clusters, application servers, associated datastores, and various developer-facing services. You will be expected to participate in 24/7 incident response and on-call rotations. This position requires close collaboration with other SRE team members to maintain and enhance our systems, as well as interaction with non-SRE teams such as Security, Release, and Software Engineering, all working together to advance our projects and technologies.

We utilize a broad range of technologies, and we encourage you to apply even if your experience doesn't perfectly align with every keyword listed. If our work interests you and you are drawn to the prospect of working with a globally distributed team managing a Top-15 website, we might be the right fit for you.

Responsibilities

• Design, implementation, and maintenance of public-facing infrastructure and services.
• Utilization of configuration management and deployment tools.
• Architectural design and operation at scale.
• Monitoring of systems and services, including performance and resource utilization optimization.
• Proactive identification of instability sources in distributed systems and analysis of complex system failure modes from a reliability and resilience perspective.
• Execution of common operating system tasks, such as logging and backup/restore procedures.
• Implementation of cookbooks/runbooks for routine maintenance actions.
• Participation in 24/7 on-call rotations and escalations for production issue resolution.
• Leading incident response efforts and post-incident reviews, contributing to failure analysis and implementing preventive measures.
• Automation and streamlining of tasks, as well as identifying process gaps.
• Collaboration with a global team that communicates asynchronously (remote work experience is not required; we will support your transition).
• Mentoring peers in areas of technical and operational expertise.
• Expected travel domestically or potentially internationally 2-3 times per year for team gatherings and conferences.

Skills and Experience

• 5+ years of experience in an SRE/Operations/DevOps role.
• Experience operating highly available infrastructure.
• Experience running applications and services at scale.
• Experience implementing containerization solutions (e.g., Docker, Kubernetes).
• Proficiency in shell scripting and a programming language commonly used in SRE/Operations engineering (e.g., Python, Go, Ruby).
• Comfort with open-source configuration management and orchestration tools (e.g., Puppet, Ansible, Terraform).
• Communicative technical English.

Additionally, we’d love it if you have:

• Experience with package management for operating systems (e.g., Debian).
• We are avid supporters (and users) of o... (Note: This sentence appears incomplete in the original description).

Company Information

Wikimedia Foundation is hiring.