Engineering Manager - Ads Experimentation Platform

Position Overview

• Location Type: Not Specified
• Job Type: Not Specified
• Salary: Not Specified

Reddit is a community of communities. It’s built on shared interests, passion, and trust and is home to the most open and authentic conversations on the internet. Every day, Reddit users submit, vote, and comment on the topics they care most about. With 100,000+ active communities and approximately 101M+ daily active unique visitors, Reddit is one of the internet’s largest sources of information. For more information, visit redditinc.com.

The Ads Experimentation Platform (AEXP) team builds tools for our engineers and scientists to accurately test new products and efficiently verify new ideas in the ads marketplace. Have you heard about Budget Cannibalization? Have you wondered why a user-split A/B testing is insufficient in testing products in the Ads marketplace? How can we make the most out of rare and sparse samples (e.g. conversion events)? How can I explore 100 algorithmic candidates at the same time? These are the typical problems we are dealing with on a daily basis. Meanwhile, Ads Pacing plays a crucial role in both supporting the dynamism of the marketplace and accommodating the budget-aware A/B experiments, therefore our team is also committed to building scalable and flexible pacing infrastructures that improves the market efficiency and budget-aware testing capacity.

As the manager of the Ads Experimentation Platform team, you will work closely with engineers, data science partners and product leaders to improve the capacity and efficiency of existing experimentation tools, research and build new testing methodologies and tools, and lead the experimentation practices across the Ads marketplace teams. You will be responsible for the quality and technical approach within the team; partner with other managers in direction setting, planning, and overseeing engineering designs and executions; establish and contribute to the group’s culture and processes. You will collaborate with cross-functional engineering, product, and data science teams to ensure that our customers’ voices are represented in your work.

Requirements

• Not Specified

Responsibilities

• Improve the capacity and efficiency of existing experimentation tools.
• Research and build new testing methodologies and tools.
• Lead the experimentation practices across the Ads marketplace teams.
• Be responsible for the quality and technical approach within the team.
• Partner with other managers in direction setting, planning, and overseeing engineering designs and executions.
• Establish and contribute to the group’s culture and processes.
• Collaborate with cross-functional engineering, product, and data science teams.

Project Highlights

• An A/B testing framework that split both users and ad budgets to create parallel and mutually exclusive control/treatment sub ads marketplaces.
• An exploration tool that can run multiple (hundreds of) A/B testing pairs of campaigns over multiple algorithmic candidates for pacing.
• Partitioning the pacing and budgeting systems to accommodate the growing number of advertisers and maintain the refreshing rate above once per minute.