AI Agent Software Engineer - Agent Performance Engineering

Position Overview

• Location Type: Remote
• Job Type: Full-Time
• Salary: $120K - $270K

Assembled builds the infrastructure for exceptional customer support, enabling companies like CashApp, Etsy, and Robinhood to deliver faster, better service at scale. Our solutions for workforce management, BPO collaboration, and AI-powered issue resolution simplify modern support operations by integrating in-house, outsourced, and AI-powered agents into a single operating system. Backed by $70M in funding from NEA, Emergence Capital, and Stripe, and driven by a passionate team, we are at the forefront of support operations technology.

The Agent Intelligence team is dedicated to maximizing AI agent performance and driving volume automation across voice, chat, and email channels. This team is crucial for our vision of creating the ultimate omnichannel AI customer support agent and is experiencing rapid growth. We are one of the largest companies deploying AI agents at massive scale, working with enterprise clients like Canva, Etsy, and Ramp. The engineering challenges are significant and largely unsolved, including building sophisticated evaluation systems, automating knowledge generation, optimizing AI agent accuracy, and creating infrastructure for continuous improvement at scale.

As part of the Agent Performance Engineering team, you will work on core systems that enhance the intelligence, accuracy, and capability of our AI agents in handling complex customer interactions. You will build evaluation frameworks, knowledge automation systems, and intelligence optimization tools that directly impact our ability to automate customer support at enterprise scale.

Responsibilities

• Build foundational evaluation infrastructure: Develop comprehensive evaluation systems from the ground up, including golden dataset creation, automated benchmarking, and model comparison tools. Create frameworks to measure and optimize AI agent performance across all communication channels.
• Automate knowledge generation: Design and implement systems that automatically create synthetic guides, documentation, and metadata to improve agent knowledge bases. Work on cutting-edge approaches to knowledge extraction and augmentation.
• Optimize AI agent accuracy: Enhance retrieval systems, implement advanced prompt optimization techniques, and build tools for continuous improvement of agent responses through automated evaluation and refinement.
• Develop intelligence infrastructure: Architect systems for rapid model upgrades, A/B testing of different AI approaches, and scalable evaluation pipelines supporting enterprise deployment.
• Drive volume automation: Focus on maximizing automated resolutions across voice, chat, and email by building the intelligence systems that enable this.

About You

You might be a good fit if you:

• Have 5+ years of experience in software engineering as an individual contributor.
• Have experience with AI evaluation systems, data pipelines, or AI model optimization.
• Are passionate about building systems that measure and improve AI performance.
• Have worked with retrieval systems, knowledge bases, or information extraction.
• Enjoy building tools and infrastructure that empower other engineers and AI systems.
• Are highly ambitious and driven, setting high goals for yourself and others.
• Put customers first, focusing on solving real problems that impact support quality.
• Thrive in fast-paced environments and can quickly adapt to new insights.
• Possess a maverick streak that enables creative solutions.
• Have made a noticeable impact on small teams and have solid experience in startups or smaller companies.
• Stay humble, are open to feedback, value teamwork, and are always ready to learn.