Assembled · 3 months ago
AI Agent Software Engineer - Agent Performance Engineering
San Francisco, CA
Full-time
Hybrid
Senior Level
$135K/yr - $280K/yr
5+ years expAssembled is a company that builds the infrastructure for exceptional customer support, working with notable clients like CashApp and Etsy. The AI Agent Software Engineer will be part of the Agent Performance Engineering team, focusing on enhancing AI agent performance through building evaluation frameworks, automating knowledge generation, and optimizing accuracy for complex customer interactions.
Customer ServiceInformation TechnologySchedulingSoftware
Responsibilities
Build foundational evaluation infrastructure: Develop comprehensive evaluation systems from the ground up, including golden dataset creation, automated benchmarking, and model comparison tools. You'll help create the frameworks that enable us to measure and optimize AI agent performance across all communication channels
Automate knowledge generation: Design and implement systems that automatically create synthetic guides, documentation, and metadata to improve agent knowledge bases. You'll work on cutting-edge approaches to knowledge extraction and augmentation
Optimize AI agent accuracy: Enhance our retrieval systems, implement advanced prompt optimization techniques, and build tools that continuously improve agent responses through automated evaluation and refinement
Develop intelligence infrastructure: Architect systems that enable rapid model upgrades, A/B testing of different AI approaches, and scalable evaluation pipelines that support enterprise deployment
Drive volume automation: Focus on the north star goal of maximizing automated resolutions across voice, chat, and email by building the intelligence systems that make it possible
Qualification
Required
5+ years of experience in software engineering as an individual contributor
Experience with AI evaluation systems, data pipelines, or AI model optimization
Passionate about building systems that measure and improve AI performance
Worked with retrieval systems, knowledge bases, or information extraction
Enjoy building tools and infrastructure that enable other engineers and AI systems to perform better
Highly ambitious and driven, setting high goals for yourself and others
Put customers first, focusing on solving real problems that impact support quality
Enjoy fast-paced environments and can quickly adjust when new insights emerge
Have a bit of a maverick streak that helps you come up with creative solutions
Made a noticeable impact on small teams and have solid experience in startups or smaller companies
Stay humble and open to feedback, value teamwork, and are always ready to learn and grow
Preferred
Experience with Python/Golang for evaluation systems and data processing
Experience with LLMs and prompt optimization frameworks
Experience with vector databases and retrieval systems
Experience with AI evaluation and benchmarking tools
Experience with data pipeline and automation infrastructure
Benefits
Generous medical, dental, and vision benefits
Paid company holidays, sick time, and unlimited time off
Monthly credits to spend on each: professional development, general wellness, Assembled customers, and commuting
Paid parental leave
Hybrid work model with catered lunches everyday (M-F), snacks, and beverages in our SF & NY offices
401(k) plan enrollment
Company
Assembled
Assembled is a workforce management platform that helps modern organizations scale customer support teams.
51-200 employees
H1B Sponsorship
Assembled has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (4)
2024 (4)
2023 (4)
2022 (2)
2021 (1)
Funding
Current Stage
Growth StageTotal Funding
$70.7MKey Investors
New Enterprise AssociatesEmergence CapitalStripe
2022-05-26Series B· $51M
2021-03-11Series A· $16.6M
2020-03-11Seed· $3.1M
Leadership Team
Ryan Wang
Co-founder/CEO
John Wang
Cofounder and CTO
Recent News
2025-08-27
Talk Business & Politics
2024-05-20
Company data provided by crunchbase