Apply on Employer Site

Basata · 3 days ago

AI Agent Performance Engineer

Tempe, AZ

Full-time

Onsite

Mid, Senior Level

3+ years exp

Basata is a company focused on rethinking the way healthcare practices operate through AI-powered tools. They are seeking an AI Agent Performance Engineer to ensure the quality, reliability, and performance of their conversational agents, building observability and evaluation systems to meet performance benchmarks.

Artificial Intelligence (AI)Document ManagementHealth Care

Responsibilities

Build and maintain observability and monitoring systems that track agent latency, accuracy, call outcomes, call-flow behavior, and failure modes in production

Develop rigorous QA and regression testing frameworks for Voice AI agents, including automated evaluation pipelines and scripted test scenarios

Define, instrument, and own key performance metrics and ensure agents consistently meet or exceed them

Analyze large volumes of agent interactions to identify trends, misclassifications, drift, prompt issues, or reliability gaps

Partner closely with Product, AI/ML, and Deployment teams to convert findings into concrete product, prompt, and architecture improvements

Build internal dashboards, alerts, and reports that make agent performance transparent for internal teams and customers

Conduct root-cause investigations for production issues and drive sustainable, long-term fixes

Document QA processes, performance baselines, and best practices to support fast scaling across specialties

Qualification

Software EngineeringApplied AILLM-based ApplicationsPythonAnalytics ToolingQA SystemsObservability SystemsLarge-scale Data LogsFast-paced Environments