Akamai Technologies · 1 month ago
Principal Software Development in Test - Akamai Inference Cloud (Remote)
United States
Full-time
Remote
Senior Level, Lead/Staff
$148K/yr - $308K/yr
12+ years expAkamai Technologies is a leader in digital experience solutions, and they are seeking a Principal Software Development in Test for their Inference Cloud team. In this role, you will establish quality frameworks and methodologies for evaluating AI inference systems, ensuring performance and reliability at scale.
Cyber SecurityDigital MediaSaaSSoftwareVideo Streaming
Responsibilities
Establishing quality frameworks and standards for AI inference, including model accuracy evaluation, performance benchmarking, and output quality validation
Designing and implementing sophisticated testing strategies for LLM and inference workloads, including accuracy testing, regression detection, and model behavior validation
Providing thought leadership on AI quality metrics, evaluation methodologies, and best practices for ensuring reliable AI systems at scale
Building tools and automation for continuous model validation, performance testing, and quality assessment across global deployments
Collaborating with engineering and product teams to define quality requirements, SLOs, and acceptance criteria for AI services
Qualification
Required
12 years of relevant experience and a Bachelor's degree or its equivalent
Have expertise in AI/ML quality assessment, model evaluation, and performance testing of inference systems
Demonstrate extensive knowledge of LLM evaluation frameworks, metrics (perplexity, BLEU, ROUGE, etc.), and methodologies for assessing model accuracy and output quality
Show mastery of test automation, CI/CD integration, and building quality infrastructure for distributed systems at scale
Possess solid opinions on AI quality standards including model explainability, bias detection, safety validation, and responsible AI practices
Demonstrate excellent technical leadership with ability to influence quality culture, mentor engineers, and establish testing best practices across teams
Preferred
Have a background in chaos engineering and reliability testing for distributed systems
Possess experience with prompt engineering, RAG evaluation, AI application testing patterns
Have knowledge of GPU performance profiling and inference optimization validation
Have experience with observability tools and performance analytics for AI workloads
Have knowledge with compliance testing frameworks (FedRAMP, SOX, GDPR)
Benefits
Healthcare
401K savings plan
Company holidays
Vacation (in the form of PTO)
Sick time
Family friendly benefits including parental leave
Employee assistance program including a focus on mental and financial wellness
Company
Akamai Technologies
Akamai is a provider of cloud services that help enterprises provide secure, high-performing user experiences on any device.
5001-10000 employees
H1B Sponsorship
Akamai Technologies has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (219)
2024 (204)
2023 (227)
2022 (236)
2021 (247)
2020 (279)
Funding
Current Stage
Public CompanyTotal Funding
$2.63BKey Investors
IVP
2025-05-14Post Ipo Debt· $1.5B
2023-08-15Post Ipo Debt· $1.1B
2001-08-01Post Ipo Equity
Leadership Team
Tom Leighton
Chief Executive Officer
Charlie Gero
CTO
Recent News
Small Business Trends
2026-01-06
Digital Commerce 360
2025-12-30
Benzinga.com
2025-12-29
Company data provided by crunchbase