Apple · 1 day ago
Applied Scientist, AI Evaluation Platform
Seattle, WA
Full-time
Onsite
Senior Level
$140K/yr - $210K/yrApple is a company that values individual imaginations and diversity, driving innovation in every product and service they create. They are seeking an Applied Scientist to design and develop automated benchmarking methodologies for AI-powered code assistant tools, collaborating with various teams to ensure high-quality evaluation frameworks.
AppsArtificial Intelligence (AI)BroadcastingDigital EntertainmentFoundational AIMedia and EntertainmentMobile DevicesOperating SystemsTVWearables
Responsibilities
Design scientifically grounded benchmarking methodologies for code assistants, covering multiple dimensions of quality (e.g. correctness, performance) across several use cases
Developing automated evaluation pipelines that collect, automatically judge, and analyze model outputs at scale
Create and curate datasets, tasks, and coding scenarios that represent realistic developer workflows across multiple languages and domains
Define and validate new metrics for complex phenomena such as tool reliability, reasoning quality, or multi-turn developer interaction patterns
Apply statistical rigor and reproducibility to above mentioned metrics
Work closely with engineering and research teams to translate experimental findings into actionable model improvements
Publish internal reports and external papers
Monitor evolving industry practices and academic work to ensure benchmarks remain relevant
Qualification
Required
Advanced degree (MS or PhD) in Computer Science, Software Engineering, or equivalent research/work experience
Strong research background in empirical evaluation, experimental design, or benchmarking
Strong proficiency in Python
Intermediate proficiency in Swift
Deep familiarity with software engineering workflows and developer tools
Experience working with or evaluating AI/ML models, preferably LLMs or program synthesis systems
Strong analytical and communication skills, including the ability to write clear reports
Preferred
Publications in ML evaluation or related fields
Experience with automated testing frameworks
Experience constructing human-in-the-loop or multi-turn evaluation setups
Prior work on agentic developer tools
Benefits
Comprehensive medical and dental coverage
Retirement benefits
A range of discounted products and free services
Reimbursement for certain educational expenses — including tuition
Discretionary bonuses or commission payments
Relocation
Company
Apple
Apple is a technology company that designs, manufactures, and markets consumer electronics, personal computers, and software.
Founded in 1976
10001+ employees
H1B Sponsorship
Apple has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (6998)
2024 (3766)
2023 (3939)
2022 (4822)
2021 (4060)
2020 (3656)
Funding
Current Stage
Public CompanyTotal Funding
$5.67BKey Investors
Berkshire HathawayMicrosoftSequoia Capital
2025-05-05Post Ipo Debt· $4.5B
2025-01-16Post Ipo Debt· $0.31M
2021-04-30Post Ipo Equity
Leadership Team
Tim Cook
CEO
Craig Federighi
SVP, Software Engineering
Recent News
Venrock
2025-12-01
2025-09-25
Mac Daily News
2025-09-25
Company data provided by crunchbase