Microsoft · 9 hours ago
Relevance Metrics Data or Applied Scientist
United States, California, Mountain View
Full-time
Onsite
Senior Level, Lead/Staff
$140K/yr - $275K/yr
5+ years expMicrosoft is seeking an accomplished Principal Data Scientist to lead the development and evaluation of offline search metrics for Bing. This role will involve leveraging BigData technologies to analyze search logs, formulate hypotheses about search relevance, and deliver insights to leadership, significantly influencing Bing’s search quality.
Agentic AIApplication Performance ManagementArtificial Intelligence (AI)Business DevelopmentDevOpsInformation ServicesInformation TechnologyManagement Information SystemsNetwork SecuritySoftware
Responsibilities
Sample large, representative datasets from extensive search log repositories utilizing BigData Map-Reduce frameworks and distributed data platforms
Architect and implement scalable data processing and analysis pipelines for offline metric computation, leveraging modern data engineering best practices
Formulate, test, and validate hypotheses regarding search result quality using advanced statistical methods and machine learning models
Design labeling protocols and manage trained crowd judges and auditors for high-fidelity data annotation and validation of search results
Adapt traditional evaluation workflows to incorporate LLM-as-a-judge and fine-tuned LLMs, ensuring robust and scalable quality assessments
Develop and deliver custom reports, visualizations, and presentations to communicate insights and recommendations to senior leadership
Collaborate across multidisciplinary teams to extend offline metric methodologies and support innovative search experiences
Qualification
Required
Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 5+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 7+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10+ years data science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
OR equivalent experience
Preferred
Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 8+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
OR Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 10+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
OR Bachelor's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field AND 12+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
OR equivalent experience
Extensive experience in Information Retrieval, including designing, evaluating, and optimizing search and ranking algorithms
Demonstrated ability to develop and validate metrics for Information Retrieval systems, ensuring robust measurement of relevance, accuracy, and user satisfaction
Expertise in crowdscience methodologies, including designing and running large-scale crowdsourcing experiments for data annotation and model evaluation
Proven track record in architecting and deploying large-scale data pipelines for real-time and batch processing of heterogeneous data sources
Solid background in experimental design, statistical analysis, and A/B testing for data-driven product improvements
Ability to lead cross-functional teams and mentor junior scientists in best practices for data science and machine learning
Ability to work independently, solid collaboration and communication skills
Familiarity with Python, T-SQL, (HTML/JS for dashboarding)
Company
Microsoft
Microsoft is a software corporation that develops, manufactures, licenses, supports, and sells a range of software products and services.
10001+ employees
H1B Sponsorship
Microsoft has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (9192)
2024 (9343)
2023 (7677)
2022 (11403)
2021 (7210)
2020 (7852)
Funding
Current Stage
Public CompanyTotal Funding
$1MKey Investors
Technology Venture Investors
2022-12-09Post Ipo Equity
1986-03-13IPO
1981-09-01Series Unknown· $1M
Leadership Team
Satya Nadella
Chairman and CEO
Vukani Mngxati
Chief Executive Officer - Microsft South Africa
Recent News
2026-01-16
Morningstar.com
2026-01-16
Company data provided by crunchbase