National Institute of Standards and Technology (NIST) · 2 days ago
AI Evaluations Scientist, Biological AI Models
San Francisco, CA
Full-time
Onsite
Mid, Senior Level
1+ years expThe National Institute of Standards and Technology (NIST) is seeking a candidate for the position of AI Evaluations Scientist, focusing on biological AI models. The role involves designing and executing evaluations related to national security and biological AI, adapting models, curating datasets, and collaborating with various experts.
GovernmentLife ScienceNanotechnology
No H1B
U.S. Citizen Only
Responsibilities
Design, develop, and execute national security-related evaluations of frontier biological AI models, such as biological language models and biomolecular modeling and design tools
Adapt biological AI models for evaluation purposes
Curate relevant benchmarks and ground-truth datasets
Develop and refine scalable software workflows, custom tooling, and evaluation pipelines
Quantify uncertainty and statistical significance of evaluation results
Analyze biosecurity/national security implications of demonstrated model capabilities
Continuously adapt evaluation strategies to align with the state-of-the-art AI architectures and modalities
Collaborate with cross-functional teams of biology, machine learning, and policy subject matter experts
Communicate complex technical findings to technical and non-technical stakeholders
Qualification
Required
Experience must be IT related; the experience may be demonstrated by paid or unpaid experience and/or completion of specific, intensive training (for example, IT certification), as appropriate
Attention to Detail - Is thorough when performing work and conscientious about attending to detail
Customer Service - Works with clients and customers to assess their needs, provide information or assistance, resolve their problems, or satisfy their expectations; knows about available products and services; is committed to providing quality products and services
Oral Communication - Expresses information to individuals or groups effectively, taking into account the audience and nature of the information; makes clear and convincing oral presentations; listens to others, attends to nonverbal cues, and responds appropriately
Problem Solving - Identifies problems; determines accuracy and relevance of information; uses sound judgment to generate and evaluate alternatives, and to make recommendations
ZP-2210-IV: In addition to the basic requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-12 level (ZP-III at NIST). Specialized experience is defined as experience conducting research related to biological Artificial Intelligence (AI)/Machine Learning (ML) modeling approaches (e.g., numerous relevant scientific publications); developing, adapting, and/or evaluating biological AI models; demonstrated understanding of biosecurity and/or national security implications of biological AI models; collaborating and coordinating with teams at the intersection of AI and biology
ZP-2210-V: In addition to the above requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-14 level (ZP-IV at NIST). Specialized experience is defined as experience demonstrating exceptional leadership in research related to biological Artificial Intelligence (AI)/Machine Learning (ML) modeling approaches (e.g., recognition as a technical authority; numerous highly-cited relevant scientific publications); highly skilled in developing, adapting, and/or evaluating biological AI models (e.g., having authored a widely-used benchmark or model); demonstrated understanding of biosecurity and/or national security implications of biological AI models; independently executed projects at the intersection of AI and biology that have had significant influence and impact
Company
National Institute of Standards and Technology (NIST)
We are the National Institute of Standards and Technology (NIST), a non-regulatory federal agency within the U.S. Department of Commerce.
1001-5000 employees
Funding
Current Stage
Late StageTotal Funding
unknownKey Investors
U.S. Department of Homeland Security
2023-01-17Grant
Leadership Team
George Jenkins, CPA, MBA, FAC-P/PM
Chief Financial Officer
Anita Vanek
Deputy Chief Operating Officer
Recent News
Help Net Security
2025-12-22
2025-12-18
NYT > Technology
2025-12-06
Company data provided by crunchbase