Apply on Employer Site

NetSuite · 3 months ago

Principal Applied Scientist (OCI/GEN AI- Data Evaluation)

United States

Full-time

Remote

Senior Level, Lead/Staff

3+ years exp

NetSuite is seeking a Principal Applied Scientist to evaluate LLM and LMM models, focusing on innovative solutions for real-world problems. The role involves conducting research, producing production-ready code, and collaborating with cross-functional teams to integrate evaluation capabilities into applications and products.

Cloud ComputingComputerCRMiOSSaaSSoftware

No H1B

Responsibilities

Conduct in-depth research on foundation model evaluation

Produce production-ready code for handoff to engineering counterparts

Help build and mentor a high-performing team of scientists and engineers

Work closely with cross-functional teams to integrate evaluation capabilities into various applications and products

Identify new opportunities for evaluation and explore emerging technologies

Stay up-to-date with industry trends and advancements in evaluation

Design and execute experiments, researching new algorithms, and optimizing risk, profitability, and customer experience

Qualification

Machine Learning TechniquesFoundation Model EvaluationPythonTensorFlowPyTorchResearch SkillsExperiment DesignStaying Up-to-dateCross-functional CollaborationMentoring

Required

3 to 5+ years of experience in product development

Ability to read, write, and speak English

Deep understanding of machine learning techniques, including foundation model evaluation, traditional NLP/CV metrics, LLMaaJ techniques, human evaluation, confidence estimation, agentive application evaluation, and RAG application evaluation

Experience in conducting in-depth research

Ability to produce production-ready code

Experience collaborating with cross-functional teams to integrate evaluation capabilities into various applications and products

Strong background in programming languages, such as Python

Experience with machine learning frameworks, such as TensorFlow or PyTorch

Ability to design and execute experiments

Experience researching new algorithms

Ability to optimize risk, profitability, and customer experience

Preferred

PhD in Computer Science, Mathematics, Statistics, Physics, Linguistics, or a related field, with a dissertation or thesis centered on machine learning techniques

Master's or Bachelor's degree with relevant experience