Apply on Employer Site

Oracle · 3 months ago

Principal Applied Scientist (OCI/GEN AI- Data Evaluation)

United States

Full-time

Remote

Senior Level, Lead/Staff

3+ years exp

Oracle is seeking a Principal Applied Scientist to evaluate LLM and LMM models, creating innovative solutions to real-world problems. The role involves conducting research, producing code, and collaborating with teams to integrate evaluation capabilities into applications.

Data GovernanceData ManagementEnterprise SoftwareInformation TechnologySaaSSoftware

No H1B

Responsibilities

Conducting in-depth research on foundation model evaluation

Producing production-ready code for handoff to engineering counterparts

Helping to build and mentor a high-performing team of scientists and engineers

Working closely with cross-functional teams to integrate evaluation capabilities into various applications and products

Identifying new opportunities for evaluation and exploring emerging technologies

Staying up-to-date with industry trends and advancements in evaluation

Designing and executing experiments

Researching new algorithms

Finding new ways of optimizing risk, profitability, and customer experience

Qualification

Machine Learning TechniquesFoundation Model EvaluationPythonTensorFlowPyTorchResearch SkillsCross-functional CollaborationTeam Mentoring

Required

3 to 5+ years of experience in machine learning techniques

Deep understanding of foundation model evaluation, traditional NLP/CV metrics, LLMaaJ techniques, human evaluation, confidence estimation, agentive application evaluation, and RAG application evaluation

Experience in conducting in-depth research

Ability to produce production-ready code

Experience collaborating with cross-functional teams

Strong background in programming languages, such as Python

Experience with machine learning frameworks, such as TensorFlow or PyTorch

Ability to design and execute experiments

Researching new algorithms

Finding new ways of optimizing risk, profitability, and customer experience

Staying up-to-date with industry trends and advancements in evaluation

Preferred

PhD in Computer Science, Mathematics, Statistics, Physics, Linguistics, or a related field with a dissertation or thesis centered on machine learning techniques

Master's or Bachelor's degree with relevant experience