ChatGPT Jobs ยท 6 days ago
Senior Data Scientist - Generative AI
Mountain View, CA
Full-time
Hybrid
Senior Level
5+ years expChatGPT Jobs is seeking a Senior Data Scientist specializing in Generative AI. The role involves designing and operationalizing evaluation frameworks for GenAI systems, conducting product analytics, and developing methodologies to improve AI model performance and data generation flows.
Computer Software
Responsibilities
Develop and improve evaluation frameworks for GenAI features (text, image, 3D, 4D, agentic workflow), including eval experiment design, eval dataset design, label reliability analysis, results analysis, and online evaluation based on user behavior and feedback
Establish best practices and guidelines for GenAI evaluation
Conduct product analytics, online experiments (A/B tests) and causal analyses to quantify GenAI feature impact and identify opportunities
Build automated evaluation systems, such as research and implement LLM-as-judge and VLM-as-judge methods
Research and apply state-of-the-art methodologies in GenAI evaluation
Advance reproducible evaluation tooling to lift evaluation rigor and efficiency at the company
Proactively explore and develop solutions to improve the AI model and data generation flow, ensuring high-quality input for training and deployment
Design and implement agentic workflows and AI agents to enable teams to effectively access data, extract data insights, and follow best data practices
Partner closely with cross-functional teams to align goals, plans, and execution
Qualification
Required
An advanced Degree and/or PhD in Statistics, Economics, Operations Research, Computer Science, Applied Math, Physics, Engineering, or another quantitative field
5+ years of experience in data science or a related field
Familiarity with GenAI models and GenAI evaluation methods
Passion for the GenAI field and enthusiasm for continuously improving methods and practices to drive product quality and business impact
Ability to effectively use AI tools to enhance productivity in research, ideation, coding, and documentation
Strong learning agility, experience conducting applied research or writing technical papers is a plus
Proficiency in SQL, Hive, or Spark for transforming and manipulating large datasets
Experience with scripting languages such as Python or R
A demonstrated track record of solving open-ended data science and modeling problems that drive business impact and improve user experience
Company
ChatGPT Jobs
We find the best job offers for experts in ChatGPT and related technologies.
2-10 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase