Laivly · 1 month ago
Sr. Machine Learning Scientist & Engineer
United States
Full-time
Remote
Mid, Senior Level
2+ years expLaivly is an ambitious company of innovators building and shaping the future of customer service technology. They are seeking a Sr. Machine Learning Scientist & Engineer to lead prompt optimization and validation efforts for LLM-powered features, working closely with engineering and product teams to design and evaluate robust AI experiments.
Artificial Intelligence (AI)Business IntelligenceChatbotCustomer ServiceGenerative AIInformation TechnologySoftware
Responsibilities
Lead prompt design, optimization, versioning, and documentation
Generate and maintain synthetic datasets for pre-release validation and testing
Design and run experiments to measure accuracy, robustness, and safety of LLM behavior
Define and monitor key metrics such as hallucination rates, latency, token cost, and regression risk
Build evaluation pipelines, dashboards, and drift detection tools
Implement rigorous statistical methods and produce reproducible reports with effect sizes and confidence intervals
Investigate regressions or prompt failures in production and coordinate mitigation with stakeholders
Collaborate with cross-functional teams on model integration, rate limiting, and cost controls
Maintain prompt libraries, test suites, and operational runbooks
Qualification
Required
Bachelor's degree in Computer Science, Computational Linguistics, Statistics, or related field; Master's preferred
2+ years working directly with LLMs, prompt design, or applied NLP in production
Experience creating synthetic datasets using programmatic generation or controlled sampling
Deep knowledge of experimental design, statistical inference, and power analysis
Strong Python programming skills and familiarity with statistical/ML frameworks
Experience defining observability for model outputs and instrumenting alerting systems
Familiarity with cloud environments (AWS, Azure), Kubernetes, and production deployment practices
Proficiency in SQL and experience with reproducible analysis pipelines
Excellent communication skills and strong documentation practices
Company
Laivly
Laivly’s platform, Sidd, enables contact center teams to build, deploy, and manage their own AI agents, securely and at scale
51-200 employees
Funding
Current Stage
Growth StageLeadership Team
Jeff Fettes
Chief Executive Officer
Recent News
2025-08-21
Company data provided by crunchbase