MillenniumSoft Inc · 1 month ago
LLM - Full Stack Python + JS - Remote (Stron Exp in LLM Coding tools)
United States
Full-time
Remote
Senior Level
6+ years expMillenniumSoft Inc is a coding-focused team serving as a research partner for a Frontier AI Lab. They are seeking a Full Stack Python + JS developer to build coding tasks, evaluations, datasets, and tooling that help train and improve large language models (LLMs). The role involves writing and debugging production-quality code, designing evaluations, and collaborating with engineers and researchers.
Staffing & Recruiting
No H1B
Responsibilities
Write, review, and debug code across multiple languages
Design tasks and evaluation scenarios for coding, reasoning, and debugging
Investigate LLM outputs and identify hallucinations, regressions, and failure modes
Build reproducible dev environments using Docker + automation tools
Develop scripts, pipelines, and tools for data generation, scoring, and validation
Produce structured annotations, judgments, and high-quality datasets
Run systematic evaluations that help improve model reliability and reasoning
Qualification
Required
Experience using LLM coding tools (Cursor, Copilot, Code Whisperer)
Strong hands-on coding experience (professional or research-based) in one or more of: Python, JavaScript / Node.js, TypeScript (Additional languages like Go, Java, C++, C#, Rust, SQL, R, Dart, etc. are a plus)
Solid experience with Linux + Bash, scripting, and automation
Strong with Docker, reproducible environments, and dev containers
Advanced Git skills (branching, diffs, patches, conflict resolution)
Solid understanding of testing and QA (unit, integration, negative, edge-case focused)
Ability to reliably overlap with 8am-12pm PT
Preferred
Experience using LLM coding tools (Cursor, Copilot, Code Whisperer)
Experience with dataset creation, annotation, evaluation, or ML pipelines
Familiarity with benchmarks like SWE Bench or Terminal Bench
Background in QA automation, DevOps, ML systems, or data engineering
Bachelor's degree in a technical field with 6+ years' experience
Master's degree in a technical field with 4+ years' experience
PhD in a technical field with 2+ years' experience
Company
MillenniumSoft Inc
Hire us to hire for you Staffing is not a straightforward activity.
51-200 employees
Funding
Current Stage
Growth StageLeadership Team
Sailaja Mashetty
Partner - Customer Sucess
Company data provided by crunchbase