AssemblyAI · 3 weeks ago
Senior Software Engineer, Go - LLM Team
United States
Full-time
Remote
Senior Level
$180K/yr - $240K/yr
5+ years expAssemblyAI builds best-in-class Speech AI models and is looking for a Senior Software Engineer to join their LLM team. This role focuses on building and maintaining the LLM gateway service, ensuring reliable access to multiple LLM providers while solving customer challenges and improving workflows.
Artificial Intelligence (AI)Developer APIsFoundational AIGenerative AIInformation TechnologySpeech Recognition
Responsibilities
Solve Customer Problems Partner closely with the customer success team to understand customer use cases, challenges, and integration needs
Translate customer pain points surfaced by the CS team into technical solutions and product improvements
Build features and tooling that directly address customer needs and improve their workflows
Provide technical guidance and expertise to the customer success team to help them support customers effectively
Drive Technical Execution Own and deliver complete features and integrations within our LLM gateway service
Build and maintain integrations with multiple LLM providers and AI services (OpenAI, Anthropic, Google Vertex, AWS Bedrock etc.)
Write clean, maintainable, well-tested code following best practices
Design and implement scalable, fault-tolerant solutions with appropriate abstractions
Proactively identify and address technical debt, reliability issues, and code quality concerns
Participate in on-call rotation to ensure service reliability and rapid incident response
Elevate Engineering Standards Conduct thorough code reviews focused on maintainability, testing, reliability, and architectural concerns
Ensure proper test coverage across unit, integration, and end-to-end testing levels
Improve code maintainability and extensibility through targeted refactoring
Contribute to runbooks, incident postmortems, customer-facing documentation, and knowledge sharing
Identify bottlenecks and propose solutions to improve team velocity and system reliability
Build and Collaborate Work closely with product, customer success, and engineering teams to deliver solutions
Collaborate with external API providers to troubleshoot integration issues and improve reliability
Translate ambiguous customer requirements into well-scoped technical work
Proactively communicate status, risks, and blockers
Mentor junior engineers on coding standards, reliability practices, and customer-centric thinking
Participate in architectural discussions and contribute to technical decisions
Qualification
Required
5+ years of professional software engineering experience
Strong proficiency in Go with demonstrated ability to write production-quality code
Hands-on experience with LLM APIs (OpenAI, Anthropic, etc.) and understanding of prompt engineering concepts
Experience building or working with API gateways, proxies, and rate limiting systems
Deep experience with prompt engineering, chain-of-thought, few-shot learning, and other LLM optimization techniques
Experience building applications with llms or ai agents with cascading architectures
Solid understanding of software engineering fundamentals: Data structures and algorithms, System design and architectural patterns, Testing strategies (unit, integration, end-to-end), Code review practices and technical collaboration
Experience with: RESTful APIs and distributed systems concepts, Building or integrating with third-party APIs, SQL and database design, Version control (Git) and CI/CD pipelines, Containerization (Docker) and cloud infrastructure
Strong communication skills and ability to collaborate effectively with cross-functional teams
Track record of delivering high-quality, reliable software in a fast-paced, team environment
Ability to thrive in a startup environment with changing priorities and rapid iteration
Preferred
Experience with AWS services (ECS, Lambda, ElastiCache/Redis, RDS, S3, CloudWatch, API Gateway)
Experience with monitoring and observability tools (Datadog, CloudWatch)
Experience with request routing, load balancing, and traffic management
Experience with circuit breakers, retry logic, fallback strategies, and fault tolerance patterns
Understanding of API design, versioning, and backward compatibility
Experience working closely with customer success or support teams
Benefits
100% employer-paid benefits
401k match up to 4% for US-based full time team members
Company
AssemblyAI
AssemblyAI is an artificial intelligence company that engages in building a platform of APIs to transcribe and understand audio data.
51-200 employees
H1B Sponsorship
AssemblyAI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2024 (1)
Funding
Current Stage
Growth StageTotal Funding
$113.12MKey Investors
AccelInsight PartnersAccel,Daniel Gross,John Collison,Nat Friedman
2023-12-03Series C· $50M
2022-07-14Series B· $30M
2022-03-04Series A· $28M
Leadership Team
Dylan Fox
Founder & CEO
Recent News
2025-11-24
Crunchbase News
2025-09-10
EIN Presswire
2025-07-15
Company data provided by crunchbase