Assembled · 3 months ago
Software Engineer - AI Infrastructure
New York City, NY
Full-time
Hybrid
Senior Level
$135K/yr - $280K/yr
6+ years expAssembled is a company that builds infrastructure for exceptional customer support, enabling companies to deliver efficient service at scale. They are seeking a Software Engineer to join the Infrastructure team, responsible for developing and operating core systems for their AI agent platform, which automates support workflows and aims to scale infrastructure to meet growing demand.
Customer ServiceInformation TechnologySchedulingSoftware
Responsibilities
Manage and scale the infrastructure that serves LLM-powered agents across chat, email, and voice
Select inference strategies, integrate with model providers (e.g. OpenAI, Anthropic), and dynamically route traffic for performance and cost efficiency
Own highly-available, fast-access storage and indexing layers optimized for real-time AI interactions
Build systems like network-level intrusion detection (IDS/IPS), audit logging, and LLM usage policy enforcement to meet enterprise expectations
Operate systems that surface key metrics—token usage, latency, cost per response, and quality signals
Explore and evangelize the use of AI to accelerate internal engineering workflows through internal chat agents, pair programming tools, and intelligent automation
Qualification
Required
6+ years of engineering experience, with past ownership of high-scale, production-critical infrastructure
Experience with distributed systems and container orchestration (especially Kubernetes)
Worked with AI/ML platforms or are excited to build foundational infrastructure for LLM-based applications
Thrive in fast-paced environments with shifting requirements and ambiguous problem spaces
Motivated by impact, enjoy deep technical challenges, and want to work cross-functionally across security, AI, and product
Strong familiarity with one or more parts of our tech stack: Cloud provider: AWS, Orchestration: Kubernetes + Karpenter, LLM integration: Experience with OpenAI, Anthropic, or open-source model serving (e.g., vLLM, HuggingFace TGI, Ray Serve), Prompt & embedding infrastructure: Vector databases (e.g., Pinecone, Weaviate, PGVector), semantic search, prompt templating systems, Datastores: Postgres + PgBouncer, Snowflake, Redis, Languages: Go and Python, Monitoring & CI/CD: Datadog, Mezmo, CloudWatch, Buildkite, CircleCI
Benefits
Generous medical, dental, and vision benefits
Paid company holidays, sick time, and unlimited time off
Monthly credits to spend on each: professional development, general wellness, Assembled customers, and commuting
Paid parental leave
Hybrid work model with catered lunches everyday (M-F), snacks, and beverages in our SF & NY offices
401(k) plan enrollment
Company
Assembled
Assembled is a workforce management platform that helps modern organizations scale customer support teams.
51-200 employees
H1B Sponsorship
Assembled has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (4)
2024 (4)
2023 (4)
2022 (2)
2021 (1)
Funding
Current Stage
Growth StageTotal Funding
$70.7MKey Investors
New Enterprise AssociatesEmergence CapitalStripe
2022-05-26Series B· $51M
2021-03-11Series A· $16.6M
2020-03-11Seed· $3.1M
Leadership Team
Ryan Wang
Co-founder/CEO
John Wang
Cofounder and CTO
Recent News
2025-08-27
Talk Business & Politics
2024-05-20
Company data provided by crunchbase