ByteDance · 21 hours ago
Tech Lead Software Engineer - AI Inference Infrastructure
Seattle, WA
Full-time
Onsite
Senior Level
$198K/yr - $416K/yr
5+ years expByteDance is a global technology company known for its innovative products like TikTok. The Tech Lead Software Engineer will be responsible for designing and building large-scale AI inference infrastructure, collaborating across teams to deliver advanced solutions, and ensuring high performance and scalability in cloud-native systems.
ContentData MiningFoundational AIInternetSocial Media
Responsibilities
Design and build large-scale, container-based cluster management and orchestration systems with extreme performance, scalability, and resilience
Architect next-generation cloud-native GPU and AI accelerator infrastructure to deliver cost-efficient and secure ML platforms
Collaborate across teams to deliver world-class inference solutions using vLLM, SGLang, TensorRT-LLM, and other LLM engines
Stay current with the latest advances in open source (Kubernetes, Ray, etc.), AI/ML and LLM infrastructure, and systems research; integrate best practices into production systems
Write high-quality, production-ready code that is maintainable, testable, and scalable
Qualification
Required
B.S./M.S. in Computer Science, Computer Engineering, or related fields with 5 years of relevant experience (Ph.D. with strong systems/ML publications also considered)
Strong understanding of large model inference, distributed and parallel systems, and/or high-performance networking systems
Hands-on experience building cloud or ML infrastructure in areas such as resource management, scheduling, request routing, monitoring, or orchestration
Solid knowledge of container and orchestration technologies (Docker, Kubernetes)
Proficiency in at least one major programming language (Go, Rust, Python, or C++)
Preferred
Experience contributing to or operating large-scale cluster management systems (e.g., Kubernetes, Ray)
Experience with workload scheduling, GPU orchestration, scaling, and isolation in production environments
Hands-on experience with GPU programming (CUDA) or inference engines (vLLM, SGLang, TensorRT-LLM)
Familiarity with public cloud providers (AWS, Azure, GCP) and their ML platforms (SageMaker, Azure ML, Vertex AI)
Strong knowledge of ML systems (Ray, DeepSpeed, PyTorch) and distributed training/inference platforms
Excellent communication skills and ability to collaborate across global, cross-functional teams
Passion for system efficiency, performance optimization, and open-source innovation
Benefits
Medical, dental, and vision insurance
401(k) savings plan with company match
Paid parental leave
Short-term and long-term disability coverage
Life insurance
Wellbeing benefits
10 paid holidays per year
10 paid sick days per year
17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure)
Company
ByteDance
ByteDance is a technology company that develops content creation platforms and services.
10001+ employees
H1B Sponsorship
ByteDance has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1350)
2024 (1123)
2023 (775)
2022 (487)
2021 (417)
2020 (245)
Funding
Current Stage
Late StageTotal Funding
$9.8BKey Investors
Capital TodayG42Tiger Global Management
2025-11-20Secondary Market· $300M
2024-07-25Secondary Market
2023-03-14Secondary Market· $100M
Leadership Team
Jochen Bischoff
Head of Global Business Solutions - Africa
Matty Lin
General Manager, Global Business Solutions, KR
Recent News
2026-01-08
Company data provided by crunchbase