FriendliAI · 11 hours ago
Software Engineer - Inference Engine
New York, NY
Full-time
Hybrid
Senior Level
$160K/yr - $230K/yr
5+ years expFriendliAI is a Redwood City-based startup focused on building a next-generation AI inference platform. The role of Inference Engine Engineer involves optimizing GPU kernels and supporting infrastructure for generative and agentic AI workloads, directly impacting performance for customers.
Artificial Intelligence (AI)Generative AIInformation TechnologyInternetSaaSSoftware
Hiring Manager
Woojin Lee
Responsibilities
Design and optimize custom GPU kernels for AI (e.g., transformer and diffusion) workloads
Contribute to the development of FriendliAI’s kernel compiler, memory planner, runtime, and other core components
Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performance
Analyze performance bottlenecks across the software and hardware stack, and implement targeted optimizations
Drive support for new model architectures and tensor compute patterns
Maintain production-grade performance infrastructure, including profiling, benchmarking, and validation tools
Qualification
Required
5+ years of experience in production or high-impact research environments
Production-level expertise in Python and C++
Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
Experience developing machine learning frameworks or performance-critical runtime systems
Hands-on experience writing and optimizing GPU kernels
Hands-on experience profiling GPU kernels
Experience working with generative AI models such as transformer and diffusion models
Preferred
Experience developing machine learning compilers or code generation systems
Familiarity with dynamic shape compilation, memory planning, and kernel fusion
Contributions to inference engines, compilers, or high-performance numerical libraries
Understanding of multi-GPU and distributed inference strategies
Benefits
Flexible working hours
Daily lunch and dinner provided
Unlimited snacks and beverages
Supportive work environment
Health check-up support
Top-tier equipment support
Competitive compensation
Startup equity
Health insurance
Other benefits
Company
FriendliAI
FriendliAI is an AI infrastructure company that enables deployment, scaling, and monitoring of large language and multimodal models.
11-50 employees
Funding
Current Stage
Early StageTotal Funding
$26.75MKey Investors
Capstone Partners
2025-08-28Seed· $20M
2021-12-15Seed· $6.75M
Leadership Team
Byung-Gon Chun
Chief Executive Officer
Gyeong-In Yu
Chief Technology Officer
Recent News
Inside HPC & AI News | High-Performance Computing & Artificial Intelligence
2025-12-20
2025-12-16
2025-10-28
Company data provided by crunchbase