Samsung Semiconductor · 5 hours ago
Principal engineer, AI Serving Framework Architect (Software)
San Jose, California, United States
Full-time
Onsite
Lead/Staff
$219K/yr - $351K/yr
15+ years expSamsung Semiconductor is a global leader in technology solutions, dedicated to pushing the boundaries of innovation. They are seeking a Principal AI System Architect to develop system-level performance models and drive architecture-level design decisions to enhance AI workloads and system architecture.
Semiconductor
Responsibilities
Leading research teams in Korea and proposing technical direction
Research on dynamic scheduling methodologies for maximizing AI inference performance in multi-rack scale memory-centric systems, comprised of heterogeneous compute-capable memory and hierarchical memory
Investigating methods to accelerate search operations in RAG’s vector DB and AI Agent’s knowledge-graph by leveraging compute-capable memory
Studying strategies for optimally placing KVCache and a vector DB in hierarchical memory to minimize frequent SSD accesses and reduce IO stalls
Proposing SW design for implementing the derived optimization algorithms on open-source platforms such as vLLM
Qualification
Required
PhD in Computer Science or a related field with 15+ years of experience in AI Serving Framework for large-scale computing, with focusing on the AI workloads
Led a project to build and optimize a Large Language Model (LLM) Inference Software Stack on a multi-rack scale system to deliver AI Inference services to over 100,000 users
Extensive experience in designing AI Inference Software Stacks for heterogeneous devices
In-depth understanding of the internal architecture and operation mechanisms of inference engines such as vLLM
Proficiency in AI Inference System Profiling and optimization
Knowledge and practical experience with future AI workloads, including reasoning models, multi-modal solutions, AI agents, and world models
Strong understanding of compute, memory, and networking bottlenecks in AI systems
Required skillsets: PyTorch, Python, and C++
A collaborative mindset, curiosity, and resilience in solving complex challenges
Excellent verbal, presentation, and written communication skills
Preferred
Native or fluent Korean speakers are preferred
Benefits
Medical/Dental/Vision/401k
Charitable giving match
4+ weeks of paid time off a year
Stipend for fertility care or adoption
Medical travel support
Virtual vet care for your fur babies
On-demand apps and free confidential therapy sessions
Onsite Café and gym
Virtual classes
Flexible environment
Company
Samsung Semiconductor
Samsung Semiconductor, Inc. (SSI) is a multi-billion dollar wide range of industry-leading semiconductor solutions.
Founded in 1974San Jose, California, USA
10001+ employees
H1B Sponsorship
Samsung Semiconductor has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (130)
2024 (110)
2023 (153)
2022 (134)
2021 (124)
2020 (134)
Funding
Current Stage
Late StageLeadership Team
Daehee Lee
Principal Engineer
Eric Hibbard
Director, Product Planning – Security
Recent News
Android Authority
2025-12-22
2025-11-07
PhAndroid.com
2025-11-07
Company data provided by crunchbase