Staff Software Engineer: Microservice Infrastructure & Real-Time ML Inference jobs in United States
cer-icon
Apply on Employer Site
company-logo

Sanas · 1 month ago

Staff Software Engineer: Microservice Infrastructure & Real-Time ML Inference

positionPalo Alto, CA
timeFull-time
remoteOnsite
senioritySenior Level, Lead/Staff
date7+ years exp

Sanas is pioneering the future of human communication with its real-time speech transformation platform. The Staff Software Engineer will design and build the next generation of real-time translation infrastructure, focusing on architecting microservices that support low-latency audio/video processing and scaling the platform for millions of concurrent sessions.

Artificial Intelligence (AI)Language LearningSaaSSoftwareTranslation Service
check
H1B Sponsor Likelynote

Responsibilities

Lead the design for high-throughput, low-latency microservices that enable bidirectional streaming in Sanas’ audio/video pipelines
Build event/telemetry/feature pipelines (Kafka/Redis/DynamoDB) that support near-real-time decisions and model features at scale
Productionize model serving (Triton/vLLM/TorchServe), implement autoscaling/batching/shadow-deploys, and enforce p99/p999 budgets
Establish SLOs/error budgets, graceful degradation (keep call quality first), idempotency, circuit breakers, retries with jitter, and chaos drills
Lead Sanas-wide logging/metrics/tracing (OpenTelemetry), RED/USE dashboards, and symptom-based alerting
Drive cross-team designs, mentor seniors, lead postmortems/design reviews, and lay the foundation for shared libraries and patterns (auth, interceptors, tracing, schema rollout)

Qualification

Distributed architecturePythonGoContainerizationOrchestrationML InferenceTechnical leadershipGRPC/protobufData streamingSoft skills

Required

7+ years of Software Engineering experience, with a focus on distributed architecture and technical leadership
Strong proficiency in Python or Go; strong async/concurrency (asyncio/futures), profiling, and GC/heap tuning
Strong proficiency in Containerization and Orchestration: AWS/Azure, Terraform, Kubernetes, IaaC patterns and node pools. (CPU/GPU)
Experience in ML Inference: Triton/vLLM/TorchServe; GPU scheduling/packing, batching, A/B and shadow traffic
Experience with gRPC/protobuf at scale (versioning, interceptors, performance tuning, and compatibility testing)

Preferred

Experience with WebRTC/SRTP, RTP/RTCP, NAT traversal STUN/TURN,, SIP interop; FFmpeg/codec tradeoffs
Experience in data streaming with Kafka, Redis, DynamoDB; exactly-once/at-least-once patterns; stream-batch bridges

Company

Sanas

twittertwitter
company-logo
Sanas is a real-time speech-understanding platform that modulates accents while preserving voices and emotions for natural interactions.
dateFounded in 2020
location
Palo Alto, California, USA
people-size51-200 employees
web-link
https://sanas.ai

H1B Sponsorship

Sanas has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (2)
2024 (1)

Funding

Current Stage
Growth Stage
Total Funding
$117.72M
Key Investors
Insight PartnersHuman CapitalVillage Global
2025-02-19Series B· $65M
2023-03-23Series Unknown· $14.72M
2022-03-29Series A· $32M

Leadership Team

leader-logo
Shawn Zhang
Co-Founder & CTO
linkedin
leader-logo
Maxim Serebryakov
Founder
linkedin

Recent News

EIN Presswire
Telephone Call Centers Market to Reach US $150.51 Billion by 2029
2025-11-19
IGEL
IGEL Catalyses a New Era of EUC Transformation and Trusted Endpoint Security
2025-11-07
Canada NewsWire
Frost & Sullivan Recognizes Leading Organizations at the 2025 Best Practices Recognition Gala
2025-10-23
Company data provided by crunchbase