Apply on Employer Site

Pryon · 2 months ago

Senior Engineering Manager - Accelerated Compute Memory Systems

United States

Full-time

Remote

Senior Level, Lead/Staff

$220K/yr - $250K/yr

10+ years exp

Pryon is a team of AI, technology, and language experts building an industry-leading knowledge management and Retrieval-Augmented Generation platform. They are seeking a Senior Engineering Manager to lead the team in designing distributed systems for large-scale AI memory workloads and ensure scalability and reliability in a fast-growing startup environment.

Artificial Intelligence (AI)Computer VisionGenerative AIKnowledge ManagementMachine Learning

Growth Opportunities

No H1B

Responsibilities

Build and lead a team delivering cloud-native ingestion, retrieval, and inference layers that will power mission-critical deployments for commercial and federal entities with millions of public users

Architect and deliver scalable, fault-tolerant distributed systems capable of handling billions of documents and burst loads of 30K+ concurrent users on managed cloud infrastructure and on-premises deployments

Guide implementation of multimodal ingestion pipelines (PDF, HTML, DOCX, JSON, XML, PPTX, TIFF) optimized for cloud-scale AI memory workloads

Oversee design and optimization of LLM-driven data ingestion and retrieval workflows using modern orchestration frameworks

Own optimization and tuning of high-throughput, low-latency production environments via async orchestration and resource management

Establish performance benchmarking, compliance frameworks, and automated testing strategies for petabyte-scale systems

Balance technical leadership with people leadership—guiding architecture decisions at the application and service layer while scaling and mentoring a high-performing team

Collaborate cross-functionally with Product, Executive Leadership, and Customer Success in a dynamic startup environment

Qualification

PythonDistributed systemsCloud platformsAI/ML systemsML orchestration frameworksVector databasesMessage queuing systemsMultimodal ingestion pipelinesLLM optimizationInfrastructure-as-codeStartup dynamicsTeam leadershipCommunication skills

Required

10+ years in software engineering, 5+ years in management roles delivering large-scale AI/ML systems and cloud infrastructure

Expert-level proficiency in Python, with strong experience in at least one systems language (Go, Rust, C++, or Java)

5+ years building production-grade distributed systems on cloud platforms (AWS, GCP, or Azure)

Hands-on experience with modern ML orchestration frameworks (Ray, Kubeflow, Airflow, or similar open-source tools)

Production experience with vector databases (Pinecone, Weaviate, Qdrant, Milvus, or similar)

Deep understanding of message queuing and streaming systems (Kafka, Pulsar, RabbitMQ, Kinesis)

Proven track record designing and operating scalable, fault-tolerant distributed architectures in cloud environments

Direct experience building multimodal ingestion pipelines for knowledge management platforms

Experience optimizing LLM inference and retrieval workloads at the application/framework level (PyTorch, TensorFlow, vLLM, or similar)

Previous success managing engineering teams delivering production-scale AI infrastructure in startup or high-growth environments

Deep understanding of cloud-native distributed systems architecture: compute orchestration (Kubernetes/EKS/GKE), storage systems, networking, observability, security, disaster recovery, and cost optimization

Strong knowledge of AI memory and knowledge management system design patterns, embedding models, retrieval strategies, and LLM integration patterns

Experience with infrastructure-as-code (Terraform, CloudFormation, Pulumi) and modern DevOps practices

Familiarity with distributed tracing, metrics, and logging systems (Datadog, Prometheus, Grafana, CloudWatch)

Experience with parallel programming models (e.g., MPI, OpenMP, CUDA)

Custom hardware accelerator design or bare-metal cluster management

On-premises datacenter operations or HPC cluster management tools (e.g., Slurm)

Demonstrated ability to mentor engineers and build high-performing teams in ambiguous, fast-paced environments

Strong communication skills with ability to translate technical decisions for executive and product stakeholders

Comfort with startup dynamics: rapid iteration, evolving requirements, and wearing multiple hats

Proven track record of candor and transparency when discussing technical tradeoffs and knowledge boundaries

Experience balancing technical excellence with pragmatic delivery in resource-constrained environments