Arch · 3 months ago
Executive AI Solutions Architect/Engineer — Federal/Civilian
United States
Full-time
Remote
Senior Level, Lead/Staff
$120K/yr - $140K/yr
10+ years expArch Systems, LLC is seeking an Executive AI Solutions Architect/Engineer to design and deliver secure AI systems for federal civilian missions. This hands-on role involves coding, leading solution development, and presenting to federal stakeholders while ensuring compliance and security standards are met.
Cloud InfrastructureSoftwareUX Design
No H1B
Security Clearance RequiredU.S. Citizen Only
Responsibilities
Write production Python; build FastAPI services; implement RAG pipelines (embeddings + vector DB); fine-tune/evaluate models; containerize (Docker) and deploy to Kubernetes with Terraform and GitHub/GitLab CI
Spend ≥50% of build phases actively coding, pairing, and conducting design/code reviews
Lead discovery, fit-gap, and make/buy; define requirements and acceptance criteria; produce reference architectures (data → model → serving → security → observability) with trade-off analyses (cost/latency/accuracy/privacy)
Build decks and demos for federal audiences; clearly describe benefits and mission impact with KPIs (cycle-time reduction, precision/recall, cost per query, latency, compliance posture)
Contribute to RFIs/RFPs, technical volumes, and orals; handle Q&A with mixed technical/non-technical stakeholders
Stand up MLflow/Databricks/SageMaker/Azure ML; manage model registry, feature store, CI/CD, gated promotions, canary/rollback; automate drift/bias/robustness monitoring and alerts
Build secure ingestion/curation pipelines; implement PII/PHI controls (de-identification, tokenization), data quality checks, catalogs/lineage, RBAC/least privilege, and data contracts
Engineer prompt flows, grounding, guardrails, and policy enforcement; define offline/online evaluation with golden sets and human-in-the-loop; monitor factuality, relevance, toxicity; optimize context and token/latency budgets
Embed NIST 800-53/37 (RMF), 800-171, FedRAMP controls; implement authN/Z, encryption, secrets, logging/auditing, boundary protections; contribute to SSPs, POA&Ms, continuous monitoring; coordinate with ISSO/3PAO
Instrument with logs/metrics/traces (OpenTelemetry); set SLIs/SLOs; build dashboards/alerts; run game days and blameless post-mortems; design HA/DR patterns to reduce MTTR and improve availability
Model TCO; track cloud spend and cost/query; improve with right-sizing, autoscaling, caching, batching, distillation/quantization; present ROI trade-offs to stakeholders
Publish reference implementations, IaC modules, pipeline/evaluation templates, and security baselines; drive adoption and measure reuse impact across programs
Lead cross-functional agile teams; groom technical backlogs; ensure DoD (“definition of done”) includes tests, docs, security scans, and performance baselines; mentor engineers via pairing and reviews
Maintain runbooks, ADRs, data contracts, and user guides meeting IV&V standards; manage versioning, release notes, stakeholder sign-offs; ensure Section 508 for end-user UIs/content
Qualification
Required
10+ years software/ML engineering; 5+ years architecting and shipping enterprise AI/ML solutions
Recent, hands-on delivery with Python, PyTorch or TensorFlow, API development (FastAPI), containers (Docker), and microservices on Kubernetes
Demonstrated GenAI/RAG delivery: embeddings, vector databases (FAISS/Milvus/pgvector), prompt orchestration/guardrails, evaluation and optimization
Proven solutioning experience: leading discovery, estimating, architecture decisions, risk/cost trade-offs, and creating exec-ready artifacts that communicate impact and benefits
Implemented MLOps on AWS and/or Azure using MLflow/Databricks/SageMaker/Azure ML; CI/CD for models and data pipelines; model registry; automated testing
Federal delivery with working knowledge of FISMA/FedRAMP/RMF and ATO processes; ability to brief mixed technical/non-technical federal audiences
Excellent communication/storytelling with quantitative framing (KPIs, SLAs/SLOs, ROI)
Architecture: 5 years (Required)
Security clearance: Confidential (Required)
Preferred
Public Trust or clearance eligibility; prior support within HHS, DHS, USDA, NOAA, or IRS
Certifications: AWS Solutions Architect, Azure Solutions Architect/DP-203/DP-100; security/architecture (CISSP, CCSP, TOGAF) a plus
Experience with model risk management, adversarial testing/red teaming, and Section 508 evaluation practices
Technical oversight is a plus (while remaining hands-on)
AI software/ML engineering: 7 years (Preferred)
License/Certification: AWS Solutions Architect or Azure Solutions Architect (Preferred)
Company
Arch
Arch is a market provider of SAP usability software and delivering solutions to enhance the SAP user experience.
11-50 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase