JPS Tech Solutions · 10 hours ago
AI Quality Engineer
United States
Full-time
Remote
Senior Level, Lead/Staff
10+ years expJPS Tech Solutions is seeking a highly experienced AI Quality Engineer to lead the testing, validation, and lifecycle management of their AI and automation systems. The role requires expertise in AI/ML, RPA, and cloud-native technologies, focusing on ensuring the quality and compliance of AI systems.
Information TechnologyIT InfrastructureSoftware Engineering
No H1B
Responsibilities
Develop and execute testing frameworks for AI models, workflows, and automation solutions
Validate AI model performance using advanced metrics like F1-score, Precision-Recall, and MAE
Perform adversarial testing to check model robustness and data security
Monitor AI models in production for drift and performance issues
Ensure AI outputs are ethical, unbiased, and compliant with standards
Audit datasets for bias, noise, and completeness; implement automated quality checks
Collaborate with data scientists and engineers to improve model validation
Design test cases and plans for UiPath RPA implementations and integrations
Validate hand-offs between AI agents and RPA bots for accurate task execution
Track defects, document issues, and manage resolutions
Act as AI governance and compliance lead, ensuring adherence to ethical and regulatory guidelines
Promote best practices in prompt engineering, automated testing, and AI quality assurance
Qualification
Required
10+ years of experience in software quality engineering, with at least 3–5 years focused on AI/ML and RPA (UiPath) automation testing
Strong understanding of OCR (Optical Character Recognition), NLP (Natural Language Processing), and LLM architectures
Proficient in Java and/or Python scripting
Hands-on experience with Salesforce and Google Cloud Platform (GCP)
Skilled in system integrations, including SQL, REST APIs, and Google Kubernetes Engine (GKE)
Experience in drift detection, fairness monitoring, and performance validation of AI models
Strong knowledge of test automation, CI/CD pipelines, and software quality best practices
Excellent collaboration, communication, and leadership skills for cross-functional team engagement
Company
JPS Tech Solutions
JPS Tech Solutions offers software engineering, cloud, mobile development, and IT infrastructure solutions.
201-500 employees
Funding
Current Stage
Growth StageCompany data provided by crunchbase