Celonis · 2 days ago

Staff Quality Engineer

Raleigh, NC

Full-time

Onsite

Mid Level

Wonder how qualified you are to the job?

Maximize your interview chances

AnalyticsBig Data

Actively Hiring

Insider Connection @Celonis

Discover valuable connections within the company who might provide insights and potential referrals, giving your job application an inside edge.

Responsibilities

Operate independently as a member of a cross-functional team, consistently advocating to push quality left and drive customer satisfaction of our product offerings

Own responsibility for one or more complex areas of testing

Drive robust and high-quality test coverage with minimal overlap in the team, ensuring the definition of done includes minimal but sufficient automation and recognize the level of automation necessary to reach the goals

Research, recommend and implement tools and processes as needed with the goal of increasing automation and test coverage

Collaborate across the EPE Chapter, mentoring other team members and contributing to efforts to improve overall process and quality

Live by data not opinions, ensuring your team consistently meets auditability goals and that your team KPIs reflect the truth of overall quality status

Serve in a capacity as a test coach to developers; reviewing unit tests and suggesting methods for better coverage

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Quality AssuranceAgile ProcessesTest AutomationML AlgorithmsLarge Language ModelsModel EvaluationValidationBias DetectionPrompt EngineeringAI/ML Performance MetricsProgramming LanguagesPythonPyTestJavaRestAssuredLLM Response Evaluation ToolsPromptfooCI/CDMonitoring ToolsDataDogAnalyticalProblem-Solving

Required

Strong understanding of Quality Assurance, Agile Processes and Test Automation

Proficiency in traditional ML algorithms as well as large language models (LLMs)

Proficiency in Model Evaluation and Validation as well as knowledge bias detection and mitigation techniques

Knowledge of prompt engineering techniques

Familiarity with AI/ML performance metrics

Competence in programming languages and test automation tools (Python, PyTest, Java, RestAssured or similar), knowledge about LLM response evaluation tools (e.g. Promptfoo) will be considered a plus

Competence with CI/CD as well as monitoring tools (e.g. DataDog)

Strong analytical and problem-solving skills