Apply on Employer Site

Straive · 1 day ago

Evaluator - Economics, Econometrics & Finance

United States

Contract

Remote

Entry, Mid Level

2+ years exp

Straive is a global leader in enterprise-grade data analytics and AI solutions, committed to empowering businesses across various industries. The Evaluator role involves reviewing and validating AI responses to assess their usefulness, quality, and safety, while staying current with advancements in the domain and maintaining confidentiality of data.

CRMOutsourcing

Hiring Manager

Jeffrey Evans M Victor

Responsibilities

Review and Validate AI Responses:

Evaluators will work via our online platform

All decisions of the Evaluators must be guided towards evaluating this central theme: usefulness of the AI response. This is the primary metric for measuring AI response quality:

Is the AI response useful in addressing the question posed in the user prompt

The goal is to assess the quality, safety, and utility of AI-generated content- particularly in use cases like literature summarization, clinical question answering, and decision support. Each AI generated response should be scored across 10 key criteria

Evaluators must use a Likert scale (4=Great, 3=Good, 2=Fair, 1=Poor) on the first six criterion, and pass’ or ‘fail’ on the remaining four

A commentary must be provided on the AI response, and flag critical issues (e.g., hallucinations, or, ethical/safety concerns)

Commentaries must be error-free, concise and consistent with established guidelines, evidence-based practices, and current industry standards. Rewriting the commentary and revisiting the evaluation may also be within scope based on feedback from quality checks

Stay Current with latest in their domain: Stay up-to-date with the latest research, guidelines, and advancements in your area of expertise, ensuring that their validation and feedback are informed by the most current and accurate information

Maintain Confidentiality: Adhere to strict confidentiality of the data received for processing and comply with data protection protocols and other relevant laws and guidelines

Qualification

PhDMastersResearch ExperienceSubject Matter ExpertiseAICommunication Skills

Required

PhD or Masters Economics, Econometrics or Finance

Have a minimum of 2 years of research or work experience in a reputed institution or an organization

Strong subject matter expertise

Possess excellent written skills, with the ability to clearly articulate complex curriculum concepts and provide concise commentaries on shortcomings of AI response

Have a basic understanding of Generative AI usage pertaining to economics information processing