Apply on Employer Site

Productive Playhouse · 1 month ago

German AI Model Rater

United States

Full-time

Remote

Mid Level

$24/hr - $24/hr

Productive Playhouse is a global data company specializing in language and data services. They are seeking experienced German speakers with coding knowledge to join their AI LLM evaluation project, focusing on assessing the quality of AI-generated responses that utilize integrated applications and tools (APIs).

BroadcastingFilmMedia and Entertainment

No H1B

Responsibilities

Analyze and rate AI-generated responses and their associated tool outputs against a detailed set of quality guidelines

Verify that the AI successfully integrates information from different digital services (APIs) as required by the prompt

Evaluate the accuracy, completeness, and overall quality of the AI's response and tool execution

Follow provided instructions to achieve task goals, with all evaluations recorded in English

Qualification

API knowledgeGerman fluencyEnglish comprehensionTechnical guideline adherenceDetail-oriented

Required

Advanced/Expert fluency in the German language, as spoken in Germany (ISO: de_de)

Strong comprehension of English (read and write)

Strong conceptual knowledge of APIs and digital services (ability to understand how AI tools interact with external apps like search, maps, or calendar, without needing to write code)

Ability to follow complex technical guidelines precisely

All participants must have, or be willing to create, an Upwork account

Applicants must pass a quick skill verification assessment (MC, duration ~10 minutes)

Work must be performed on participant's own devices

You must use your own laptop, and smartphone (if required)

A Gmail account may be required to access certain tools. If you do not have a Gmail account, you must be willing to create one

Using AI during the assessment and work is STRICTLY forbidden, and results/inputs will be monitored to detect AI usage