Apply on Employer Site

iMerit Technology · 1 day ago

English Language Specialist

United States

Contract

Remote

Entry Level

1+ years exp

iMerit Technology is seeking detail-oriented and analytically minded Multimodal GenAI Evaluation Analysts to perform evaluations of AI system outputs across different modalities. The role involves assessing the accuracy, appropriateness, and quality of model outputs, ensuring alignment with project standards and real-world use cases.

Information Technology & Services

H1B Sponsor Likely

Responsibilities

Evaluate outputs generated by LLMs across multiple modalities (text, image captions, video descriptions, and multimodal prompts)

Assess quality against project-specific criteria such as correctness, coherence, completeness, style, cultural appropriateness, and safety

Identify subtle errors, hallucinations, or biases in AI responses

Apply domain expertise and logical reasoning to resolve ambiguous or unclear outputs

Provide detailed written feedback, tagging, and scoring of outputs to ensure consistency across the evaluation team

Escalate unclear cases and contribute to refining evaluation guidelines

Collaborate with Project Managers and Quality Leads to meet accuracy, reliability, and turnaround benchmarks

Qualification

LLMs familiarityMultimodal systems knowledgeData annotation experienceCritical reading skillsCultural awarenessEnglish comprehensionAttention to detailAdaptability

Required

Bachelor's degree/ diploma or equivalent educational qualification

1+ years of experience in data annotation, LLM evaluation, content moderation, or related AI/ML domains

Demonstrated experience working with data annotation tools and software platforms

Strong understanding of language and multimodal communication (instruction following in image generation, fact-checking, narrative coherence in video, etc.)

Ability to adapt quickly to changing project directions and fast-paced work environments

Previous experience creating or annotating complex data specifically for Large Language Model (LLM) training

Candidates should be aware that occasional exposure to NSFW or otherwise sensitive content may occur due to imperfections in client-provided datasets

Preferred

Excellent English comprehension (CEFR B2 or above); additional languages a plus

Familiarity with LLMs, generative AI, and multimodal systems

Strong attention to detail and ability to apply guidelines consistently

Awareness of cultural and linguistic nuances, including potential bias and harm in AI outputs

Comfort with evolving workflows, rapid feedback cycles, and complex quality frameworks

Prior exposure to generative AI, prompt engineering, or LLM fine-tuning workflows is a plus

Benefits

Opportunities to shape the evaluation standards for next-generation multimodal AI systems.

Innovative and supportive global working environment.

Competitive compensation and flexible remote working arrangements.

Continuous learning and growth in applied AI evaluation.

Company

iMerit Technology

iMerit is a leading AI data solutions company providing high-quality data across computer vision, natural language processing, and content services that power machine learning and artificial intelligence applications for large enterprises.

Founded in 2012

San Jose, California, US

1001-5000 employees

http://www.imerit.net

H1B Sponsorship

iMerit Technology has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2022 (1)