JobsPassport · 2 months ago
Generalist Rater
United States
Full-time, Contract
Remote
Mid Level
3+ years expJobsPassport is a company that focuses on computer industry roles, and they are seeking a Generalist Rater to design and evaluate prompts for LLMs. The role involves generating creative text formats and collaborating with teams to improve model performance and best practices in prompt engineering.
Computer Software
Responsibilities
Design and write effective prompts that elicit desired responses from LLMs, considering different tasks, contexts, and target audiences
Generate creative and informative text formats, including stories, articles, summaries, dialogues based on guidelines
Evaluate and compare the quality of LLM-generated responses based on criteria such as fluency, coherence, factuality, creativity, and bias
Analyze LLM outputs to identify strengths and weaknesses, and provide feedback to improve model performance
Collaborate with engineers, researchers, and product managers to refine prompting strategies and evaluation metrics
Contribute to the development of best practices and guidelines for prompt engineering and LLM evaluation
Qualification
Required
Bachelor's degree in English, Creative Writing, Linguistics, Computer Science, or a related field
3+ years of experience in content writing, copywriting, or a related field
Strong understanding of LLM capabilities and limitations
Excellent writing and editing skills, with the ability to adapt to different styles and tones
Strong analytical and problem-solving skills
Ability to work independently and as part of a team
Can-do attitude, willingness to learn, and growth mindset
Preferred
experience with video generation tasks
Experience with prompt engineering and evaluating LLM outputs is a plus
Understanding of statistics and data analysis is a plus
Familiarity with data annotation and labeling tools is a plus
Experience working with different LLM architectures and frameworks
Knowledge of NLP concepts and techniques
Familiarity with human evaluation methods for LLMs
Experience in a research or academic setting
Tech savvy: ability to read json not required, but we need candidates willing to grow their technical abilities
Bilingual preferred (Chinese, Spanish, Portuguese, Arabic, French, German)
Company
JobsPassport
JobsPassport Breaking barriers, connecting talents worldwide! Jobs Passport is a one-stop international recruitment portal that connects potential talent with business organizations looking forward to building a diverse, inclusive & competitive workforce with global talent.
2-10 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase