Crossing Hurdles · 16 hours ago
Software Engineer | $80/hr | Remote
United States
Contract
Remote
Mid Level
$45/hr - $80/hrCrossing Hurdles is seeking a Software Engineering & Systems Design Expert to evaluate LLM-generated responses to software engineering queries. The role involves validating technical correctness, assessing code quality, and providing structured feedback on model outputs.
Staffing & Recruiting
Responsibilities
Evaluate LLM-generated responses to software engineering and coding queries for accuracy, reasoning, clarity, and completeness
Validate technical correctness through fact-checking and by executing code where appropriate
Assess code quality, readability, algorithmic efficiency, and adherence to engineering best practices
Identify logical errors, edge cases, inefficiencies, and misleading explanations in model outputs
Annotate responses with structured feedback highlighting strengths, gaps, and conceptual or factual inaccuracies
Ensure outputs align with expected conversational behavior and system guidelines
Apply consistent evaluation standards using defined taxonomies, benchmarks, and evaluation frameworks
Qualification
Required
Bachelor's, Master's, or PhD degree in Computer Science or a closely related field
Strong real-world experience in software engineering or related technical roles
Expertise in at least one major programming language (e.g., Python, Java, C++, JavaScript, Go, Rust)
Ability to independently solve Medium to Hard–level HackerRank or LeetCode problems
Experience contributing to well-known open-source projects, including merged pull requests
Hands-on experience using LLMs during software development and understanding their limitations
Strong attention to detail and ability to evaluate complex technical reasoning and subtle bugs
Company
Crossing Hurdles
At Crossing Hurdles, we specialise in customised recruitment and staffing solutions designed to drive success for businesses and professionals.
Founded in 2022
11-50 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase