Pyramid Consulting, Inc · 9 hours ago
LLM-GenAI Model Evaluator
Austin, TX
Contract
Hybrid
Mid Level
$50/hr - $55/hrPyramid Consulting, Inc is a leading IT Industry, and they are seeking a LLM-GenAI Model Evaluator. This role involves evaluating generative AI models and requires a strong understanding of LLMs and AI/ML frameworks.
ConsultingInformation TechnologyLegalProfessional ServicesSoftwareStaffing Agency
No H1B
U.S. Citizen Only
Qualification
Required
Key Skills: - Artificial Intelligence, Machine Learning, AI/ML frameworks (PyTorch, TensorFlow, HuggingFace, LangChain)
Looking for GC and US Citizens
Strong understanding of LLMs, generative AI, and transformer-based architectures
Experience with Python, data analysis, and model evaluation frameworks
Familiarity with prompt engineering, embeddings, RLHF/RLAIF, and LLM-based scoring methods
Experience building evaluation datasets and working with annotation platforms
Understanding of safety alignment, bias detection, and adversarial testing
ML/AI frameworks: PyTorch, TensorFlow, HuggingFace, LangChain
Evaluation/annotation tools: Scale AI, GroundTruth, Labelbox, Prodigy
Prompt testing tools: Weights & Biases, MLflow, OpenAI evals, LLM-as-a-judge pipelines
Benefits
Health insurance (medical, dental, vision)
401(k) plan
Paid sick leave (depending on work location)
Company
Pyramid Consulting, Inc
Pyramid Consulting, a global leader in workforce and technology solutions, empowers individuals and organizations to transform and thrive in the most challenging and competitive markets.
Founded in 1996
5001-10000 employees
Funding
Current Stage
Late StageLeadership Team
Ramesh Maturu
President and Co-Founder
Manish Kaushik
Chief Financial Officer
Recent News
2025-08-18
Company data provided by crunchbase