Machine Learning Evaluation Specialist
Alignerr
Location
🇺🇸 United States
Type
contractor
Salary
$200–$400
Posted
2d ago
Job Description
Machine Learning Evaluation Specialist (AI Training)
About The Role
What if your years of research and domain expertise could directly shape how the next generation of AI models think and reason? That's exactly what this role offers. We're looking for ML experts and domain specialists to design fiendishly difficult evaluation problems that push state-of-the-art AI systems to their limits. This isn't about textbook prompts — it's about crafting the kind of challenges that only someone with deep, hard-earned expertise can build and judge. Your work will directly influence how leading AI models are measured, improved, and deployed. • Organization: Alignerr • Type: Hourly Contract • Location: Remote • Commitment: 10–40 hours/week
What You'll Do
- Design complex, original machine learning problems rooted in your specialized domain of expertise
- Craft evaluation tasks that go beyond standard ML pipelines and require genuine advanced knowledge to solve
- Draw from your own research experience to build challenges that would stump a highly capable AI
- Write clear, precise problem statements with defined evaluation criteria and gold-standard solutions
- Assess AI-generated ML solutions for correctness, creativity, and methodological rigor
- Document problem difficulty, required domain knowledge, and expected failure modes
- Collaborate asynchronously with a global team of researchers and engineers