LLM Engineering Expert – AI Evaluation

Vraify is hiring experienced LLM Engineering Experts to create and validate challenging, simulation-based engineering design problems for evaluating advanced AI agents. This remote contractor opportunity is open only to candidates residing in the United States or Canada. You will design multi-constraint tasks, configure open-source simulation tools, analyze agent execution logs, diagnose reasoning and tool-use failures, and build objective automated graders across electrical, mechanical, aerospace, control systems, systems engineering, and robotics. Important experience note This is a senior specialist role requiring at least 8 years of directly relevant engineering experience. To help candidates focus on opportunities aligned with their background, applications that do not meet this minimum will be automatically screened out. Freshers and early-career applicants are therefore not eligible for this role, and we warmly encourage them to consider opportunities better suited to their current experience level.