Remote | Machine Learning & NLP Research Specialist

We are sharing a specialised part-time consulting opportunity for US-based machine learning and natural language processing professionals with hands-on experience in Python, model training and evaluation, transformers, large language models, retrieval systems, and applied ML pipelines. This role supports an advanced AI initiative focused on identifying reasoning and capability gaps in frontier models. Selected professionals will design challenging real-world ML and NLP tasks, develop executable reference solutions, evaluate model performance, and analyse failures across language understanding, generation, retrieval, training workflows, and applied machine learning systems.

ML & NLP Task Development • Design challenging machine learning and natural language processing problems based on practical research or industry experience • Create tasks involving model training, evaluation, language understanding, generation, retrieval, or applied ML pipelines • Target specific reasoning, implementation, and capability gaps in advanced AI models • Define clear specifications, expected behaviour, constraints, datasets, and evaluation criteria Reference Solutions & Python Development • Develop accurate reference solutions and supporting materials using Python • Integrate tasks into agent-based development and evaluation environments • Create executable tests, validation scripts, scoring logic, or model pipelines where appropriate • Ensure reference implementations are technically sound, reproducible, and appropriately challenging Model Evaluation & Failure Analysis • Evaluate model and agent performance across assigned ML and NLP tasks • Compare generated outputs with reference solutions and expected results • Identify tasks where models demonstrate meaningful limitations or inconsistent behaviour • Classify failures involving reasoning, implementation, retrieval, language understanding, generation, or instruction adherence • Document findings through clear and technically detailed written analysis Quality Calibration & Collaboration • Review tasks and evaluation methods developed by other machine learning specialists • Maintain consistent standards for difficulty, accuracy, realism, and technical quality • Participate in calibration and peer-review activities • Collaborate with other subject matter experts to improve evaluation coverage and reliability Ideal Profile Strong candidates may have: • Deep hands-on experience in machine learning, natural language processing, or both • Practical proficiency in Python demonstrated through professional, academic, or open-source work • Strong understanding of modern ML and NLP methods, including transformers and large language models • Experience with model training, fine-tuning, evaluation, retrieval, or production ML pipelines • Familiarity with PyTorch, TensorFlow, JAX, Hugging Face, or comparable frameworks and tooling • Ability to design realistic technical problems and develop complete reference solutions • Strong written communication and the ability to explain complex model behaviour clearly • Reliable availability for approximately 20 hours per week