AI Scientist

About the Company

Maki People is shaping the future of hiring through innovation, rigour, and collaboration. The Science team drives the development of high-quality content, evaluating AI-driven scoring algorithms and large language models to ensure fairness, accuracy, and transparency.

Responsibilities

• Assess the statistical accuracy and reliability of LLMs used for automated scoring
• Evaluate and calibrate psychometric models (e.g. CTT, IRT, CFA) to ensure scientific validity
• Test model robustness against prompt variation and adversarial behaviour
• Design research comparing AI-scored assessments with expert human judgments
• Detect and analyse potential biases in AI-generated or psychometric scores across demographic groups
• Develop predictive models that estimate success probabilities and identify behavioural predictors
• Implement continuous monitoring systems to track model performance and drift
• Explore prompt engineering, evaluation methods, and model optimisation techniques

Requirements

• Advanced degree (PhD/MSc) in Data Science, Machine Learning, Psychometrics, Computational Linguistics, or Psychology
• Proven expertise in AI model evaluation and psychometric validation
• Knowledge of psychometric modelling (IRT, CFA, CAT)
• Familiarity with LLMs and NLP techniques
• Experience applying fairness and bias testing methodologies
• Proficiency in Python or R
• Experience with statistical software and cloud databases like BigQuery
• Strong grounding in ethical AI and data governance
• Ability to communicate technical findings to non-technical stakeholders