AI Model Behaviour Analyst

Job Description

The AI Model Behaviour Analyst is responsible for studying and evaluating how artificial intelligence systems respond to different instructions, user interactions, contextual conditions, and testing scenarios. The role focuses on identifying patterns in model behaviour, understanding response inconsistencies, examining how models handle complex requests, and assessing whether their outputs remain reliable across different situations.

The successful candidate will analyse AI-generated responses to determine how models interpret instructions, maintain conversational context, respond to corrections, handle uncertainty, and adapt to changes in user requirements. They will examine behavioural patterns such as excessive confidence, inconsistent explanations, unexpected refusals, repetitive responses, inappropriate assumptions, and failures to follow instructions.

The analyst will also conduct comparative evaluations across different prompts, tasks, and model versions to determine whether behavioural changes are expected or indicate potential performance problems. Findings will be documented in detailed reports that help research and development teams understand model strengths, weaknesses, and opportunities for improvement.

This position requires a combination of analytical thinking, AI evaluation knowledge, research skills, and strong written communication. The successful candidate must be capable of identifying subtle behavioural patterns and explaining their significance in a clear and evidence-based manner.

Key Responsibilities:

Analyse AI-generated responses across different tasks and interaction scenarios.
Identify patterns in model behaviour, including consistency and response quality.
Evaluate how AI systems interpret and follow user instructions.
Examine model behaviour when handling ambiguous or conflicting requests.
Assess whether AI systems maintain relevant conversational context.
Identify unexpected refusals, repetitive outputs, and inconsistent responses.
Review how models communicate uncertainty and limitations.
Compare behavioural performance across different model versions.
Prepare detailed reports on observed behavioural patterns and weaknesses.
Recommend improvements to AI training, evaluation, and response-generation processes.
Qualifications and Requirements:

Bachelor’s degree in Computer Science, Psychology, Data Science, Cognitive Science, Linguistics, or a related discipline is preferred.
Strong analytical, observational, and research abilities.
Understanding of artificial intelligence and language model behaviour.
Excellent written communication and technical reporting skills.
Ability to identify patterns and inconsistencies across large volumes of responses.
Familiarity with AI testing and evaluation platforms is an advantage.
Knowledge of human-computer interaction or behavioural research is beneficial.
Strong attention to detail, objectivity, and professional judgement.