AI Hallucination Detection Analyst

Job Description

The AI Hallucination Detection Analyst is responsible for identifying and analysing instances where artificial intelligence systems generate information that is fabricated, unsupported, misleading, or inconsistent with available evidence. The position focuses on detecting situations in which AI models present incorrect information as fact, invent sources or references, misrepresent research findings, or provide confident answers without adequate factual support.

The successful candidate will carefully examine AI-generated responses and verify whether the information can be supported by credible sources. They will investigate questionable claims, check cited publications, confirm whether named organisations and individuals exist in the stated context, and identify fabricated statistics, quotations, events, or technical details.

The analyst will also examine patterns in hallucination behaviour to help determine the circumstances under which AI systems are most likely to generate unreliable information. This may include evaluating responses to ambiguous questions, requests involving specialised knowledge, questions about recent events, and tasks requiring detailed references or multi-step reasoning.

In addition to identifying hallucinations, the analyst will document the nature and severity of each error, explain why the response is unreliable, and provide feedback that supports improvements in model training, factual grounding, and response generation.

Key Responsibilities:

Identify fabricated, unsupported, and misleading claims in AI-generated content.
Verify names, dates, statistics, quotations, publications, and cited sources.
Detect invented references, nonexistent research papers, and inaccurate citations.
Assess whether AI-generated statements are supported by available evidence.
Identify cases where AI models confuse facts, events, individuals, or organisations.
Investigate hallucination patterns across different subjects and task types.
Classify detected errors according to their nature and potential severity.
Document evidence supporting each hallucination finding.
Prepare reports identifying recurring model weaknesses.
Recommend improvements to AI evaluation and factual reliability processes.
Collaborate with research, quality assurance, and AI development teams.
Qualifications and Requirements:

Bachelor’s degree in Computer Science, Data Science, Research, Journalism, Information Science, or a related field is preferred.
Strong investigative research and fact-checking abilities.
Excellent analytical reasoning and information-verification skills.
Ability to distinguish between factual errors, unsupported claims, and legitimate uncertainty.
Familiarity with AI language models and their common limitations.
Experience using academic databases, search engines, and reference-verification tools.
Excellent written communication and report-writing skills.
Strong attention to detail and professional judgement.
Previous experience in AI evaluation, research, quality assurance, or fact-checking is an advantage.