Home Office asylum AI fails accuracy checks in one of five summaries
A new report finds the UK government's AI tool for summarizing asylum interviews has a 22 percent failure rate, raising concerns about oversight.
Key points
- Open Rights Group report finds 22 percent of AI-generated asylum interview summaries failed expert accuracy checks.
- The Home Office uses OpenAI’s GPT-4o model via Microsoft Azure for its Asylum Case Summarisation system.
- Experts checked only 0.3 percent of summaries between April and July 2026, according to the campaign group.
The UK Home Office’s artificial intelligence system for summarizing asylum interviews failed accuracy tests in more than one in five cases, according to a new report by the Open Rights Group. This finding contradicts ministerial promises that the technology would speed up decision-making processes for asylum seekers.
The 20-page investigation reveals that the department is continuing to deploy these AI tools despite significant concerns regarding their accuracy and the lack of adequate oversight. The report suggests that the promised efficiency gains may not be materializing as expected.
Accuracy and Oversight Concerns
The Open Rights Group based its findings on Freedom of Information requests and correspondence with government ministers. The scrutiny focused on two specific systems used by Home Office caseworkers: Asylum Case Summarisation and Asylum Policy Search. Both systems rely on OpenAI’s GPT-4o model, accessed through Microsoft’s Azure AI Foundry platform.
Data obtained by the campaign group shows that 45 out of 203 AI-generated interview summaries failed initial expert checks after the system launched in April 2026. This represents a 22 percent failure rate, which is significantly higher than the 9 percent rejection rate recorded during the original pilot phase.
An additional 19 summaries were flagged for further review. The figures also document 56 second-stage reviews, of which 30 failed to meet the required standards. These errors can include factual inaccuracies and misrepresentations of what asylum seekers stated during their interviews.
The government maintains that humans remain ultimately responsible for all asylum decisions. However, the Open Rights Group questions how reliably caseworkers can identify mistakes in AI-generated information, especially when the volume of cases is high.
Limited Review Scope
Despite the high failure rates, the Open Rights Group notes that relatively few summaries have been examined by experts. The campaign group estimates that Home Office experts checked only 0.3 percent of summaries between April and July 2026. This estimate is based on the department’s projected monthly caseload of 17,515 cases.
The report also found that summaries could pass initial checks even if they omitted important details, provided that the information they did include was accurate. This raises concerns about the completeness of the records used for decision-making.
The Open Rights Group also questioned whether the tools were actually saving time, as ministers had promised. The data suggests that the technology may be introducing additional layers of complexity rather than streamlining the process.
What should you do?
Security and technology professionals should monitor developments regarding the use of AI in government decision-making processes. Organizations deploying similar AI tools for summarization or search should implement rigorous human-in-the-loop verification protocols.
What to do and how to stay safe: OpenAI
- Review current AI deployment policies to ensure human oversight mechanisms are robust and documented.
- Audit AI-generated outputs for factual accuracy and completeness, not just surface-level correctness.
- Monitor vendor updates and security advisories for the underlying AI models and platforms.
- Assess whether AI tools are actually improving efficiency or adding complexity to existing workflows.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
What is the failure rate of the Home Office's AI asylum summaries?
According to the Open Rights Group, 22 percent of AI-generated interview summaries failed initial expert checks.
Which AI models are being used by the Home Office?
The Home Office uses OpenAI’s GPT-4o model through Microsoft’s Azure AI Foundry for its asylum processing tools.
How many summaries were checked by experts?
The Open Rights Group estimates that only 0.3 percent of summaries were checked by experts between April and July 2026.
