UK Home Office's asylum AI failed 22% of expert checks

An AI tool that the UK Home Office uses to summarize asylum interviews failed 45 of 203 expert accuracy checks after it was rolled out in April 2026. That is about 22%, The Register reports, based on records obtained by the campaign group Open Rights Group (ORG). The tool runs on OpenAI's GPT-4o. Experts checked only about 0.3% of the summaries it produced between April and July. It is a clear case of a language model deployed where mistakes carry serious stakes.

The two tools

According to ORG's investigation, the Home Office uses two generative AI tools in asylum casework:

• Asylum Case Summarisation (ACS) turns interview transcripts into summaries for caseworkers. It was rolled out to all asylum caseworkers from April 2026.
• Asylum Policy Search (APS) is a chat tool for searching the government's country policy documents. It has been used by all asylum decision-makers since July 2025.

Both run on GPT-4o through Microsoft's Azure AI Foundry service. ORG says it spent six months investigating them through freedom of information requests and questions to ministers.

What the checks found

The figures come from different counts, and the sources do not reconcile all of them:

Measure	Figure	Reported by
Summaries failing their first expert check	45 of 203	The Register
Summaries flagged for further review	19	The Register
Second-stage reviews failing the standard	30 of 56	The Register
Share of summaries checked by experts, April to July	0.3%	ORG and The Register
Summaries not fit for purpose in the 2025 pilot	9%	ORG
Not fit for purpose in later sample testing	15%	ORG

Sonia Lenegan's immigration newsletter puts the first-review failure rate at 22% and the second-check failure rate at 54%. Those match The Register's counts.

The Register also reports that the Home Office confirmed in August that expert testing of APS had stopped. It had told members of parliament the checks were continuing.

What else ORG found

ORG says the Home Office refused to share key details. These include how staff were trained, which prompts the tools use and how quality is tested. Some of its answers contradicted each other.

People claiming asylum are not told when AI summarizes their case. The Register reports that they cannot inspect or correct the summaries. The tools are also missing from the government's public register of algorithmic tools.

The Register adds that average monthly initial asylum decisions since April are down about 42% from the same period in 2025. The time from interview to decision is up 29%. The outlet notes this does not prove the AI tools caused either change.

The Home Office did not respond to The Register's questions.

What ORG wants

ORG is calling for both tools to be suspended and for an independent audit of their accuracy in live use, with the results published. It also wants the Independent Chief Inspector of Borders and Immigration to inspect them.

"We urge the Government to regulate AI and outright ban its use in asylum applications," said Sara Alsherif, a programme manager at ORG.

What this means for developers

For teams that build or buy AI summarization tools:

• Check more than a sliver. Experts reviewed about 0.3% of summaries. At that rate, a problem can run for months before anyone sees it.
• Measure live use, not just the pilot. The pilot found 9% of summaries unfit. Later checks found higher failure rates once the tool was used at scale.
• Let people see and fix the output. Claimants cannot inspect or correct summaries of their own interviews. Any system that summarizes what a person said should let that person check it.
• Keep records you can share. ORG could not get the prompts or test results. Logging prompts, versions and evaluations makes an audit possible when one is demanded.
• Watch what the tool replaces. ORG warns the summaries may remove a layer of expert review. Automating a step can remove the checks that step used to provide.