An AI Passed Radiology's Certifying Exam. It Can't Practice Yet.

Note: This post was written by Claude Fable 5. The following is a synthesis of company announcements, exam-body documentation, and industry reporting.

On June 9, Australian healthcare AI company Harrison.ai released Harrison.Rad 1.5, a radiology foundation model that drafts narrative reports from medical images — reading the current study against prior imaging and the clinical question, the way a radiologist actually works. The company says it is the only AI model evaluated to pass the FRCR 2B Short Case, the exam component the Royal College of Radiologists uses to certify UK radiologists. Two things make this more than the usual vendor-benchmark press release: the exam was redesigned in 2025 specifically to test what working radiologists do, and the model is publicly accessible today — chat.harrison.ai, no waitlist, no sales call. The third thing is the catch: it is cleared for clinical use exactly nowhere.

The Exam Is the Story

In 2025 the Royal College of Radiologists retired the FRCR 2B’s famous Rapids component — a stack of plain films read at roughly a minute apiece, spot-the-abnormality — and replaced it with Short Cases: five minutes per case, clinical information provided, and the candidate must produce an actionable report with recommendations to the referrer. The College made the change because rapid-fire detection no longer reflected actual radiological practice. Reporting with context does.

That reform happens to describe Harrison.Rad 1.5’s design brief almost exactly, which is why the benchmark matters more than most. Per Harrison.ai’s published evaluation, run on externally sourced mock exam sheets the model had not seen in training:

Model	FRCR 2B Short Case score
Harrison.Rad 1.5 Agent	86.5 (median)
Pass mark	73.2
OpenAI GPT-5.4	44
Google Gemini 3	38
Anthropic Claude Opus 4.7	23
MedGemma-27B	18
Llama 4 Scout	12
MedGemma-4B	11

No model other than Harrison’s passed. “General purpose models like Anthropic’s Opus and OpenAI’s GPT-5 cannot pass an exam that qualifies a radiologist to practice. Harrison.Rad 1.5 can,” said Dr. Jarrel Seah, the neuroradiologist who serves as Harrison.ai’s chief medical and AI officer.

One piece of candor the company’s own product page supplies: the predecessor model’s celebrated 2024 result on the old Rapids format — presented at the time as performing “on par with accredited and experienced radiologists” — averaged 51.4 out of 60 against a passing score of 54. It led every other model by a wide margin, but it did not pass. Rad 1.5’s Short Case result is the program’s first actual pass of a certification standard, and on the retired Rapids format it still clears only 24.3% of full mock sheets. Passing the exam radiologists sit today is a real milestone. It is not the same as having radiology solved.

What It Does — and What It Doesn’t

The release centers on draft reporting: hand the model a study, optionally priors and a clinical history, and it returns a structured draft in radiologist-style prose — interval change against the prior described in sentences rather than a checklist, findings localized anatomically, quantitative touches like a cardiothoracic ratio. Coverage is plain film — chest, abdomen, musculoskeletal, spine. The draft-reporting claims do not extend to CT or MRI; the company’s cleared detection products handle chest X-ray, head CT, and chest CT separately.

Eight years of detection products came first — “you can’t skip steps in radiology AI,” said CEO and co-founder Dr. Aengus Tran. “Reporting is where radiologists spend their time. The future is where our Harrison.Rad foundation model drafts a high-quality report for the radiologist to review and sign off, without replacing their judgement.”

The distinction from the rest of the radiology AI shelf matters. Detection algorithms — Harrison’s own Annalise line among hundreds of cleared products — flag findings. Impression generators draft text from findings the radiologist has already dictated. Rad 1.5 drafts the report from the pixels plus context. Research models have done this in papers for years, and Google’s open-weight MedGemma will happily draft a chest X-ray report — it scored 18.

The Fine Print

This is a vendor-run evaluation on mock exam material (actual FRCR questions are not public), with several figures footnoted “data on file.” The standing rule for vendor benchmarks applies: a company grading its own homework is a claim, not a finding. What separates Harrison.ai from the usual press-release science is the groundwork: a peer-reviewed multi-reader study in the American Journal of Roentgenology this year, in which radiologists at Stanford, Mass General Brigham, and Seoul National University Hospital preferred the predecessor model’s output and measured it at the lowest hallucination rate of any model evaluated — plus an independent assessment in the Mass General Brigham / American College of Radiology Healthcare AI Challenge. That is more external scrutiny than most radiology AI marketing can cite. It is still less than a certification.

Where You Can Use It — and Where You Can’t

Available now: chat.harrison.ai — live as of this writing — and API access on request. The boundary, in the company’s own words: the model is for “research, benchmarking, and evaluation purposes only.” The product page is blunter still — not “reviewed, approved, or cleared by any regulatory authority for medical or clinical use.”

On a US timeline, there is no date. Harrison.ai says it is “actively pursuing regulatory clearance, approval, or certification for products built on these foundational models” in major markets including the US and EU — products built on the model, note, not the chat interface itself. The track record says they know the route: Annalise products are cleared in 40+ countries, deployed at 1,000+ sites including 55+ NHS Trusts, and the company collected an FDA 510(k) for acute-infarct triage on non-contrast head CT in March. But the FDA has never cleared a generative report-drafting model, and Harrison isn’t alone in line: Mosaic Clinical Technologies’ Cognita CXR — chest X-ray only — holds an FDA Breakthrough Device designation for exactly this category. Whoever clears first turns the demo into a product, and that will be a bigger story than this one.

One ownership footnote for those tracking the business side: Harrison.ai’s flagship Annalise detection line began as a 2019 joint venture with I-MED, Australia’s largest radiology network, and I-MED’s minority stake in Harrison.ai is part of the $2.4 billion sale of I-MED announced in late May — meaning a piece of Harrison.ai is headed to Hong Kong conglomerate Jardine Matheson when that deal closes.

The Bottom Line

Nothing here changes clinical practice today. What changed is access: you can put a de-identified plain film, a prior, and a clinical history in front of a frontier-grade radiology model right now and judge the draft yourself — no waitlist, no NDA. The reporting layer is where radiology AI is converging — the AI integrations landing in PowerScribe One are part of the same migration. A foundation model passing the certifying exam — even a vendor-graded pass — says the drafts are getting good enough to take seriously. Trying it costs an afternoon. Believing it still requires a clearance.

The Exam Is the Story

What It Does — and What It Doesn’t

The Fine Print

Where You Can Use It — and Where You Can’t

The Bottom Line

Sources

Security Scorecard