AI Reference Checker

Why AI tools invent citations

Large language models generate text by predicting what comes next, not by consulting a database. A citation is a highly regular pattern – author surnames, a year, a plausible title, a real journal name, a DOI-shaped string – so a model can produce one that looks exactly right while the paper behind it does not exist. A 2024 study of systematic-review reference retrieval measured fabrication rates of 39.6% for GPT-3.5, 28.6% for GPT-4, and 91.4% for Google Bard (Chelli et al., 2024). The fabrications are convincing precisely because they borrow real author names and real journals; it is the specific combination that is fake.

The confusion reaches librarians too. Reviewing patron questions received over two weeks in October 2025, one Library of Virginia librarian found that 15% were generated from AI chatbot summaries (Library of Virginia, broadSIDE 2026 issue 1).

What each status means

• Verified (green): the reference matched a record in Crossref or PubMed. The matched title and DOI are shown so you can confirm it is the paper you meant.
• Not found (red): no record with a similar title exists in the databases we query, and any DOI in the reference does not resolve. Strong signal of a hallucinated reference – but see the limitations below before deleting it.
• Mismatch (amber): something resolved, but it is not the paper the reference claims. The classic case is a real DOI attached to a different article – the most common form of AI citation hallucination.
• Unparseable (gray): we could not extract a title or DOI from the entry, so there was nothing to verify. Reformat the entry and run it again.

How the checker works

Each entry becomes a claim with structured fields (title, authors, year, journal, DOI). If the reference claims a DOI, we resolve it against the Crossref registry and compare the registered title to the claimed title. If there is no DOI, we search Crossref by title and first author, run a fuzzy bibliographic search on the whole entry, and query PubMed for biomedical references. Candidate matches are scored with a normalized token-set similarity: 0.75 or higher counts as verified, 0.4 to 0.75 is reported as a mismatch with the near-match attached as evidence, and anything lower is not found. Up to 30 references are checked per run, and nothing you paste is stored.

Limitations

Crossref and PubMed index journal articles, conference papers, and preprints with DOIs. Books, book chapters, theses, reports, legal sources, and most websites are outside their coverage, so a real book can come back as not found. A verified status also says nothing about formatting – the paper exists, but the page numbers, year, or author order in your entry may still be wrong. Use the verdicts as triage: anything red or amber deserves a manual check against the original source before your work is submitted.

Frequently asked questions

How do I check if a citation is real?

Paste the reference above and select Check references. Any DOI is resolved against Crossref and the title is searched in Crossref and PubMed; the verdict tells you whether a matching record exists.

Why does ChatGPT make up citations?

Language models predict plausible text rather than querying a database, so they assemble author-year-title-journal combinations that look real but were never published.

What does the mismatch status mean?

A record resolved, but it is not the paper your reference claims – most often a real DOI pointing at a different article.

Does the checker work for books and websites?

Coverage is strongest for journal articles. Books and websites are often absent from Crossref and PubMed, so treat not found as a flag to verify manually, not proof of fabrication.

Is the reference checker free?

Yes – free, no sign-up, up to 30 references per check, and your references are not stored.