Inside Our Fact-Check Verification Ladder: How AuthenAI Cross-Checks Academic Citations

Verifying a citation sounds simple: look up the paper, confirm it exists, done. In practice it’s harder, because a bad citation fails in three different ways, and only one of them is “this paper doesn’t exist.” A reference can be entirely fabricated — a title, author list, and journal never published anywhere, the kind of thing a language model produces when asked for a source and it generates something plausible instead of recalling a real one. It can be subtly wrong: a real paper, correctly titled, cited for a claim it never actually makes. Or it can simply be hard to find — a real paper a narrow search misses because of an odd title, a non-standard author-name format, or an unindexed journal.

A single API call can’t tell these cases apart. A DOI lookup that comes back empty might mean the citation is fake, or just that this database doesn’t carry that journal. A web search that finds nothing conclusive might mean nothing exists, or that the search terms didn’t match how the paper is indexed. That ambiguity is why AuthenAI’s fact-check pipeline doesn’t rely on one lookup — it runs a layered search, trying progressively broader sources until it gets a confident answer or genuinely exhausts what’s checkable, rather than treating one query’s silence as proof of anything.

The nine-layer ladder

AuthenAI’s reference verification tries strategies in a fixed order, stopping as soon as one returns a confident match. In the codebase this is literally called a 9-layer search strategy, and the order matters: it starts with the most authoritative, structured sources and only falls back to looser, general ones once those come up empty.

The ladder starts with crossref_doi: if the reference includes a DOI, this layer looks it up directly in CrossRef, the registry publishers use to register DOIs, and checks whether the title on file matches the cited paper. When that check comes back clean, the same layer also records a doi_title_verified marker — not a separate next check, but a confirmation flag that the DOI genuinely points to the cited paper. The marker is absent on a mismatch; a real DOI pointed at the wrong paper is instead caught by crossref_doi itself returning a low match score. From there, crossref_title searches CrossRef by title alone — it runs whenever a title is available and the prior layer hasn’t already reached high confidence, so it also catches a DOI that exists but didn’t resolve cleanly.

The next tier moves to dedicated academic search APIs: semantic_scholar, openalex, and pubmed, in that order — three large academic indexes covering most peer-reviewed literature, with PubMed strong for biomedical and life-sciences references the more general indexes sometimes miss.

If none of that resolves it, the ladder falls back to general web search as a last resort, in three progressively looser passes: web_doi (searching the web for the DOI string, in case it resolves somewhere the academic APIs didn’t index), web_exact_title (a precise title search), and web_author_year_keywords (a looser combined search on author, year, and title keywords, for mangled titles). The final layer, dblp, checks the DBLP computer-science bibliography — useful for citations the academic-API layers are least likely to carry.

The important part isn’t just that there are nine layers — it’s that the ladder stops the moment it doesn’t need to keep going. As soon as any single layer returns a match with confidence of 80 or higher, verification ends there and reports that result. There’s no reason to burn through PubMed, web search, and DBLP once CrossRef has already confirmed a DOI’s title matches cleanly; the later layers exist for references the early ones don’t catch, not to double-check ones they already did.

Why a backend outage should never look like a fake citation

Every layer depends on a real, external service — CrossRef, Semantic Scholar, OpenAlex, PubMed, DBLP, the web search backend. Any of those can have a bad moment: a rate limit, a timeout, a server error — no matter how well the ladder is designed.

What matters is what happens next. When a layer hits an upstream fault, that fault is tracked separately from an ordinary “no match found” result — it does not get folded into evidence that a citation doesn’t exist. A reference where several layers couldn’t be reached, because a source API happened to be down during that check, gets pushed toward an uncertain verdict, not a “this doesn’t exist” verdict. The distinction sounds small, but it’s the difference between an honest “we couldn’t check this” and a false accusation of fabrication. A backend hiccup on CrossRef’s end should never read as evidence someone invented a source.

What this catches that a single search wouldn’t

Layered, structured verification catches problems a single fuzzy search tends to miss, or wave through. A garbled or paraphrased title — a word dropped, a subtitle reworded — can defeat one exact-match lookup, but a looser layer further down the ladder, checking author and year alongside title keywords, can still find the real paper. A DOI that resolves and looks legitimate but actually points to a different paper than claimed is invisible to a check that only confirms “does this DOI exist” — catching that mismatch takes comparing the DOI’s registered title against the citation, which is exactly what crossref_doi does.

The layered approach also cross-checks structured metadata — title, authors, journal — against what a source has on record, rather than trusting a superficially similar result. That matters most for the case easiest to get wrong with a single web search: a real, existing paper cited for a claim it doesn’t actually support. Confirming a paper exists and confirming it says what it’s credited with saying are two different questions, and a lookup answering only the first can wave through a citation that misleads in exactly the way that erodes academic trust.

Closing: why this level of rigor matters

None of this complexity is decorative. A citation-checking tool exists to help someone decide whether a source can be trusted, and a tool that can’t reliably tell “doesn’t exist” apart from “couldn’t check it right now” isn’t precise enough to support that decision. Flattening those two outcomes into one result risks turning an ordinary API hiccup into an accusation of fabrication — a serious claim to get wrong. Building verification as a ladder — authoritative sources first, general search as a genuine last resort, upstream failures tracked honestly instead of swept into “not found” — is what makes it possible to say, with real confidence, which citations check out and which don’t.