Building Trustworthy AI Verification: Lessons From FakeCheck

The instinct is to output a judgement: true, false, misleading. It demos well and it is almost always the wrong product decision.

A binary verdict hides the reasoning, gives the user nothing to evaluate, and fails badly at the edges — which is exactly where verification matters. Claims are rarely cleanly true or false. They are outdated, partially correct, missing context, or technically accurate and misleading.

What works better is showing the shape of the evidence: what was found, how consistent it is, how recent, and where it came from. The user forms the judgement, faster than they could have alone, with the reasoning visible.

This also changes the failure mode. A wrong verdict is a lie. A weak evidence set that the user can see is weak is just an honest limitation.

If there is one lesson worth carrying to any verification product, it is this: users trust the sources, not the model.

That has concrete design consequences:

• Every claim shown must be traceable to something the user can open and read. A statement without a source is worse than no statement.
• Source quality has to be visible, not silently weighted. Users need to see that one result is a primary source and another is an aggregator.
• Recency must be explicit. A large share of "misinformation" is simply information that used to be true.
• Disagreement should be surfaced, not resolved. When credible sources conflict, showing the conflict is the honest answer.

Engineering-wise this means the retrieval layer carries far more metadata than a typical implementation, and the model's job shrinks to organising and explaining rather than deciding.

In most products you design the happy path and then handle errors. In verification the uncertain path is the common path, so we designed it first.

The states that needed real design attention:

• Not enough evidence. Very common, and the temptation to fill the screen with something is strong. The correct response is a clear, calm statement that the system could not find enough to say anything useful.
• Evidence conflicts. Presented side by side rather than averaged into mush.
• Claim is not checkable. Opinions, predictions, and personal statements need to be recognised and handed back, not processed.
• Ambiguous claim. The system asks a clarifying question rather than guessing which interpretation the user meant.

Getting these right made the confident answers more believable, because the product had demonstrated it would say when it did not know.

Verification products are used in a slightly adversarial emotional state. Someone is checking a claim because they suspect it, or because they want to prove someone else wrong.

Language that reads as smug, political, or authoritative in the wrong way will get the product dismissed regardless of correctness. What worked:

• Describing evidence rather than pronouncing on claims — "three sources published since 2024 report a different figure" instead of "this is false".
• Neutral, non-editorialising phrasing with no rhetorical framing.
• Consistency of tone between the confident and uncertain states, so the product does not sound triumphant when it agrees and defensive when it does not.

We treated this as a specification, not a copywriting pass — the constraints were written down and tested alongside accuracy.

You cannot evaluate verification with a simple accuracy score, because the ground truth is contested. What we could measure:

• Source relevance. Did retrieval return material that a reasonable person would consider on-topic? This is objective enough to grade reliably.
• Faithfulness. Does every statement in the output trace to a retrieved source, with nothing invented? This catches the most damaging failure mode.
• Calibration. When the system expressed low confidence, was the evidence genuinely thin? Overconfidence is the real risk.
• Appropriate refusal. Did it decline the claims it should have declined?

Faithfulness turned out to be the most valuable of these. A system that never invents a fact and shows its sources can be usefully wrong. One that fabricates a citation is unrecoverable.

If you are building anything where an AI system makes an assertion the user will act on:

1. Ship reasoning, not conclusions. Your competitive advantage is helping someone think faster, not thinking for them.
2. Invest in retrieval and metadata before the model. Source quality determines output quality more than model choice does.
3. Design "I don't know" as a first-class screen. It will be one of your most-seen states.
4. Measure fabrication above everything. One invented citation costs more trust than ten thin answers.
5. Write the tone rules down. They are product requirements, not style preferences.

Trust is not a layer you add at the end. It is the thing you are actually building.