How to check an AI answer

What it isCopy Anchor Link

Models write confident text whether they are right or wrong, so you need your own checks. Ask for sources and open them, because a made-up link looks just like a real one. Ask for a short quote from the source and search for that quote. Cross-check the key fact in a second, independent place. Recompute numbers yourself or with a calculator. Ask the model to list what it assumed and what it is unsure about. Try it first on something you already know, to learn how reliable it is for that kind of question.

A well-known fact-checking routine called SIFT fits here: stop and think before sharing, investigate who is behind a source, find better coverage, and trace claims back to where they started.

A worked exampleCopy Anchor Link

You ask for "three studies showing that sleep improves memory", and the answer lists three titles with authors and years. You search for each title. Two exist. The third does not, and it was a plausible invention. In 2023 a lawyer submitted a legal brief that cited six court cases a chatbot had invented, and none of them existed. A search takes a minute. The cost of skipping it can be very large.

How much to checkCopy Anchor Link

Low stakes, such as a joke or a recipe idea: a quick glance. Medium, such as a homework fact or a work email: confirm the key claims. High, such as health, money, law or safety: use an authoritative source or a professional, and treat the model as a starting point only.

Why models get things wrongCopy Anchor Link

A language model writes the most likely next words given your question. It does not look facts up unless it is connected to a search tool. When it has little reliable pattern to draw on, such as a rare name or a recent event, it still produces fluent text. That is called a hallucination. The sentence sounds the same whether it is true or invented, so tone tells you nothing.

A second example: a wrong calculationCopy Anchor Link

You ask: "A jacket costs $80 with 15% off. What do I pay?" A model might answer $65. Check it yourself: 15% of 80 is 12, and 80 minus 12 is 68. The model was close, which makes the slip easy to miss. Models do arithmetic by predicting text, not by calculating, so numbers deserve a calculator. If a model can run code or a calculator tool, results improve, but still look at what it did.

What goes wrong when you checkCopy Anchor Link

Checking can fail too. A search that only repeats the same claim is not independent. Ten websites copying one source count as one source. Also beware asking the model to confirm itself: "Is this accurate?" tends to produce agreement. Instead, search for the specific name, number or quote in a place the model did not write.

How to tell your checking is workingCopy Anchor Link

Run a small test. Ask a model five questions where you already know the answers, including one trick question about something that does not exist, such as a made-up book title. Count how many it gets right and whether it admits it does not know the made-up one. That gives you a rough feel for how far to trust it on that kind of question. The sample is tiny, so treat it as a hint, not a measurement.

Common mistakeCopy Anchor Link

Asking the model "are you sure?" and accepting the reply. It will often agree or reverse itself without checking anything. Check against the world, not against the model.