How we measure and fix Answer Fidelity when AI describes you wrongly

AI assistants describe most companies roughly right and get a few facts wrong. We count the wrong statements, trace each one to the page it came from and correct that page.

What can you do when ChatGPT gets facts about your company wrong?

Write down the facts a buyer might check, ask each assistant questions that name your company, and mark every statement as correct, outdated, wrong or unverifiable. Torke calls the share that is correct Answer Fidelity. Then fix the sources the errors come from, old pages, conflicting directories and missing facts, because assistants repeat what they read.

When ChatGPT has wrong information about your company, the error rarely looks dramatic. It lists a product you discontinued, places your headquarters in the wrong city, credits you with a certification you never held or leaves out the one you spent a year earning, and it says all of it in the same confident tone as the facts it gets right. A technical buyer who asks an assistant about you before a first call takes that description into the meeting, and a single wrong specification can end an evaluation that a sponsor wanted to continue.

We call the measure of this Answer Fidelity: when an AI assistant is asked about your company by name, the share of its factual statements that are correct against your approved facts. It sits next to Machine Recall, which asks whether an assistant names you when the buyer doesn't, and the two can move in opposite directions, since a company mentioned often and described wrongly takes more damage than one mentioned rarely and described well.

Wrong answers have three usual causes, and each points at a different repair. The first is old pages, such as an archived product sheet, an old press release or a directory entry nobody updated, which the assistant still finds because they are still public. The second is pages that contradict each other, such as a website stating one founding year while a funding database states another, which leaves the assistant to choose. The third is a gap, a fact that appears nowhere in plain text, such as which industries you serve or which standards you comply with, which the assistant fills with what similar companies usually do.

The measurement starts with a written list of approved facts, twenty to forty statements a buyer might check, such as what you sell, where you operate, which certifications you hold, which industries you serve and which integrations you support, each signed off by the person who owns it. Then ask each assistant five to ten questions that name your company, such as what it sells, how it compares with a named competitor and what its system can and can't do, and split every answer into single factual statements. Mark each statement as correct, outdated, wrong or unverifiable against the list, and divide the correct ones by all the statements you could judge.

Take four assistants and eight questions, which might produce around 160 factual statements in all. If 128 of them match the approved list, Answer Fidelity is 80 percent, and the 32 that don't become the work list. When several assistants repeat the same error, it usually comes from one shared source, so a single outdated product name can account for a large share of the 32, and correcting that one source moves the score more than any new page would.

Fixing means correcting the source rather than the answer, because you can't edit what an assistant says, only what it reads. For each wrong statement, find the page it most likely came from, by asking the assistant for its sources or by searching the wrong phrase, then update, redirect or remove that page, correct the directory and database entries you control, and write the missing facts in plain sentences on the pages buyers and assistants read first. Assistants that search the web at the moment of the question can reflect a correction once they retrieve the updated page, while answers drawn from training data change only when the model is retrained, so an error can disappear from one assistant and stay in another for months.

The measure has limits. Some statements can't be judged against a list, such as an assistant's view that a competitor suits small sites better, and those belong in a separate note rather than in the score. Assistants also decline to answer some questions about smaller companies, and a refusal is neither right nor wrong, so we count refusals separately. And forty approved facts cover only what you thought to write down, so read the statements marked unverifiable too, because they show what buyers ask that your list doesn't answer yet.

Torke works from approved claims only, and the same discipline belongs here, because an assistant is now one of the places a buyer reads your specification before they read your own sheet. The audit works for anything a buyer reads about you without you in the room, from analyst summaries to partner pages. Write your list of approved facts this week, ask four assistants five questions that name your company, and count the statements that are wrong, since each one points to a page somewhere that still says it.

Questions readers ask

Each answer stands on its own, so you can forward one to a colleague, and an assistant asked the same question can quote it as it is.

Why does ChatGPT say my company sells a product we discontinued?

Most likely because an old page describing the product is still public, such as an archived product sheet or a directory entry, and the assistant retrieves it or learned from it. Updating, redirecting or removing that page is the repair.

How long does it take for AI assistants to show corrected company information?

Assistants that search the web when asked can reflect a correction once they retrieve the updated page. Answers drawn from training data change only when the model is retrained, so the timing differs from one assistant to another.

What belongs on a list of approved company facts for AI assistants?

Twenty to forty statements a buyer might check: what you sell, where you operate, certifications, industries served, integrations, founding year and leadership, each signed off by the person who owns that fact.

Bring us the problem, and we bring the solution to it

Tell us what your team sells, who buys it and where deals stop moving. You'll get a written reply saying which way of working fits you, or whether a Diagnostic makes sense, and if neither does, the reason.

What would you like to talk about?

Your answers go straight to Vincent's inbox at vincent@torke.tech, and no third-party form service handles them. So we can see which pages lead to a conversation, the site keeps your company name, the topic you chose, the pages you read before writing and the network, city and IP address of your request in a database in the EU. The IP address is deleted after 90 days and the rest after 395, and none of it is sold or shared.