confidently incorrect: investing with LLMs

July 2026 · by vellestrae

An AI audit of six companies’ financials — and what it says about trusting the machine with your money

In October 1994, a number theorist named Thomas Nicely noticed his sums weren’t adding up.

He was at Lynchburg College, grinding through calculations on twin primes — the kind of work where you check a value against itself a dozen different ways before breakfast. One result kept coming back as incorrect. Not wildly incorrect. Incorrect in the fifth significant digit.  Through trial and error, Nicely ruled out his code, his compiler, his own arithmetic, and was left with the suspect nobody suspects: the chip. Intel’s brand-new Pentium was dividing certain numbers incorrectly. A lookup table inside the floating-point unit was missing five entries out of 1,066, and for a thin band of operands, the machine built to do arithmetic had gotten the arithmetic wrong.

The most interesting element of this story is that the gap wasn’t huge.  4195835 ÷ 3145727 should give you 1.333820. The Pentium returned 1.333739. Sit with those two numbers. Nothing about the wrong one announces itself — right length, right shape, precise to the same decimal place. You’d never catch it unless you already knew the answer, which is the only reason that Nicely caught it at all.

It never looked wrong (therein lies the problem).

Intel knew about the flaw before Nicely went public with his finding, and the company’s first instinct was to manage the optics: the average user, they said, would hit the bug once every 27,000 years. Then IBM halted Pentium shipments, and the story went national.  The PR fallout was huge. On December 20, 1994, Intel agreed to replace every chip — a $475 million charge to their books. The lesson that priced in wasn’t “machines fail.” It was costlier than that. A machine can be wrong in a way that looks exactly like being right.

Taking the metaphor to 2026, if you’re pasting “what was Exxon-Mobil’s gross margin last quarter?” into a chatbot and trading on the answer, you want to know if there are tiny inaccuracies and choices in the output that, slowly but surely, carry you further away from the wholly accurate truth.

The AI conversation is obsessed with generation. Can it write the investment memo, build the DCF, draft the thesis. Fair questions. But finance has always run on a second, quieter discipline, and the models are weakest exactly there: verification. Not can it produce a number, but can you trust the number it produced.

When I worked as an analyst on an M&A team, a transposed digit in a merger model wasn’t an oops. It moved a purchase price, or it moved a fairness opinion, or it surfaced in a deposition. So you checked the number against the filing, every time, because the cost of being confidently wrong was asymmetric and the filing didn’t care how sure you felt.  And because M&A often comes down to trust, if your numbers are off, you can deal yourself a long-term reputation hit that lasts a lot longer than “oh, let me just fix that typo.”

The chatbot feels very sure. That’s the trouble. These models aren’t dangerous because they’re bad at financial data — if they were obviously bad, you’d distrust them and check. They’re dangerous because they’re good enough that you stop checking and trust what they’re telling you. Right nine times, and the tenth arrives wearing the same confident, four-significant-digit costume as the nine.

In my little personal AI experiment studio, I decided to run an audit to find out: could I find that tenth, confident blunderer?

Three models — Claude, GPT, Gemini. Six companies, picked across sectors on purpose, because a bank’s financials and an oil major’s financials and a chipmaker’s financials don’t read as identical, and I wanted to see where each model slipped: Apple, NVIDIA, JPMorgan, Costco, Eli Lilly, Exxon.

For each, I asked all three models for the same six figures from the most recent reported quarter: revenue, net income, GAAP diluted EPS, adjusted (non-GAAP) EPS, gross margin, and forward guidance. Then I checked every answer against the primary source — the actual 10-Q and 8-K on the SEC’s EDGAR. Not against a data aggregator, not against a second model. Against the filing the company is legally on the hook for.

I asked them the way you would: each app at its out-of-the-box defaults, web search on, no special toggles and no instructions about how to find anything. However you’d type the question yourself, that’s how they got it.

One ground rule made this fair: the models had to give a number, not a hedge. “Consult the latest filing” is the correct answer (and also a useless one if you’re trying to decide something today).

Every model got every hard number right.

That is not the sentence I expected to write. Revenue, net income, GAAP EPS, the gross margins that come off a clean line, next-quarter guidance — across all six companies and all three models, every figure with a single answer sitting in a filing came back matching the filing. Apple’s $111.18 billion in revenue. NVIDIA’s $2.39 in GAAP EPS, correctly above its $1.87 adjusted number, which is backwards from the usual and the kind of thing a model loves to “fix.” Exxon’s three different earnings-per-share figures, all three, named. No stale quarter served up as current. No guidance invented from nothing. Of the 36 figures I asked each model for, 26 had one right answer in a document — and all three models went 26 for 26.

So the tenth wasn’t hiding where I went looking. It was hiding in the ten numbers that don’t exist.

Headline accuracy, by model:

The revealing cut — the other ten figures, the ones with no single answer:

Two catches carry this, and neither is a wrong fact. They’re the opposite.

Costco. I asked three models for one company’s gross margin and got two different answers, both correct. Claude and GPT said 11.04% — Costco’s own figure from the 10-Q, net sales minus the cost of merchandise. Gemini said 12.77% — the same arithmetic run over total revenue, membership fees folded in. Neither is wrong. Costco doesn’t report “gross margin” as a line you can point to; the analyst must calculate it for herself. So the number depends on a denominator that relies on the analyst’s judgment. The models picked differently, silently, and handed the result back to the hundredth of a percent. (I’ll admit the trap nearly caught me, too: my own scoring sheet first flagged Gemini’s 12.77% as the error, before I remembered there was no single right number to bench against.)

Exxon. Worse, and cleaner. Exxon is an integrated oil major; it has no gross-margin line at all, because the concept barely applies to a company that pumps, ships, refines, and sells the same barrel. But if you ask anyway, the machine will not tell you the field is empty. GPT built a number: 26.6%. Gemini built one: 17.9%. Nine percentage points apart, on a quantity that does not exist, each delivered clean and self-assured. Only Claude refused — it said Exxon doesn’t report a gross margin, handed over operating and net margin instead, and left the empty field empty.

It never looked wrong. On these two, it couldn’t look wrong, because there was no right answer standing next to it. 

The patterns I went in watching for — last year’s number served as this quarter’s, the adjusted figure smuggled in as the reported one, guidance spun from air — barely appeared. Web search closed those gaps. What replaced them is quieter and, if you trade on the answer, worse: false precision on fields with no single truth. The model’s confidence never flickers. It reads exactly the same handing you Apple’s audited revenue and inventing Exxon’s gross margin. The four-significant-digit costume fits both.

The naive takeaway is “AI is unreliable, don’t use it.” Wrong lesson — and this audit makes it more wrong, not less. On the facts, the models were flawless. Every hard number a company files, all three returned correctly. If your fear was that the chatbot would misquote revenue or flip an EPS, you can forget that. That problem is largely solved.

But the tendency toward error doesn’t disappear entirely. It moves — to the one place you’re least attentive to catching it: the field where there’s no filing to check against, because the number was never in a filing. Costco’s gross margin, Exxon’s gross margin. The machine builds these, quietly chooses a method, and reports the construction with the same certainty it reports an audited figure. You can verify a number that exists. But you cannot as easily verify a number that the model built, because there is nothing to hold it against, except another model’s equally confident, differently-built number.

Which is the Pentium again, in a new package. The chip wasn’t usually wrong; it was wrong in a narrow band, invisibly, while looking right. The band has moved — it’s no longer the reported figures, it’s the derived ones — but the shape of the risk is identical. You manage it the way Nicely did: not by distrusting the machine wholesale, but by knowing which band to check. And the band to check is the number that had to be calculated, never the number that was simply filed. 

If you’re using these tools to research investment targets, the asymmetry from the M&A desk hasn’t changed; it’s just something you have to manage personally. One swapped GAAP-for-adjusted EPS reshapes a P/E. One stale guidance figure rebuilds a thesis on quicksand. The model won’t flag it, because to the model, the wrong number and the right number are the same kind of object — plausible text. The judgment about which is which is still on you, the investor.  (To my mind, this is also why expertise in a field still insulates human workers, at least for some time: you have to know where to be alert for AI’s confident gaffes.  In most fields, that Spidey sense is hard-won, through years in the field and steadily built expertise.)

Treat every AI financial figure as a lead, not a fact. Three habits that cost a minute and save a thesis:

1. Anchor the hard numbers to the filing. Revenue, EPS, guidance: see them in the 10-Q or 8-K on EDGAR before you trade on them. Free, primary, and the one source that’s legally accountable.

2. Always ask GAAP or adjusted. If the model doesn’t say, assume it doesn’t know;  that’s the swap most likely to flatter a valuation and lead you astray.

3. Distrust the clean number most. False precision is the tell. A figure suspiciously exact to the dollar is the one to check first, not last. And when the number is one a company never actually files — a gross margin for a bank or an oil major — remember the model will build you one anyway, and won’t mention that it’s making pivotal assumptions along the way.

It never looks wrong. That’s still the whole problem — and the minute you spend checking is the cheapest insurance in your process.

Next in this series: Investing with Open-Source.  The three models above are the ones with Silicon Valley price tags. But the open-weight models everyone’s suddenly running — DeepSeek, Qwen, Zhipu’s GLM — took the same six-company test, on the same filings, the same week. One of them handed me two-year-old numbers with total composure. Another answered for a quarter that hasn’t happened yet. Same questions, a very different report card — next week.

(Hit reply and tell me the worst number a model ever handed you with a straight face. I’m collecting them for posterity.)