How we check

The short version

Headroom answers two questions: will a model fit, and how fast will it write. The first is arithmetic on published facts. The second is an estimate we haven’t yet checked against real runs.

Of 75 figures on the site, 32 have been checked by hand. Each one carries a date, and this count updates itself as they are.

What the labels mean

Every chart and table says where its numbers came from with one of five labels. They run from most to least certain.

Measured: We ran it ourselves on real hardware.

We ran it ourselves, on a named machine, with named tool versions. The only kind we treat as settled.

Tested by others: Measured by a named independent tester, not by us.

Measured by a named independent tester, such as Artificial Analysis for cloud services. Real, but on their setup, not ours.

Spec sheet: From the manufacturer's published specifications.

Copied from Apple, NVIDIA or Ollama. Usually right, but it says what the hardware can do, not what you will get.

Published: From the model's published architecture.

Read from the file a model’s makers publish alongside it, such as its number of layers. These are facts about the model.

Estimate: Calculated from a stated rule of thumb. Real results vary.

Worked out from a stated rule of thumb. Good enough to compare options; not good enough to quote.

Unverified: A stand-in value we have not confirmed yet.

A stand-in we chose so the tools work. It will be replaced by a measurement before launch.

A figure also shows when it was last checked. Anything we have not yet checked says not yet verified, rather than leaving the date out.

Will it fit?

A model needs memory for three things. Here is the sum for the home page’s starting example, Qwen3 8B at 4-bit with 8K tokens of conversation on a Mac mini M4 with 16 GB.

Then we compare the two.

Here that is 63%, so the answer is runs well. The 85% line leaves room for the browser and everything else you have open. A Mac has no “runs slowly” band: once its memory is full there is nowhere else for the model to go.

How fast will it write?

To write each tokenTokenA piece of a word, about three-quarters of an English word on average. Models read and write in tokens.Read the full entry, the computer reads the whole model out of memory once. Reading is slow and the maths is quick, so memory speed sets the limit. The chart on the home page shows what that means for every model.

Two adjustments, both estimates:

• Mixture of expertsMixture of expertsA model split into many small “experts”. Each token goes through only a few of them, so far less is read per token than the model’s full size.Read the full entry models (Gemma 4 E4B and gpt-oss 20B and Gemma 4 26B-A4B and Qwen 3.6 35B-A3B) only read a small part of themselves for each token, so we use that part instead of the full size. Gemma 4 E4B has 8 billion numbers in memory but reads about 4.5 billion per token.
• Models that spill out of a graphics card are read partly from the card and partly from the PC’s main memory, at about 60 GB/s. We combine the two speeds by how much of the model sits in each, which is why a model that is mostly on the card can still slow right down.

We only estimate writing speed. Before writing, the model first reads your message; that is much faster per token, but a long document can still take a few seconds before the first word appears.

Where our estimates are weakest

We would rather tell you than have you find out.

• The 45%–75% range is a guess. It is the single most important number on the site, and we have not measured it yet. Every speed you see depends on it.
• Mixture-of-experts speeds are probably too high. Choosing experts for each token costs time that the simple sum leaves out. For gpt-oss 20B on an RTX 4090 we currently show 225–375 tokens a second, which we expect real runs to undercut.
• Speeds for models that spill are probably too low. Ollama splits the work between the card and the processor in ways our sum doesn’t capture.
• Gemma 3’s conversation memory is overstated. Most of its layers only look at recent tokens, which needs less memory than our formula assumes.
• We assume conversation memory at full precision. Some apps can store it at lower precision, which would let longer chats fit.
• Other apps are not counted. A browser with many tabs can take several gigabytes, especially on a Mac, where it shares memory with the model.

How we test the guides

Every guide opens with a Last tested box: the date, the machines, and the versions of every tool used. A guide counts as tested when someone has followed it start to finish, as written, on a real machine. That means:

• A user account with no AI tools installed, so nothing is already set up.
• A stopwatch on every step, so the times we print are real, and slow steps get flagged.
• Every place the instructions didn’t match the screen, written down and fixed.

After 90 days the box turns into a warning, because apps change their menus and commands faster than that. Right now no guide has been through this yet, and each one says so at the top.

Every figure, in one place

The same data the checker uses, with its source and the date it was last checked.

Models

Machines

Precision

Our assumptions

Conversation lengths offered in the checker: 4K, 8K, 32K, 128K tokens.

What we will measure next

Two machines will turn most of the estimates above into measurements:

• A PC with an RTX 4090 (24 GB) and 64 GB of main memory, on Windows 11. This will give the real share of memory speed on a graphics card, how much memory Windows keeps back, and how fast a model that spills really runs.
• A Mac mini M4 with 16 GB. This will give the same share for Apple’s chips and confirm how much memory macOS lets a model use.

For each run we will publish the model, settings, tool versions and the raw speed Ollama reports, so you can repeat it on your own machine.