Headroom
Colour theme

Your first local model, in about 30 minutes

From "what do I have?" to chatting with a model that runs entirely on your computer. No coding and no accounts.

About 30 minutes, most of it waiting for a download

Last tested
Not yet Written from documentation; it will be timed on real machines.
With
Ollama (version to record)
Show steps for
Your operating system

By the end of this page you’ll have an AI model running on your own computer. It works offline, and what you type never leaves your machine.

You’ll use Ollama, a free app that downloads and runs models. The rest of this site builds on it.

  1. Check what you have

    About 2 minutes

    Open the Apple menu and choose About This Mac. Note two things: the Chip (for example Apple M4) and the Memory (for example 16 GB).

    Any Mac with Apple silicon (M1 or later) works well. On an Intel Mac, models run on the processor alone, which is much slower.

    Press Ctrl Shift Esc to open Task Manager, choose Performance, then GPU. Note the card’s name and its Dedicated GPU memory.

    An NVIDIA card with 8 GB or more works well. Without one, models still run on the processor, just much more slowly.

    If you have an NVIDIA card, run this in a terminal and note the card’s name and memory:

    Terminal
    nvidia-smi

    Without an NVIDIA card, models still run on the processor, just much more slowly.

    Pick your machine below for the best first model and its download time.

    Your internet
    Your internet speed

    Start with

    Gemma 4 E4B

    It fits comfortably in your Mac with room for a long conversation, and should write about 19–31 tokens a second. Small, and can see and hear.

    Download Spec sheet: From the manufacturer's published specifications.
    9.5 GB
    Time to download Estimate: Calculated from a stated rule of thumb. Real results vary.
    About 15 min

  2. Install Ollama

    About 5 minutes

    Download Ollama for macOS from ollama.com/download, open the download and drag Ollama into your Applications folder. Open it from Applications. A small llama icon appears in the menu bar: that means it is running.

    Then open Terminal (press ⌘ Space and type Terminal) and check it is installed:

    Terminal
    ollama --version
    You should see
    ollama version is 0.x.x

    Download OllamaSetup.exe from ollama.com/download and run it. When it finishes, a small llama icon appears in the system tray near the clock: that means it is running.

    Then open PowerShell (press the Windows key and type PowerShell) and check it is installed:

    PowerShell
    ollama --version
    You should see
    ollama version is 0.x.x

    Run the official install script. It installs Ollama and starts it as a background service.

    Terminal
    curl -fsSL https://ollama.com/install.sh | sh
    Terminal
    ollama --version
    You should see
    ollama version is 0.x.x

  3. Download a model and chat

    About 5 to 20 minutes, depending on your internet

    Run this. The first time, Ollama downloads the model and shows a progress bar; after that it starts in seconds.

    Terminal
    ollama run gemma4:e4b
    You should see
    >>> Send a message (/? for help)

    When you see the >>> prompt, type a question and press ReturnEnter. Try: Explain what you are in two sentences.

    To leave the chat, type /bye. The model stays downloaded, so next time ollama run starts it straight away.

  4. See how fast it really is

    About 2 minutes

    Run the same model with --verbose. After each answer, Ollama prints timing details.

    Terminal
    ollama run gemma4:e4b --verbose

    Ask anything, then look for the line that says eval rate. That is how many TokenSmall pieces of text a model reads and writes. A token is roughly three-quarters of a word.Read the full entry per second your machine writes. If it’s far below the home page estimate, the next check usually explains why.

    While the chat is open, run this in a second window:

    Terminal
    ollama ps

    The PROCESSOR column should say 100% GPU. If it shows a share on the CPU, the model did not fit in fast memory. Close other large apps, or try a smaller model.

If something goes wrong

The download is very slow or stops. Run the same ollama run command again; it picks up where it left off.

ollama is not recognised. Make sure the Ollama app is running (look for the llama icon), then open a new terminal window.

Answers come out very slowly. Run ollama ps. If it does not say 100% GPU, the model is too big for your fast memory. Pick a smaller one in step 1.

An error says the model needs more memory. The model does not fit. Choose a smaller one, or close other apps and try again.

Where to go next

  • Running models on a Mac: how much memory you really have, which models to pick, and how to make them faster.
  • How I run it: serve models from one PC to every device in the house.