What is a "small" AI model, and how is it different from ChatGPT?

"AI model" is the part of an AI tool that actually writes the words. ChatGPT, Copilot and Gemini run very large models on servers in data centres. A small model is one that has been made compact enough to run on an ordinary laptop, with no internet connection at all.

What "small" means

Models are measured in parameters, the numbers they learned during training. The open models WorkOffline uses come from the Qwen3 family, published by Alibaba's Qwen team under the Apache 2.0 licence. According to their model cards on Hugging Face, they range from 4.0 billion parameters (Qwen3 4B Instruct 2507) to 14.8 billion (Qwen3 14B).

To fit on a laptop they are stored in a compressed 4-bit format, which shrinks each model to roughly a quarter of its original size with only a small loss in quality. That is how a 8.2 billion parameter model becomes a 5.0 GB file. The companies behind ChatGPT do not publish how big their models are, but they need data-centre hardware, far beyond a laptop.

Small model vs ChatGPT

Small models have improved quickly. On its own model card, Qwen reports that Qwen3-4B-Instruct-2507 scores ahead of OpenAI's GPT-4.1 nano on most of the tests it chose. Those are the publisher's own results on the publisher's choice of tests, so read them as a sign of progress, not a guarantee.

What small models do well

• Writing: drafting an email, making a message shorter or friendlier, turning bullet points into a paragraph.
• Proofreading: spelling, grammar and clumsy sentences in your own text.
• Summarising: a report, meeting notes or a long email thread you give it.
• Your own documents: "What is the notice period in this contract?" works well when the app finds the right page and hands it to the model. The answer can then point to the file and page so you can check.

Where they fall short

• Facts from memory: dates, figures, names and quotes are where small models slip most. They can sound confident and still be wrong.
• Anything recent: a model only knows what was in its training data. Offline, it cannot look anything up.
• Long reasoning: multi-step maths, tricky logic and complex code are better left to large models.
• Specialist advice: medical, legal and financial specifics need a professional or an up to date source.

How to get good results

1. Give it the material. "Summarise this document" beats "What do you know about this topic?".
2. Ask for one thing at a time, then refine: "Shorter." "More formal." "Bullet points."
3. Check any figure or date it gives from memory before you rely on it.
4. Choose the biggest model your laptop runs comfortably. See the Mac and Windows guides.

How WorkOffline handles this

WorkOffline is built around what small models do well. When you ask about your files, the app searches them first and the answer cites the file and page. If nothing is found, it says so. When a general question is answered without your files, the app itself adds a label: "Limited general knowledge: this small offline AI model can be wrong or out of date." It does not rely on the model to label itself. Questions about current events and medical, legal or financial specifics are declined, with an option to save them to check later.

Every ticket includes chat with your files and writing help, priced at one flat white. See tickets and prices.

Frequently asked questions

An evergreen guide from WorkOffline. No tracking scripts on this page.