A GeForce RTX 5080 is a fast card. A 16 GB card is also the one people buy first for a local LLM, because it costs less than a 32 GB card. For an office assistant that is a trap, and the reason is memory, not speed.
VRAM is the memory on the graphics card. A model runs fast only when it fits there. If you want the formula for how much it needs, we cover it in our guide to GPU memory for local LLMs. This post is about what happens when it does not fit.
The chip is not the problem
Public llama.cpp results show how fast a 16 GB card is when the model fits. On a 7B model in 4-bit form, an RTX 5080 generates about 182 to 185 tok/s (tokens per second, roughly words per second), against about 290 to 300 for an RTX 5090 and about 169 for an RTX PRO 4500 in the same test. The 5080 is not slow. It simply holds half the memory.
What happens when a 27B model meets 16 GB
Qwen3.8 27B is a dense model, which means it uses every weight for every word. At 4-bit and a 32,000-token context (roughly 24,000 words) it needs an estimated 21.5 GB. The card has 15.4 GB usable. The rest, about 35% of the model, stays in ordinary system RAM. This is called offload. For every word, the CPU has to work through that part from system RAM while the card waits.
Qwen3.8 27B, 4-bit, 32K context: the same model on three cards
Estimate. Tokens per second for one user. Higher is faster.
Source: EMD Lab AI selector estimate, calculated from model files and card specifications. Not a measurement.
That is a drop from about 35 to about 5 tok/s, a factor of seven, for a model that is only 6 GB too large. It is a cliff, not a slope. At 5 tok/s a long answer takes a minute or more, and people stop using the tool. Gemma 4 31B is worse: about 4 tok/s on the 16 GB card against about 30 on the RTX PRO 4500.
Why the drop is so steep
Speed in the memory-bound case is set by how fast the card can read the weights. A GDDR7 card reads at several hundred GB/s. System RAM on a desktop board is many times slower. Any share of the model that lives there sets the pace for every word. A little offload is a small penalty; a third of the model is a large one.
MoE and small models degrade more gently
MoE means mixture-of-experts: the model has many weights but uses only a few billion for each word. Less data moves per word, so offload hurts less. On the same 16 GB card the selector estimates:
| Model | Type | Fits in 16 GB? | Speed |
|---|---|---|---|
| gpt-oss-20b | MoE | Yes (14 GB) | about 153 tok/s |
| Gemma 4 26B-A4B | MoE | No, offload | about 36 tok/s |
| Qwen3-Coder 30B-A3B | MoE | No, offload | about 28 tok/s |
| Qwen3.8 27B | Dense | No, offload | about 5 tok/s |
| Gemma 4 31B | Dense | No, offload | about 4 tok/s |
So a 16 GB card is a good home for gpt-oss-20b, which fits with room to spare. But by the selector's count it has room for only about two long conversations at once. For an office, that is a personal tool, not a shared assistant.
The real floor
For an office assistant on 27B-class models, the floor is 24 GB, because the 4-bit file plus a long context needs about 21.5 GB. That leaves little room for a second user or a longer document. 32 GB is the comfortable size: it holds the model, a 32K context and a few more users. At 8-bit quality, or with many users, the answer is 96 GB. The 8 GB cards are not AI cards: they fit only the smallest models.
What 16 GB is still good for
A 16 GB card is not a bad purchase. It is the right size for jobs other than chat, using the selector's workload estimates:
- Whisper speech to text: about 4 GB comfortable. It handles Indonesian.
- OCR (reading text from scans and invoices): PaddleOCR-VL about 8 GB comfortable.
- Embeddings for document search: bge-m3 about 4 GB.
- 8B vision models that read screenshots and documents, in quantised form.
- Quantised image generation: models such as Z-Image Turbo (about 12 GB comfortable) and FLUX.2 [klein] 4B (about 13 GB).
- Small chat models up to about 20B MoE, for one person.
These are estimates. Image and video models run on one card, so the 16 GB limit applies to them too.
Which card to pick
If the assistant is the goal and the team is more than one person, buy 32 GB. The RTX PRO 4500 in the standard Studio build has 32 GB, and the RTX 5090 is faster at the same capacity. If you only need OCR, Whisper and search, a 16 GB card does the job and you save the money.
The short version
- The chip is fast; the memory is the limit. An RTX 5080 reaches about 182 to 185 tok/s on a 7B model.
- A 27B dense model on 16 GB falls from about 35 to about 5 tok/s (estimate). It is a cliff.
- MoE models fall more gently, and gpt-oss-20b fits and flies at about 153 tok/s.
- The floor for an office assistant is 24 GB; 32 GB is comfortable.
- 16 GB suits Whisper, OCR, embeddings, 8B vision and small models.
All speed and memory figures from the selector are estimates. Check your own model in the model finder.