A new kind of small computer has 128 GB of memory that its graphics chip can use. The best graphics cards have 32 or 96 GB. So the box wins, right? Not quite: it holds more, but it reads that memory far more slowly.
The two kinds of memory trade capacity against speed. Knowing which one your work needs saves you from buying the wrong machine.
Two ways to give an AI chip memory
VRAM is memory soldered onto the graphics card (GDDR7 on current cards). It is very fast, but the card has a fixed amount, and the rest of the computer cannot add to it. When a model is larger than VRAM, part of it runs from ordinary system memory and speed collapses. This is called offload.
Unified memory is one pool of LPDDR5X that the CPU and the GPU share. There is no separate card, so the GPU can use most of the pool. That is how the NVIDIA DGX Spark, the ASUS Ascent GX10 (same GB10 chip), Ryzen AI Max+ 395 boxes and Apple M-series computers hold 100 GB-class models on a desk.
The cost is bandwidth, how many gigabytes per second the chip can read from memory. For every word a model writes, the chip reads the active part of the model once. So bandwidth, more than raw compute, sets the writing speed in tok/s (tokens per second, roughly words per second).
Capacity against bandwidth
| Hardware | Memory type | Memory for AI | Bandwidth |
|---|---|---|---|
| RTX PRO 4500 Blackwell | VRAM, GDDR7 | 32 GB | 896 GB/s |
| GeForce RTX 5090 | VRAM, GDDR7 | 32 GB | 1,792 GB/s |
| RTX PRO 6000 Max-Q | VRAM, GDDR7 | 96 GB | 1,792 GB/s |
| DGX Spark / ASUS Ascent GX10 | Unified, LPDDR5X | 128 GB, about 115 GB usable | 273 GB/s |
| Ryzen AI Max+ 395 box | Unified, LPDDR5X | 128 GB, about 120 GB usable on Linux | 256 GB/s |
| Apple M5 Max | Unified | Up to 128 GB | 614 GB/s |
Read the last column. A box moves 256 to 273 GB/s. An RTX PRO 4500 moves 896. An RTX 5090 or an RTX PRO 6000 moves 1,792, about six to seven times as much as a box. The Apple chip sits in between at 614. On the other hand, no single graphics card in the table holds more than 96 GB, and the box holds more than 110 GB for the model.
Generation speed follows bandwidth
Public llama.cpp results use the same small model on every chip, a 7B model in 4-bit form, so they isolate the hardware. The order matches the bandwidth column almost exactly.
Text generation speed, 7B model, 4-bit (llama.cpp tg128)
Tokens per second. Higher is better. One chip each. The 7B model is small on purpose; bigger models run slower on every bar.
Source: llama.cpp community results (Llama 2 7B Q4_0, token generation). Not EMD Lab measurements. The RTX PRO 6000 result is the full-power Workstation Edition; the Max-Q card in our AI machines is not in this data.
The speed is not exactly proportional, because software and chip design also matter. But the pattern is clear: when the model fits, more bandwidth means more words per second.
What the 128 GB actually buys
Memory size decides which models run at all. A 120B-class model such as gpt-oss-120b needs about 67 GB in 4-bit form at a 32K context, according to the selector. That is more than any 32 GB card holds. It fits in a 128 GB box, and fits in a single 96 GB RTX PRO 6000. So the box is not the only way to run it. It is the compact, one-box way.
It also depends on the software. The Ryzen box shares one pool, and the GPU only gets a share of it: AMD's documentation sets that share with a kernel limit, and Windows allows less than Linux. We use 96 GB on Windows and about 120 GB on Linux when we size it. The GX10 has a fixed 128 GB with about 115 GB usable.
The box also costs you speed on anything big and dense: a dense 27B model writes at about 10 tok/s on a box, by the selector's estimate. Mixture-of-experts (MoE) models, which read only a small part of their weights for each word, avoid most of that cost.
What a box is good at, and what a tower is good at
A box fits:
- One person or a small group who want a large mixture-of-experts model such as gpt-oss-120b in the office, with files that stay in the building.
- Developers who need CUDA, NVIDIA's software stack, on a small Arm machine. That is the case for the ASUS Ascent GX10.
- A compact machine on a desk, instead of a tower with one or more large cards.
A tower fits:
- Speed. A 32 GB card writes a 27B model several times faster than a box does.
- Teams. Many people at once need bandwidth for every conversation.
- Image and video work, where raw compute matters more than memory size.
- Upgrades. A tower takes a bigger card later; a box is fixed.
A Mac with an M-series chip is a good machine for one person. We mention it for comparison; it has no CUDA, and the model tooling is different.
Which one for which job
Start from the model. If the model fits in 32 GB, a card is faster and simpler: our Studio workstation takes an RTX PRO 4500 or an RTX 5090. If you want a 100B-class mixture-of-experts model for yourself, a GX10 or a Ryzen box does it in a small case; ask us and we will tell you what we can quote. If many people share the model, or you want dense 70B models at a usable speed, you need VRAM: one RTX PRO 6000 Max-Q gives 96 GB, and our AI workstation has two for 192 GB.
The short version
- Unified memory gives you capacity: about 115 to 120 GB usable for the model in a box. VRAM gives you speed: 896 to 1,792 GB/s on a card, against 256 to 273 GB/s in a box.
- Writing speed follows bandwidth, so a card writes several times faster than a box when the model fits.
- Choose a box for big mixture-of-experts models for one person or a small group, and for CUDA development on the GX10.
- Choose a tower for speed, teams, and image or video work.
Speeds in the chart are public community results on a 7B model. Usable-memory figures are selector estimates. Try your own model in the model finder, or see the cards and boxes side by side on which hardware runs which AI.