In this article
Short answer
Dedicated VRAM is fast but limited in size, while unified memory offers larger pools, usually with less bandwidth than a high-end card. A model that fits in VRAM usually runs faster on a discrete GPU. A model that does not fit often runs faster on a machine with a big unified pool. Capacity decides what loads, and bandwidth decides how quickly it answers.
- Text generation speed follows memory bandwidth; prompt processing follows compute.
- Apple, AMD and NVIDIA's GB10 use a shared pool for processor and graphics.
- A model that spills out of VRAM can slow down sharply, as Puget and Hardware Corner show.
- The right choice follows the size of the model you want to run.
What each design means#
A discrete graphics card carries its own memory, called VRAM, soldered next to the GPU. NVIDIA lists the desktop RTX 5090 with 32 GB of GDDR7 on a 512-bit interface, the RTX 5080 with 16 GB on 256-bit, and the RTX 5070 Ti with 16 GB on 256-bit. The processor has its own separate system memory, and data crosses a slower link between the two.
Unified memory gives the processor and the graphics engine a single shared pool. Apple, AMD and NVIDIA's GB10 chip all use this idea. The graphics engine reads the same memory as the CPU, so a model is not limited to a small private pool.
Three makers show how this looks in practice.
| Platform | Memory design | Memory interface and bandwidth | Largest pool |
|---|---|---|---|
| Desktop RTX 5090 | Dedicated GDDR7 | 512-bit | 32 GB |
| Apple Mac mini (M6) | Unified | Up to 170 GB/s | 32 GB |
| AMD Ryzen AI Max+ 395 | Shared LPDDR5x | 256-bit LPDDR5x-8000 | 128 GB |
| NVIDIA DGX Spark (GB10) | Unified LPDDR5x | 256-bit, 273 GB/s | 128 GB |
| Apple MacBook Pro (M5 Max) | Unified | 614 GB/s | 128 GB |
| Apple Mac Studio (M5 Ultra) | Unified | 1.2 TB/s | 512 GB |
NVIDIA lists bus width, not bandwidth, for its desktop cards, so the table gives bus width only.
| Item | Value |
|---|---|
| Mac mini M6 | 170 GB/s |
| Ryzen AI Max+ 395 | 256 GB/s |
| DGX Spark (GB10) | 273 GB/s |
| MacBook Pro M5 Max | 614 GB/s |
| Mac Studio M5 Ultra | 1200 GB/s |
Why capacity decides what loads#
A language model has to sit in memory before it can answer. If it fits in graphics memory, everything stays on the fast path. If it does not, the runtime places some layers in system memory and the processor handles them, which is much slower.
AMD gives concrete examples for its own platform. It says a 128 GB Ryzen AI Max+ 395 machine can dedicate up to 96 GB to graphics, which it says is enough for 4-bit models up to 128 billion parameters, or FP16 models up to 32 billion. AMD adds that Llama 4 109B in Q4_K_M needs roughly 96 GB. AMD also shows how quantisation shrinks files, quoting llama.cpp sizes for a 7B model of 3.80 GB at Q4_K_M, 6.70 GB at Q8_0 and 13 GB at F16.
NVIDIA says its 128 GB DGX Spark handles inference for models of up to 200 billion parameters. These are maker claims about what can load, not about speed. The explainer on the NVIDIA DGX Spark covers that machine in detail.
For a desktop card, the same logic applies at a smaller scale. A 16 GB card holds models that a 12 GB card cannot, as the how much VRAM for an LLM explainer shows by model size.
| Item | Value |
|---|---|
| RTX 5080 | 16 GB |
| RTX 5090 | 32 GB |
| Mac mini M6 | 32 GB |
| Ryzen AI Max+ 395 | 128 GB |
| DGX Spark (GB10) | 128 GB |
| MacBook Pro M5 Max | 128 GB |
| Mac Studio M5 Ultra | 512 GB |
Why bandwidth decides speed#
Once a model is loaded, the speed of text generation depends on how fast memory can stream its weights. Tom's Hardware explains it simply: each new token passes through every layer, and every layer's active weights are read from memory for each token. The compute per token is small, so memory bandwidth sets the pace.
Prompt processing is different. The whole prompt is processed together, which is work for the compute units, not just the memory link. Tom's Hardware notes that a benchmark made mostly of prompt processing favours raw compute. Image generation is also compute-bound, it says.
This is why the two numbers in a benchmark can point in different directions. A machine with more bandwidth can lead on generation while a machine with more compute leads on prompt processing. Puget Systems saw exactly that in one test, described below.
Bandwidth is also not the whole story. In Tom's Hardware's llama.cpp tests, an M4 Max Mac Studio (546 GB/s, per Tom's Hardware) with about double the bandwidth of the GB10 and the Ryzen AI Max+ 395 produced only about 25 percent more generation speed on a mixture-of-experts model. On a dense 12B model it was about 1.8 times the GB10 and 2.26 times the AMD chip. On gpt-oss-120b the gaps were about 1.5 and 1.6 times. Tom's Hardware concludes that bandwidth alone is a poor proxy for delivered speed, and that the gain depends on model design.
When the model fits in VRAM#
Puget Systems compared an RTX 4090 Mobile laptop with 16 GB of VRAM (576 GB/s) against a 64 GB M3 Max MacBook Pro (400 GB/s) on llama.cpp. Llama 3 8B at Q4_K_M fits in 16 GB, so the discrete card ran at full speed.
Meta-Llama-3-8B-Instruct Q4_K_M (fits in 16GB VRAM): token generation (tg128) (tokens/s, higher is better)
2 products. Highest: Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600, 67.5 tokens/s. Lowest: Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified), 55.2 tokens/s.
| Product | tokens/s |
|---|---|
| Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600 | 67.5 |
| Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified) | 55.2 |
About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.
The NVIDIA laptop generated 67.5 tokens per second against 55.2 on the Mac, about 22 percent ahead, and the lead grew with heavier formats, reaching 38 percent at FP16. Puget also measured prompt processing at more than seven times faster on the PC at FP16. This is the case discrete VRAM is built for. The model fits, the bandwidth is higher, and the extra compute helps.
LMSYS found the same pattern with a desktop-class comparison. On gpt-oss-20b under Ollama, a model small enough for every card tested, the DGX Spark generated 49.7 tokens per second. The RTX 5090 Founders Edition managed 205 and the RTX PRO 6000 Blackwell 215.
GPT-OSS 20B MXFP4 (fits in every card tested): decode (tokens/s, higher is better)
3 products. Highest: NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB, 215 tokens/s. Lowest: NVIDIA DGX Spark GB10 128GB LPDDR5x (273 GB/s), 49.7 tokens/s.
| Product | tokens/s |
|---|---|
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 215 |
| NVIDIA GeForce RTX 5090 Founders Edition 32GB GDDR7 | 205 |
| NVIDIA DGX Spark GB10 128GB LPDDR5x (273 GB/s) | 49.7 |
About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.
LMSYS names the Spark's 273 GB/s of LPDDR5x bandwidth as the main limit. Prompt processing showed the same gap, at 2,053 tokens per second for the Spark against 8,519 for the RTX 5090. A big shared pool does not make a model faster when it already fits somewhere faster.
Unified Memory or VRAM for AI (last checked November 2026).
When the model does not fit#
Puget's second test is the revealing one. Llama 3 70B at Q4_K_M is too large for 16 GB of VRAM, so the laptop offloaded part of the model to system memory. The Mac's 64 GB pool held the whole model.
Meta-Llama-3-70B-Instruct Q4_K_M (too big for 16GB VRAM): token generation (tg128) (tokens/s, higher is better)
2 products. Highest: Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified), 7.5 tokens/s. Lowest: Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600, 1.4 tokens/s.
| Product | tokens/s |
|---|---|
| Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified) | 7.5 |
| Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600 | 1.4 |
About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.
The Mac generated 7.5 tokens per second against 1.4 for the NVIDIA laptop, a little over five times as fast. The lead from the previous test reversed. Puget says unified memory shows its strengths when loading large models, and that more offloading to non-unified memory costs more speed.
The PC still led on prompt processing, at 99.8 tokens per second against 71.2 for the Mac. That fits the compute argument above. For long prompts the discrete card helps. For answer speed on a model that spills, the single pool helps.
Hardware Corner found a similar penalty on desktop cards. gpt-oss-120b does not fit in the 32 GB of an RTX 5090. With a plain split of layers between GPU and system memory, generation ran at about 3.4 tokens per second. Moving only the mixture-of-experts layers into system memory, a setting called --n-cpu-moe, raised it to about 9.6.
gpt-oss-120b (F16 GGUF, MXFP4 weights) on RTX 5090 32GB; model does not fit in VRAM: token generation, ~86 token prompt (tokens/s, higher is better)
2 products. Highest: NVIDIA RTX 5090 32GB, --n-cpu-moe 21 (expert layers in system RAM, attention in VRAM), 9.6 tokens/s. Lowest: NVIDIA RTX 5090 32GB, standard layer split (17 layers on GPU, rest on CPU/system RAM), 3.4 tokens/s.
| Product | tokens/s |
|---|---|
| NVIDIA RTX 5090 32GB, --n-cpu-moe 21 (expert layers in system RAM, attention in VRAM) | 9.6 |
| NVIDIA RTX 5090 32GB, standard layer split (17 layers on GPU, rest on CPU/system RAM) | 3.4 |
About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.
Hardware Corner's reasoning is that the attention layers gain most from fast memory and so belong on the GPU, while the bulky expert layers can sit in system memory. Tuning where layers go can recover a lot of speed, but it takes effort and still runs far below a model that fits entirely.
Unified Memory or VRAM for AI (last checked November 2026).
Context length adds to the load#
Context length is the amount of text a model keeps in memory while it answers, and it uses memory too. Hardware Corner reported that an RTX 5090 could not load 64K context for a 27B dense model in llama.cpp, while an M5 Max MacBook Pro with 128 GB handled it. It attributed the difference to the larger unified pool. Those M5 Max numbers came from a community user running MLX, as Hardware Corner reported, so treat the comparison as indicative.
On the same 27B model at 6-bit, Hardware Corner found the RTX 5090 about two to three times faster at generation and 3.5 to 3.9 times faster at prompt processing when everything fit. On gpt-oss-120b, which fits in the M5 Max pool, the RTX PRO 6000 with 96 GB was about 2.5 to 2.8 times faster at generation, and the M5 Max still produced 64.5 to 87.9 tokens per second. A bigger pool lets you run the model at a usable speed, but a faster card still wins when both fit.
A short way to decide#
Pick the biggest model you plan to use, then see which row of the table it lands in.
| Your model | Better fit | Reason from the sources |
|---|---|---|
| Fits in 16 to 32 GB of VRAM | Discrete RTX card | Higher bandwidth and compute, as in the Puget and LMSYS results |
| Larger than any card's VRAM but under 128 GB | Large unified or shared pool | Avoids slow offloading, as Puget's 70B test shows |
| Needs very long context | Larger pool | Hardware Corner reports a 64K context loaded on the M5 Max but not on the RTX 5090 |
| Needs CUDA-only software | NVIDIA GPU or GB10 | CUDA runs only on NVIDIA hardware |
Many buyers sit in the first row, and for them a card with enough VRAM is the simpler answer. The second row is where a Mac or a 128 GB Ryzen AI Max+ 395 machine earns its place.
Software and ecosystem#
Memory design is only part of the choice. CUDA, NVIDIA's software stack, runs only on NVIDIA GPUs. Apple chips run MLX and llama.cpp with Metal. AMD's Ryzen AI Max+ platform runs llama.cpp with Vulkan, and AMD says its driver update allows the full 96 GB of dedicated graphics memory in LM Studio on Windows.
Tom's Hardware also found that software can outweigh hardware. On one very large model, a two-unit DGX Spark cluster using a different inference engine led a Mac running llama.cpp on prompt processing by a wide margin. Tom's Hardware also reported that the Mac needed a workaround for an FP8 failure in an image-generation test. Compatibility needs checking for each tool you plan to use.
What this means when buying#
Match the memory design to the model you want to run, not to a headline spec.
- Models that fit in 16 to 32 GB run fastest on a discrete RTX card. The best GPU for local LLM and best high VRAM GPU for AI guides cover the cards.
- Models of 70 billion parameters and above need a large pool. A Mac with big unified memory or a 128 GB Ryzen AI Max+ 395 machine can load them. Our mini PC guide for local models lists the options.
- Laptops follow the same split. The best laptop for machine learning guide and the AI laptop vs desktop explainer cover the trade-offs.
- System memory matters for offloading. See how much RAM for a local LLM for sizing.
The wider AI workstations and PCs hub lists every machine type in one place.
Apple laptops and desktops are sold in India through Apple's own store and authorised resellers, and their memory is fixed, so choose the configuration carefully before you buy.
Indian buyers comparing a Mac with a graphics card desktop should price both complete systems in rupees, since a Mac includes memory that a desktop sells separately.
Frequently asked questions#
What is unified memory?
Unified memory is one pool shared by the processor and the graphics engine. Apple chips, AMD's Ryzen AI Max+ 395 and NVIDIA's GB10 all use it, so a model is not limited to a small private graphics memory.
Is unified memory slower than VRAM?
Often, but not always. Most shared pools have less bandwidth than a high-end card, though Apple lists 1.2 TB/s for the M5 Ultra. LMSYS measured 49.7 tokens per second on the DGX Spark against 205 on an RTX 5090 for the same model. A bigger pool can still win when the model does not fit in VRAM.
How much unified memory can the GPU use?
It depends on the platform. AMD says a 128 GB Ryzen AI Max+ 395 can dedicate up to 96 GB to graphics. Apple lists its memory as one unified pool.
What happens when a model does not fit in VRAM?
The runtime moves some layers to system memory, which is much slower. Puget measured 1.4 tokens per second on a 16 GB laptop GPU against 7.5 on a 64 GB Mac for Llama 3 70B.
Does more memory bandwidth always mean faster AI?
No. Tom's Hardware found double the bandwidth gave only about 25 percent more generation speed on one mixture-of-experts model. Model design and software change the result.
Is a Mac or an RTX card better for local LLMs?
Neither wins everywhere. An RTX card is faster when the model fits in its memory. A Mac with a large unified pool is better for models that exceed a card's VRAM.
Does prompt processing depend on memory type?
Mostly on compute. Tom's Hardware notes that prompt-heavy work favours raw compute, and Puget saw an RTX laptop lead a Mac on prompt processing even when the Mac won on generation.
Can I add more unified memory later?
Apple lists memory as configurations chosen at purchase, and the Ryzen AI Max+ 395 machines and DGX Spark use fixed memory. Choose the capacity you need when you buy.
Sources & credits#
We have not tested these products ourselves. Specs come from the makers; benchmark figures come from the sources we name.
- NVIDIA India, DGX Spark (opens in a new tab)
- NVIDIA India, desktop GeForce RTX comparison (opens in a new tab)
- NVIDIA India, GeForce RTX 5090 (opens in a new tab)
- AMD, Ryzen AI Max+ 395 (opens in a new tab)
- AMD blog, Variable Graphics Memory, VRAM, AI model sizes and quantisation (opens in a new tab)
- AMD blog, Ryzen AI Max+ upgraded for up to 128B parameter LLMs in LM Studio (opens in a new tab)
- Apple India, Mac mini tech specs (opens in a new tab)
- Apple India, MacBook Pro tech specs (opens in a new tab)
- Apple India, Mac Studio tech specs (opens in a new tab)
- Tom's Hardware, Mac Studio and M4 Max local AI performance (opens in a new tab)
- Tom's Hardware, Mac Studio with M5 Ultra local AI performance (opens in a new tab)
- LMSYS Org, NVIDIA DGX Spark in-depth review (opens in a new tab)
- Puget Systems, Puget Mobile 17 vs M3 Max MacBook Pro 16 for AI workflows (opens in a new tab)
- Hardware Corner, Apple M5 Max for local LLMs (opens in a new tab)
- Hardware Corner, GPT-OSS 120B MoE offloading (opens in a new tab)
Facts and figures in this article come from these sources. Articles are compiled from the makers' published specifications and the benchmarks named above, and checked by the NextAIGrid Editorial Team.
Disclaimer
This article is for information only. Specifications and availability change, so check details on the maker's page before buying. Found an error? Tell us.
How was this explainer put together?
It draws on maker specification pages and published third-party benchmarks, named in the article. We have not tested these products ourselves. Specs come from the makers; benchmark figures come from the sources we name. Specifications and sources were last checked on 17 November 2026. Read how we research and our editorial policy.
Spotted an error? Tell us. Confirmed fixes are listed on the corrections page.