Explainer

Unified Memory or VRAM for AI: Which One Fits Your Model

In this article
  1. What each design means
  2. Why capacity decides what loads
  3. Why bandwidth decides speed
  4. When the model fits in VRAM
  5. When the model does not fit
  6. Context length adds to the load
  7. A short way to decide
  8. Software and ecosystem
  9. What this means when buying
  10. Frequently asked questions
  11. Sources & credits

Short answer

Dedicated VRAM is fast but limited in size, while unified memory offers larger pools, usually with less bandwidth than a high-end card. A model that fits in VRAM usually runs faster on a discrete GPU. A model that does not fit often runs faster on a machine with a big unified pool. Capacity decides what loads, and bandwidth decides how quickly it answers.

  • Text generation speed follows memory bandwidth; prompt processing follows compute.
  • Apple, AMD and NVIDIA's GB10 use a shared pool for processor and graphics.
  • A model that spills out of VRAM can slow down sharply, as Puget and Hardware Corner show.
  • The right choice follows the size of the model you want to run.

What each design means#

A discrete graphics card carries its own memory, called VRAM, soldered next to the GPU. NVIDIA lists the desktop RTX 5090 with 32 GB of GDDR7 on a 512-bit interface, the RTX 5080 with 16 GB on 256-bit, and the RTX 5070 Ti with 16 GB on 256-bit. The processor has its own separate system memory, and data crosses a slower link between the two.

Unified memory gives the processor and the graphics engine a single shared pool. Apple, AMD and NVIDIA's GB10 chip all use this idea. The graphics engine reads the same memory as the CPU, so a model is not limited to a small private pool.

Three makers show how this looks in practice.

Table
PlatformMemory designMemory interface and bandwidthLargest pool
Desktop RTX 5090Dedicated GDDR7512-bit32 GB
Apple Mac mini (M6)UnifiedUp to 170 GB/s32 GB
AMD Ryzen AI Max+ 395Shared LPDDR5x256-bit LPDDR5x-8000128 GB
NVIDIA DGX Spark (GB10)Unified LPDDR5x256-bit, 273 GB/s128 GB
Apple MacBook Pro (M5 Max)Unified614 GB/s128 GB
Apple Mac Studio (M5 Ultra)Unified1.2 TB/s512 GB

NVIDIA lists bus width, not bandwidth, for its desktop cards, so the table gives bus width only.

Memory bandwidth for shared-memory platforms: makers' figures, except the Ryzen AI Max+ 395 figure, which is from Tom's HardwareMac mini M6170 GB/sRyzen AI Max+ 395256 GB/sDGX Spark (GB10)273 GB/sMacBook Pro M5 Max614 GB/sMac Studio M5 Ultra1200 GB/s
Memory bandwidth for shared-memory platforms: makers' figures, except the Ryzen AI Max+ 395 figure, which is from Tom's Hardware
ItemValue
Mac mini M6170 GB/s
Ryzen AI Max+ 395256 GB/s
DGX Spark (GB10)273 GB/s
MacBook Pro M5 Max614 GB/s
Mac Studio M5 Ultra1200 GB/s
Memory bandwidth for shared-memory platforms: makers' figures, except the Ryzen AI Max+ 395 figure, which is from Tom's Hardware

Why capacity decides what loads#

A language model has to sit in memory before it can answer. If it fits in graphics memory, everything stays on the fast path. If it does not, the runtime places some layers in system memory and the processor handles them, which is much slower.

AMD gives concrete examples for its own platform. It says a 128 GB Ryzen AI Max+ 395 machine can dedicate up to 96 GB to graphics, which it says is enough for 4-bit models up to 128 billion parameters, or FP16 models up to 32 billion. AMD adds that Llama 4 109B in Q4_K_M needs roughly 96 GB. AMD also shows how quantisation shrinks files, quoting llama.cpp sizes for a 7B model of 3.80 GB at Q4_K_M, 6.70 GB at Q8_0 and 13 GB at F16.

NVIDIA says its 128 GB DGX Spark handles inference for models of up to 200 billion parameters. These are maker claims about what can load, not about speed. The explainer on the NVIDIA DGX Spark covers that machine in detail.

For a desktop card, the same logic applies at a smaller scale. A 16 GB card holds models that a 12 GB card cannot, as the how much VRAM for an LLM explainer shows by model size.

Largest memory pool listed by the makers for each platformRTX 508016 GBRTX 509032 GBMac mini M632 GBRyzen AI Max+ 395128 GBDGX Spark (GB10)128 GBMacBook Pro M5 Max128 GBMac Studio M5 Ultra512 GB
Largest memory pool listed by the makers for each platform
ItemValue
RTX 508016 GB
RTX 509032 GB
Mac mini M632 GB
Ryzen AI Max+ 395128 GB
DGX Spark (GB10)128 GB
MacBook Pro M5 Max128 GB
Mac Studio M5 Ultra512 GB
Largest memory pool listed by the makers for each platform

Why bandwidth decides speed#

Once a model is loaded, the speed of text generation depends on how fast memory can stream its weights. Tom's Hardware explains it simply: each new token passes through every layer, and every layer's active weights are read from memory for each token. The compute per token is small, so memory bandwidth sets the pace.

Prompt processing is different. The whole prompt is processed together, which is work for the compute units, not just the memory link. Tom's Hardware notes that a benchmark made mostly of prompt processing favours raw compute. Image generation is also compute-bound, it says.

This is why the two numbers in a benchmark can point in different directions. A machine with more bandwidth can lead on generation while a machine with more compute leads on prompt processing. Puget Systems saw exactly that in one test, described below.

Bandwidth is also not the whole story. In Tom's Hardware's llama.cpp tests, an M4 Max Mac Studio (546 GB/s, per Tom's Hardware) with about double the bandwidth of the GB10 and the Ryzen AI Max+ 395 produced only about 25 percent more generation speed on a mixture-of-experts model. On a dense 12B model it was about 1.8 times the GB10 and 2.26 times the AMD chip. On gpt-oss-120b the gaps were about 1.5 and 1.6 times. Tom's Hardware concludes that bandwidth alone is a poor proxy for delivered speed, and that the gain depends on model design.

When the model fits in VRAM#

Puget Systems compared an RTX 4090 Mobile laptop with 16 GB of VRAM (576 GB/s) against a 64 GB M3 Max MacBook Pro (400 GB/s) on llama.cpp. Llama 3 8B at Q4_K_M fits in 16 GB, so the discrete card ran at full speed.

Meta-Llama-3-8B-Instruct Q4_K_M (fits in 16GB VRAM): token generation (tg128) (tokens/s, higher is better)

Meta-Llama-3-8B-Instruct Q4_K_M (fits in 16GB VRAM): token generation (tg128)Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600: 67.5 tokens/sPuget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s…67.5 tokens/sApple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified): 55.2 tokens/sApple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 G…55.2 tokens/s

2 products. Highest: Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600, 67.5 tokens/s. Lowest: Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified), 55.2 tokens/s.

Meta-Llama-3-8B-Instruct Q4_K_M (fits in 16GB VRAM): token generation (tg128)
Producttokens/s
Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-560067.5
Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified)55.2
Source: Puget Systems, Puget Mobile 17 vs M3 Max MacBook Pro 16 for AI workflows (28 May 2024) (opens in a new tab). llama-bench, llama.cpp build 628b2991 (2794), Metal on Mac / CUDA 12.2 on PC, 5 runs averaged, plugged in; pp=512 tokens, tg=128 tokens. Values read from Puget bar-chart labels. Model fits in VRAM on both..

About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.

The NVIDIA laptop generated 67.5 tokens per second against 55.2 on the Mac, about 22 percent ahead, and the lead grew with heavier formats, reaching 38 percent at FP16. Puget also measured prompt processing at more than seven times faster on the PC at FP16. This is the case discrete VRAM is built for. The model fits, the bandwidth is higher, and the extra compute helps.

LMSYS found the same pattern with a desktop-class comparison. On gpt-oss-20b under Ollama, a model small enough for every card tested, the DGX Spark generated 49.7 tokens per second. The RTX 5090 Founders Edition managed 205 and the RTX PRO 6000 Blackwell 215.

GPT-OSS 20B MXFP4 (fits in every card tested): decode (tokens/s, higher is better)

GPT-OSS 20B MXFP4 (fits in every card tested): decodeNVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB: 215 tokens/sNVIDIA RTX PRO 6000 Blackwell Workstation Edition 9…215 tokens/sNVIDIA GeForce RTX 5090 Founders Edition 32GB GDDR7: 205 tokens/sNVIDIA GeForce RTX 5090 Founders Edition 32GB GDDR7205 tokens/sNVIDIA DGX Spark GB10 128GB LPDDR5x (273 GB/s): 49.7 tokens/sNVIDIA DGX Spark GB10 128GB LPDDR5x (273 GB/s)49.7 tokens/s

3 products. Highest: NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB, 215 tokens/s. Lowest: NVIDIA DGX Spark GB10 128GB LPDDR5x (273 GB/s), 49.7 tokens/s.

GPT-OSS 20B MXFP4 (fits in every card tested): decode
Producttokens/s
NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB215
NVIDIA GeForce RTX 5090 Founders Edition 32GB GDDR7205
NVIDIA DGX Spark GB10 128GB LPDDR5x (273 GB/s)49.7
Source: LMSYS Org, NVIDIA DGX Spark In-Depth Review (13 Oct 2025) (opens in a new tab). Ollama, batch 1; review says results may date as software improves. Numbers as stated in LMSYS text. Model fits in memory on all three..

About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.

LMSYS names the Spark's 273 GB/s of LPDDR5x bandwidth as the main limit. Prompt processing showed the same gap, at 2,053 tokens per second for the Spark against 8,519 for the RTX 5090. A big shared pool does not make a model faster when it already fits somewhere faster.

Unified Memory or VRAM for AI (last checked November 2026).

When the model does not fit#

Puget's second test is the revealing one. Llama 3 70B at Q4_K_M is too large for 16 GB of VRAM, so the laptop offloaded part of the model to system memory. The Mac's 64 GB pool held the whole model.

Meta-Llama-3-70B-Instruct Q4_K_M (too big for 16GB VRAM): token generation (tg128) (tokens/s, higher is better)

Meta-Llama-3-70B-Instruct Q4_K_M (too big for 16GB VRAM): token generation (tg128)Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified): 7.5 tokens/sApple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 G…7.5 tokens/sPuget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600: 1.4 tokens/sPuget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s…1.4 tokens/s

2 products. Highest: Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified), 7.5 tokens/s. Lowest: Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-5600, 1.4 tokens/s.

Meta-Llama-3-70B-Instruct Q4_K_M (too big for 16GB VRAM): token generation (tg128)
Producttokens/s
Apple MacBook Pro 16 M3 Max 40-core GPU 64GB (400 GB/s unified)7.5
Puget Mobile 17 RTX 4090 Mobile 16GB VRAM (576 GB/s) + 64GB DDR5-56001.4
Source: Puget Systems, Puget Mobile 17 vs M3 Max MacBook Pro 16 for AI workflows (28 May 2024) (opens in a new tab). llama-bench, llama.cpp build 628b2991 (2794), Metal on Mac / CUDA 12.2 on PC, 5 runs averaged, plugged in; pp=512 tokens, tg=128 tokens. Model does not fit 16GB VRAM; PC offloads part to system RAM (Puget text). Values read from chart labels..

About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.

The Mac generated 7.5 tokens per second against 1.4 for the NVIDIA laptop, a little over five times as fast. The lead from the previous test reversed. Puget says unified memory shows its strengths when loading large models, and that more offloading to non-unified memory costs more speed.

The PC still led on prompt processing, at 99.8 tokens per second against 71.2 for the Mac. That fits the compute argument above. For long prompts the discrete card helps. For answer speed on a model that spills, the single pool helps.

Hardware Corner found a similar penalty on desktop cards. gpt-oss-120b does not fit in the 32 GB of an RTX 5090. With a plain split of layers between GPU and system memory, generation ran at about 3.4 tokens per second. Moving only the mixture-of-experts layers into system memory, a setting called --n-cpu-moe, raised it to about 9.6.

gpt-oss-120b (F16 GGUF, MXFP4 weights) on RTX 5090 32GB; model does not fit in VRAM: token generation, ~86 token prompt (tokens/s, higher is better)

gpt-oss-120b (F16 GGUF, MXFP4 weights) on RTX 5090 32GB; model does not fit in VRAM: token generation, ~86 token promptNVIDIA RTX 5090 32GB, --n-cpu-moe 21 (expert layers in system RAM, attention in VRAM): 9.6 tokens/sNVIDIA RTX 5090 32GB, --n-cpu-moe 21 (expert layers…9.6 tokens/sNVIDIA RTX 5090 32GB, standard layer split (17 layers on GPU, rest on CPU/system RAM): 3.4 tokens/sNVIDIA RTX 5090 32GB, standard layer split (17 laye…3.4 tokens/s

2 products. Highest: NVIDIA RTX 5090 32GB, --n-cpu-moe 21 (expert layers in system RAM, attention in VRAM), 9.6 tokens/s. Lowest: NVIDIA RTX 5090 32GB, standard layer split (17 layers on GPU, rest on CPU/system RAM), 3.4 tokens/s.

gpt-oss-120b (F16 GGUF, MXFP4 weights) on RTX 5090 32GB; model does not fit in VRAM: token generation, ~86 token prompt
Producttokens/s
NVIDIA RTX 5090 32GB, --n-cpu-moe 21 (expert layers in system RAM, attention in VRAM)9.6
NVIDIA RTX 5090 32GB, standard layer split (17 layers on GPU, rest on CPU/system RAM)3.4
Source: Hardware Corner, GPT-OSS 120B: Offloading MoE Layers to CPU Boosts RTX 3090 and 5090 Performance (Allan Witt, updated 10 Nov 2025) (opens in a new tab). llama.cpp server, --ctx-size 8192, flash-attn, n-gpu-layers 99. Article does not restate the 5090 host system; the RTX 3090 test used EPYC 7343 + 64GB DDR4-3200. Article lists the same command for both 5090 runs, with differences described in text. Layer placement set by hand (--n-cpu-moe 21 for the second row)..

About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.

Hardware Corner's reasoning is that the attention layers gain most from fast memory and so belong on the GPU, while the bulky expert layers can sit in system memory. Tuning where layers go can recover a lot of speed, but it takes effort and still runs far below a model that fits entirely.

Unified Memory or VRAM for AI (last checked November 2026).

Context length adds to the load#

Context length is the amount of text a model keeps in memory while it answers, and it uses memory too. Hardware Corner reported that an RTX 5090 could not load 64K context for a 27B dense model in llama.cpp, while an M5 Max MacBook Pro with 128 GB handled it. It attributed the difference to the larger unified pool. Those M5 Max numbers came from a community user running MLX, as Hardware Corner reported, so treat the comparison as indicative.

On the same 27B model at 6-bit, Hardware Corner found the RTX 5090 about two to three times faster at generation and 3.5 to 3.9 times faster at prompt processing when everything fit. On gpt-oss-120b, which fits in the M5 Max pool, the RTX PRO 6000 with 96 GB was about 2.5 to 2.8 times faster at generation, and the M5 Max still produced 64.5 to 87.9 tokens per second. A bigger pool lets you run the model at a usable speed, but a faster card still wins when both fit.

A short way to decide#

Pick the biggest model you plan to use, then see which row of the table it lands in.

Table
Your modelBetter fitReason from the sources
Fits in 16 to 32 GB of VRAMDiscrete RTX cardHigher bandwidth and compute, as in the Puget and LMSYS results
Larger than any card's VRAM but under 128 GBLarge unified or shared poolAvoids slow offloading, as Puget's 70B test shows
Needs very long contextLarger poolHardware Corner reports a 64K context loaded on the M5 Max but not on the RTX 5090
Needs CUDA-only softwareNVIDIA GPU or GB10CUDA runs only on NVIDIA hardware

Many buyers sit in the first row, and for them a card with enough VRAM is the simpler answer. The second row is where a Mac or a 128 GB Ryzen AI Max+ 395 machine earns its place.

Software and ecosystem#

Memory design is only part of the choice. CUDA, NVIDIA's software stack, runs only on NVIDIA GPUs. Apple chips run MLX and llama.cpp with Metal. AMD's Ryzen AI Max+ platform runs llama.cpp with Vulkan, and AMD says its driver update allows the full 96 GB of dedicated graphics memory in LM Studio on Windows.

Tom's Hardware also found that software can outweigh hardware. On one very large model, a two-unit DGX Spark cluster using a different inference engine led a Mac running llama.cpp on prompt processing by a wide margin. Tom's Hardware also reported that the Mac needed a workaround for an FP8 failure in an image-generation test. Compatibility needs checking for each tool you plan to use.

What this means when buying#

Match the memory design to the model you want to run, not to a headline spec.

The wider AI workstations and PCs hub lists every machine type in one place.

Apple laptops and desktops are sold in India through Apple's own store and authorised resellers, and their memory is fixed, so choose the configuration carefully before you buy.

Indian buyers comparing a Mac with a graphics card desktop should price both complete systems in rupees, since a Mac includes memory that a desktop sells separately.

Frequently asked questions#

What is unified memory?

Unified memory is one pool shared by the processor and the graphics engine. Apple chips, AMD's Ryzen AI Max+ 395 and NVIDIA's GB10 all use it, so a model is not limited to a small private graphics memory.

Is unified memory slower than VRAM?

Often, but not always. Most shared pools have less bandwidth than a high-end card, though Apple lists 1.2 TB/s for the M5 Ultra. LMSYS measured 49.7 tokens per second on the DGX Spark against 205 on an RTX 5090 for the same model. A bigger pool can still win when the model does not fit in VRAM.

How much unified memory can the GPU use?

It depends on the platform. AMD says a 128 GB Ryzen AI Max+ 395 can dedicate up to 96 GB to graphics. Apple lists its memory as one unified pool.

What happens when a model does not fit in VRAM?

The runtime moves some layers to system memory, which is much slower. Puget measured 1.4 tokens per second on a 16 GB laptop GPU against 7.5 on a 64 GB Mac for Llama 3 70B.

Does more memory bandwidth always mean faster AI?

No. Tom's Hardware found double the bandwidth gave only about 25 percent more generation speed on one mixture-of-experts model. Model design and software change the result.

Is a Mac or an RTX card better for local LLMs?

Neither wins everywhere. An RTX card is faster when the model fits in its memory. A Mac with a large unified pool is better for models that exceed a card's VRAM.

Does prompt processing depend on memory type?

Mostly on compute. Tom's Hardware notes that prompt-heavy work favours raw compute, and Puget saw an RTX laptop lead a Mac on prompt processing even when the Mac won on generation.

Can I add more unified memory later?

Apple lists memory as configurations chosen at purchase, and the Ryzen AI Max+ 395 machines and DGX Spark use fixed memory. Choose the capacity you need when you buy.

Sources & credits#

We have not tested these products ourselves. Specs come from the makers; benchmark figures come from the sources we name.

Facts and figures in this article come from these sources. Articles are compiled from the makers' published specifications and the benchmarks named above, and checked by the NextAIGrid Editorial Team.

Disclaimer

This article is for information only. Specifications and availability change, so check details on the maker's page before buying. Found an error? Tell us.

How was this explainer put together?

It draws on maker specification pages and published third-party benchmarks, named in the article. We have not tested these products ourselves. Specs come from the makers; benchmark figures come from the sources we name. Specifications and sources were last checked on 17 November 2026. Read how we research and our editorial policy.

Spotted an error? Tell us. Confirmed fixes are listed on the corrections page.

What questions do readers ask next?

Back to top