Explainer

How Much RAM for a Local LLM? Sizes by Model

In this article
  1. The sizing rule
  2. RAM needed by model size
  3. What changes the number
  4. What each RAM tier gives you
  5. Platform limits and module layout
  6. Reading quantisation names
  7. Worked examples
  8. How layers and offload change the maths
  9. Common sizing mistakes
  10. Planning for the next model
  11. What this means when buying
  12. Frequently asked questions
  13. Sources & credits

Short answer

Take the model's file size and add working room for the system. A 70B model at Q4_K_M is 43 GB in the Ollama library, so 64 GB is a safe floor for it. Smaller models run in 16 GB to 32 GB. Models near 120B need 96 GB, and a 70B model at 8-bit needs 128 GB.

  • Model file size sets the minimum, and quantisation level changes that size more than anything else.
  • Corsair recommends 64 GB as a minimum for AI systems and 96 GB for future-proofing.
  • Any part of the model that fits in VRAM does not need system RAM.
  • Desktop platforms list 256 GB as the maximum, but four modules lower the supported speed.

The sizing rule#

Local LLM software loads the whole model into memory, either VRAM, system RAM or a mix. The file you download is almost the amount of memory it takes. So the first step is to read the file size from the model's library page.

Then add room for three things. The operating system and your other programs take their share. The context cache, which holds the conversation so far, grows as the chat gets longer. And the runtime needs some working memory of its own. Makers do not publish one number for these, and it varies with the software and the context length you choose.

The practical method is to take the file size and pick the next RAM size up. A 43 GB model sits comfortably in 64 GB. A 58 GB model fits in 64 GB only with little left over, so 96 GB is the sensible choice for that one.

RAM needed by model size#

The table uses file sizes published in the Ollama library. Each row names the smallest common RAM size that holds the file with room to work.

Table
ModelQuantisationFile sizeRAM that suits it
Qwen3 8BQ4_K_M5.2 GB16 GB
Qwen3 14BQ4_K_M9.3 GB16 GB to 32 GB
gpt-oss 20BOllama default14 GB32 GB
Qwen3 32BQ4_K_M20 GB32 GB
Qwen3 32BQ8_035 GB64 GB
Llama 3.3 70BQ4_K_M43 GB64 GB
Llama 3.3 70BQ6_K58 GB96 GB
gpt-oss 120BOllama default65 GB96 GB
Llama 3.3 70BQ8_075 GB128 GB
Llama 3.3 70BFP16141 GBMore than 128 GB
Qwen3 235B-A22B (MoE)Q4_K_M142 GBMore than 128 GB

The right-hand column is our reading of the published sizes plus working room. It is a guide, not a maker specification.

Corsair's own article on DRAM for AI gives a published benchmark of the same limit. It reports that 32 GB and 48 GB could not fit a 70B DeepSeek-R1 distill running CPU-only, and that 32 GB is on the edge even with a GeForce RTX 4090 installed. It recommends 64 GB at minimum and 96 GB for future-proofing.

What changes the number#

Four things move the figure up or down.

Quantisation. Fewer bits per weight shrinks the file. Llama 3.3 70B falls from 141 GB at FP16 to 75 GB at Q8_0, 43 GB at Q4_K_M and 26 GB at Q2_K.

Context length. A longer conversation or a large document needs more cache. Plan extra room if you work with long inputs.

Mixture-of-experts design. Qwen3 30B-A3B is 19 GB at Q4_K_M and Qwen3 235B-A22B is 142 GB. The memory has to hold the whole file, even though only part of it works on each token.

GPU offload. Every layer you place on the graphics card leaves system RAM. If the full model fits in VRAM, system RAM only needs to cover the system. Our guide to how much VRAM an LLM needs covers that side.

What each RAM tier gives you#

The tiers below follow the table. They describe what loads, not how fast it runs.

Table
System RAMWhat it runsLimit
16 GBSmall models at 4-bit, up to about 14BLittle room for other programs
32 GBMid-size models such as gpt-oss 20B and Qwen3 32B at 4-bitCannot hold a 70B model
64 GBA 70B model at Q4_K_MCannot hold a 70B model at 8-bit
96 GBA 70B model at 6-bit and gpt-oss 120BCannot hold FP16 or the largest mixture-of-experts models
128 GBA 70B model at 8-bitQwen3 235B at 4-bit needs 142 GB

Speed is a separate question. A model in system RAM runs at memory bandwidth. Maxim Saplin measured Mistral 7B at Q6_K on a CPU at 9.66 tokens per second with DDR5-4800 and 11.34 with DDR5-6000, in a community test.

Mistral 7B Q6_K (5.94 GB) CPU-only generation, prompt: Tell me about Mars: Token generation speed (tokens/s, higher is better)

Mistral 7B Q6_K (5.94 GB) CPU-only generation, prompt: Tell me about Mars: Token generation speedDDR5-6000: 11.3 tokens/sDDR5-600011.3 tokens/sDDR5-4800: 9.66 tokens/sDDR5-48009.66 tokens/s

2 products. Highest: DDR5-6000, 11.34 tokens/s. Lowest: DDR5-4800, 9.66 tokens/s.

Mistral 7B Q6_K (5.94 GB) CPU-only generation, prompt: Tell me about Mars: Token generation speed
Producttokens/s
DDR5-600011.34
DDR5-48009.66
Source: Maxim Saplin (DEV Community) (opens in a new tab). LM Studio 0.3.4, 12 threads, 2 x 32 GB DIMMs. Intel Core i5-13600KF (overclocked), Windows 11, LM Studio 0.3.4, 12 threads, 100% CPU offload; post dated 12 Oct 2024, edited 17 Feb 2025; community blog post, not a publication lab.

About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.

Bigger models read more data per token, so they run slower. The Mistral file in that test is 5.94 GB and the Llama file is 16.07 GB, which is why the first ran at more than twice the rate of the second. In the same test, Llama 3.1 8B at f16 produced 4.00 and 4.74 tokens per second. A 70B model in RAM will be slower again, and the published data on that is thin. Read DDR5 speed versus capacity for AI for the full picture.

Platform limits and module layout#

Capacity is limited by your processor and board. AMD lists 256 GB as the maximum for the Ryzen 9 9950X, at DDR5-5600 with two DIMMs and a lower DDR5-3600 with four. Intel lists 256 GB and up to DDR5-6400 on two channels for the Core Ultra 9 285K.

The layout matters as well. Saplin's four-stick 96 GB setup could not hold its 6000 profile and was stable only at 4800. At that speed it was slightly slower than two sticks. For capacity, buy two large modules and leave the other slots free.

Channel count has an effect of its own. InsiderLLM measured Gemma 3 4B at Q4_K_M on an older DDR4 mini PC: 5.89 tokens/s on a single stick against 9.15 on a pair. Treat that as a demonstration of the effect.

Gemma 3 4B Q4_K_M, llama.cpp tg128: Token generation speed (tokens/s, higher is better)

Gemma 3 4B Q4_K_M, llama.cpp tg128: Token generation speedDual channel: 9.15 tokens/sDual channel9.15 tokens/sSingle channel: 5.89 tokens/sSingle channel5.89 tokens/s

2 products. Highest: Dual channel, 9.15 tokens/s. Lowest: Single channel, 5.89 tokens/s.

Gemma 3 4B Q4_K_M, llama.cpp tg128: Token generation speed
Producttokens/s
Dual channel9.15
Single channel5.89
Source: InsiderLLM (opens in a new tab). DDR4-2133 SODIMM; llama.cpp b10326 CPU backend, tg128, 4 threads. Intel i7-6700T (4C/8T) Lenovo ThinkCentre M710q, DDR4-2133 SODIMM, llama.cpp b10326 CPU backend, tg128, 4 threads; page dated 11 Aug 2026; self-measured blog on old low-end hardware, shows the channel effect only, not DDR5 desktop values.

About this data: figures are copied from the named source, not measured by us, and the test system and settings are those the source states above. See all benchmark sources and credits.

Before you buy, check the motherboard maker's compatibility list. TeamGroup advises doing so, and it also advises against mixing kits. For 64 GB modules, check that the board maker lists them as supported.

How Much RAM for a Local LLM? Sizes by Model (last checked November 2026).

Reading quantisation names#

Model pages use short codes such as Q4_K_M and Q8_0. The number after the Q is the rough number of bits stored per weight. Fewer bits give a smaller file and a small drop in quality. FP16 keeps the original 16-bit weights and is the largest.

The Ollama library lists one model in many sizes, which shows the effect clearly.

Table
Llama 3.3 70B quantisationFile size
Q2_K26 GB
Q3_K_M34 GB
Q4_K_M43 GB
Q5_K_M50 GB
Q6_K58 GB
Q8_075 GB
FP16141 GB

Many users settle on Q4_K_M or Q5_K_M because they balance size and quality. The table shows what each step costs in memory. Moving from Q4_K_M to Q8_0 adds 32 GB, which is the difference between a 64 GB kit and a 128 GB kit. Whether that gain in precision is worth the extra memory is a choice for your task.

Worked examples#

Three buyer profiles show how the sizing rule plays out.

A student with a mid-range graphics card. The goal is to run 8B and 14B models for coding help and notes. The Qwen3 8B file is 5.2 GB and the 14B file is 9.3 GB in the Ollama library, both at Q4_K_M. Either one fits in the card's memory or in 16 GB of system RAM. A 32 GB kit leaves room for a browser, an editor and a Python environment at the same time. Going beyond that adds cost without opening new models.

A developer who wants one strong local assistant. The goal is a 32B model at a higher precision, or a 70B model at 4-bit. Qwen3 32B at Q8_0 is 35 GB and Llama 3.3 70B at Q4_K_M is 43 GB. Both need 64 GB in practice, and the 70B file is close enough to the limit that it leaves only modest room for context. This buyer gains from 96 GB if long documents are part of the plan.

A researcher who compares large models. The goal is to keep gpt-oss 120B, at 65 GB, and a 70B model loaded together, or to run a 70B model at Q8_0, which is 75 GB. That asks for 128 GB. Corsair recommends 64 GB as a minimum and 96 GB for future-proofing. Even then, the biggest mixture-of-experts files, such as Qwen3 235B at 142 GB, stay out of reach on a desktop.

In all three cases the order is the same. Choose the model, read its size, pick the RAM tier, and only then compare speeds.

How layers and offload change the maths#

Most runtimes divide a model into layers. Each layer holds a slice of the weights, and the runtime decides how many layers go to the graphics card and how many stay in system RAM. A card with enough VRAM takes them all. A card with less takes some, and system RAM takes the rest.

This gives a practical way to size memory. Subtract the VRAM you can use from the model's file size. What remains is roughly what system RAM must hold, plus working room. A 43 GB model on a 16 GB card leaves about 27 GB for system memory, which any 64 GB kit covers with room to spare. The same model on an 8 GB card leaves about 35 GB, which still fits in 64 GB.

The cost is speed. The layers in system RAM set the pace for every token, because the graphics card has to wait for them. Saplin's tests, which were CPU-only, show the ceiling of that pace. A model run from RAM alone produced about 4 to 11 tokens per second in his tests, depending on size and memory speed. That is a reference for small models, not a forecast for large ones.

If speed matters as much as fit, add VRAM before adding RAM. Our guide to VRAM for LLMs sets out the sizes.

A related point comes from Gaessler's llama.cpp notes, which say that about five threads are enough to saturate dual-channel desktop memory. Buying a processor with many more cores will not make a model in system RAM run noticeably faster. Spending on capacity or on a graphics card with more VRAM does more.

Common sizing mistakes#

  • Sizing from the parameter count alone. An 8B model can be 5.2 GB or 16 GB depending on quantisation. Read the file size.
  • Ignoring the context cache. Long chats and long documents need more memory than short ones.
  • Forgetting the rest of the system. The operating system and other programs share the same memory.
  • Filling all four slots for capacity. AMD lists a lower speed for four DIMMs, so two larger modules are usually the better route.
  • Buying for the largest model on the market. The biggest open models need server-class memory. Pick the size for models you will actually run.
  • Assuming faster RAM fixes a small kit. A model that does not fit gains nothing from speed.

Planning for the next model#

The Ollama library already lists the Qwen3 235B model at 250 GB for Q8_0, well beyond any desktop. Two large modules with two free slots leave a path to more capacity, although AMD lists a lower supported speed for four DIMMs on the Ryzen 9 9950X.

What this means when buying#

Start from your largest model, then pick the size from the table. For a 64 GB build, our best 64GB DDR5 RAM kit guide compares three options. For 96 GB and 128 GB, see the guide to large kits for local models. Our overview of RAM for AI work covers all three sizes together. Browse the whole range on the RAM and memory page.

Match the capacity to your largest model first, then check Indian listings and warranty for a kit with the speed your processor supports.

For a dependable 32 GB option, see the best 32GB DDR5 RAM kit for AI and ML guide.

Indian retailers list many kits in India as two-module sets, which suits the dual-channel advice here, so pick the total capacity first.

Frequently asked questions#

How much RAM do I need to run Llama 3.3 70B?

At Q4_K_M the file is 43 GB in the Ollama library, so 64 GB is the sensible minimum. At Q8_0 it is 75 GB, which needs 128 GB.

Is 32 GB enough for a local LLM?

For models up to about 32B at 4-bit, yes. Corsair reports that 32 GB could not fit a 70B model running CPU-only. Choose 64 GB if you want larger models or plan to keep other heavy programs open.

Does a graphics card reduce the RAM I need?

Yes. Layers placed in VRAM do not use system RAM, so a card with more VRAM lowers the amount of system memory the model needs.

Can 128 GB run the biggest open models?

Not the very largest. The Ollama library lists the Qwen3 235B file at 142 GB for Q4_K_M, which is above 128 GB. Smaller models and heavier compression fit, but the biggest open models need more memory than this.

Do I need DDR5 for local LLMs?

DDR5 is standard on current AMD and Intel desktop platforms, and its bandwidth helps CPU inference. Check that your board takes the kit you choose.

Is more RAM or faster RAM better?

More, up to the size your largest model needs. After that, choose the fastest speed your processor lists. Capacity decides which models you can load at all, while speed only changes how fast they respond.

What happens when a model is bigger than my RAM?

Software may swap data to storage, which is very slow. Choose enough memory to hold the file. A model that does not fit can make the machine unresponsive, so match RAM to your biggest planned model.

How do I check a model's size before I download it?

Open the model page in the Ollama library and read the size listed beside each quantisation tag, for example 43 GB for Llama 3.3 70B at Q4_K_M. Add room for the operating system and context, then choose the next RAM size up.

Sources & credits#

We have not tested these products ourselves. Specs come from the makers; benchmark figures come from the sources we name.

Facts and figures in this article come from these sources. Articles are compiled from the makers' published specifications and the benchmarks named above, and checked by the NextAIGrid Editorial Team.

Disclaimer

This explainer is for information only. Specifications change, so check the maker page. Found an error? Tell us.

How was this explainer put together?

It draws on maker specification pages and published third-party benchmarks, named in the article. We have not tested these products ourselves. Specs come from the makers; benchmark figures come from the sources we name. Specifications and sources were last checked on 17 November 2026. Read how we research and our editorial policy.

Spotted an error? Tell us. Confirmed fixes are listed on the corrections page.

What questions do readers ask next?

Back to top