AI Hardware Glossary
The spec terms you will meet in AI hardware guides, in plain words.
Last updated:
What is TOPS?
TOPS means trillions of operations per second. Makers quote it for AI chips and accelerators at a stated number precision, such as INT8, so two TOPS figures compare only when the precision is the same.
Read more: What Is TOPS? AI Chip Performance Explained.
What is VRAM?
VRAM is the video memory on a graphics card. It holds a model, its working data and the context during use, so it decides which AI models fit on the card before speed matters.
Read more: How Much VRAM Do You Need for an LLM?.
What is NPU?
An NPU, or neural processing unit, is a block inside a processor built to run small AI tasks at low power. It speeds up on-device features but does not replace a GPU for large models.
Read more: NPU vs GPU for AI Explained.
What is AVX-512?
AVX-512 is a set of wide CPU instructions that process many numbers at once. Some processors support it, and AI software can use it to speed up CPU-based inference where it is available.
What is Tokens per second?
Tokens per second measures how fast a language model produces text. A token is a word fragment. Figures depend on the model, its quantisation, the context length and the software, so compare only like with like.
Read more: GPU Benchmarks for Local LLMs: Tokens per Second.
What is Quantisation?
Quantisation stores a model’s numbers at lower precision, such as 4-bit instead of 16-bit. The model needs less memory and often runs faster, with some loss of accuracy that grows as precision drops.