LearnViz CompSci
Home / Learn / Computer Architecture & Logic / Memory Hierarchy

Intermediate Topic 25 of 51 Computer Architecture & Logic

Memory Hierarchy

Tiers of storage trading speed for capacity.

How it works

The memory hierarchy arranges storage in layers from small, fast and expensive to large, slow and cheap: registers, L1, L2 and L3 caches, main memory (DRAM), SSDs and hard disks, and network or tape storage. It works because programs exhibit temporal and spatial locality, so most accesses hit the fast upper levels. Each level holds a subset of the one below it.

Step-by-step walkthrough

  1. Registers are fastest and smallest, about one cycle.
  2. L1 cache takes a few cycles.
  3. L2 and L3 hold more but are slower.
  4. Main memory takes hundreds of cycles.
  5. Disk is millions of times slower but huge and persistent.
  6. Locality of reference makes the hierarchy effective.

Where it is used

Processor and operating-system design, database buffer pools, file-system caches and performance tuning all use the hierarchy. Choosing data layouts that favor locality can yield large speedups.

Common pitfall

Ignoring locality, for example striding through a large 2D array by column instead of by row, can make code orders of magnitude slower. Access time differences are huge, roughly a nanosecond for cache and 100 nanoseconds for DRAM. Assuming memory is uniformly fast leads to poor performance. Measure with profilers and cache counters.

Where it fits in the course

Memory Hierarchy is topic 25 of 51 in Computer Architecture & Logic, rated Intermediate. It follows Addressing Modes. The next topic is Cache Memory.

Practice and explore

How to study this topic

  1. Read the explanation above and restate the definition in your own words.
  2. Open the interactive lesson and answer its mini quiz.
  3. Read the common pitfall, then write a short example that avoids it.
  4. Check your understanding in the Computer Architecture & Logic quiz, and revisit the topic if you miss a question.

Frequently asked questions

What is Memory Hierarchy?
Tiers of storage trading speed for capacity. It belongs to Computer Architecture & Logic and is an intermediate-level topic.
Where do people use Memory Hierarchy?
Processor and operating-system design, database buffer pools, file-system caches and performance tuning all use the hierarchy.
What is the most common mistake with Memory Hierarchy?
Ignoring locality, for example striding through a large 2D array by column instead of by row, can make code orders of magnitude slower.
What should I learn before this topic?
Start with Addressing Modes, the topic just before it in Computer Architecture & Logic.
How can I practice it?
Answer the mini quiz in the interactive lesson, then take the Computer Architecture & Logic quiz.
Is Memory Hierarchy hard to learn?
It is an Intermediate topic. Be comfortable with the Beginner topics in this area first, then read the common pitfall carefully, since it is where most mistakes happen.
How long does it take to learn Memory Hierarchy?
Most learners can read the explanation and finish the interactive lesson in under an hour. Fluency comes from applying the idea, so plan a few short practice sessions with the quiz.
Which topics are related to Memory Hierarchy?
Addressing Modes, Cache Memory, Assembly Language are closely connected. The full list is in the related topics section of this page.
What should I study after Memory Hierarchy?
Continue with Cache Memory, the next topic in Computer Architecture & Logic.

Related topics