Concurrency vs Parallelism
Concurrency is about structuring work so tasks can overlap; parallelism is about running them at the same instant. Watch one thread juggle three requests, then use Amdahl's law to see how far parallelism can take you.
🎯 What you'll learn
- Concurrency vs parallelism: two different questions
- Event loops and overlapping I/O
- The four combinations
- Amdahl's law and what makes a core 'concurrent'
📚 Before you start
This topic opens the Advanced Engineering stack because almost every pattern that follows — race conditions, deadlocks, resilience patterns — is a consequence of running more than one thing "at once." Before any of that makes sense, the two different meanings of "at once" have to be pulled apart. This is a language-agnostic, general-engineering treatment; JavaScript's specific single-threaded model is covered in languages/javascript/34-concurrency and its accidental-serialization bugs in languages/javascript/33-parallel_vs_sequential — this topic gives the concepts underneath both.
1Two Different Questions
Concurrency and parallelism answer two different questions, and conflating them is the single biggest source of confusion in this area:
- Concurrency asks: is my program structured to make progress on multiple tasks over an overlapping period of time? It is a property of program design.
- Parallelism asks: are multiple tasks physically executing at the exact same instant? It is a property of execution hardware.
A single CPU core with an operating system that rapidly switches between tasks gives you concurrency without parallelism. A GPU running the same instruction across thousands of cores gives you parallelism that barely needs concurrency (the "tasks" don't even coordinate with each other). Most real backend and browser systems need to reason about both at once.
2Concurrency: Structuring for Overlap
A concurrent program is one where multiple tasks are in progress simultaneously, without necessarily executing simultaneously. The correctness of a concurrent design must not depend on which order the interleaved steps happen to run in — if it does, that is a race condition (its own topic, next in this stack).
Task A: [-----wait for DB-----][compute][respond]
Task B: [-----wait for DB-----][compute][respond]
Task C: [-----wait for DB-----][compute][respond]None of A, B, C need a dedicated CPU core to be "in progress" together — most of their time is spent waiting on I/O, during which the CPU is free to advance a different task. This is exactly why a single Node.js process, with one thread, can serve thousands of concurrent HTTP requests: the requests are concurrent (all in flight together) without being parallel (only one line of JS executes at any given instant).
3Parallelism: Physically Simultaneous Execution
Parallelism requires multiple execution units — cores, processors, or machines — each independently making progress at the same instant. It is what actually speeds up CPU-bound work: two cores computing two halves of a matrix multiplication finish in roughly half the wall-clock time of one core doing both halves, because the work is genuinely split across simultaneous execution, not merely interleaved.
Core 0: [======= compute half A =======]
Core 1: [======= compute half B =======]
both finish together, ~half the timeConcurrency without parallelism does not get this speed-up for CPU-bound work — interleaving two CPU-heavy loops on one core takes the same total core-time as running them back to back, just reshuffled (and slightly worse, due to context-switch overhead — see Under the Hood).
4The Four Combinations
| Not concurrent | Concurrent | |
|---|---|---|
| Not parallel | One task, run to completion, single core. | One core, many tasks interleaved (async I/O on a single thread). |
| Parallel | Two cores, each running one long independent task that never interacts (rare in practice). | Many tasks, many cores, a scheduler distributing interleaved work across all of them (a real multi-threaded server). |
Most production backend systems live in the bottom-right cell: concurrent in design (many in-flight requests, connections, timers) and parallel in execution (spread across multiple cores or machines) — which is exactly why race conditions and deadlocks matter: true parallel execution means two threads can genuinely be reading and writing the same memory at the identical instant, not just in some interleaved order.
5Where the Distinction Changes What You Should Do
Problem is I/O-bound (waiting on network, disk, timers)
→ concurrency helps enormously — overlap the waiting
→ parallelism is not required; extra cores don't speed up waiting
Problem is CPU-bound (heavy computation, no waiting)
→ concurrency alone does nothing — there's no idle time to fill
→ needs real parallelism (more cores/threads/processes) to go fasterReaching for async/await-style concurrency to speed up a CPU-bound loop is a common and costly mistake (detailed in the JavaScript Concurrency topic) — it reduces to the exact same single-core throughput, just with scheduling overhead added on top.
6Amdahl's Law — The Limit of Parallelizing
Not every program benefits equally from more cores. Amdahl's Law quantifies the ceiling: if a fraction P of a program's work can be parallelized and the rest, (1 - P), is inherently serial, the maximum possible speed-up with N processors is:
speedup(N) = 1 / ((1 - P) + P / N)P = 0.90 (90% parallelizable), N = 4 → speedup = 1 / (0.10 + 0.90/4) = 1 / 0.325 ≈ 3.08x
P = 0.90, N = ∞ → speedup → 1 / 0.10 = 10x MAXIMUM, ever, regardless of cores
P = 0.50 (only half parallelizable), N = ∞ → speedup → 2x MAXIMUMThis is why "just add more cores" has diminishing, then negligible, returns: the serial portion (setup, coordination, merging results, contention on shared state) caps the achievable speed-up no matter how much hardware you throw at it. It is the mathematical justification behind why reducing the serial/coordination portion of an algorithm often matters more than adding parallelism.
Under the HoodWhat Actually Makes One Core "Concurrent"▾
The OS scheduler and time-slicing
On a single core, the illusion of multiple threads running "together" is produced by the operating system's scheduler, which gives each runnable thread a small slice of CPU time (a quantum, typically single-digit to low double-digit milliseconds) before forcibly suspending it and running another. This is preemptive multitasking — the OS decides when a thread's turn ends, the thread itself does not choose to yield (contrast with JavaScript's single-threaded event loop, which is cooperative: code voluntarily yields control back at await points, never preempted mid-expression — see languages/javascript/27-event_loop).
Context switches are not free
Every time the scheduler swaps which thread is running on a core, it performs a context switch: saving the outgoing thread's full register state, program counter, and stack pointer, then loading the incoming thread's saved state. This costs real, measurable time — typically on the order of low microseconds — plus a secondary cost from cache invalidation: the outgoing thread's hot data in the CPU's L1/L2 cache gets evicted by the incoming thread's data, so the incoming thread runs slower for a while until its own working set repopulates the cache. This is precisely why creating far more threads than you have cores can make a CPU-bound workload slower, not faster — the overhead of constant switching and cache-thrashing outweighs any benefit, since none of it is genuine simultaneous execution anyway.
Hyper-threading is not a second core
A hyper-threaded (SMT) core exposes two logical threads per physical core by duplicating some internal state (registers) while sharing the core's actual execution units (ALUs, cache). This gives the OS two schedulable threads, and can improve throughput when one logical thread stalls waiting on memory and the other has independent work ready — but it is not two independent execution pipelines the way two physical cores are; CPU-bound work on both logical threads of the same core contends for the same underlying execution resources and does not scale the way true additional cores do.
Where parallelism actually comes from
True parallelism requires the hardware to have multiple genuinely independent instruction pipelines: multiple physical cores, multiple CPU sockets, a GPU's many execution units, or multiple separate machines coordinated over a network. The runtime's job (a language's thread pool, a Worker Threads implementation, a distributed job scheduler) is to map the concurrent, in-progress units of work your program describes onto however many of those independent pipelines actually exist — which is why "how many cores does this machine have" is a real, load-bearing question for CPU-bound work, and mostly irrelevant for I/O-bound work.
⚠️ Common Pitfalls
- Using "concurrent" and "parallel" interchangeably — they answer different questions (design vs. execution); a correct concurrent design can run on one core with zero parallelism.
- Expecting concurrency to speed up CPU-bound work — interleaving heavy computation on one core does not reduce total core-time; only real parallelism (more cores) does.
- Assuming more threads always means more speed — beyond the number of available cores, extra threads mostly add context-switch and cache-thrashing overhead for CPU-bound work, not more throughput.
- Ignoring Amdahl's Law when planning to "just parallelize it" — a workload with a large serial/coordination fraction hits a low speed-up ceiling no matter how many cores are added; profile first (Performance Optimization topic) to find out how parallelizable the work actually is.
- Forgetting that parallel execution reopens correctness problems concurrency alone can hide — code that is merely concurrent on one core never has two operations touch memory at the literal same instant; genuinely parallel execution can, which is precisely the precondition for race conditions (next topic).
📋 Quick Reference
Concurrency = design property — tasks IN PROGRESS over overlapping time (1+ cores)
Parallelism = execution property — tasks running at the EXACT same instant (2+ cores required)
I/O-bound work → concurrency helps a lot, parallelism optional
CPU-bound work → concurrency alone does nothing, needs real parallelism
speedup(N cores) = 1 / ((1 - P) + P/N) — Amdahl's Law; serial fraction caps max speedup
Context switch: real cost (register save/restore + cache eviction), not free
Hyper-threading: 2 logical threads, shared execution units — not equivalent to 2 physical cores🧠 Key takeaways
- Concurrency: structuring overlapping tasks (event loops, async). Parallelism: simultaneous execution on multiple cores.
- Event loops give great concurrency for I/O-bound work on one thread; CPU-bound work needs parallelism.
- You can have either, both or neither; choose based on whether you wait on I/O or compute.
- Amdahl's law: the serial fraction caps speedup (90% parallel → max 10×).
🎤 Interview Q&A
Q1What's the difference between concurrency and parallelism?core▾
Concurrency is structuring a program to deal with many tasks that overlap in time (interleaving, e.g. handling thousands of connections on one thread). Parallelism is executing multiple tasks at the same instant on multiple cores. Concurrency is about structure; parallelism is about execution.
- Can you have one without the other? Yes: Node.js is concurrent on one thread; a parallel matrix multiply needn't be 'concurrent' in design.
Q2I/O-bound vs CPU-bound work: how does it change your approach?core▾
I/O-bound work benefits from async/non-blocking concurrency (event loops, async/await) because threads would mostly wait. CPU-bound work needs parallelism across cores (thread pools, processes, worker threads); async alone won't speed it up.
- Python example? asyncio for I/O; multiprocessing for CPU because of the GIL.
Q3Threads vs async/await vs processes?senior▾
Processes: isolated memory, safest, expensive to create and communicate. Threads: shared memory, cheaper, need synchronization. Async/await: cooperative concurrency on few threads, very scalable for I/O, but blocking calls stall everything.
- What's a green thread / virtual thread? A lightweight thread scheduled by the runtime (Go goroutines, Java virtual threads).
Q4What does Amdahl's law tell you?core▾
The speedup from parallelism is limited by the serial part: with a fraction p parallelizable, max speedup = 1 / ((1 − p) + p/N). If 10% is serial, you can never exceed 10x no matter how many cores.
- Practical lesson? Reduce the serial part (locks, coordination) before adding cores.
Q5What are common concurrency hazards?core▾
Race conditions, deadlocks, livelocks, starvation, visibility issues (stale CPU caches without memory barriers), and thread-pool exhaustion from blocking calls.
- How do you avoid most of them? Prefer immutability, message passing, and higher-level concurrency primitives.
🛠️ Practice exercises
Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.
✅ Check yourself
Answer the questions — feedback is instant.