How Node.js works: one thread, thousands of connections

Node.js runs JavaScript on the server, using the same single-threaded language from the browser. And yet one Node process can comfortably serve thousands of users at once. How does one thread keep up with that many connections, and why can a single bad request make every other user wait?

This post covers what Node is made of, how it handles I/O without blocking, how its event loop differs from the browser's, and the one rule that keeps a Node server fast. There's a demo where you can block a server yourself.

What Node actually is

Node isn't a language or a framework. It's a runtime: a program that runs JavaScript and gives it access to the outside world. It's built from a few parts:

• V8, the same JavaScript engine as Chrome. It parses, compiles and runs your code, exactly as described in How JavaScript works.
• libuv, a C library that provides the event loop, asynchronous networking and file access, and a small thread pool.
• Bindings, C++ glue that exposes those features to JavaScript.
• The standard library (fs, http, crypto, stream and friends), mostly written in JavaScript on top of the bindings.

The browser gives JavaScript the DOM and fetch. Node gives it files, sockets, processes and the operating system instead. The language and the engine are the same.

The big idea: don't wait, get called back

A traditional server might give every connection its own thread. While that thread waits for the database, it sits idle but still costs memory. With ten thousand connections you need ten thousand threads.

Node does the opposite. Your JavaScript runs on one main thread, and it never sits waiting for I/O. When you query a database or read a file, Node hands the job to the operating system or to libuv and moves straight on to the next piece of work. When the result is ready, your callback (or the code after await) is queued, and the event loop runs it.

import { createServer } from "node:http";
 
createServer(async (req, res) => {
  const user = await db.findUser(req.url); // the thread is free while the DB works
  res.end(JSON.stringify(user));            // runs later, when the result is back
}).listen(3000);

While one request waits on the database, the same thread is parsing the next request, sending another response and running a timer. Most web servers spend most of their time waiting, so one thread that never waits goes a long way.

Where the waiting actually happens

"Non-blocking" doesn't mean the work disappears. Someone still has to do it. libuv sends it down one of two completely different paths, and mixing them up is one of the most common misunderstandings about Node.

Path 1: networking goes to the operating system

Sockets (incoming HTTP requests, fetch, database and Redis connections) are handled by the operating system kernel, not by extra threads. Every major OS lets a single thread register thousands of sockets and later ask "which of these have data ready?":

• epoll on Linux
• kqueue on macOS
• IOCP on Windows

So with 10,000 users connected, Node doesn't use 10,000 threads, or even 4. The main thread tells the kernel which sockets to watch, then in the event loop's poll phase it asks which ones are ready and runs their callbacks. Nothing sits blocked waiting on a socket. Networking never uses the thread pool, and that's why one Node process can hold so many connections cheaply.

Path 2: the thread pool, for work the OS can't do asynchronously

Some operations have no good "tell me when you're done" interface in the operating system, or they're just heavy CPU work. libuv runs those on its thread pool, which has 4 threads by default:

A pool thread does the blocking call or the CPU work. When it's finished, it queues the result, and the main thread runs your callback (or continues your await) on a later loop iteration. Your JavaScript itself never runs on a pool thread.

              your JavaScript: one main thread running the event loop
                 │                                   │
   sockets       │ "watch these"                     │  fs, dns.lookup, crypto, zlib
                 ▼                                   ▼
  OS kernel (epoll / kqueue / IOCP)        libuv thread pool (4 threads)
                 │                                   │
                 └────────────── "done" ─────────────┘
                                   ▼
                  callback queued, then run on the main thread

Why "only 4" matters

All of those operations share the same 4 threads. Start five password hashes at once:

const crypto = require("node:crypto");
const start = Date.now();
 
for (let i = 1; i <= 5; i++) {
  crypto.pbkdf2("secret", "salt", 100_000, 64, "sha512", () => {
    console.log(`hash ${i}: ${Date.now() - start} ms`);
  });
}
 
// Example output on a machine with 4+ cores (exact times vary):
// hash 1: 98 ms
// hash 2: 99 ms
// hash 3: 101 ms
// hash 4: 102 ms
// hash 5: 197 ms   ← waited for a free thread

The first four run in parallel, one per thread. The fifth has to queue until one of them finishes, so it takes about twice as long. On a real server, the side effect is easy to miss: a burst of logins can slow down unrelated file reads and DNS lookups, because they're queuing for the same threads. Ordinary HTTP traffic keeps flowing, because sockets don't use the pool. Try it:

What you can do about it:

• Raise the pool size with the UV_THREADPOOL_SIZE environment variable, for example UV_THREADPOOL_SIZE=8 node server.js. Set it before Node starts. For CPU work like hashing, it only helps up to about your number of CPU cores.
• Never use the *Sync versions (pbkdf2Sync, readFileSync, scryptSync) in request handlers. They skip the pool and run on the main thread, which blocks every user instead of just the queue.
• Cut down on DNS lookups. Reuse connections with keep-alive (Node's HTTP client does this by default since Node 19), so you don't do a DNS lookup and new connection for every request.
• Move sustained CPU work out of the process. Worker threads or a separate job queue keep heavy work away from both the main thread and the shared pool.

Node's event loop has phases

The browser's event loop has tasks, microtasks and rendering. Node's has no rendering, but it splits tasks into phases that it cycles through in order:

1. Timers. Callbacks from setTimeout and setInterval whose time is up.
2. Pending callbacks. Some system-level callbacks deferred from the previous loop.
3. Poll. Fetch new I/O events and run their callbacks: incoming requests, finished reads, database responses. If there's nothing else to do, the loop waits here.
4. Check. Callbacks from setImmediate, which always run right after the poll phase.
5. Close callbacks. For example a socket's "close" event.

Between every callback, Node empties two queues first: the process.nextTick queue, then the promise microtask queue. So nextTick runs before promises, and both run before the loop moves on:

setTimeout(() => console.log("timeout"), 0);
setImmediate(() => console.log("immediate"));
process.nextTick(() => console.log("nextTick"));
Promise.resolve().then(() => console.log("promise"));
console.log("sync");
 
// sync, nextTick, promise, then timeout and immediate

Those last two can come out in either order when called from the main module, because it depends on whether the 1 ms timer has expired by the time the loop starts. Inside an I/O callback, setImmediate always runs first, because the check phase comes straight after poll.

The one rule: don't block the main thread

Everything above depends on your JavaScript finishing quickly. While a callback runs, nothing else can: no new requests are accepted, no responses are sent, no timers fire. In the browser, a long task freezes one user's page. On a server, it freezes every user at once.

Try it. Five users hit the same server. One requests a heavy report, and the other four want something quick:

When the heavy work is async I/O, the other users barely notice, because the main thread is free while it waits. When the same amount of work is synchronous JavaScript, everyone queues behind it. Moving that CPU work into a worker thread gives the same result as I/O: the main thread stays free.

Common ways to block a Node server by accident:

• *Sync functions in request handlers: readFileSync, execSync, crypto.pbkdf2Sync. They're fine at startup, never per request.
• Huge JSON.parse / JSON.stringify calls on multi-megabyte payloads.
• CPU-heavy loops: image processing, big sorts, report generation, compression in JavaScript.
• Regular expressions with catastrophic backtracking, where a crafted input makes a regex take seconds. That's a real denial-of-service vector called ReDoS.

Using more than one core

One Node process runs your JavaScript on one CPU core. To use the rest:

• Worker threads (node:worker_threads) run JavaScript in parallel inside the same process. Each has its own V8 instance and event loop, and they talk by passing messages. Use them for CPU-heavy jobs.
• Multiple processes, one per core, behind a load balancer. That's what the cluster module, PM2, or simply running several containers does. Each process is independent, so shared state belongs in a database or Redis, not in memory. It's the same idea as scaling out in What is system design?.

Streams: handling data bigger than memory

Reading a 2 GB file into a string will crash a server or bring it to a crawl. Streams process data in chunks as it arrives:

import { createReadStream } from "node:fs";
import { createGzip } from "node:zlib";
import { pipeline } from "node:stream/promises";
 
// Compress a huge log file and send it, a chunk at a time, with constant memory use
await pipeline(createReadStream("app.log"), createGzip(), res);

pipeline also handles backpressure: if the client downloads slowly, it pauses reading from disk instead of buffering the whole file in memory. It also cleans up every stream if one of them fails.

Why this matters in practice

• Keep every callback short. If a request needs heavy CPU work, give it to a worker thread or a job queue, and keep the main thread for coordinating.
• Know which calls use the thread pool. File access, dns.lookup, compression and crypto all share those 4 threads.
• Measure event loop delay. monitorEventLoopDelay() from node:perf_hooks tells you how late the loop is running. If it climbs under load, something is blocking.
• Scale out with processes, not bigger machines. One process per core, with state kept outside the process.
• Stream anything large instead of loading it all at once.

If you're building that server in TypeScript (and most Node backends now are), How TypeScript works explains what the compiler checks, and what it can't check once your code is running.