Async Survival: Defusing Event Loop Blocking

Table of Contents

1. Challenge

   Introduction to event loop blocking

   Node.js can handle large amounts of concurrent I/O because it does not dedicate one operating-system thread to every request. But that model has a sharp edge: your JavaScript executes on one main thread. If one request handler spends 300 milliseconds in a synchronous CPU loop, it does not only make that one request slow. It also prevents timers, health checks, socket callbacks, and other requests from running.

   In this lab, the expensive work is a similarity scorer. It loops over a batch of candidate vectors and computes scores synchronously. That is a reasonable shape for AI-native backend work: ranking retrieved context, scoring callback payloads, filtering candidates, post-processing model output, or applying local business rules after an LLM response. The specific math is less important than the operational failure mode: CPU-bound JavaScript monopolizes the event loop.

   In a Terminal, start the service:

   npm start
   

   In a second Terminal, run the load driver:

   npm run load
   

   At the beginning of the lab, the numbers will not be very helpful because the metrics functions are still stubs. By the end of the lab, the same load driver will show a clear difference between blocking and chunked execution:

   MODE=blocking npm run load
   MODE=chunked npm run load
   

   Info: If you get stuck at any point, the solutions/ folder contains the completed code for every task.

2. Challenge

   Detect and measure the blocking

   Before you can fix event-loop blocking, you need to prove that it is happening. CPU usage by itself can be misleading: a single blocked Node process may show only one busy core on a large machine, while the service is still unable to answer health checks.

   In this step, you will instrument the loop with Node's perf_hooks APIs and capture a baseline you can compare against later. ### Task 2.1: Measure event loop delay

   monitorEventLoopDelay() creates a histogram that samples how late the event loop is when it wakes up. If synchronous JavaScript monopolizes the thread, the histogram records that delay. The values are stored in nanoseconds, so you will convert them to milliseconds before reporting them.

   In this task, you will build the loop-delay monitor and summarize its readings. ### Task 2.2: Add event loop utilization sampling

   Delay tells you the loop woke up late. Event loop utilization helps corroborate why: it estimates how much time the loop spent active instead of idle.

   For this lab, you will sample utilization as a delta between two points in time so the load driver can report what happened during the recent interval. ### Task 2.3: Capture a baseline snapshot

   Now that the raw metrics work, capture a small "before" snapshot and persist it to outputs/. This gives you a concrete artifact to compare against after the blocking scorer is refactored.

3. Challenge

   Task scheduling and ordering

   Not every asynchronous-looking primitive yields to the event loop in the same way. process.nextTick() and microtasks run before the loop moves on to timers or I/O callbacks. That makes them useful for very small follow-up work, but dangerous for recursive CPU work. A loop that keeps scheduling more next-tick or microtask callbacks can starve the rest of the service while still looking "async" in code review. ### Task 3.1: Verify scheduling order

   You will schedule several callback types from inside an I/O callback and record the order in which they run. Scheduling from an I/O callback is deliberate: setImmediate() vs. setTimeout(0) can be platform-sensitive from top-level code, but the order is deterministic from the poll phase. ### Task 3.2: Fix next-tick starvation

   The file includes runNextTickStarvation(), which recursively schedules CPU work with process.nextTick(). That function is deferred, but it does not let the event loop breathe. You will replace that pattern with bounded chunks and a real loop yield.

4. Challenge

   Defuse the CPU blocking and verify

   The scorer still does all of its CPU work in one synchronous pass. In this step, you will apply the same technique from the previous step to the service workload: process a bounded amount of CPU work, yield to the event loop, and then resume. This approach does not make the CPU work disappear. It trades some throughput for much better responsiveness, which is often the right trade for request-serving code. ### Task 4.1: Chunk the similarity scorer

   The blocking scorer in src/workload.js is intentionally left intact so you can compare behavior. Your job is to implement a chunked equivalent in defuse.js that processes the batch in bounded slices and yields to the event loop between them. ### Task 4.2: Route the endpoint to the chunked scorer

   The HTTP service already passes a mode value into scoreRequest(). In this task, you will make mode=chunked select the yielding implementation. That lets the load driver compare the old and new behavior without changing the service code. ### Task 4.3: Decide when to escalate to a worker thread

   Chunking is a mitigation, not magic. It improves responsiveness by sharing the main thread more fairly, but the CPU work still runs on the main thread. Once a job is large enough, the right answer is to move it to a worker thread or an external worker service. In this task, you will encode a simple decision rule.