Batch require cache deletion to avoid quadratic scanning

[lukesandberg]

What?

Rewrites deleteFromRequireCache() in packages/next/src/server/dev/require-cache.ts to accept an array of file paths and perform a single scan of require.cache, instead of one full scan per file. Adds a deleteCacheBatch() export and updates all three batch call sites to use it.

Why?

During server-side HMR, deleteCache() is called in loops — once per server path in the turbopack hot-reloader (5-20 files), up to 15 times per writeManifests() in the manifest-loader, and 2 + N times (runtime chunks + page entries) in the webpack plugin's afterEmit hook. Each call scans the entire require.cache to clean up parent-child references, making the overall cost O(N × C) where C is the cache size. In large apps with thousands of cached modules, this becomes a meaningful bottleneck on every HMR update.

How?

Core change (require-cache.ts): deleteFromRequireCache now accepts string[]. It resolves all paths upfront, collects target modules into a Set<NodeModule>, then does one scan of require.cache using Set.has() (O(1) lookup) to filter children arrays. For single-item callers (deleteCache), the overhead of a 1-element Set is negligible.

Call site updates:

• hot-reloader-turbopack.ts: Collects files to delete into an array during the serverPaths loop, calls deleteCacheBatch() once after. clearModuleContext stays per-file (separate system).
• manifest-loader.ts: Adds pendingCacheDeletes array to TurbopackManifestLoader. All ~15 deleteCache() calls in write* methods become push() calls. Flushed at the end of writeManifests() with a single deleteCacheBatch(). Moving cache deletion to after all writeFileAtomic calls is safe (synchronous code, no interleaving) and slightly better (new files on disk before cache is cleared).
• nextjs-require-cache-hot-reloader.ts (webpack): Batches the afterEmit hook — collects runtime chunk + page entry paths, calls deleteCacheBatch() once. The assetEmitted tap callback stays as individual deleteCache().

Theoretical improvement:

Call Site	Before (cache scans)	After	Improvement
turbopack clearRequireCache	N (5-20)	1	5-20× fewer scans
manifest-loader writeManifests	up to 15	1	up to 15× fewer scans
webpack afterEmit	2 + page count	1	up to 50× fewer scans

Benchmark

Tested with a generated 250-route app with 50 API route handlers importing server external packages (zod, lodash, express, pg, ioredis) plus middleware. API route handlers are key because unlike bundled app-router pages, they create real require.cache depth with native Node.js module loading.

Results (steady-state, 18 paths / 7 found / 781 nodes / ~1800 edges):

Mode	Time per batch
Batched (1 scan)	~0.24ms
Unbatched (18 individual scans)	~0.70ms
Speedup	~3×

Raw data (representative samples):

# Batched (1 scan of require.cache)
[PERF] deleteCacheBatch(batch=true): 18 paths (7 found), 781 nodes, 1772 edges, 0.283ms
[PERF] deleteCacheBatch(batch=true): 18 paths (7 found), 781 nodes, 1800 edges, 0.250ms
[PERF] deleteCacheBatch(batch=true): 18 paths (7 found), 781 nodes, 1912 edges, 0.283ms

# Unbatched (18 individual scans)
[PERF] deleteCacheBatch(batch=false): 18 paths (7 found), 781 nodes, 1772 edges, 0.671ms
[PERF] deleteCacheBatch(batch=false): 18 paths (7 found), 781 nodes, 1800 edges, 0.649ms
[PERF] deleteCacheBatch(batch=false): 18 paths (7 found), 781 nodes, 1912 edges, 0.716ms

The speedup scales with cache complexity — in production apps with more server externals (ORMs, validation libs, etc.) the cache would be larger and the improvement proportionally greater. With only bundled app-router pages (few edges in require.cache graph), the difference is negligible since most of the graph complexity comes from native Node.js module loading.