The previous implementation used a regex pre-check to determine which files needed deeper inspection, then called vscode.workspace.openTextDocument() on matching files. While this avoided parsing every file, openTextDocument is still an expensive operation - triggering language parsing, syntax highlighting metadata, and complex VS Code infrastructure. For log detection, we only need line-by-line text access.

TurboTextDocument is a minimal wrapper around raw file content that implements only the API surface needed for detectAll operations. It uses synchronous file reads to eliminate async overhead and skips all VS Code document processing.

Result: ~1000x faster file parsing for log detection operations.

The previous implementation used recursive directory traversal with fs.readdir - processing directories one-by-one in sequence. This meant scanning 40,000 files required 40,000 sequential I/O operations.

fast-glob replaces this with parallel glob pattern matching. Instead of manually walking the directory tree, we declare what we want (e.g., **/*.{js,ts,jsx,tsx,php}) and let the highly-optimized glob engine handle discovery - using all available CPU cores.

Result: TypeScript repo (40,000 files) scanned in 403ms instead of multiple seconds.

File filtering uses a two-stage approach that balances performance with accuracy:

This strategy means we never waste time scanning massive node_modules directories, while still respecting project-specific ignore rules.

Once we have the file list, we process them with p-limit set to 16 concurrent operations. This balances throughput with system resources - allowing parallel file reads without overwhelming the disk I/O subsystem.

Each file goes through a fast regex prefilter before running the full AST detection - skipping 97.5% of files that don't contain any log statements. Result: 40,000 files processed in ~4 seconds on M1 MacBook Pro.