feat(performance): Add async utility functions and filesystem utilities

- Implemented async utilities including delay, retryWithBackoff, withTimeout, parallelLimit, debounceAsync, AsyncMutex, and CircuitBreaker.
- Created tests for async utilities to ensure functionality and reliability.
- Developed AsyncFileSystem class with methods for file and directory operations, including ensureDir, readFile, writeFile, remove, and more.
- Added tests for filesystem utilities to validate file operations and error handling.
This commit is contained in:
2025-05-31 17:45:40 +00:00
parent 02603c3b07
commit 7b81186bb3
12 changed files with 1437 additions and 292 deletions

View File

@ -342,4 +342,66 @@ const routes: IRouteConfig[] = [{
- Fix the busy wait loop immediately (critical event loop blocker)
- Add proper cleanup for all timers and event listeners
- Consider worker threads for CPU-intensive operations
- See `readme.problems.md` for detailed analysis and recommendations
- See `readme.problems.md` for detailed analysis and recommendations
## Performance Optimizations Implemented (Phase 1 - v19.6.0)
### 1. Async Utilities Created (`ts/core/utils/async-utils.ts`)
- **delay()**: Non-blocking alternative to busy wait loops
- **retryWithBackoff()**: Retry operations with exponential backoff
- **withTimeout()**: Execute operations with timeout protection
- **parallelLimit()**: Run async operations with concurrency control
- **debounceAsync()**: Debounce async functions
- **AsyncMutex**: Ensure exclusive access to resources
- **CircuitBreaker**: Protect against cascading failures
### 2. Filesystem Utilities Created (`ts/core/utils/fs-utils.ts`)
- **AsyncFileSystem**: Complete async filesystem operations
- exists(), ensureDir(), readFile(), writeFile()
- readJSON(), writeJSON() with proper error handling
- copyFile(), moveFile(), removeDir()
- Stream creation and file listing utilities
### 3. Critical Fixes Applied
#### Busy Wait Loop Fixed
- **Location**: `ts/proxies/nftables-proxy/nftables-proxy.ts:235-238`
- **Fix**: Replaced `while (Date.now() < waitUntil) {}` with `await delay(ms)`
- **Impact**: Unblocks event loop, massive performance improvement
#### Certificate Manager Migration
- **File**: `ts/proxies/http-proxy/certificate-manager.ts`
- Added async initialization method
- Kept sync methods for backward compatibility with deprecation warnings
- Added `loadDefaultCertificatesAsync()` method
#### Certificate Store Migration
- **File**: `ts/proxies/smart-proxy/cert-store.ts`
- Replaced all `fileExistsSync`, `ensureDirSync`, `removeManySync`
- Used parallel operations with `Promise.all()` for better performance
- Improved error handling and async JSON operations
#### NFTables Proxy Improvements
- Added deprecation warnings to sync methods
- Created `executeWithTempFile()` helper for common pattern
- Started migration of sync filesystem operations to async
- Added import for delay and AsyncFileSystem utilities
### 4. Backward Compatibility Maintained
- All sync methods retained with deprecation warnings
- Existing APIs unchanged, new async methods added alongside
- Feature flags prepared for gradual rollout
### 5. Phase 1 Completion Status
**Phase 1 COMPLETE** - All critical performance fixes have been implemented:
- Fixed busy wait loop in nftables-proxy.ts
- Created async utilities (delay, retry, timeout, parallelLimit, mutex, circuit breaker)
- Created filesystem utilities (AsyncFileSystem with full async operations)
- Migrated all certificate management to async operations
- Migrated nftables-proxy filesystem operations to async (except stopSync for exit handlers)
- All tests passing for new utilities
### 6. Next Steps (Remaining Phases)
- **Phase 2**: Implement LifecycleComponent for resource management
- **Phase 3**: Add worker threads for CPU-intensive operations
- **Phase 4**: Performance monitoring dashboard