feat(proxy): Start implementing PROXY protocol support and WrappedSocket class for enhanced client IP handling

2025-06-01 21:30:37 +00:00 · 2025-06-01 21:30:37 +00:00 · 01e1153fb8
commit 01e1153fb8
parent fa9166be4b
6 changed files with 1000 additions and 248 deletions
--- a/readme.hints.md
+++ b/readme.hints.md
@ -661,4 +661,145 @@ Updated all forwarding handlers to use the new centralized socket utilities:
 - Reduced code duplication
 ### Migration Notes
-No user-facing changes. All forwarding handlers now use the same robust socket handling as the main SmartProxy connection handler.
+No user-facing changes. All forwarding handlers now use the same robust socket handling as the main SmartProxy connection handler.
 ## WrappedSocket Class Evaluation for PROXY Protocol (v19.5.19+)
 ### Current Socket Handling Architecture
 - Sockets are handled directly as `net.Socket` instances throughout the codebase
 - Socket augmentation via TypeScript module augmentation for TLS properties
 - Metadata tracked separately in `IConnectionRecord` objects
 - Socket utilities provide helper functions but don't encapsulate the socket
 - Connection records track extensive metadata (IDs, timestamps, byte counters, TLS state, etc.)
 ### Evaluation: Should We Introduce a WrappedSocket Class?
 **Yes, a WrappedSocket class would make sense**, particularly for PROXY protocol implementation and future extensibility.
 ### Design Considerations for WrappedSocket
 ```typescript
 class WrappedSocket {
  private socket: net.Socket;
  private connectionId: string;
  private metadata: {
    realClientIP?: string;     // From PROXY protocol
    realClientPort?: number;   // From PROXY protocol
    proxyIP?: string;          // Immediate connection IP
    proxyPort?: number;        // Immediate connection port
    bytesReceived: number;
    bytesSent: number;
    lastActivity: number;
    isTLS: boolean;
    // ... other metadata
  };
  // PROXY protocol handling
  private proxyProtocolParsed: boolean = false;
  private pendingData: Buffer[] = [];
  constructor(socket: net.Socket) {
    this.socket = socket;
    this.setupHandlers();
  }
  // Getters for clean access
  get remoteAddress(): string {
    return this.metadata.realClientIP || this.socket.remoteAddress || '';
  }
  get remotePort(): number {
    return this.metadata.realClientPort || this.socket.remotePort || 0;
  }
  get isFromTrustedProxy(): boolean {
    return !!this.metadata.realClientIP;
  }
  // PROXY protocol parsing
  async parseProxyProtocol(trustedProxies: string[]): Promise<boolean> {
    // Implementation here
  }
  // Delegate socket methods
  write(data: any): boolean {
    this.metadata.bytesSent += Buffer.byteLength(data);
    return this.socket.write(data);
  }
  destroy(error?: Error): void {
    this.socket.destroy(error);
  }
  // Event forwarding
  on(event: string, listener: Function): this {
    this.socket.on(event, listener);
    return this;
  }
 }
 ```
 ### Implementation Benefits
 1. **Encapsulation**: Bundle socket + metadata + behavior in one place
 2. **PROXY Protocol Integration**: Cleaner handling without modifying existing socket code
 3. **State Management**: Centralized socket state tracking and validation
 4. **API Consistency**: Uniform interface for all socket operations
 5. **Future Extensibility**: Easy to add new socket-level features (compression, encryption, etc.)
 6. **Type Safety**: Better TypeScript support without module augmentation
 7. **Testing**: Easier to mock and test socket behavior
 ### Implementation Drawbacks
 1. **Major Refactoring**: Would require changes throughout the codebase
 2. **Performance Overhead**: Additional abstraction layer (minimal but present)
 3. **Compatibility**: Need to maintain event emitter compatibility
 4. **Learning Curve**: Developers need to understand the wrapper
 ### Recommended Approach: Phased Implementation
 **Phase 1: PROXY Protocol Only** (Immediate)
 - Create minimal `ProxyProtocolSocket` wrapper for new connections from trusted proxies
 - Use in connection handler when receiving from trusted proxy IPs
 - Minimal disruption to existing code
 ```typescript
 class ProxyProtocolSocket {
  constructor(
    public socket: net.Socket,
    public realClientIP?: string,
    public realClientPort?: number
  ) {}
  get remoteAddress(): string {
    return this.realClientIP || this.socket.remoteAddress || '';
  }
  get remotePort(): number {
    return this.realClientPort || this.socket.remotePort || 0;
  }
 }
 ```
 **Phase 2: Gradual Migration** (Future)
 - Extend wrapper with more functionality
 - Migrate critical paths to use wrapper
 - Add performance monitoring
 **Phase 3: Full Adoption** (Long-term)
 - Complete migration to WrappedSocket
 - Remove socket augmentation
 - Standardize all socket handling
 ### Decision Summary
 ✅ **Implement minimal ProxyProtocolSocket for immediate PROXY protocol support**
 - Low risk, high value
 - Solves the immediate proxy chain connection limit issue
 - Sets foundation for future improvements
 - Can be implemented alongside existing code
 📋 **Consider full WrappedSocket for future major version**
 - Cleaner architecture
 - Better maintainability
 - But requires significant refactoring
--- a/readme.plan.md
+++ b/readme.plan.md
@ -0,0 +1,517 @@
 # PROXY Protocol Implementation Plan
 ## Overview
 Implement PROXY protocol support in SmartProxy to preserve client IP information through proxy chains, solving the connection limit accumulation issue where inner proxies see all connections as coming from the outer proxy's IP.
 ## Problem Statement
 - In proxy chains, the inner proxy sees all connections from the outer proxy's IP
 - This causes the inner proxy to hit per-IP connection limits (default: 100)
 - Results in connection rejections while outer proxy accumulates connections
 ## Solution Design
 ### 1. Core Features
 #### 1.1 PROXY Protocol Parsing
 - Support PROXY protocol v1 (text format) initially
 - Parse incoming PROXY headers to extract:
  - Real client IP address
  - Real client port
  - Proxy IP address
  - Proxy port
  - Protocol (TCP4/TCP6)
 #### 1.2 PROXY Protocol Generation
 - Add ability to send PROXY protocol headers when forwarding connections
 - Configurable per route or target
 #### 1.3 Trusted Proxy IPs
 - New `proxyIPs` array in SmartProxy options
 - Auto-enable PROXY protocol acceptance for connections from these IPs
 - Reject PROXY protocol from untrusted sources (security)
 ### 2. Configuration Schema
 ```typescript
 interface ISmartProxyOptions {
  // ... existing options
  // List of trusted proxy IPs that can send PROXY protocol
  proxyIPs?: string[];
  // Global option to accept PROXY protocol (defaults based on proxyIPs)
  acceptProxyProtocol?: boolean;
  // Global option to send PROXY protocol to all targets
  sendProxyProtocol?: boolean;
 }
 interface IRouteAction {
  // ... existing options
  // Send PROXY protocol to this specific target
  sendProxyProtocol?: boolean;
 }
 ```
 ### 3. Implementation Steps
 #### Phase 1: WrappedSocket Foundation
 1. Create minimal `ProxyProtocolSocket` class in `ts/core/models/proxy-protocol-socket.ts`
 2. Implement basic socket wrapper with real IP getters
 3. Update `ConnectionManager` to accept wrapped sockets
 4. Create tests for wrapped socket functionality
 #### Phase 2: PROXY Protocol Parser
 1. Create `ProxyProtocolParser` class in `ts/core/utils/proxy-protocol.ts`
 2. Implement v1 text format parsing
 3. Add validation and error handling
 4. Integrate parser into ProxyProtocolSocket
 #### Phase 3: Connection Handler Integration
 1. Modify `RouteConnectionHandler` to use ProxyProtocolSocket for trusted IPs
 2. Parse PROXY header before TLS detection
 3. Use wrapped socket throughout connection lifecycle
 4. Strip PROXY header before forwarding data
 #### Phase 4: Outbound PROXY Protocol
 1. Add PROXY header generation in `setupDirectConnection`
 2. Make it configurable per route
 3. Send header immediately after TCP connection
 4. Use ProxyProtocolSocket for outbound connections too
 #### Phase 5: Security & Validation
 1. Validate PROXY headers strictly
 2. Reject malformed headers
 3. Only accept from trusted IPs
 4. Add rate limiting for PROXY protocol parsing
 ### 4. Design Decision: Socket Wrapper Architecture
 #### Option A: Minimal Single Socket Wrapper
 - **Scope**: Wraps individual sockets with metadata
 - **Use Case**: PROXY protocol support with minimal refactoring
 - **Pros**: Simple, low risk, easy migration
 - **Cons**: Still need separate connection management
 #### Option B: Comprehensive Connection Wrapper
 - **Scope**: Manages socket pairs (incoming + outgoing) with all utilities
 - **Use Case**: Complete connection lifecycle management
 - **Pros**: 
  - Encapsulates all socket utilities (forwarding, cleanup, backpressure)
  - Single object represents entire connection
  - Cleaner API for connection handling
 - **Cons**: 
  - Major architectural change
  - Higher implementation risk
  - More complex migration
 #### Recommendation
 Start with **Option A** (ProxyProtocolSocket) for immediate PROXY protocol support, then evaluate Option B based on:
 - Performance impact of additional abstraction
 - Code simplification benefits
 - Team comfort with architectural change
 ### 5. Code Changes (Using Option A Initially)
 #### 5.1 ProxyProtocolSocket (new file)
 ```typescript
 // ts/core/models/proxy-protocol-socket.ts
 export class ProxyProtocolSocket {
  constructor(
    public readonly socket: plugins.net.Socket,
    private realClientIP?: string,
    private realClientPort?: number
  ) {}
  get remoteAddress(): string {
    return this.realClientIP || this.socket.remoteAddress || '';
  }
  get remotePort(): number {
    return this.realClientPort || this.socket.remotePort || 0;
  }
  get isFromTrustedProxy(): boolean {
    return !!this.realClientIP;
  }
  setProxyInfo(ip: string, port: number): void {
    this.realClientIP = ip;
    this.realClientPort = port;
  }
 }
 ```
 #### 4.2 ProxyProtocolParser (new file)
 ```typescript
 // ts/core/utils/proxy-protocol.ts
 export class ProxyProtocolParser {
  static readonly PROXY_V1_SIGNATURE = 'PROXY ';
  static parse(chunk: Buffer): IProxyInfo | null {
    // Implementation
  }
  static generate(info: IProxyInfo): Buffer {
    // Implementation
  }
 }
 ```
 #### 4.3 Connection Handler Updates
 ```typescript
 // In handleConnection method
 let wrappedSocket: ProxyProtocolSocket | plugins.net.Socket = socket;
 // Wrap socket if from trusted proxy
 if (this.settings.proxyIPs?.includes(socket.remoteAddress)) {
  wrappedSocket = new ProxyProtocolSocket(socket);
 }
 // Create connection record with wrapped socket
 const record = this.connectionManager.createConnection(wrappedSocket);
 // In handleInitialData method
 if (wrappedSocket instanceof ProxyProtocolSocket) {
  const proxyInfo = await this.checkForProxyProtocol(chunk);
  if (proxyInfo) {
    wrappedSocket.setProxyInfo(proxyInfo.sourceIP, proxyInfo.sourcePort);
    // Continue with remaining data after PROXY header
  }
 }
 ```
 #### 4.4 Security Manager Updates
 - Accept socket or ProxyProtocolSocket
 - Use `socket.remoteAddress` getter for real client IP
 - Transparent handling of both socket types
 ### 5. Configuration Examples
 #### Basic Setup
 ```typescript
 // Outer proxy - sends PROXY protocol
 const outerProxy = new SmartProxy({
  ports: [443],
  routes: [{
    name: 'to-inner-proxy',
    match: { ports: 443 },
    action: {
      type: 'forward',
      target: { host: '195.201.98.232', port: 443 },
      sendProxyProtocol: true  // Enable for this route
    }
  }]
 });
 // Inner proxy - accepts PROXY protocol from outer proxy
 const innerProxy = new SmartProxy({
  ports: [443],
  proxyIPs: ['212.95.99.130'],  // Outer proxy IP
  // acceptProxyProtocol: true is automatic for proxyIPs
  routes: [{
    name: 'to-backend',
    match: { ports: 443 },
    action: {
      type: 'forward',
      target: { host: '192.168.5.247', port: 443 }
    }
  }]
 });
 ```
 ### 6. Testing Plan
 #### Unit Tests
 - PROXY protocol v1 parsing (valid/invalid formats)
 - Header generation
 - Trusted IP validation
 - Connection record updates
 #### Integration Tests
 - Single proxy with PROXY protocol
 - Proxy chain with PROXY protocol
 - Security: reject from untrusted IPs
 - Performance: minimal overhead
 - Compatibility: works with TLS passthrough
 #### Test Scenarios
 1. **Connection limit test**: Verify inner proxy sees real client IPs
 2. **Security test**: Ensure PROXY protocol rejected from untrusted sources
 3. **Compatibility test**: Verify no impact on non-PROXY connections
 4. **Performance test**: Measure overhead of PROXY protocol parsing
 ### 7. Security Considerations
 1. **IP Spoofing Prevention**
   - Only accept PROXY protocol from explicitly trusted IPs
   - Validate all header fields
   - Reject malformed headers immediately
 2. **Resource Protection**
   - Limit PROXY header size (107 bytes for v1)
   - Timeout for incomplete headers
   - Rate limit connection attempts
 3. **Logging**
   - Log all PROXY protocol acceptance/rejection
   - Include real client IP in all connection logs
 ### 8. Rollout Strategy
 1. **Phase 1**: Deploy parser and acceptance (backward compatible)
 2. **Phase 2**: Enable between controlled proxy pairs
 3. **Phase 3**: Monitor for issues and performance impact
 4. **Phase 4**: Expand to all proxy chains
 ### 9. Success Metrics
 - Inner proxy connection distribution matches outer proxy
 - No more connection limit rejections in proxy chains
 - Accurate client IP logging throughout the chain
 - No performance degradation (<1ms added latency)
 ### 10. Future Enhancements
 - PROXY protocol v2 (binary format) support
 - TLV extensions for additional metadata
 - AWS VPC endpoint ID support
 - Custom metadata fields
 ## WrappedSocket Class Design
 ### Overview
 A WrappedSocket class has been evaluated and recommended to provide cleaner PROXY protocol integration and better socket management architecture.
 ### Rationale for WrappedSocket
 #### Current Challenges
 - Sockets handled directly as `net.Socket` instances throughout codebase
 - Metadata tracked separately in `IConnectionRecord` objects
 - Socket augmentation via TypeScript module augmentation for TLS properties
 - PROXY protocol would require modifying socket handling in multiple places
 #### Benefits
 1. **Clean PROXY Protocol Integration** - Parse and store real client IP/port without modifying existing socket handling
 2. **Better Encapsulation** - Bundle socket + metadata + behavior together
 3. **Type Safety** - No more module augmentation needed
 4. **Future Extensibility** - Easy to add compression, metrics, etc.
 5. **Simplified Testing** - Easier to mock and test socket behavior
 ### Implementation Strategy
 #### Phase 1: Minimal ProxyProtocolSocket (Immediate)
 Create a minimal wrapper for PROXY protocol support:
 ```typescript
 class ProxyProtocolSocket {
  constructor(
    public socket: net.Socket,
    public realClientIP?: string,
    public realClientPort?: number
  ) {}
  get remoteAddress(): string {
    return this.realClientIP || this.socket.remoteAddress || '';
  }
  get remotePort(): number {
    return this.realClientPort || this.socket.remotePort || 0;
  }
  get isFromTrustedProxy(): boolean {
    return !!this.realClientIP;
  }
 }
 ```
 Integration points:
 - Use in `RouteConnectionHandler` when receiving from trusted proxy IPs
 - Update `ConnectionManager` to accept wrapped sockets
 - Modify security checks to use `socket.remoteAddress` getter
 #### Phase 2: Connection-Aware WrappedSocket (Alternative Design)
 A more comprehensive design that manages both sides of a connection:
 ```typescript
 // Option A: Single Socket Wrapper (simpler)
 class WrappedSocket extends EventEmitter {
  private socket: net.Socket;
  private connectionId: string;
  private metadata: ISocketMetadata;
  constructor(socket: net.Socket, metadata?: Partial<ISocketMetadata>) {
    super();
    this.socket = socket;
    this.connectionId = this.generateId();
    this.metadata = { ...defaultMetadata, ...metadata };
    this.setupHandlers();
  }
  // ... single socket management
 }
 // Option B: Connection Pair Wrapper (comprehensive)
 class WrappedConnection extends EventEmitter {
  private connectionId: string;
  private incoming: WrappedSocket;
  private outgoing?: WrappedSocket;
  private forwardingActive: boolean = false;
  constructor(incomingSocket: net.Socket) {
    super();
    this.connectionId = this.generateId();
    this.incoming = new WrappedSocket(incomingSocket);
  }
  // Connect to backend and set up forwarding
  async connectToBackend(target: ITarget): Promise<void> {
    const outgoingSocket = await this.createOutgoingConnection(target);
    this.outgoing = new WrappedSocket(outgoingSocket);
    await this.setupBidirectionalForwarding();
  }
  // Built-in forwarding logic from socket-utils
  private async setupBidirectionalForwarding(): Promise<void> {
    if (!this.outgoing) throw new Error('No outgoing socket');
    // Handle data forwarding with backpressure
    this.incoming.on('data', (chunk) => {
      this.outgoing!.write(chunk, () => {
        // Handle backpressure
      });
    });
    this.outgoing.on('data', (chunk) => {
      this.incoming.write(chunk, () => {
        // Handle backpressure
      });
    });
    // Handle connection lifecycle
    const cleanup = (reason: string) => {
      this.forwardingActive = false;
      this.incoming.destroy();
      this.outgoing?.destroy();
      this.emit('closed', reason);
    };
    this.incoming.once('close', () => cleanup('incoming_closed'));
    this.outgoing.once('close', () => cleanup('outgoing_closed'));
    this.forwardingActive = true;
  }
  // PROXY protocol support
  async handleProxyProtocol(trustedProxies: string[]): Promise<boolean> {
    if (trustedProxies.includes(this.incoming.socket.remoteAddress)) {
      const parsed = await this.incoming.parseProxyProtocol();
      if (parsed && this.outgoing) {
        // Forward PROXY protocol to backend if configured
        await this.outgoing.sendProxyProtocol(this.incoming.realClientIP);
      }
      return parsed;
    }
    return false;
  }
  // Consolidated metrics
  getMetrics(): IConnectionMetrics {
    return {
      connectionId: this.connectionId,
      duration: Date.now() - this.startTime,
      incoming: this.incoming.getMetrics(),
      outgoing: this.outgoing?.getMetrics(),
      totalBytes: this.getTotalBytes(),
      state: this.getConnectionState()
    };
  }
 }
 ```
 #### Phase 3: Full Migration (Long-term)
 - Replace all `net.Socket` usage with `WrappedSocket`
 - Remove socket augmentation from `socket-augmentation.ts`
 - Update all socket utilities to work with wrapped sockets
 - Standardize socket handling across all components
 ### Integration with PROXY Protocol
 The WrappedSocket class integrates seamlessly with PROXY protocol:
 1. **Connection Acceptance**:
   ```typescript
   const wrappedSocket = new ProxyProtocolSocket(socket);
   if (this.isFromTrustedProxy(socket.remoteAddress)) {
     await wrappedSocket.parseProxyProtocol(this.settings.proxyIPs);
   }
   ```
 2. **Security Checks**:
   ```typescript
   // Automatically uses real client IP if available
   const clientIP = wrappedSocket.remoteAddress;
   if (!this.securityManager.isIPAllowed(clientIP)) {
     wrappedSocket.destroy();
   }
   ```
 3. **Connection Records**:
   ```typescript
   const record = this.connectionManager.createConnection(wrappedSocket);
   // ConnectionManager uses wrappedSocket.remoteAddress transparently
   ```
 ### Option B Example: How It Would Replace Current Architecture
 Instead of current approach with separate components:
 ```typescript
 // Current: Multiple separate components
 const record = connectionManager.createConnection(socket);
 const { cleanupClient, cleanupServer } = createIndependentSocketHandlers(
  clientSocket, serverSocket, onBothClosed
 );
 setupBidirectionalForwarding(clientSocket, serverSocket, handlers);
 ```
 Option B would consolidate everything:
 ```typescript
 // Option B: Single connection object
 const connection = new WrappedConnection(incomingSocket);
 await connection.handleProxyProtocol(trustedProxies);
 await connection.connectToBackend({ host: 'server', port: 443 });
 // Everything is handled internally - forwarding, cleanup, metrics
 connection.on('closed', (reason) => {
  logger.log('Connection closed', connection.getMetrics());
 });
 ```
 This would replace:
 - `IConnectionRecord` - absorbed into WrappedConnection
 - `socket-utils.ts` functions - methods on WrappedConnection
 - Separate incoming/outgoing tracking - unified in one object
 - Manual cleanup coordination - automatic lifecycle management
 Additional benefits with Option B:
 - **Connection Pooling Integration**: WrappedConnection could integrate with EnhancedConnectionPool for backend connections
 - **Unified Metrics**: Single point for all connection statistics
 - **Protocol Negotiation**: Handle PROXY, TLS, HTTP/2 upgrade in one place
 - **Resource Management**: Automatic cleanup with LifecycleComponent pattern
 ### Migration Path
 1. **Week 1-2**: Implement minimal ProxyProtocolSocket (Option A)
 2. **Week 3-4**: Test with PROXY protocol implementation
 3. **Month 2**: Prototype WrappedConnection (Option B) if beneficial
 4. **Month 3-6**: Gradual migration if Option B proves valuable
 5. **Future**: Complete adoption in next major version
 ### Success Criteria
 - PROXY protocol works transparently with wrapped sockets
 - No performance regression (<0.1% overhead)
 - Simplified code in connection handlers
 - Better TypeScript type safety
 - Easier to add new socket-level features
--- a/readme.problems.md
+++ b/readme.problems.md
@ -1,170 +0,0 @@
 # SmartProxy Performance Issues Report
 ## Executive Summary
 This report identifies performance issues and blocking operations in the SmartProxy codebase that could impact scalability and responsiveness under high load.
 ## Critical Issues
 ### 1. **Synchronous Filesystem Operations**
 These operations block the event loop and should be replaced with async alternatives:
 #### Certificate Management
 - `ts/proxies/http-proxy/certificate-manager.ts:29`: `fs.existsSync()` 
 - `ts/proxies/http-proxy/certificate-manager.ts:30`: `fs.mkdirSync()`
 - `ts/proxies/http-proxy/certificate-manager.ts:49-50`: `fs.readFileSync()` for loading certificates
 #### NFTables Proxy
 - `ts/proxies/nftables-proxy/nftables-proxy.ts`: Multiple uses of `execSync()` for system commands
 - `ts/proxies/nftables-proxy/nftables-proxy.ts`: Multiple `fs.writeFileSync()` and `fs.unlinkSync()` operations
 #### Certificate Store
 - `ts/proxies/smart-proxy/cert-store.ts:8`: `ensureDirSync()`
 - `ts/proxies/smart-proxy/cert-store.ts:15,31,76`: `fileExistsSync()`
 - `ts/proxies/smart-proxy/cert-store.ts:77`: `removeManySync()`
 ### 2. **Event Loop Blocking Operations**
 #### Busy Wait Loop
 - `ts/proxies/nftables-proxy/nftables-proxy.ts:235-238`: 
  ```typescript
  const waitUntil = Date.now() + retryDelayMs;
  while (Date.now() < waitUntil) {
    // busy wait - blocks event loop completely
  }
  ```
  This is extremely problematic as it blocks the entire Node.js event loop.
 ### 3. **Potential Memory Leaks**
 #### Timer Management Issues
 Several timers are created without proper cleanup:
 - `ts/proxies/http-proxy/function-cache.ts`: `setInterval()` without storing reference for cleanup
 - `ts/proxies/http-proxy/request-handler.ts`: `setInterval()` for rate limit cleanup without cleanup
 - `ts/core/utils/shared-security-manager.ts`: `cleanupInterval` stored but no cleanup method
 #### Event Listener Accumulation
 - Multiple instances of event listeners being added without corresponding cleanup
 - Connection handlers add listeners without always removing them on connection close
 ### 4. **Connection Pool Management**
 #### ConnectionPool (ts/proxies/http-proxy/connection-pool.ts)
 **Good practices observed:**
 - Proper connection lifecycle management
 - Periodic cleanup of idle connections
 - Connection limits enforcement
 **Potential issues:**
 - No backpressure mechanism when pool is full
 - Synchronous sorting operation in `cleanupConnectionPool()` could be slow with many connections
 ### 5. **Resource Management Issues**
 #### Socket Cleanup
 - Some error paths don't properly clean up sockets
 - Missing `removeAllListeners()` in some error scenarios could lead to memory leaks
 #### Timeout Management
 - Inconsistent timeout handling across different components
 - Some sockets created without timeout settings
 ### 6. **JSON Operations on Large Objects**
 - `ts/proxies/smart-proxy/cert-store.ts:21`: `JSON.parse()` on certificate metadata
 - `ts/proxies/smart-proxy/cert-store.ts:71`: `JSON.stringify()` with pretty printing
 - `ts/proxies/http-proxy/function-cache.ts:76`: `JSON.stringify()` for cache keys (called frequently)
 ## Recommendations
 ### Immediate Actions (High Priority)
 1. **Replace Synchronous Operations**
   ```typescript
   // Instead of:
   if (fs.existsSync(path)) { ... }
   // Use:
   try {
     await fs.promises.access(path);
     // file exists
   } catch {
     // file doesn't exist
   }
   ```
 2. **Fix Busy Wait Loop**
   ```typescript
   // Instead of:
   while (Date.now() < waitUntil) { }
   // Use:
   await new Promise(resolve => setTimeout(resolve, retryDelayMs));
   ```
 3. **Add Timer Cleanup**
   ```typescript
   class Component {
     private cleanupTimer?: NodeJS.Timeout;
     start() {
       this.cleanupTimer = setInterval(() => { ... }, 60000);
     }
     stop() {
       if (this.cleanupTimer) {
         clearInterval(this.cleanupTimer);
         this.cleanupTimer = undefined;
       }
     }
   }
   ```
 ### Medium Priority
 1. **Optimize JSON Operations**
   - Cache JSON.stringify results for frequently used objects
   - Consider using faster hashing for cache keys (e.g., crypto.createHash)
   - Use streaming JSON parsers for large objects
 2. **Improve Connection Pool**
   - Implement backpressure/queueing when pool is full
   - Use a heap or priority queue for connection management instead of sorting
 3. **Standardize Resource Cleanup**
   - Create a base class for components with lifecycle management
   - Ensure all event listeners are removed on cleanup
   - Add abort controllers for better cancellation support
 ### Long-term Improvements
 1. **Worker Threads**
   - Move CPU-intensive operations to worker threads
   - Consider using worker pools for NFTables operations
 2. **Monitoring and Metrics**
   - Add performance monitoring for event loop lag
   - Track connection pool utilization
   - Monitor memory usage patterns
 3. **Graceful Degradation**
   - Implement circuit breakers for backend connections
   - Add request queuing with overflow protection
   - Implement adaptive timeout strategies
 ## Impact Assessment
 These issues primarily affect:
 - **Scalability**: Blocking operations limit concurrent connection handling
 - **Responsiveness**: Event loop blocking causes latency spikes
 - **Stability**: Memory leaks could cause crashes under sustained load
 - **Resource Usage**: Inefficient resource management increases memory/CPU usage
 ## Testing Recommendations
 1. Load test with high connection counts (10k+ concurrent)
 2. Monitor event loop lag under stress
 3. Test long-running scenarios to detect memory leaks
 4. Benchmark with async vs sync operations to measure improvement
 ## Conclusion
 While SmartProxy has good architectural design and many best practices, the identified blocking operations and resource management issues could significantly impact performance under high load. The most critical issues (busy wait loop and synchronous filesystem operations) should be addressed immediately.
--- a/readme.routing.md
+++ b/readme.routing.md
@ -0,0 +1,341 @@
 # SmartProxy Routing Architecture Unification Plan
 ## Overview
 This document analyzes the current state of routing in SmartProxy, identifies redundancies and inconsistencies, and proposes a unified architecture.
 ## Current State Analysis
 ### 1. Multiple Route Manager Implementations
 #### 1.1 Core SharedRouteManager (`ts/core/utils/route-manager.ts`)
 - **Purpose**: Designed as a shared component for SmartProxy and NetworkProxy
 - **Features**:
  - Port mapping and expansion (e.g., `[80, 443]` → individual routes)
  - Comprehensive route matching (domain, path, IP, headers, TLS)
  - Route validation and conflict detection
  - Event emitter for route changes
  - Detailed logging support
 - **Status**: Well-designed but underutilized
 #### 1.2 SmartProxy RouteManager (`ts/proxies/smart-proxy/route-manager.ts`)
 - **Purpose**: SmartProxy-specific route management
 - **Issues**:
  - 95% duplicate code from SharedRouteManager
  - Only difference is using `ISmartProxyOptions` instead of generic interface
  - Contains deprecated security methods
  - Unnecessary code duplication
 - **Status**: Should be removed in favor of SharedRouteManager
 #### 1.3 HttpProxy Route Management (`ts/proxies/http-proxy/`)
 - **Purpose**: HTTP-specific routing
 - **Implementation**: Minimal, inline route matching
 - **Status**: Could benefit from SharedRouteManager
 ### 2. Multiple Router Implementations
 #### 2.1 ProxyRouter (`ts/routing/router/proxy-router.ts`)
 - **Purpose**: Legacy compatibility with `IReverseProxyConfig`
 - **Features**: Domain-based routing with path patterns
 - **Used by**: HttpProxy for backward compatibility
 #### 2.2 RouteRouter (`ts/routing/router/route-router.ts`)
 - **Purpose**: Modern routing with `IRouteConfig`
 - **Features**: Nearly identical to ProxyRouter
 - **Issues**: Code duplication with ProxyRouter
 ### 3. Scattered Route Utilities
 #### 3.1 Core route-utils (`ts/core/utils/route-utils.ts`)
 - **Purpose**: Shared matching functions
 - **Features**: Domain, path, IP, CIDR matching
 - **Status**: Well-implemented, should be the single source
 #### 3.2 SmartProxy route-utils (`ts/proxies/smart-proxy/utils/route-utils.ts`)
 - **Purpose**: Route configuration utilities
 - **Features**: Different scope - config merging, not pattern matching
 - **Status**: Keep separate as it serves different purpose
 ### 4. Other Route-Related Files
 - `route-patterns.ts`: Constants for route patterns
 - `route-validators.ts`: Route configuration validation
 - `route-helpers.ts`: Additional utilities
 - `route-connection-handler.ts`: Connection routing logic
 ## Problems Identified
 ### 1. Code Duplication
 - **SharedRouteManager vs SmartProxy RouteManager**: ~1000 lines of duplicate code
 - **ProxyRouter vs RouteRouter**: ~500 lines of duplicate code
 - **Matching logic**: Implemented in 4+ different places
 ### 2. Inconsistent Implementations
 ```typescript
 // Example: Domain matching appears in multiple places
 // 1. In route-utils.ts
 export function matchDomain(pattern: string, hostname: string): boolean
 // 2. In SmartProxy RouteManager
 private matchDomain(domain: string, hostname: string): boolean
 // 3. In ProxyRouter
 private matchesHostname(configName: string, hostname: string): boolean
 // 4. In RouteRouter
 private matchDomain(pattern: string, hostname: string): boolean
 ```
 ### 3. Unclear Separation of Concerns
 - Route Managers handle both storage AND matching
 - Routers also handle storage AND matching
 - No clear boundaries between layers
 ### 4. Maintenance Burden
 - Bug fixes need to be applied in multiple places
 - New features must be implemented multiple times
 - Testing effort multiplied
 ## Proposed Unified Architecture
 ### Layer 1: Core Routing Components
 ```
 ts/core/routing/
 ├── types.ts              # All route-related types
 ├── utils.ts              # All matching logic (consolidated)
 ├── route-store.ts        # Route storage and indexing
 └── route-matcher.ts      # Route matching engine
 ```
 ### Layer 2: Route Management
 ```
 ts/core/routing/
 └── route-manager.ts      # Single RouteManager for all proxies
    - Uses RouteStore for storage
    - Uses RouteMatcher for matching
    - Provides high-level API
 ```
 ### Layer 3: HTTP Routing
 ```
 ts/routing/
 └── http-router.ts        # Single HTTP router implementation
    - Uses RouteManager for route lookup
    - Handles HTTP-specific concerns
    - Legacy adapter built-in
 ```
 ### Layer 4: Proxy Integration
 ```
 ts/proxies/
 ├── smart-proxy/
 │   └── (uses core RouteManager directly)
 ├── http-proxy/
 │   └── (uses core RouteManager + HttpRouter)
 └── network-proxy/
    └── (uses core RouteManager directly)
 ```
 ## Implementation Plan
 ### Phase 1: Consolidate Matching Logic (Week 1)
 1. **Audit all matching implementations**
   - Document differences in behavior
   - Identify the most comprehensive implementation
   - Create test suite covering all edge cases
 2. **Create unified matching module**
   ```typescript
   // ts/core/routing/matchers.ts
   export class DomainMatcher {
     static match(pattern: string, hostname: string): boolean
   }
   export class PathMatcher {
     static match(pattern: string, path: string): MatchResult
   }
   export class IpMatcher {
     static match(pattern: string, ip: string): boolean
     static matchCidr(cidr: string, ip: string): boolean
   }
   ```
 3. **Update all components to use unified matchers**
   - Replace local implementations
   - Ensure backward compatibility
   - Run comprehensive tests
 ### Phase 2: Unify Route Managers (Week 2)
 1. **Enhance SharedRouteManager**
   - Add any missing features from SmartProxy RouteManager
   - Make it truly generic (no proxy-specific dependencies)
   - Add adapter pattern for different options types
 2. **Migrate SmartProxy to use SharedRouteManager**
   ```typescript
   // Before
   this.routeManager = new RouteManager(this.settings);
   // After
   this.routeManager = new SharedRouteManager({
     logger: this.settings.logger,
     enableDetailedLogging: this.settings.enableDetailedLogging
   });
   ```
 3. **Remove duplicate RouteManager**
   - Delete `ts/proxies/smart-proxy/route-manager.ts`
   - Update all imports
   - Verify all tests pass
 ### Phase 3: Consolidate Routers (Week 3)
 1. **Create unified HttpRouter**
   ```typescript
   export class HttpRouter {
     constructor(private routeManager: SharedRouteManager) {}
     // Modern interface
     route(req: IncomingMessage): RouteResult
     // Legacy adapter
     routeLegacy(config: IReverseProxyConfig): RouteResult
   }
   ```
 2. **Migrate HttpProxy**
   - Replace both ProxyRouter and RouteRouter
   - Use single HttpRouter with appropriate adapter
   - Maintain backward compatibility
 3. **Clean up legacy code**
   - Mark old interfaces as deprecated
   - Add migration guides
   - Plan removal in next major version
 ### Phase 4: Architecture Cleanup (Week 4)
 1. **Reorganize file structure**
   ```
   ts/core/
   ├── routing/
   │   ├── index.ts
   │   ├── types.ts
   │   ├── matchers/
   │   │   ├── domain.ts
   │   │   ├── path.ts
   │   │   ├── ip.ts
   │   │   └── index.ts
   │   ├── route-store.ts
   │   ├── route-matcher.ts
   │   └── route-manager.ts
   └── utils/
       └── (remove route-specific utils)
   ```
 2. **Update documentation**
   - Architecture diagrams
   - Migration guides
   - API documentation
 3. **Performance optimization**
   - Add caching where beneficial
   - Optimize hot paths
   - Benchmark before/after
 ## Migration Strategy
 ### For SmartProxy RouteManager Users
 ```typescript
 // Old way
 import { RouteManager } from './route-manager.js';
 const manager = new RouteManager(options);
 // New way
 import { SharedRouteManager as RouteManager } from '../core/utils/route-manager.js';
 const manager = new RouteManager({
  logger: options.logger,
  enableDetailedLogging: options.enableDetailedLogging
 });
 ```
 ### For Router Users
 ```typescript
 // Old way
 const proxyRouter = new ProxyRouter();
 const routeRouter = new RouteRouter();
 // New way
 const router = new HttpRouter(routeManager);
 // Automatically handles both modern and legacy configs
 ```
 ## Success Metrics
 1. **Code Reduction**
   - Target: Remove ~1,500 lines of duplicate code
   - Measure: Lines of code before/after
 2. **Performance**
   - Target: No regression in routing performance
   - Measure: Benchmark route matching operations
 3. **Maintainability**
   - Target: Single implementation for each concept
   - Measure: Time to implement new features
 4. **Test Coverage**
   - Target: 100% coverage of routing logic
   - Measure: Coverage reports
 ## Risks and Mitigations
 ### Risk 1: Breaking Changes
 - **Mitigation**: Extensive adapter patterns and backward compatibility layers
 - **Testing**: Run all existing tests plus new integration tests
 ### Risk 2: Performance Regression
 - **Mitigation**: Benchmark critical paths before changes
 - **Testing**: Load testing with production-like scenarios
 ### Risk 3: Hidden Dependencies
 - **Mitigation**: Careful code analysis and dependency mapping
 - **Testing**: Integration tests across all proxy types
 ## Long-term Vision
 ### Future Enhancements
 1. **Route Caching**: LRU cache for frequently accessed routes
 2. **Route Indexing**: Trie-based indexing for faster domain matching
 3. **Route Priorities**: Explicit priority system instead of specificity
 4. **Dynamic Routes**: Support for runtime route modifications
 5. **Route Templates**: Reusable route configurations
 ### API Evolution
 ```typescript
 // Future unified routing API
 const routingEngine = new RoutingEngine({
  stores: [fileStore, dbStore, dynamicStore],
  matchers: [domainMatcher, pathMatcher, customMatcher],
  cache: new LRUCache({ max: 1000 }),
  indexes: {
    domain: new TrieIndex(),
    path: new RadixTree()
  }
 });
 // Simple, powerful API
 const route = await routingEngine.findRoute({
  domain: 'example.com',
  path: '/api/v1/users',
  ip: '192.168.1.1',
  headers: { 'x-custom': 'value' }
 });
 ```
 ## Conclusion
 The current routing architecture has significant duplication and inconsistencies. By following this unification plan, we can:
 1. Reduce code by ~30%
 2. Improve maintainability
 3. Ensure consistent behavior
 4. Enable future enhancements
 The phased approach minimizes risk while delivering incremental value. Each phase is independently valuable and can be deployed separately.
--- a/ts/proxies/http-proxy/http-proxy.ts
+++ b/ts/proxies/http-proxy/http-proxy.ts
@ -2,7 +2,6 @@ import * as plugins from '../../plugins.js';
 import {
  createLogger,
  RouteManager,
  convertLegacyConfigToRouteConfig
 } from './models/types.js';
 import type {
  IHttpProxyOptions,
--- a/ts/proxies/http-proxy/models/types.ts
+++ b/ts/proxies/http-proxy/models/types.ts
@ -99,82 +99,6 @@ export interface IReverseProxyConfig {
  backendProtocol?: 'http1' | 'http2';
 }
 /**
 * Convert a legacy IReverseProxyConfig to the modern IRouteConfig format
 *
 * @deprecated This function is maintained for backward compatibility.
 * New code should create IRouteConfig objects directly.
 *
 * @param legacyConfig The legacy configuration to convert
 * @param proxyPort The port the proxy listens on
 * @returns A modern route configuration equivalent to the legacy config
 */
 export function convertLegacyConfigToRouteConfig(
  legacyConfig: IReverseProxyConfig,
  proxyPort: number
 ): IRouteConfig {
  // Create basic route configuration
  const routeConfig: IRouteConfig = {
    // Match properties
    match: {
      ports: proxyPort,
      domains: legacyConfig.hostName
    },
    // Action properties
    action: {
      type: 'forward',
      target: {
        host: legacyConfig.destinationIps,
        port: legacyConfig.destinationPorts[0]
      },
      // TLS mode is always 'terminate' for legacy configs
      tls: {
        mode: 'terminate',
        certificate: {
          key: legacyConfig.privateKey,
          cert: legacyConfig.publicKey
        }
      },
      // Advanced options
      advanced: {
        // Rewrite host header if specified
        headers: legacyConfig.rewriteHostHeader ? { 'host': '{domain}' } : {}
      }
    },
    // Metadata
    name: `Legacy Config - ${legacyConfig.hostName}`,
    priority: 0, // Default priority
    enabled: true
  };
  // Add authentication if present
  if (legacyConfig.authentication) {
    routeConfig.security = {
      authentication: {
        type: 'basic',
        credentials: [{
          username: legacyConfig.authentication.user,
          password: legacyConfig.authentication.pass
        }]
      }
    };
  }
  // Add backend protocol if specified
  if (legacyConfig.backendProtocol) {
    if (!routeConfig.action.options) {
      routeConfig.action.options = {};
    }
    routeConfig.action.options.backendProtocol = legacyConfig.backendProtocol;
  }
  return routeConfig;
 }
 /**
 * Route manager for NetworkProxy
 * Handles route matching and configuration