update

NetworkProxy-SmartProxy-Connection-Management.md

@@ -0,0 +1,153 @@
+# NetworkProxy Connection Termination and SmartProxy Connection Handling
+
+## Overview
+
+The connection management between NetworkProxy and SmartProxy involves complex coordination to handle TLS termination, connection forwarding, and proper cleanup. This document outlines how these systems work together.
+
+## SmartProxy Connection Management
+
+### Connection Tracking (ConnectionManager)
+
+1. **Connection Lifecycle**:
+   - New connections are registered in `ConnectionManager.createConnection()`
+   - Each connection gets a unique ID and tracking record
+   - Connection records track both incoming (client) and outgoing (target) sockets
+   - Connections are removed from tracking upon cleanup
+
+2. **Connection Cleanup Flow**:
+   ```
+   initiateCleanupOnce() -> cleanupConnection() -> cleanupSocket()
+   ```
+   - `initiateCleanupOnce()`: Prevents duplicate cleanup operations
+   - `cleanupConnection()`: Main cleanup logic, removes connections from tracking
+   - `cleanupSocket()`: Handles socket termination (graceful end, then forced destroy)
+
+3. **Cleanup Triggers**:
+   - Socket errors (ECONNRESET, ETIMEDOUT, etc.)
+   - Socket close events
+   - Inactivity timeouts
+   - Connection lifetime limits
+   - Manual cleanup (e.g., NFTables-handled connections)
+
+## NetworkProxy Integration
+
+### NetworkProxyBridge
+
+The `NetworkProxyBridge` class manages the connection between SmartProxy and NetworkProxy:
+
+1. **Connection Forwarding**:
+   ```typescript
+   forwardToNetworkProxy(
+     connectionId: string,
+     socket: net.Socket,
+     record: IConnectionRecord,
+     initialChunk: Buffer,
+     networkProxyPort: number,
+     cleanupCallback: (reason: string) => void
+   )
+   ```
+   - Creates a new socket connection to NetworkProxy
+   - Pipes data between client and NetworkProxy sockets
+   - Sets up cleanup handlers for both sockets
+
+2. **Cleanup Coordination**:
+   - When either socket ends or errors, both are cleaned up
+   - Cleanup callback notifies SmartProxy's ConnectionManager
+   - Proper unpipe operations prevent memory leaks
+
+## NetworkProxy Connection Tracking
+
+### Connection Tracking in NetworkProxy
+
+1. **Raw TCP Connection Tracking**:
+   ```typescript
+   setupConnectionTracking(): void {
+     this.httpsServer.on('connection', (connection: net.Socket) => {
+       // Track connections in socketMap
+       this.socketMap.add(connection);
+       
+       // Setup cleanup handlers
+       connection.on('close', cleanupConnection);
+       connection.on('error', cleanupConnection);
+       connection.on('end', cleanupConnection);
+     });
+   }
+   ```
+
+2. **SmartProxy Connection Detection**:
+   - Connections from localhost (127.0.0.1) are identified as SmartProxy connections
+   - Special counter tracks `portProxyConnections`
+   - Connection counts are updated when connections close
+
+3. **Metrics and Monitoring**:
+   - Active connections tracked in `connectedClients`
+   - TLS handshake completions tracked in `tlsTerminatedConnections`
+   - Connection pool status monitored periodically
+
+## Connection Termination Flow
+
+### Typical TLS Termination Flow:
+
+1. Client connects to SmartProxy
+2. SmartProxy creates connection record and tracks socket
+3. SmartProxy determines route requires TLS termination
+4. NetworkProxyBridge forwards connection to NetworkProxy
+5. NetworkProxy performs TLS termination
+6. Data flows through piped sockets
+7. When connection ends:
+   - NetworkProxy cleans up its socket tracking
+   - NetworkProxyBridge handles cleanup coordination
+   - SmartProxy's ConnectionManager removes connection record
+   - All resources are properly released
+
+### Cleanup Coordination Points:
+
+1. **SmartProxy Cleanup**:
+   - ConnectionManager tracks all cleanup reasons
+   - Socket handlers removed to prevent memory leaks
+   - Timeout timers cleared
+   - Connection records removed from maps
+   - Security manager notified of connection removal
+
+2. **NetworkProxy Cleanup**:
+   - Sockets removed from tracking map
+   - Connection counters updated
+   - Metrics updated for monitoring
+   - Connection pool resources freed
+
+3. **Bridge Cleanup**:
+   - Unpipe operations prevent data loss
+   - Both sockets properly destroyed
+   - Cleanup callback ensures SmartProxy is notified
+
+## Important Considerations
+
+1. **Memory Management**:
+   - All event listeners must be removed during cleanup
+   - Proper unpipe operations prevent memory leaks
+   - Connection records cleared from all tracking maps
+
+2. **Error Handling**:
+   - Multiple cleanup mechanisms prevent orphaned connections
+   - Graceful shutdown attempted before forced destruction
+   - Timeout mechanisms ensure cleanup even in edge cases
+
+3. **State Consistency**:
+   - Connection closed flags prevent duplicate cleanup
+   - Termination reasons tracked for debugging
+   - Activity timestamps updated for accurate timeout handling
+
+4. **Performance**:
+   - Connection pools minimize TCP handshake overhead
+   - Efficient socket tracking using Maps
+   - Periodic cleanup prevents resource accumulation
+
+## Best Practices
+
+1. Always use `initiateCleanupOnce()` to prevent duplicate cleanup operations
+2. Track termination reasons for debugging and monitoring
+3. Ensure all event listeners are removed during cleanup
+4. Use proper unpipe operations when breaking socket connections
+5. Monitor connection counts and cleanup statistics
+6. Implement proper timeout handling for all connection types
+7. Keep socket tracking maps synchronized with actual socket state