3.23.0

changelog.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## 2025-03-03 - 3.23.0 - feat(documentation)
+Updated documentation with architecture flow diagrams.
+
+- Added detailed architecture and flow diagrams for SmartProxy components.
+- Included HTTPS Reverse Proxy Flow diagram.
+- Integrated Port Proxy with SNI-based Routing diagram.
+- Added Let's Encrypt Certificate Acquisition flow.
+
+## 2025-03-03 - 3.22.5 - fix(documentation)
+Refactored readme for clarity and consistency, fixed documentation typos
+
+- Updated readme to improve clarity and remove redundant information.
+- Fixed minor documentation issues in the code comments.
+- Reorganized readme structure for better readability.
+- Improved sample code snippets for easier understanding.
+
 ## 2025-03-03 - 3.22.4 - fix(core)
 Addressed minor issues in the core modules to improve stability and performance.
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@push.rocks/smartproxy",
-  "version": "3.22.4",
+  "version": "3.23.0",
   "private": false,
   "description": "A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, and dynamic routing with authentication options.",
   "main": "dist_ts/index.js",

readme.md

@@ -1,228 +1,389 @@
 # @push.rocks/smartproxy
 
-A robust and versatile proxy package designed to handle high workloads, offering features like SSL redirection, port proxying, WebSocket support, and customizable routing and authentication.
+A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, and dynamic routing with authentication options.
 
-## Install
+## Architecture & Flow Diagrams
 
-To install `@push.rocks/smartproxy`, run the following command in your project's root directory:
+### Component Architecture
+The diagram below illustrates the main components of SmartProxy and how they interact:
 
-```bash
-npm install @push.rocks/smartproxy --save
+```mermaid
+flowchart TB
+    Client([Client])
+    
+    subgraph "SmartProxy Components"
+        direction TB
+        HTTP80[HTTP Port 80\nSslRedirect]
+        HTTPS443[HTTPS Port 443\nNetworkProxy]
+        PortProxy[TCP Port Proxy\nwith SNI routing]
+        IPTables[IPTablesProxy]
+        Router[ProxyRouter]
+        ACME[Port80Handler\nACME/Let's Encrypt]
+        Certs[(SSL Certificates)]
+    end
+    
+    subgraph "Backend Services"
+        Service1[Service 1]
+        Service2[Service 2]
+        Service3[Service 3]
+    end
+    
+    Client -->|HTTP Request| HTTP80
+    HTTP80 -->|Redirect| Client
+    Client -->|HTTPS Request| HTTPS443
+    Client -->|TLS/TCP| PortProxy
+    
+    HTTPS443 -->|Route Request| Router
+    Router -->|Proxy Request| Service1
+    Router -->|Proxy Request| Service2
+    
+    PortProxy -->|Direct TCP| Service2
+    PortProxy -->|Direct TCP| Service3
+    
+    IPTables -.->|Low-level forwarding| PortProxy
+    
+    HTTP80 -.->|Challenge Response| ACME
+    ACME -.->|Generate/Manage| Certs
+    Certs -.->|Provide TLS Certs| HTTPS443
+    
+    classDef component fill:#f9f,stroke:#333,stroke-width:2px;
+    classDef backend fill:#bbf,stroke:#333,stroke-width:1px;
+    classDef client fill:#dfd,stroke:#333,stroke-width:2px;
+    
+    class Client client;
+    class HTTP80,HTTPS443,PortProxy,IPTables,Router,ACME component;
+    class Service1,Service2,Service3 backend;
 ```
 
-This will add `@push.rocks/smartproxy` to your project's dependencies.
+### HTTPS Reverse Proxy Flow
+This diagram shows how HTTPS requests are handled and proxied to backend services:
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant NetworkProxy
+    participant ProxyRouter
+    participant Backend
+    
+    Client->>NetworkProxy: HTTPS Request
+    
+    Note over NetworkProxy: TLS Termination
+    
+    NetworkProxy->>ProxyRouter: Route Request
+    ProxyRouter->>ProxyRouter: Match hostname to config
+    
+    alt Authentication Required
+        NetworkProxy->>Client: Request Authentication
+        Client->>NetworkProxy: Send Credentials
+        NetworkProxy->>NetworkProxy: Validate Credentials
+    end
+    
+    NetworkProxy->>Backend: Forward Request
+    Backend->>NetworkProxy: Response
+    
+    Note over NetworkProxy: Add Default Headers
+    
+    NetworkProxy->>Client: Forward Response
+    
+    alt WebSocket Request
+        Client->>NetworkProxy: Upgrade to WebSocket
+        NetworkProxy->>Backend: Upgrade to WebSocket
+        loop WebSocket Active
+            Client->>NetworkProxy: WebSocket Message
+            NetworkProxy->>Backend: Forward Message
+            Backend->>NetworkProxy: WebSocket Message
+            NetworkProxy->>Client: Forward Message
+            NetworkProxy-->>NetworkProxy: Heartbeat Check
+        end
+    end
+```
+
+### Port Proxy with SNI-based Routing
+This diagram illustrates how TCP connections with SNI (Server Name Indication) are processed and forwarded:
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant PortProxy
+    participant Backend
+    
+    Client->>PortProxy: TLS Connection
+    
+    alt SNI Enabled
+        PortProxy->>Client: Accept Connection
+        Client->>PortProxy: TLS ClientHello with SNI
+        PortProxy->>PortProxy: Extract SNI Hostname
+        PortProxy->>PortProxy: Match Domain Config
+        PortProxy->>PortProxy: Validate Client IP
+        
+        alt IP Allowed
+            PortProxy->>Backend: Forward Connection
+            Note over PortProxy,Backend: Bidirectional Data Flow
+        else IP Rejected
+            PortProxy->>Client: Close Connection
+        end
+    else Port-based Routing
+        PortProxy->>PortProxy: Match Port Range
+        PortProxy->>PortProxy: Find Domain Config
+        PortProxy->>PortProxy: Validate Client IP
+        
+        alt IP Allowed
+            PortProxy->>Backend: Forward Connection
+            Note over PortProxy,Backend: Bidirectional Data Flow
+        else IP Rejected
+            PortProxy->>Client: Close Connection
+        end
+    end
+    
+    loop Connection Active
+        PortProxy-->>PortProxy: Monitor Activity
+        PortProxy-->>PortProxy: Check Max Lifetime
+        alt Inactivity or Max Lifetime Exceeded
+            PortProxy->>Client: Close Connection
+            PortProxy->>Backend: Close Connection
+        end
+    end
+```
+
+### Let's Encrypt Certificate Acquisition
+This diagram shows how certificates are automatically acquired through the ACME protocol:
+
+```mermaid
+sequenceDiagram
+    participant Client
+    participant Port80Handler
+    participant ACME as Let's Encrypt ACME
+    participant NetworkProxy
+    
+    Client->>Port80Handler: HTTP Request for domain
+    
+    alt Certificate Exists
+        Port80Handler->>Client: Redirect to HTTPS
+    else No Certificate
+        Port80Handler->>Port80Handler: Mark domain as obtaining cert
+        Port80Handler->>ACME: Create account & new order
+        ACME->>Port80Handler: Challenge information
+        
+        Port80Handler->>Port80Handler: Store challenge token & key authorization
+        
+        ACME->>Port80Handler: HTTP-01 Challenge Request
+        Port80Handler->>ACME: Challenge Response
+        
+        ACME->>ACME: Validate domain ownership
+        ACME->>Port80Handler: Challenge validated
+        
+        Port80Handler->>Port80Handler: Generate CSR
+        Port80Handler->>ACME: Submit CSR
+        ACME->>Port80Handler: Issue Certificate
+        
+        Port80Handler->>Port80Handler: Store certificate & private key
+        Port80Handler->>Port80Handler: Mark certificate as obtained
+        
+        Note over Port80Handler,NetworkProxy: Certificate available for use
+        
+        Client->>Port80Handler: Another HTTP Request
+        Port80Handler->>Client: Redirect to HTTPS
+        Client->>NetworkProxy: HTTPS Request
+        Note over NetworkProxy: Uses new certificate
+    end
+```
+
+## Features
+
+- **HTTPS Reverse Proxy** - Route traffic to backend services based on hostname with TLS termination
+- **WebSocket Support** - Full WebSocket proxying with heartbeat monitoring
+- **TCP Port Forwarding** - Advanced port forwarding with SNI inspection and domain-based routing
+- **HTTP to HTTPS Redirection** - Automatically redirect HTTP requests to HTTPS
+- **Let's Encrypt Integration** - Automatic certificate management using ACME protocol
+- **IP Filtering** - Control access with IP allow/block lists using glob patterns
+- **IPTables Integration** - Direct manipulation of iptables for low-level port forwarding
+- **Basic Authentication** - Support for basic auth on proxied routes
+- **Connection Management** - Intelligent connection tracking and cleanup
+
+## Installation
+
+```bash
+npm install @push.rocks/smartproxy
+```
 
 ## Usage
 
-`@push.rocks/smartproxy` is a comprehensive package that provides advanced functionalities for handling proxy tasks efficiently, including SSL redirection, port proxying, WebSocket support, and dynamic routing with authentication capabilities. Here is an extensive guide on how to utilize these features effectively, ensuring robust and secure proxy operations.
-
-### Initial Setup
-
-Before exploring the advanced features of `smartproxy`, you need to set up a basic proxy server. This setup serves as the foundation for incorporating additional functionalities later on:
+### Basic Reverse Proxy Setup
 
 ```typescript
 import { NetworkProxy } from '@push.rocks/smartproxy';
 
-// Create an instance of NetworkProxy with the desired configuration
-const myNetworkProxy = new NetworkProxy({ port: 443 });
+// Create a reverse proxy listening on port 443
+const proxy = new NetworkProxy({
+  port: 443
+});
 
-// Define reverse proxy configurations for the domains you wish to proxy
+// Define reverse proxy configurations
 const proxyConfigs = [
   {
-    destinationIp: '127.0.0.1',
-    destinationPort: '3000',
     hostName: 'example.com',
-    privateKey: `-----BEGIN PRIVATE KEY-----
-PRIVATE_KEY_CONTENT
------END PRIVATE KEY-----`,
-    publicKey: `-----BEGIN CERTIFICATE-----
-CERTIFICATE_CONTENT
------END CERTIFICATE-----`,
+    destinationIp: '127.0.0.1',
+    destinationPort: 3000,
+    publicKey: 'your-cert-content',
+    privateKey: 'your-key-content'
   },
-  // Additional configurations can be added here
+  {
+    hostName: 'api.example.com',
+    destinationIp: '127.0.0.1',
+    destinationPort: 4000,
+    publicKey: 'your-cert-content',
+    privateKey: 'your-key-content',
+    // Optional basic auth
+    authentication: {
+      type: 'Basic',
+      user: 'admin',
+      pass: 'secret'
+    }
+  }
 ];
 
-// Start the network proxy to enable forwarding
-await myNetworkProxy.start();
-
-// Apply the configurations you defined earlier
-await myNetworkProxy.updateProxyConfigs(proxyConfigs);
-
-// Optionally, you can set default headers to be included in all responses
-await myNetworkProxy.addDefaultHeaders({
-  'X-Powered-By': 'smartproxy',
-});
+// Start the proxy and update configurations
+(async () => {
+  await proxy.start();
+  await proxy.updateProxyConfigs(proxyConfigs);
+  
+  // Add default headers to all responses
+  await proxy.addDefaultHeaders({
+    'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload'
+  });
+})();
 ```
 
-### Configuring SSL Redirection
-
-A critical feature of modern proxy servers is the ability to redirect HTTP traffic to secure HTTPS endpoints. The `SslRedirect` class in `smartproxy` simplifies this process by automatically redirecting requests from HTTP port 80 to HTTPS:
+### HTTP to HTTPS Redirection
 
 ```typescript
 import { SslRedirect } from '@push.rocks/smartproxy';
 
-// Create an SslRedirect instance to listen on port 80
-const mySslRedirect = new SslRedirect(80);
-
-// Start the redirect to enforce HTTPS
-await mySslRedirect.start();
-
-// To stop HTTP redirection, use the following command:
-await mySslRedirect.stop();
+// Create and start HTTP to HTTPS redirect service on port 80
+const redirector = new SslRedirect(80);
+redirector.start();
 ```
 
-### Managing Port Proxying
-
-Port proxying is essential for forwarding traffic from one port to another, an important feature for services that require dynamic port changes without downtime. Smartproxy's `PortProxy` class efficiently handles these scenarios:
+### TCP Port Forwarding with Domain-based Routing
 
 ```typescript
 import { PortProxy } from '@push.rocks/smartproxy';
 
-// Set up a PortProxy to forward traffic from port 5000 to 3000
-const myPortProxy = new PortProxy(5000, 3000);
-
-// Initiate the port proxy
-await myPortProxy.start();
-
-// To halt the port proxy, execute:
-await myPortProxy.stop();
-```
-
-For more intricate setups—such as forwarding based on specific domain rules or IP allowances—smartproxy allows detailed configurations:
-
-```typescript
-import { PortProxy } from '@push.rocks/smartproxy';
-
-// Configure complex port proxy rules
-const advancedPortProxy = new PortProxy({
-  fromPort: 6000,
-  toPort: 3000,
-  domains: [
+// Configure port proxy with domain-based routing
+const portProxy = new PortProxy({
+  fromPort: 443,
+  toPort: 8443,
+  targetIP: 'localhost', // Default target host
+  sniEnabled: true,      // Enable SNI inspection
+  globalPortRanges: [{ from: 443, to: 443 }],
+  defaultAllowedIPs: ['*'], // Allow all IPs by default
+  domainConfigs: [
     {
-      domain: 'api.example.com',
-      allowedIPs: ['192.168.0.*', '127.0.0.1'],
-      targetIP: '192.168.1.100'
+      domains: ['example.com', '*.example.com'], // Glob patterns for matching domains
+      allowedIPs: ['192.168.1.*'],               // Restrict access by IP
+      blockedIPs: ['192.168.1.100'],             // Block specific IPs
+      targetIPs: ['10.0.0.1', '10.0.0.2'],       // Round-robin between multiple targets
+      portRanges: [{ from: 443, to: 443 }]
     }
-    // Additional domain rules can be added as needed
   ],
-  sniEnabled: true,  // Server Name Indication (SNI) support
-  defaultAllowedIPs: ['*'],
+  maxConnectionLifetime: 3600000, // 1 hour in milliseconds
+  preserveSourceIP: true
 });
 
-// Activate the proxy with conditional rules
-await advancedPortProxy.start();
+portProxy.start();
 ```
 
-### WebSocket Handling
-
-With real-time applications becoming more prevalent, effective WebSocket handling is crucial in a proxy server. Smartproxy natively incorporates WebSocket support to manage WebSocket traffic securely and efficiently:
-
-```typescript
-import { NetworkProxy } from '@push.rocks/smartproxy';
-
-// Create a NetworkProxy instance for WebSocket traffic
-const wsNetworkProxy = new NetworkProxy({ port: 443 });
-
-// Define proxy configurations targeted for WebSocket traffic
-const websocketConfig = [
-  {
-    destinationIp: '127.0.0.1',
-    destinationPort: '8080',
-    hostName: 'socket.example.com',
-    // Include SSL details if necessary
-  }
-];
-
-// Start the proxy and apply WebSocket settings
-await wsNetworkProxy.start();
-await wsNetworkProxy.updateProxyConfigs(websocketConfig);
-
-// Set heartbeat intervals to maintain WebSocket connections
-wsNetworkProxy.heartbeatInterval = setInterval(() => {
-  // Logic for connection health checks
-}, 60000); // every minute
-
-// Capture and handle server errors for resiliency
-wsNetworkProxy.httpsServer.on('error', (error) => console.log('Server Error:', error));
-```
-
-### Advanced Routing and Custom Features
-
-Smartproxy shines with its dynamic routing capabilities, allowing for custom and advanced request routing based on the request's destination. This enables extensive flexibility, such as directing API requests or facilitating intricate B2B integrations:
-
-```typescript
-import { NetworkProxy } from '@push.rocks/smartproxy';
-
-// Instantiate a proxy with dynamic routing
-const routeProxy = new NetworkProxy({ port: 8443 });
-
-routeProxy.router.setNewProxyConfigs([
-  {
-    destinationIp: '192.168.1.150',
-    destinationPort: '80',
-    hostName: 'dynamic.example.com',
-    authentication: {
-      type: 'Basic',
-      user: 'admin',
-      pass: 'password123'
-    }
-  }
-]);
-
-// Activate the routing proxy
-await routeProxy.start();
-```
-
-For those who require granular traffic control, integrating tools like `iptables` offers additional power over network management:
+### IPTables Port Forwarding
 
 ```typescript
 import { IPTablesProxy } from '@push.rocks/smartproxy';
 
-// Set up IPTables for sophisticated network traffic management
-const iptablesProxy = new IPTablesProxy({
-  fromPort: 8081,
+// Configure IPTables to forward from port 80 to 8080
+const iptables = new IPTablesProxy({
+  fromPort: 80,
   toPort: 8080,
-  deleteOnExit: true // Clean up rules when the server shuts down
+  toHost: 'localhost',
+  preserveSourceIP: true,
+  deleteOnExit: true  // Automatically clean up rules on process exit
 });
 
-// Enable routing through IPTables
-await iptablesProxy.start();
+iptables.start();
 ```
 
-### Integrating SSL and HTTP/HTTPS Credentials
-
-Handling sensitive data like SSL keys and certificates securely is crucial in proxy configurations:
+### Automatic HTTPS Certificate Management
 
 ```typescript
-import { loadDefaultCertificates } from '@push.rocks/smartproxy';
+import { Port80Handler } from '@push.rocks/smartproxy';
 
-try {
-  const { privateKey, publicKey } = loadDefaultCertificates(); // Adjust path if necessary
-  console.log('SSL certificates loaded successfully.');
-  // Use these credentials in your configurations
-} catch (error) {
-  console.error('Error loading certificates:', error);
-}
+// Create an ACME handler for Let's Encrypt
+const acmeHandler = new Port80Handler();
+
+// Add domains to manage certificates for
+acmeHandler.addDomain('example.com');
+acmeHandler.addDomain('api.example.com');
 ```
 
-### Testing and Validation
+## Configuration Options
 
-Smartproxy supports extensive testing to ensure your proxy configurations operate as expected. Leveraging `tap` alongside TypeScript testing frameworks supports quality assurance:
+### NetworkProxy Options
 
-```typescript
-import { expect, tap } from '@push.rocks/tapbundle';
-import { NetworkProxy } from '@push.rocks/smartproxy';
+| Option         | Description                                       | Default |
+|----------------|---------------------------------------------------|---------|
+| `port`         | Port to listen on for HTTPS connections           | -       |
 
-tap.test('Check proxied request returns status 200', async () => {
-  // Testing logic
-});
+### PortProxy Settings
 
-tap.start();
-```
+| Option                   | Description                                            | Default     |
+|--------------------------|--------------------------------------------------------|-------------|
+| `fromPort`               | Port to listen on                                      | -           |
+| `toPort`                 | Destination port to forward to                         | -           |
+| `targetIP`               | Default destination IP if not specified in domainConfig | 'localhost' |
+| `sniEnabled`             | Enable SNI inspection for TLS connections              | false       |
+| `defaultAllowedIPs`      | IP patterns allowed by default                         | -           |
+| `defaultBlockedIPs`      | IP patterns blocked by default                         | -           |
+| `preserveSourceIP`       | Preserve the original client IP                        | false       |
+| `maxConnectionLifetime`  | Maximum time in ms to keep a connection open           | 600000      |
+| `globalPortRanges`       | Array of port ranges to listen on                      | -           |
+| `forwardAllGlobalRanges` | Forward all global range connections to targetIP       | false       |
+| `gracefulShutdownTimeout`| Time in ms to wait during shutdown                     | 30000       |
 
-### Conclusion
+### IPTablesProxy Settings
 
-`@push.rocks/smartproxy` is designed for both simple and complex proxying demands, offering tools for high-performance and secure proxy management across diverse environments. Its efficient configurations are capable of supporting SSL redirection, WebSocket traffic, dynamic routing, and other advanced functionalities, making it indispensable for developers seeking robust and adaptable proxy solutions. By integrating these capabilities with ease of use, `smartproxy` stands out as an essential tool in modern software architecture.
+| Option            | Description                                 | Default     |
+|-------------------|---------------------------------------------|-------------|
+| `fromPort`        | Source port to forward from                 | -           |
+| `toPort`          | Destination port to forward to              | -           |
+| `toHost`          | Destination host to forward to              | 'localhost' |
+| `preserveSourceIP`| Preserve the original client IP             | false       |
+| `deleteOnExit`    | Remove iptables rules when process exits    | false       |
+
+## Advanced Features
+
+### Connection Management and Monitoring
+
+The `PortProxy` class includes built-in connection tracking and monitoring:
+
+- Automatic cleanup of idle connections
+- Timeouts for connections that exceed maximum lifetime
+- Detailed logging of connection states
+- Termination statistics
+
+### WebSocket Support
+
+The `NetworkProxy` class provides WebSocket support with:
+
+- WebSocket connection proxying
+- Automatic heartbeat monitoring
+- Connection cleanup for inactive WebSockets
+
+### SNI-based Routing
+
+The `PortProxy` class can inspect the SNI (Server Name Indication) field in TLS handshakes to route connections based on the requested domain:
+
+- Multiple backend targets per domain
+- Round-robin load balancing
+- Domain-specific allowed IP ranges
+- Protection against SNI renegotiation attacks
 
 ## License and Legal Information
 

ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartproxy',
-  version: '3.22.4',
+  version: '3.23.0',
   description: 'A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, and dynamic routing with authentication options.'
 }

ts/classes.portproxy.ts

@@ -663,7 +663,7 @@ export class PortProxy {
       const checkInterval = setInterval(() => {
         if (this.connectionRecords.size === 0) {
           clearInterval(checkInterval);
-          resolve();
+          resolve(); // lets resolve here as early as we reach 0 remaining connections
         }
       }, 1000);