push.rocks/smartproxy
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
smartproxy/readme.md

615 lines
24 KiB
Markdown
Raw Permalink Blame History

# @push.rocks/smartproxy
A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, and dynamic routing with authentication options.
## Architecture & Flow Diagrams
### Component Architecture
The diagram below illustrates the main components of SmartProxy and how they interact:
```mermaid
flowchart TB
Client([Client])
subgraph "SmartProxy Components"
direction TB
HTTP80[HTTP Port 80\nSslRedirect]
HTTPS443[HTTPS Port 443\nNetworkProxy]
PortProxy[TCP Port Proxy\nwith SNI routing]
NfTables[NfTablesProxy]
Router[ProxyRouter]
ACME[Port80Handler\nACME/Let's Encrypt]
Certs[(SSL Certificates)]
end
subgraph "Backend Services"
Service1[Service 1]
Service2[Service 2]
Service3[Service 3]
end
Client -->|HTTP Request| HTTP80
HTTP80 -->|Redirect| Client
Client -->|HTTPS Request| HTTPS443
Client -->|TLS/TCP| PortProxy
HTTPS443 -->|Route Request| Router
Router -->|Proxy Request| Service1
Router -->|Proxy Request| Service2
PortProxy -->|Direct TCP| Service2
PortProxy -->|Direct TCP| Service3
NfTables -.->|Low-level forwarding| PortProxy
HTTP80 -.->|Challenge Response| ACME
ACME -.->|Generate/Manage| Certs
Certs -.->|Provide TLS Certs| HTTPS443
classDef component fill:#f9f,stroke:#333,stroke-width:2px;
classDef backend fill:#bbf,stroke:#333,stroke-width:1px;
classDef client fill:#dfd,stroke:#333,stroke-width:2px;
class Client client;
class HTTP80,HTTPS443,PortProxy,IPTables,Router,ACME component;
class Service1,Service2,Service3 backend;
```
### HTTPS Reverse Proxy Flow
This diagram shows how HTTPS requests are handled and proxied to backend services:
```mermaid
sequenceDiagram
participant Client
participant NetworkProxy
participant ProxyRouter
participant Backend
Client->>NetworkProxy: HTTPS Request
Note over NetworkProxy: TLS Termination
NetworkProxy->>ProxyRouter: Route Request
ProxyRouter->>ProxyRouter: Match hostname to config
alt Authentication Required
NetworkProxy->>Client: Request Authentication
Client->>NetworkProxy: Send Credentials
NetworkProxy->>NetworkProxy: Validate Credentials
end
NetworkProxy->>Backend: Forward Request
Backend->>NetworkProxy: Response
Note over NetworkProxy: Add Default Headers
NetworkProxy->>Client: Forward Response
alt WebSocket Request
Client->>NetworkProxy: Upgrade to WebSocket
NetworkProxy->>Backend: Upgrade to WebSocket
loop WebSocket Active
Client->>NetworkProxy: WebSocket Message
NetworkProxy->>Backend: Forward Message
Backend->>NetworkProxy: WebSocket Message
NetworkProxy->>Client: Forward Message
NetworkProxy-->>NetworkProxy: Heartbeat Check
end
end
```
### Port Proxy with SNI-based Routing
This diagram illustrates how TCP connections with SNI (Server Name Indication) are processed and forwarded:
```mermaid
sequenceDiagram
participant Client
participant PortProxy
participant Backend
Client->>PortProxy: TLS Connection
alt SNI Enabled
PortProxy->>Client: Accept Connection
Client->>PortProxy: TLS ClientHello with SNI
PortProxy->>PortProxy: Extract SNI Hostname
PortProxy->>PortProxy: Match Domain Config
PortProxy->>PortProxy: Validate Client IP
alt IP Allowed
PortProxy->>Backend: Forward Connection
Note over PortProxy,Backend: Bidirectional Data Flow
else IP Rejected
PortProxy->>Client: Close Connection
end
else Port-based Routing
PortProxy->>PortProxy: Match Port Range
PortProxy->>PortProxy: Find Domain Config
PortProxy->>PortProxy: Validate Client IP
alt IP Allowed
PortProxy->>Backend: Forward Connection
Note over PortProxy,Backend: Bidirectional Data Flow
else IP Rejected
PortProxy->>Client: Close Connection
end
end
loop Connection Active
PortProxy-->>PortProxy: Monitor Activity
PortProxy-->>PortProxy: Check Max Lifetime
alt Inactivity or Max Lifetime Exceeded
PortProxy->>Client: Close Connection
PortProxy->>Backend: Close Connection
end
end
```
### Let's Encrypt Certificate Acquisition
This diagram shows how certificates are automatically acquired through the ACME protocol:
```mermaid
sequenceDiagram
participant Client
participant Port80Handler
participant ACME as Let's Encrypt ACME
participant NetworkProxy
Client->>Port80Handler: HTTP Request for domain
alt Certificate Exists
Port80Handler->>Client: Redirect to HTTPS
else No Certificate
Port80Handler->>Port80Handler: Mark domain as obtaining cert
Port80Handler->>ACME: Create account & new order
ACME->>Port80Handler: Challenge information
Port80Handler->>Port80Handler: Store challenge token & key authorization
ACME->>Port80Handler: HTTP-01 Challenge Request
Port80Handler->>ACME: Challenge Response
ACME->>ACME: Validate domain ownership
ACME->>Port80Handler: Challenge validated
Port80Handler->>Port80Handler: Generate CSR
Port80Handler->>ACME: Submit CSR
ACME->>Port80Handler: Issue Certificate
Port80Handler->>Port80Handler: Store certificate & private key
Port80Handler->>Port80Handler: Mark certificate as obtained
Note over Port80Handler,NetworkProxy: Certificate available for use
Client->>Port80Handler: Another HTTP Request
Port80Handler->>Client: Redirect to HTTPS
Client->>NetworkProxy: HTTPS Request
Note over NetworkProxy: Uses new certificate
end
```
## Features
- **HTTPS Reverse Proxy** - Route traffic to backend services based on hostname with TLS termination
- **WebSocket Support** - Full WebSocket proxying with heartbeat monitoring
- **TCP Port Forwarding** - Advanced port forwarding with SNI inspection and domain-based routing
- **Enhanced TLS Handling** - Robust TLS handshake processing with improved certificate error handling
- **HTTP to HTTPS Redirection** - Automatically redirect HTTP requests to HTTPS
- **Let's Encrypt Integration** - Automatic certificate management using ACME protocol
- **IP Filtering** - Control access with IP allow/block lists using glob patterns
- **NfTables Integration** - Direct manipulation of nftables for advanced low-level port forwarding
- **Basic Authentication** - Support for basic auth on proxied routes
- **Connection Management** - Intelligent connection tracking and cleanup with configurable timeouts
- **Browser Compatibility** - Optimized for modern browsers with fixes for common TLS handshake issues
## Installation
```bash
npm install @push.rocks/smartproxy
```
## Usage
### Basic Reverse Proxy Setup
```typescript
import { NetworkProxy } from '@push.rocks/smartproxy';
// Create a reverse proxy listening on port 443
const proxy = new NetworkProxy({
port: 443
});
// Define reverse proxy configurations
const proxyConfigs = [
{
hostName: 'example.com',
destinationIp: '127.0.0.1',
destinationPort: 3000,
publicKey: 'your-cert-content',
privateKey: 'your-key-content'
},
{
hostName: 'api.example.com',
destinationIp: '127.0.0.1',
destinationPort: 4000,
publicKey: 'your-cert-content',
privateKey: 'your-key-content',
// Optional basic auth
authentication: {
type: 'Basic',
user: 'admin',
pass: 'secret'
}
}
];
// Start the proxy and update configurations
(async () => {
await proxy.start();
await proxy.updateProxyConfigs(proxyConfigs);
// Add default headers to all responses
await proxy.addDefaultHeaders({
'Strict-Transport-Security': 'max-age=31536000; includeSubDomains; preload'
});
})();
```
### HTTP to HTTPS Redirection
```typescript
import { SslRedirect } from '@push.rocks/smartproxy';
// Create and start HTTP to HTTPS redirect service on port 80
const redirector = new SslRedirect(80);
redirector.start();
```
### TCP Port Forwarding with Domain-based Routing
```typescript
import { PortProxy } from '@push.rocks/smartproxy';
// Configure port proxy with domain-based routing
const portProxy = new PortProxy({
fromPort: 443,
toPort: 8443,
targetIP: 'localhost', // Default target host
sniEnabled: true, // Enable SNI inspection
// Enhanced reliability settings
initialDataTimeout: 60000, // 60 seconds for initial TLS handshake
socketTimeout: 3600000, // 1 hour socket timeout
maxConnectionLifetime: 3600000, // 1 hour connection lifetime
inactivityTimeout: 3600000, // 1 hour inactivity timeout
maxPendingDataSize: 10 * 1024 * 1024, // 10MB buffer for large TLS handshakes
// Browser compatibility enhancement
enableTlsDebugLogging: false, // Enable for troubleshooting TLS issues
// Port and IP configuration
globalPortRanges: [{ from: 443, to: 443 }],
defaultAllowedIPs: ['*'], // Allow all IPs by default
// Socket optimizations for better connection stability
noDelay: true, // Disable Nagle's algorithm
keepAlive: true, // Enable TCP keepalive
enableKeepAliveProbes: true, // Enhanced keepalive for stability
// Domain-specific routing configuration
domainConfigs: [
{
domains: ['example.com', '*.example.com'], // Glob patterns for matching domains
allowedIPs: ['192.168.1.*'], // Restrict access by IP
blockedIPs: ['192.168.1.100'], // Block specific IPs
targetIPs: ['10.0.0.1', '10.0.0.2'], // Round-robin between multiple targets
portRanges: [{ from: 443, to: 443 }],
connectionTimeout: 7200000 // Domain-specific timeout (2 hours)
}
],
preserveSourceIP: true
});
portProxy.start();
```
### NfTables Port Forwarding
```typescript
import { NfTablesProxy } from '@push.rocks/smartproxy';
// Basic usage - forward single port
const basicProxy = new NfTablesProxy({
fromPort: 80,
toPort: 8080,
toHost: 'localhost',
preserveSourceIP: true,
deleteOnExit: true // Automatically clean up rules on process exit
});
// Forward port ranges
const rangeProxy = new NfTablesProxy({
fromPort: { from: 3000, to: 3010 }, // Forward ports 3000-3010
toPort: { from: 8000, to: 8010 }, // To ports 8000-8010
protocol: 'tcp', // TCP protocol (default)
ipv6Support: true, // Enable IPv6 support
enableLogging: true // Enable detailed logging
});
// Multiple port specifications with IP filtering
const advancedProxy = new NfTablesProxy({
fromPort: [80, 443, { from: 8000, to: 8010 }], // Multiple ports/ranges
toPort: [8080, 8443, { from: 18000, to: 18010 }],
allowedSourceIPs: ['10.0.0.0/8', '192.168.1.0/24'], // Only allow these IPs
bannedSourceIPs: ['192.168.1.100'], // Explicitly block these IPs
useIPSets: true, // Use IP sets for efficient IP management
forceCleanSlate: false // Clean all NfTablesProxy rules before starting
});
// Advanced features: QoS, connection tracking, and NetworkProxy integration
const advancedProxy = new NfTablesProxy({
fromPort: 443,
toPort: 8443,
toHost: 'localhost',
useAdvancedNAT: true, // Use connection tracking for stateful NAT
qos: {
enabled: true,
maxRate: '10mbps', // Limit bandwidth
priority: 1 // Set traffic priority (1-10)
},
netProxyIntegration: {
enabled: true,
redirectLocalhost: true, // Redirect localhost traffic to NetworkProxy
sslTerminationPort: 8443 // Port where NetworkProxy handles SSL
}
});
// Start any of the proxies
await basicProxy.start();
```
### Automatic HTTPS Certificate Management
```typescript
import { Port80Handler } from '@push.rocks/smartproxy';
// Create an ACME handler for Let's Encrypt
const acmeHandler = new Port80Handler();
// Add domains to manage certificates for
acmeHandler.addDomain({
domainName: 'example.com',
sslRedirect: true,
acmeMaintenance: true
});
acmeHandler.addDomain({
domainName: 'api.example.com',
sslRedirect: true,
acmeMaintenance: true
});
// Support for glob pattern domains for routing (certificates not issued for glob patterns)
acmeHandler.addDomain({
domainName: '*.example.com',
sslRedirect: true,
acmeMaintenance: false, // Can't issue certificates for wildcard domains via HTTP-01
forward: { ip: '192.168.1.10', port: 8080 } // Forward requests to this target
});
```
## Configuration Options
### NetworkProxy Options
| Option | Description | Default |
|----------------|---------------------------------------------------|---------|
| `port` | Port to listen on for HTTPS connections | - |
### PortProxy Settings
| Option | Description | Default |
|---------------------------|--------------------------------------------------------|-------------|
| `fromPort` | Port to listen on | - |
| `toPort` | Destination port to forward to | - |
| `targetIP` | Default destination IP if not specified in domainConfig | 'localhost' |
| `sniEnabled` | Enable SNI inspection for TLS connections | false |
| `defaultAllowedIPs` | IP patterns allowed by default | - |
| `defaultBlockedIPs` | IP patterns blocked by default | - |
| `preserveSourceIP` | Preserve the original client IP | false |
| `maxConnectionLifetime` | Maximum time in ms to keep a connection open | 3600000 |
| `initialDataTimeout` | Timeout for initial data/handshake in ms | 60000 |
| `socketTimeout` | Socket inactivity timeout in ms | 3600000 |
| `inactivityTimeout` | Connection inactivity check timeout in ms | 3600000 |
| `inactivityCheckInterval` | How often to check for inactive connections in ms | 60000 |
| `maxPendingDataSize` | Maximum bytes to buffer during connection setup | 10485760 |
| `globalPortRanges` | Array of port ranges to listen on | - |
| `forwardAllGlobalRanges` | Forward all global range connections to targetIP | false |
| `gracefulShutdownTimeout` | Time in ms to wait during shutdown | 30000 |
| `noDelay` | Disable Nagle's algorithm | true |
| `keepAlive` | Enable TCP keepalive | true |
| `keepAliveInitialDelay` | Initial delay before sending keepalive probes in ms | 30000 |
| `enableKeepAliveProbes` | Enable enhanced TCP keep-alive probes | false |
| `enableTlsDebugLogging` | Enable detailed TLS handshake debugging | false |
| `enableDetailedLogging` | Enable detailed connection logging | false |
| `enableRandomizedTimeouts`| Randomize timeouts slightly to prevent thundering herd | true |
### NfTablesProxy Settings
| Option | Description | Default |
|-----------------------|---------------------------------------------------|-------------|
| `fromPort` | Source port(s) or range(s) to forward from | - |
| `toPort` | Destination port(s) or range(s) to forward to | - |
| `toHost` | Destination host to forward to | 'localhost' |
| `preserveSourceIP` | Preserve the original client IP | false |
| `deleteOnExit` | Remove nftables rules when process exits | false |
| `protocol` | Protocol to forward ('tcp', 'udp', or 'all') | 'tcp' |
| `enableLogging` | Enable detailed logging | false |
| `logFormat` | Format for logs ('plain' or 'json') | 'plain' |
| `ipv6Support` | Enable IPv6 support | false |
| `allowedSourceIPs` | Array of IP addresses/CIDR allowed to connect | - |
| `bannedSourceIPs` | Array of IP addresses/CIDR blocked from connecting | - |
| `useIPSets` | Use nftables sets for efficient IP management | true |
| `forceCleanSlate` | Clear all NfTablesProxy rules before starting | false |
| `tableName` | Custom table name | 'portproxy' |
| `maxRetries` | Maximum number of retries for failed commands | 3 |
| `retryDelayMs` | Delay between retries in milliseconds | 1000 |
| `useAdvancedNAT` | Use connection tracking for stateful NAT | false |
| `qos` | Quality of Service options (object) | - |
| `netProxyIntegration` | NetworkProxy integration options (object) | - |
#### NfTablesProxy QoS Options
| Option | Description | Default |
|----------------------|---------------------------------------------------|---------|
| `enabled` | Enable Quality of Service features | false |
| `maxRate` | Maximum bandwidth rate (e.g. "10mbps") | - |
| `priority` | Traffic priority (1-10, 1 is highest) | - |
| `markConnections` | Mark connections for easier management | false |
#### NfTablesProxy NetworkProxy Integration Options
| Option | Description | Default |
|----------------------|---------------------------------------------------|---------|
| `enabled` | Enable NetworkProxy integration | false |
| `redirectLocalhost` | Redirect localhost traffic to NetworkProxy | false |
| `sslTerminationPort` | Port where NetworkProxy handles SSL termination | - |
## Advanced Features
### TLS Handshake Optimization
The enhanced `PortProxy` implementation includes significant improvements for TLS handshake handling:
- Robust SNI extraction with improved error handling
- Increased buffer size for complex TLS handshakes (10MB)
- Longer initial handshake timeout (60 seconds)
- Detection and tracking of TLS connection states
- Optional detailed TLS debug logging for troubleshooting
- Browser compatibility fixes for Chrome certificate errors
```typescript
// Example configuration to solve Chrome certificate errors
const portProxy = new PortProxy({
// ... other settings
initialDataTimeout: 60000, // Give browser more time for handshake
maxPendingDataSize: 10 * 1024 * 1024, // Larger buffer for complex handshakes
enableTlsDebugLogging: true, // Enable when troubleshooting
});
```
### Connection Management and Monitoring
The `PortProxy` class includes built-in connection tracking and monitoring:
- Automatic cleanup of idle connections with configurable timeouts
- Timeouts for connections that exceed maximum lifetime
- Detailed logging of connection states
- Termination statistics
- Randomized timeouts to prevent "thundering herd" problems
- Per-domain timeout configuration
### WebSocket Support
The `NetworkProxy` class provides WebSocket support with:
- WebSocket connection proxying
- Automatic heartbeat monitoring
- Connection cleanup for inactive WebSockets
### SNI-based Routing
The `PortProxy` class can inspect the SNI (Server Name Indication) field in TLS handshakes to route connections based on the requested domain:
- Multiple backend targets per domain
- Round-robin load balancing
- Domain-specific allowed IP ranges
- Protection against SNI renegotiation attacks
### Enhanced NfTables Management
The `NfTablesProxy` class offers advanced capabilities compared to the previous IPTablesProxy:
- Support for multiple port ranges and individual ports
- More efficient IP filtering using nftables sets
- IPv6 support with full feature parity
- Quality of Service (QoS) features including bandwidth limiting and traffic prioritization
- Advanced connection tracking for stateful NAT
- Robust error handling with retry mechanisms
- Structured logging with JSON support
- NetworkProxy integration for SSL termination
- Comprehensive cleanup on shutdown
### Port80Handler with Glob Pattern Support
The `Port80Handler` class now includes support for glob pattern domain matching:
- Supports wildcard domains like `*.example.com` for HTTP request routing
- Detects glob patterns and skips certificate issuance for them
- Smart routing that first attempts exact matches, then tries pattern matching
- Supports forwarding HTTP requests to backend services
- Separate forwarding configuration for ACME challenges
## Troubleshooting
### Browser Certificate Errors
If you experience certificate errors in browsers, especially in Chrome, try these solutions:
1. **Increase Initial Data Timeout**: Set `initialDataTimeout` to 60 seconds or higher
2. **Increase Buffer Size**: Set `maxPendingDataSize` to 10MB or higher
3. **Enable TLS Debug Logging**: Set `enableTlsDebugLogging: true` to troubleshoot handshake issues
4. **Enable Keep-Alive Probes**: Set `enableKeepAliveProbes: true` for better connection stability
5. **Check Certificate Chain**: Ensure your certificate chain is complete and in the correct order
```typescript
// Configuration to fix Chrome certificate errors
const portProxy = new PortProxy({
// ... other settings
initialDataTimeout: 60000,
maxPendingDataSize: 10 * 1024 * 1024,
enableTlsDebugLogging: true,
enableKeepAliveProbes: true
});
```
### Connection Stability
For improved connection stability in high-traffic environments:
1. **Set Appropriate Timeouts**: Use longer timeouts for long-lived connections
2. **Use Domain-Specific Timeouts**: Configure per-domain timeouts for different types of services
3. **Enable TCP Keep-Alive**: Ensure `keepAlive` is set to `true`
4. **Monitor Connection Statistics**: Enable detailed logging to track termination reasons
5. **Fine-tune Inactivity Checks**: Adjust `inactivityCheckInterval` based on your traffic patterns
### IPTables Troubleshooting
If you're experiencing issues with IPTablesProxy:
1. **Enable Detailed Logging**: Set `enableLogging: true` to see all rule operations
2. **Force Clean Slate**: Use `forceCleanSlate: true` to remove any lingering rules
3. **Use Custom Chains**: Enable `addJumpRule: true` for cleaner rule management
4. **Check Permissions**: Ensure your process has sufficient permissions to modify iptables
5. **Verify IPv6 Support**: If using `ipv6Support: true`, ensure ip6tables is available
## License and Legal Information
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
### Company Information
Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
Reference in New Issue View Git Blame Copy Permalink