serve.zone/dcrouter
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(smartproxy): Update documentation and configuration guides to adopt new route-based SmartProxy architecture

Browse Source
This commit is contained in:
Philipp Kunz 2025-05-16 15:50:46 +00:00
parent 0ad5dfd6ee
commit fb424d814c
8 changed files with 785 additions and 1516 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
changelog.md
View File

@ -1,5 +1,13 @@
# Changelog
## 2025-05-16 - 2.12.0 - feat(smartproxy)
Update documentation and configuration guides to adopt new route-based SmartProxy architecture
- Revise SmartProxy implementation hints in readme.hints.md to describe route-based configuration with glob pattern matching
- Add migration examples showing transition from old direct configuration to new route-based style
- Update DcRouter and SMTP port configuration to generate SmartProxy routes for email handling (ports 25, 587, 465 mapped to internal services)
- Enhance integration documentation with examples for HTTP and email services using the new SmartProxy routes
## 2025-05-16 - 2.11.2 - fix(dependencies)
Update dependency versions and adjust test imports to use new packages

168
readme.hints.md
View File

@ -2,31 +2,76 @@
## SmartProxy Usage
### New Route-Based Architecture (v18+)
- SmartProxy now uses a route-based configuration system
- Routes define match criteria and actions instead of simple port-to-port forwarding
- All traffic types (HTTP, HTTPS, TCP, WebSocket) are configured through routes
```typescript
// NEW: Route-based SmartProxy configuration
const smartProxy = new plugins.smartproxy.SmartProxy({
routes: [
{
name: 'https-traffic',
match: {
ports: 443,
domains: ['example.com', '*.example.com']
},
action: {
type: 'forward',
target: {
host: 'backend.server.com',
port: 8080
}
},
tls: {
mode: 'terminate',
certificate: 'auto'
}
}
],
defaults: {
target: {
host: 'fallback.server.com',
port: 8080
}
},
acme: {
accountEmail: 'admin@example.com',
enabled: true,
useProduction: true
}
});
```
### Migration from Old to New
```typescript
// OLD configuration style (deprecated)
{
fromPort: 443,
toPort: 8080,
targetIP: 'backend.server.com',
domainConfigs: [...]
}
// NEW route-based style
{
routes: [{
name: 'main-route',
match: { ports: 443 },
action: {
type: 'forward',
target: { host: 'backend.server.com', port: 8080 }
}
}]
}
```
### Direct Component Usage
- Use SmartProxy components directly instead of creating your own wrappers
- SmartProxy already includes Port80Handler and NetworkProxy functionality
- When using SmartProxy, configure it directly rather than instantiating Port80Handler or NetworkProxy separately
```typescript
// PREFERRED: Use SmartProxy with built-in ACME support
const smartProxy = new plugins.smartproxy.SmartProxy({
fromPort: 443,
toPort: targetPort,
targetIP: targetServer,
sniEnabled: true,
acme: {
port: 80,
enabled: true,
autoRenew: true,
useProduction: true,
renewThresholdDays: 30,
accountEmail: contactEmail
},
globalPortRanges: [{ from: 443, to: 443 }],
domainConfigs: [/* domain configurations */]
});
```
### Certificate Management
- SmartProxy has built-in ACME certificate management
- Configure it in the `acme` property of SmartProxy options
@ -48,15 +93,48 @@ const value = await qenv.getEnvVarOnDemand('ENV_VAR_NAME');
### SmartProxy Interfaces
- Always check the interfaces from the node_modules to ensure correct property names
- Important interfaces:
- `ISmartProxyOptions`: Main configuration for SmartProxy
- Important interfaces for the new architecture:
- `ISmartProxyOptions`: Main configuration with `routes` array
- `IRouteConfig`: Individual route configuration
- `IRouteMatch`: Match criteria for routes
- `IRouteTarget`: Target configuration for forwarding
- `IAcmeOptions`: ACME certificate configuration
- `IDomainConfig`: Domain-specific configuration
- `TTlsMode`: TLS handling modes ('passthrough' | 'terminate' | 'terminate-and-reencrypt')
### New Route Configuration
```typescript
interface IRouteConfig {
name: string;
match: {
ports: number | number[];
domains?: string | string[];
path?: string;
headers?: Record<string, string | RegExp>;
};
action: {
type: 'forward' | 'redirect' | 'block' | 'static';
target?: {
host: string | string[] | ((context) => string);
port: number | 'preserve' | ((context) => number);
};
};
tls?: {
mode: TTlsMode;
certificate?: 'auto' | { key: string; cert: string; };
};
security?: {
authentication?: IRouteAuthentication;
rateLimit?: IRouteRateLimit;
ipAllowList?: string[];
ipBlockList?: string[];
};
}
```
### Required Properties
- Remember to include all required properties in your interface implementations
- For `ISmartProxyOptions`, `globalPortRanges` is required
- For `ISmartProxyOptions`, `routes` array is the main configuration
- For `IAcmeOptions`, use `accountEmail` for the contact email
- Routes must have `name`, `match`, and `action` properties
## Testing
@ -93,4 +171,42 @@ tap.test('stop', async () => {
### Component Integration
- Leverage built-in integrations between components (like SmartProxy's ACME handling)
- Use parallel operations for performance (like in the `stop()` method)
- Separate concerns clearly (HTTP handling vs. SMTP handling)
- Separate concerns clearly (HTTP handling vs. SMTP handling)
## Email Integration with SmartProxy
### Architecture
- Email traffic is routed through SmartProxy using automatic route generation
- Email server runs on internal ports and receives forwarded traffic from SmartProxy
- SmartProxy handles external ports (25, 587, 465) and forwards to internal ports
### Email Route Generation
```typescript
// Email configuration automatically generates SmartProxy routes
emailConfig: {
ports: [25, 587, 465],
hostname: 'mail.example.com',
domainRules: [...]
}
// Generates routes like:
{
name: 'smtp-route',
match: { ports: [25] },
action: {
type: 'forward',
target: { host: 'localhost', port: 10025 }
},
tls: { mode: 'passthrough' } // STARTTLS handled by email server
}
```
### Port Mapping
- External port 25 → Internal port 10025 (SMTP)
- External port 587 → Internal port 10587 (Submission)
- External port 465 → Internal port 10465 (SMTPS)
### TLS Handling
- Ports 25 and 587: Use 'passthrough' mode (STARTTLS handled by email server)
- Port 465: Use 'terminate' mode (SmartProxy handles TLS termination)
- Domain-specific TLS can be configured per email rule

1662
readme.plan.md
View File

File diff suppressed because it is too large Load Diff

107
readme.smartlog.md
View File

@ -1,107 +0,0 @@
# Smartlog Improvement Plan
## Overview
This document outlines a plan for enhancing the `@push.rocks/smartlog` module to incorporate the advanced features currently implemented in the custom `EnhancedLogger` wrapper. By moving these features directly into `smartlog`, we can eliminate the need for wrapper classes while providing a more comprehensive logging solution.
## Current Limitations in Smartlog
- Limited context management (no hierarchical contexts)
- No correlation ID tracking for distributed tracing
- No built-in filtering or log level management
- No log sampling capabilities
- No middleware for HTTP request/response logging
- No timing utilities for performance tracking
- No child logger functionality with context inheritance
## Proposed Enhancements
### 1. Context Management
- Add hierarchical context support
- Implement methods for manipulating context:
- `setContext(context, overwrite = false)`
- `addToContext(key, value)`
- `removeFromContext(key)`
### 2. Correlation ID Tracking
- Add correlation ID support for distributed tracing
- Implement methods for correlation management:
- `setCorrelationId(id = null)`
- `getCorrelationId()`
- `clearCorrelationId()`
### 3. Log Filtering
- Implement configurable log filtering based on:
- Minimum log level
- Pattern-based exclusion rules
- Custom filtering functions
### 4. Log Sampling
- Add probabilistic log sampling for high-volume environments
- Support for enforcing critical logs (e.g., errors) regardless of sampling
### 5. Child Loggers
- Support creating child loggers with inherited context
- Allow context overrides in child loggers
### 6. Timing Utilities
- Add methods for timing operations:
- `logTimed(level, message, fn, context)`
- Support for both async and sync operations
### 7. HTTP Request Logging
- Add middleware for Express/Fastify/other HTTP frameworks
- Auto-capture request/response data
- Auto-propagate correlation IDs
### 8. Log Standardization
- Ensure consistent output format
- Add standard fields like timestamp, correlation ID
- Support for custom formatters
## Implementation Plan
1. **Core Enhancements**
- Implement context management
- Add correlation ID tracking
- Develop filtering and sampling capabilities
2. **Extended Features**
- Build child logger functionality
- Create timing utility methods
- Implement HTTP middleware
3. **Compatibility**
- Ensure backward compatibility
- Provide migration guide
- Add TypeScript declarations
4. **Documentation**
- Update README with new features
- Add examples for each feature
- Document best practices
## Migration Path
After implementing these enhancements to `smartlog`, the migration would involve:
1. Update to latest `smartlog` version
2. Replace `EnhancedLogger` instances with `smartlog.Smartlog`
3. Update configuration to use new capabilities
4. Replace middleware with `smartlog`'s built-in solutions
## Benefits
- Simplified dependency tree
- Better maintainability with single logging solution
- Improved performance with native implementation
- Enhanced type safety through TypeScript
- Standardized logging across projects

2
ts/00_commitinfo_data.ts
View File

@ -3,6 +3,6 @@
*/
export const commitinfo = {
name: '@serve.zone/platformservice',
version: '2.11.2',
version: '2.12.0',
description: 'A multifaceted platform service handling mail, SMS, letter delivery, and AI services.'
}

252
ts/classes.dcrouter.ts
View File

@ -1,6 +1,5 @@
import * as plugins from './plugins.js';
import * as paths from './paths.js';
import { SmtpPortConfig, type ISmtpPortSettings } from './classes.smtp.portconfig.js';
// Certificate types are available via plugins.tsclass
@ -53,7 +52,7 @@ export interface IDcRouterOptions {
*/
export interface PortProxyRuleContext {
proxy: plugins.smartproxy.SmartProxy;
configs: plugins.smartproxy.IPortProxySettings['domainConfigs'];
routes: plugins.smartproxy.IRouteConfig[];
}
export class DcRouter {
@ -84,17 +83,15 @@ export class DcRouter {
console.log('Starting DcRouter services...');
try {
// Set up SmartProxy for HTTP/HTTPS and general TCP/SNI traffic
if (this.options.smartProxyConfig) {
await this.setupSmartProxy();
}
// Set up SmartProxy for HTTP/HTTPS and all traffic including email routes
await this.setupSmartProxy();
// Set up unified email handling if configured
if (this.options.emailConfig) {
await this.setupUnifiedEmailHandling();
}
// 3. Set up DNS server if configured
// Set up DNS server if configured
if (this.options.dnsServerConfig) {
this.dnsServer = new plugins.smartdns.DnsServer(this.options.dnsServerConfig);
await this.dnsServer.start();
@ -111,41 +108,175 @@ export class DcRouter {
}
/**
* Set up SmartProxy with direct configuration
* Set up SmartProxy with direct configuration and automatic email routes
*/
private async setupSmartProxy(): Promise<void> {
if (!this.options.smartProxyConfig) {
return;
let routes: plugins.smartproxy.IRouteConfig[] = [];
let acmeConfig: plugins.smartproxy.IAcmeOptions | undefined;
// If user provides full SmartProxy config, use it directly
if (this.options.smartProxyConfig) {
routes = this.options.smartProxyConfig.routes || [];
acmeConfig = this.options.smartProxyConfig.acme;
}
console.log('Setting up SmartProxy with direct configuration');
// If email config exists, automatically add email routes
if (this.options.emailConfig) {
const emailRoutes = this.generateEmailRoutes(this.options.emailConfig);
routes = [...routes, ...emailRoutes];
}
// Create SmartProxy instance with full configuration
this.smartProxy = new plugins.smartproxy.SmartProxy(this.options.smartProxyConfig);
// Merge TLS/ACME configuration if provided at root level
if (this.options.tls && !acmeConfig) {
acmeConfig = {
accountEmail: this.options.tls.contactEmail,
enabled: true,
useProduction: true,
autoRenew: true,
renewThresholdDays: 30
};
}
// Set up event listeners
this.smartProxy.on('error', (err) => {
console.error('SmartProxy error:', err);
});
if (this.options.smartProxyConfig.acme) {
this.smartProxy.on('certificate-issued', (event) => {
console.log(`Certificate issued for ${event.domain}, expires ${event.expiryDate}`);
// If we have routes or need a basic SmartProxy instance, create it
if (routes.length > 0 || this.options.smartProxyConfig) {
console.log('Setting up SmartProxy with combined configuration');
// Create SmartProxy configuration
const smartProxyConfig: plugins.smartproxy.ISmartProxyOptions = {
...this.options.smartProxyConfig,
routes,
acme: acmeConfig
};
// Create SmartProxy instance
this.smartProxy = new plugins.smartproxy.SmartProxy(smartProxyConfig);
// Set up event listeners
this.smartProxy.on('error', (err) => {
console.error('SmartProxy error:', err);
});
this.smartProxy.on('certificate-renewed', (event) => {
console.log(`Certificate renewed for ${event.domain}, expires ${event.expiryDate}`);
});
if (acmeConfig) {
this.smartProxy.on('certificate-issued', (event) => {
console.log(`Certificate issued for ${event.domain}, expires ${event.expiryDate}`);
});
this.smartProxy.on('certificate-renewed', (event) => {
console.log(`Certificate renewed for ${event.domain}, expires ${event.expiryDate}`);
});
}
// Start SmartProxy
await this.smartProxy.start();
console.log(`SmartProxy started with ${routes.length} routes`);
}
// Start SmartProxy
await this.smartProxy.start();
console.log('SmartProxy started successfully');
}
/**
* Generate SmartProxy routes for email configuration
*/
private generateEmailRoutes(emailConfig: IEmailConfig): plugins.smartproxy.IRouteConfig[] {
const emailRoutes: plugins.smartproxy.IRouteConfig[] = [];
// Create routes for each email port
for (const port of emailConfig.ports) {
// Handle different email ports differently
switch (port) {
case 25: // SMTP
emailRoutes.push({
name: 'smtp-route',
match: {
ports: [25]
},
action: {
type: 'forward',
target: {
host: 'localhost', // Forward to internal email server
port: 10025 // Internal email server port
},
// No TLS termination for port 25 (STARTTLS handled by email server)
tls: {
mode: 'passthrough'
}
}
});
break;
case 587: // Submission
emailRoutes.push({
name: 'submission-route',
match: {
ports: [587]
},
action: {
type: 'forward',
target: {
host: 'localhost',
port: 10587
},
tls: {
mode: 'passthrough' // STARTTLS handled by email server
}
}
});
break;
case 465: // SMTPS
emailRoutes.push({
name: 'smtps-route',
match: {
ports: [465]
},
action: {
type: 'forward',
target: {
host: 'localhost',
port: 10465
},
tls: {
mode: 'terminate', // Terminate TLS and re-encrypt to email server
certificate: 'auto'
}
}
});
break;
}
}
// Add domain-specific email routes if configured
if (emailConfig.domainRules) {
for (const rule of emailConfig.domainRules) {
// Extract domain from pattern (e.g., "*@example.com" -> "example.com")
const domain = rule.pattern.split('@')[1];
if (domain && rule.mode === 'forward' && rule.target) {
emailRoutes.push({
name: `email-forward-${domain}`,
match: {
ports: emailConfig.ports,
domains: [domain]
},
action: {
type: 'forward',
target: {
host: rule.target.server,
port: rule.target.port || 25
},
tls: {
mode: rule.target.useTls ? 'terminate-and-reencrypt' : 'passthrough'
}
}
});
}
}
}
return emailRoutes;
}
/**
* Check if a domain matches a pattern (including wildcard support)
* @param domain The domain to check
@ -213,7 +344,7 @@ export class DcRouter {
// Update configuration
this.options.smartProxyConfig = config;
// Start new SmartProxy with updated configuration
// Start new SmartProxy with updated configuration (will include email routes if configured)
await this.setupSmartProxy();
console.log('SmartProxy configuration updated');
@ -231,15 +362,31 @@ export class DcRouter {
if (!this.options.emailConfig) {
throw new Error('Email configuration is required for unified email handling');
}
const emailConfig = this.options.emailConfig;
// Map external ports to internal ports
const portMapping = {
25: 10025, // SMTP
587: 10587, // Submission
465: 10465 // SMTPS
};
// Create internal email server configuration
const internalEmailConfig: IEmailConfig = {
...emailConfig,
ports: emailConfig.ports.map(port => portMapping[port] || port + 10000),
hostname: 'localhost' // Listen on localhost for SmartProxy forwarding
};
try {
// Create domain router for pattern matching
this.domainRouter = new DomainRouter({
domainRules: this.options.emailConfig.domainRules,
defaultMode: this.options.emailConfig.defaultMode,
defaultServer: this.options.emailConfig.defaultServer,
defaultPort: this.options.emailConfig.defaultPort,
defaultTls: this.options.emailConfig.defaultTls
domainRules: emailConfig.domainRules,
defaultMode: emailConfig.defaultMode,
defaultServer: emailConfig.defaultServer,
defaultPort: emailConfig.defaultPort,
defaultTls: emailConfig.defaultTls
});
// Initialize the rate limiter
@ -255,11 +402,11 @@ export class DcRouter {
// Initialize the unified delivery queue
const queueOptions: IQueueOptions = {
storageType: this.options.emailConfig.queue?.storageType || 'memory',
persistentPath: this.options.emailConfig.queue?.persistentPath,
maxRetries: this.options.emailConfig.queue?.maxRetries,
baseRetryDelay: this.options.emailConfig.queue?.baseRetryDelay,
maxRetryDelay: this.options.emailConfig.queue?.maxRetryDelay
storageType: emailConfig.queue?.storageType || 'memory',
persistentPath: emailConfig.queue?.persistentPath,
maxRetries: emailConfig.queue?.maxRetries,
baseRetryDelay: emailConfig.queue?.baseRetryDelay,
maxRetryDelay: emailConfig.queue?.maxRetryDelay
};
this.deliveryQueue = new UnifiedDeliveryQueue(queueOptions);
@ -274,18 +421,18 @@ export class DcRouter {
this.deliverySystem = new MultiModeDeliverySystem(this.deliveryQueue, deliveryOptions);
await this.deliverySystem.start();
// Initialize the unified email server
// Initialize the unified email server with internal configuration
this.unifiedEmailServer = new UnifiedEmailServer({
ports: this.options.emailConfig.ports,
hostname: this.options.emailConfig.hostname,
maxMessageSize: this.options.emailConfig.maxMessageSize,
auth: this.options.emailConfig.auth,
tls: this.options.emailConfig.tls,
domainRules: this.options.emailConfig.domainRules,
defaultMode: this.options.emailConfig.defaultMode,
defaultServer: this.options.emailConfig.defaultServer,
defaultPort: this.options.emailConfig.defaultPort,
defaultTls: this.options.emailConfig.defaultTls
ports: internalEmailConfig.ports,
hostname: internalEmailConfig.hostname,
maxMessageSize: emailConfig.maxMessageSize,
auth: emailConfig.auth,
tls: emailConfig.tls,
domainRules: emailConfig.domainRules,
defaultMode: emailConfig.defaultMode,
defaultServer: emailConfig.defaultServer,
defaultPort: emailConfig.defaultPort,
defaultTls: emailConfig.defaultTls
});
// Set up event listeners
@ -303,7 +450,8 @@ export class DcRouter {
// Start the unified email server
await this.unifiedEmailServer.start();
logger.log('info', `Unified email handling configured with ${this.options.emailConfig.domainRules.length} domain rules`);
logger.log('info', `Unified email handling configured with ${emailConfig.domainRules.length} domain rules on internal ports`);
logger.log('info', `Email server listening on ports: ${internalEmailConfig.ports.join(', ')}`);
} catch (error) {
logger.log('error', `Error setting up unified email handling: ${error.message}`);
throw error;

100
ts/classes.smtp.portconfig.ts
View File

@ -239,73 +239,57 @@ export class SmtpPortConfig {
}
/**
* Apply port configurations to SmartProxy settings
* @param smartProxy SmartProxy instance
* Convert port configurations to SmartProxy routes
* @returns Array of SmartProxy routes
*/
public applyToSmartProxy(smartProxy: plugins.smartproxy.SmartProxy): void {
if (!smartProxy) return;
public toSmartProxyRoutes(): plugins.smartproxy.IRouteConfig[] {
const enabledPorts = this.getEnabledPortConfigs();
const settings = smartProxy.settings;
const routes: plugins.smartproxy.IRouteConfig[] = [];
// Initialize globalPortRanges if needed
if (!settings.globalPortRanges) {
settings.globalPortRanges = [];
}
// Add configured ports to globalPortRanges
// Add configured ports as routes
for (const portConfig of enabledPorts) {
// Add port to global port ranges if not already present
if (!settings.globalPortRanges.some((r) => r.from <= portConfig.port && portConfig.port <= r.to)) {
settings.globalPortRanges.push({ from: portConfig.port, to: portConfig.port });
}
// Apply TLS settings at SmartProxy level
// Create a route for each SMTP port
const route: plugins.smartproxy.IRouteConfig = {
name: `smtp-port-${portConfig.port}`,
match: {
ports: [portConfig.port]
},
action: {
type: 'forward',
target: {
host: 'localhost',
port: portConfig.port + 10000 // Map to internal port (e.g., 25 -> 10025)
}
}
};
// Apply TLS settings
if (portConfig.port === 465 && portConfig.tls?.enabled) {
// For implicit TLS on port 465
settings.sniEnabled = true;
route.action.tls = {
mode: 'terminate',
certificate: 'auto'
};
} else if (portConfig.tls?.useStartTls) {
// For STARTTLS on ports 25 and 587
route.action.tls = {
mode: 'passthrough'
};
}
routes.push(route);
}
// Group ports by TLS configuration to log them
const starttlsPorts = enabledPorts
.filter(p => p.tls?.enabled && p.tls?.useStartTls)
.map(p => p.port);
return routes;
}
const implicitTlsPorts = enabledPorts
.filter(p => p.tls?.enabled && !p.tls?.useStartTls)
.map(p => p.port);
const nonTlsPorts = enabledPorts
.filter(p => !p.tls?.enabled)
.map(p => p.port);
if (starttlsPorts.length > 0) {
console.log(`Configured STARTTLS SMTP ports: ${starttlsPorts.join(', ')}`);
}
if (implicitTlsPorts.length > 0) {
console.log(`Configured Implicit TLS SMTP ports: ${implicitTlsPorts.join(', ')}`);
}
if (nonTlsPorts.length > 0) {
console.log(`Configured Plain SMTP ports: ${nonTlsPorts.join(', ')}`);
}
// Setup connection listeners for different port types
smartProxy.on('connection', (connection) => {
const port = connection.localPort;
// Check which type of port this is
if (implicitTlsPorts.includes(port)) {
console.log(`Implicit TLS SMTP connection on port ${port} from ${connection.remoteIP}`);
} else if (starttlsPorts.includes(port)) {
console.log(`STARTTLS SMTP connection on port ${port} from ${connection.remoteIP}`);
} else if (nonTlsPorts.includes(port)) {
console.log(`Plain SMTP connection on port ${port} from ${connection.remoteIP}`);
}
});
console.log(`Applied SMTP port configurations to SmartProxy: ${enabledPorts.map(p => p.port).join(', ')}`);
/**
* Apply port configurations to SmartProxy settings
* @param smartProxy SmartProxy instance
* @deprecated Use toSmartProxyRoutes() instead to generate routes
*/
public applyToSmartProxy(smartProxy: plugins.smartproxy.SmartProxy): void {
console.warn('SmtpPortConfig.applyToSmartProxy() is deprecated. Use toSmartProxyRoutes() instead.');
// This method is deprecated and no longer functional
}
}

2
ts_web/00_commitinfo_data.ts
View File

@ -3,6 +3,6 @@
*/
export const commitinfo = {
name: '@serve.zone/platformservice',
version: '2.11.2',
version: '2.12.0',
description: 'A multifaceted platform service handling mail, SMS, letter delivery, and AI services.'
}