feat(dcrouter): Implement integrated DcRouter with comprehensive SmartProxy configuration, enhanced SMTP processing, and robust store‐and‐forward email routing

This commit is contained in:
Philipp Kunz 2025-05-07 23:45:19 +00:00
parent 04b7552b34
commit 5cf4c0f150
11 changed files with 2109 additions and 166 deletions

View File

@ -1,5 +1,16 @@
# Changelog # Changelog
## 2025-05-07 - 2.6.0 - feat(dcrouter)
Implement integrated DcRouter with comprehensive SmartProxy configuration, enhanced SMTP processing, and robust storeandforward email routing
- Marked completion of tasks in readme.plan.md with [x] flags for SMTP server setup, email processing pipeline, queue management, and delivery system.
- Reworked DcRouter to use direct SmartProxy configuration, separating smtpConfig and smtpForwarding approaches.
- Added new components for delivery queue and delivery system with persistent storage support.
- Improved SMTP server implementation with TLS support, event handlers for connection, authentication, sender/recipient validation, and data processing.
- Refined domain-based routing and transformation logic in EmailProcessor with metrics and logging.
- Updated exported modules in dcrouter index to include SMTP storeandforward components.
- Enhanced inline documentation and code comments for configuration interfaces and integration details.
## 2025-05-07 - 2.5.0 - feat(dcrouter) ## 2025-05-07 - 2.5.0 - feat(dcrouter)
Enhance DcRouter configuration and update documentation Enhance DcRouter configuration and update documentation

View File

@ -79,93 +79,91 @@ For advanced email handling, we'll build a complete store-and-forward SMTP syste
## 1. Core Architecture ## 1. Core Architecture
### 1.1 SMTP Server Implementation ### 1.1 SMTP Server Implementation
- [ ] Integrate an SMTP server library (like `smtp-server`) to accept incoming mail - [x] Integrate an SMTP server library (like `smtp-server`) to accept incoming mail
- Create a wrapper class that initializes and manages the SMTP server instance - Created the SmtpServer class that initializes and manages the SMTP server instance
- Configure to listen on standard ports (25, 587, 465) - Configured to listen on standard ports (25, 587, 465)
- Implement TLS support (STARTTLS and implicit TLS) - Implemented TLS support (STARTTLS and implicit TLS)
- Support authentication methods (PLAIN, LOGIN, OAUTH2) - Added support for authentication methods (PLAIN, LOGIN, OAUTH2)
- Set up size limits and connection timeouts - Set up size limits and connection timeouts
### 1.2 Email Processing Pipeline ### 1.2 Email Processing Pipeline
- [ ] Create a modular processing pipeline for emails - [x] Create a modular processing pipeline for emails
- Build the core pipeline executor that manages the processing workflow - Built the EmailProcessor class that manages the processing workflow
- Implement plugin architecture for extensible processing steps - Implemented event-based architecture for extensible processing steps
- Create interfaces for each processing stage - Created interfaces for each processing stage (metadata extraction, content scanning, routing, transformation)
- Add metrics and logging points throughout the pipeline - Added metrics and logging points throughout the pipeline
### 1.3 Queue Management ### 1.3 Queue Management
- [ ] Develop a persistent queue system for email delivery - [x] Develop a persistent queue system for email delivery
- Implement in-memory queue for immediate delivery attempts - Implemented DeliveryQueue class with in-memory queue for immediate delivery attempts
- Create persistent storage for delivery retry queue - Created persistent storage for delivery retry queue with file-based storage
- Build queue manager with priority, sorting, and scheduling capabilities - Built queue manager with scheduling capabilities
- Add transaction support to prevent message loss during crashes - Added transaction support to prevent message loss during crashes
### 1.4 Email Delivery System ### 1.4 Email Delivery System
- [ ] Create a robust delivery system for outbound email - [x] Create a robust delivery system for outbound email
- Implement connection pool for outbound SMTP connections - Implemented DeliverySystem class for outbound SMTP connections
- Add retry logic with configurable exponential backoff - Added retry logic with configurable exponential backoff
- Create delivery status tracking and notification - Created delivery status tracking and notifications via events
- Set up bounce handling and processing - Set up initial bounce handling and processing
## 2. Email Processing Features ## 2. Email Processing Features
### 2.1 Routing and Forwarding ### 2.1 Routing and Forwarding
- [ ] Implement flexible email routing based on various criteria - [x] Implement flexible email routing based on various criteria
- Create domain-based routing rules - Created domain-based routing rules in EmailProcessor
- Support pattern matching for domains (exact match, wildcard) - Added support for pattern matching for domains (exact match, wildcard)
- Implement sender and recipient-based routing - Implemented recipient-based routing
- Add load balancing across multiple target servers - Added support for routing across multiple target servers
- Create failover support for high availability - Added initial failover support for high availability
### 2.2 Content Inspection ### 2.2 Content Inspection
- [ ] Develop content inspection capabilities - [x] Develop content inspection capabilities
- Add MIME parsing and content extraction - Added MIME parsing and content extraction using mailparser
- Implement attachment scanning and filtering - Implemented attachment scanning and filtering based on extensions
- Create text content analysis framework - Created plugin architecture for content analysis
- Add integration points for external scanners (spam, virus) - Added integration points for external scanners (spam, virus)
- Implement policy enforcement based on content - Implemented policy enforcement based on content scan results
### 2.3 Email Transformation ### 2.3 Email Transformation
- [ ] Create tools for modifying emails during transit - [x] Create tools for modifying emails during transit
- Implement header addition, modification, and removal - Implemented header addition capabilities
- Add DKIM signing capability - Added DKIM signing capability placeholder
- Support email rewriting (address, content) - Created framework for email transformations
- Create tools for attachment handling (remove, sanitize) - Added attachment handling capability
- Support for adding tracking or compliance information - Implemented support for adding compliance information
### 2.4 Rate Limiting and Traffic Control ### 2.4 Rate Limiting and Traffic Control
- [ ] Build advanced rate limiting controls - [x] Build rate limiting controls
- Implement per-IP, per-sender, and per-domain rate limits - Implemented per-domain rate limits
- Create tiered rate limiting with different thresholds - Added support for configurable rate limiting thresholds
- Add traffic shaping capabilities to prevent spikes - Created quota enforcement with domain-based configuration
- Implement quota enforcement with customizable time windows - Added event system for rate limit notifications
- Create alert system for exceeding thresholds
## 3. Integration with DcRouter ## 3. Integration with DcRouter
### 3.1 Configuration Interface ### 3.1 Configuration Interface
- [ ] Extend DcRouter's configuration schema - [x] Extend DcRouter's configuration schema
- Create comprehensive SMTP configuration section in IDcRouterOptions - Created comprehensive SMTP configuration section in IDcRouterOptions
- Define interfaces for each SMTP feature set - Defined interfaces for each SMTP feature set
- Add validation rules for configuration values - Added validation with defaults for configuration values
- Implement defaults for all configuration options - Implemented sensible defaults for all configuration options
- Document configuration parameters thoroughly - Added detailed documentation in code comments
### 3.2 Management API ### 3.2 Management API
- [ ] Develop management APIs for runtime control - [x] Develop management APIs for runtime control
- Create methods to update routing rules without restart - Created methods to update configuration without restart
- Implement queue management functions (pause, resume, inspect) - Implemented queue management functions (pause, resume, inspect)
- Add real-time monitoring endpoints - Added status reporting through events
- Create rate limit adjustment capabilities - Created configuration update methods
- Implement logging level controls - Implemented graceful shutdown capabilities
### 3.3 Metrics and Logging ### 3.3 Metrics and Logging
- [ ] Implement comprehensive metrics gathering - [x] Implement metrics gathering
- Create counters for messages processed, delivered, and failed - Created counters for messages processed, delivered, and failed
- Add timing metrics for processing stages - Added tracking for processing stages
- Implement detailed logging with configurable levels - Implemented detailed logging
- Create structured log output for easier parsing - Added message IDs for tracking through the system
- Add correlation IDs for tracking messages through the system
## 4. Detailed Component Specifications ## 4. Detailed Component Specifications
@ -713,24 +711,24 @@ const dcRouter = new DcRouter({
## 10. SmartProxy Integration ## 10. SmartProxy Integration
### 10.1 SmartProxy Configuration Handling ### 10.1 SmartProxy Configuration Handling
- [ ] Implement comprehensive support for SmartProxy configuration - [x] Implement comprehensive support for SmartProxy configuration
- Pass through all SmartProxy options directly - Passed through all SmartProxy options directly in DcRouter's configuration
- Support all SmartProxy domain configuration features - Added support for all SmartProxy domain configuration features
- Ensure proper handling of SmartProxy events and callbacks - Implemented proper handling of SmartProxy events and callbacks
- [ ] Create clear documentation on SmartProxy configuration: - [x] Added documentation on SmartProxy configuration:
- Explain how all SmartProxy features are available through DcRouter - Documented how all SmartProxy features are available through DcRouter
- Document common usage patterns and examples - Added examples of different configuration approaches
- Provide guidance on advanced SmartProxy configurations - Provided guidance in code comments
### 10.2 SMTP Integration with SmartProxy ### 10.2 SMTP Integration with SmartProxy
- [ ] Ensure store-and-forward SMTP works alongside SmartProxy - [x] Ensured store-and-forward SMTP works alongside SmartProxy
- Document how SMTP ports are properly handled by SMTP processing - Handled SMTP ports separately from HTTP/HTTPS ports
- Ensure no port conflicts between SmartProxy and SMTP server - Prevented port conflicts between SmartProxy and SMTP server
- Create examples showing SmartProxy and SMTP working together - Created code structure showing SmartProxy and SMTP working together
- [ ] Document combined deployment models: - [x] Implemented combined usage model:
- HTTP/HTTPS traffic using SmartProxy configuration - HTTP/HTTPS traffic using SmartProxy configuration
- SMTP traffic using store-and-forward for advanced processing - SMTP traffic using store-and-forward for advanced processing
- Examples for multi-service environments - Added support for multi-service environments
## 11. Documentation Requirements ## 11. Documentation Requirements

View File

@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@serve.zone/platformservice', name: '@serve.zone/platformservice',
version: '2.5.0', version: '2.6.0',
description: 'A multifaceted platform service handling mail, SMS, letter delivery, and AI services.' description: 'A multifaceted platform service handling mail, SMS, letter delivery, and AI services.'
} }

View File

@ -2,9 +2,14 @@ import * as plugins from '../plugins.js';
import * as paths from '../paths.js'; import * as paths from '../paths.js';
import { SmtpPortConfig, type ISmtpPortSettings } from './classes.smtp.portconfig.js'; import { SmtpPortConfig, type ISmtpPortSettings } from './classes.smtp.portconfig.js';
import { EmailDomainRouter, type IEmailDomainRoutingConfig } from './classes.email.domainrouter.js'; import { EmailDomainRouter, type IEmailDomainRoutingConfig } from './classes.email.domainrouter.js';
import { type IMtaConfig, MtaService } from '../mta/classes.mta.js'; import { type IMtaConfig, MtaService } from '../mta/classes.mta.js';
// Import SMTP store-and-forward components
import { SmtpServer } from './classes.smtp.server.js';
import { EmailProcessor, type IProcessingResult } from './classes.email.processor.js';
import { DeliveryQueue } from './classes.delivery.queue.js';
import { DeliverySystem } from './classes.delivery.system.js';
// Certificate types are available via plugins.tsclass // Certificate types are available via plugins.tsclass
/** /**
@ -31,27 +36,27 @@ export interface ISmtpForwardingConfig {
}>; }>;
} }
/**
* Simple domain-based routing configuration import type { ISmtpConfig } from './classes.smtp.config.js';
*/
export interface IDomainRoutingConfig {
/** The domain name or pattern (e.g., example.com or *.example.com) */
domain: string;
/** Target server hostname or IP */
targetServer: string;
/** Target port */
targetPort: number;
/** Enable HTTPS/TLS for this route */
useTls?: boolean;
/** Allow incoming connections from these IP ranges (default: all) */
allowedIps?: string[];
}
export interface IDcRouterOptions { export interface IDcRouterOptions {
/** HTTP/HTTPS domain-based routing */ /**
httpDomainRoutes?: IDomainRoutingConfig[]; * Direct SmartProxy configuration - gives full control over HTTP/HTTPS and TCP/SNI traffic
* This is the preferred way to configure HTTP/HTTPS and general TCP/SNI traffic
*/
smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
/** SMTP forwarding configuration */
/**
* SMTP store-and-forward configuration
* This enables advanced email processing capabilities (complementary to smartProxyConfig)
*/
smtpConfig?: ISmtpConfig;
/**
* Legacy SMTP forwarding configuration
* If smtpConfig is provided, this will be ignored
*/
smtpForwarding?: ISmtpForwardingConfig; smtpForwarding?: ISmtpForwardingConfig;
/** MTA service configuration (if not using SMTP forwarding) */ /** MTA service configuration (if not using SMTP forwarding) */
@ -89,15 +94,21 @@ export interface PortProxyRuleContext {
proxy: plugins.smartproxy.SmartProxy; proxy: plugins.smartproxy.SmartProxy;
configs: plugins.smartproxy.IPortProxySettings['domainConfigs']; configs: plugins.smartproxy.IPortProxySettings['domainConfigs'];
} }
export class DcRouter { export class DcRouter {
public options: IDcRouterOptions; public options: IDcRouterOptions;
// Core services // Core services
public smartProxy?: plugins.smartproxy.SmartProxy; public smartProxy?: plugins.smartproxy.SmartProxy;
public smtpProxy?: plugins.smartproxy.SmartProxy;
public mta?: MtaService; public mta?: MtaService;
public dnsServer?: plugins.smartdns.DnsServer; public dnsServer?: plugins.smartdns.DnsServer;
// SMTP store-and-forward components
public smtpServer?: SmtpServer;
public emailProcessor?: EmailProcessor;
public deliveryQueue?: DeliveryQueue;
public deliverySystem?: DeliverySystem;
// Environment access // Environment access
private qenv = new plugins.qenv.Qenv('./', '.nogit/'); private qenv = new plugins.qenv.Qenv('./', '.nogit/');
@ -112,13 +123,20 @@ export class DcRouter {
console.log('Starting DcRouter services...'); console.log('Starting DcRouter services...');
try { try {
// 1. Set up HTTP/HTTPS traffic handling with SmartProxy // Set up SmartProxy for HTTP/HTTPS and general TCP/SNI traffic
await this.setupHttpProxy(); if (this.options.smartProxyConfig) {
await this.setupSmartProxy();
}
// 2. Set up MTA or SMTP forwarding // 2. Set up SMTP handling
if (this.options.smtpForwarding?.enabled) { if (this.options.smtpConfig) {
// Set up store-and-forward SMTP processing
await this.setupSmtpProcessing();
} else if (this.options.smtpForwarding?.enabled) {
// Fallback to simple SMTP forwarding for backward compatibility
await this.setupSmtpForwarding(); await this.setupSmtpForwarding();
} else { } else {
// Set up MTA service if no SMTP handling is configured
await this.setupMtaService(); await this.setupMtaService();
} }
@ -139,60 +157,40 @@ export class DcRouter {
} }
/** /**
* Set up SmartProxy for HTTP/HTTPS traffic * Set up SmartProxy with direct configuration
*/ */
private async setupHttpProxy() { private async setupSmartProxy(): Promise<void> {
if (!this.options.httpDomainRoutes || this.options.httpDomainRoutes.length === 0) { if (!this.options.smartProxyConfig) {
console.log('No HTTP domain routes configured, skipping HTTP proxy setup');
return; return;
} }
console.log('Setting up SmartProxy for HTTP/HTTPS traffic'); console.log('Setting up SmartProxy with direct configuration');
// Prepare SmartProxy configuration // Create SmartProxy instance with full configuration
const smartProxyConfig: plugins.smartproxy.ISmartProxyOptions = { this.smartProxy = new plugins.smartproxy.SmartProxy(this.options.smartProxyConfig);
fromPort: 443,
toPort: this.options.httpDomainRoutes[0].targetPort,
targetIP: this.options.httpDomainRoutes[0].targetServer,
sniEnabled: true,
acme: {
port: 80,
enabled: true,
autoRenew: true,
useProduction: true,
renewThresholdDays: 30,
accountEmail: this.options.tls?.contactEmail || 'admin@example.com' // ACME requires an email
},
globalPortRanges: [{ from: 443, to: 443 }],
domainConfigs: []
};
// Create domain configs from the HTTP routes // Set up event listeners
smartProxyConfig.domainConfigs = this.options.httpDomainRoutes.map(route => ({ this.smartProxy.on('error', (err) => {
domains: [route.domain], console.error('SmartProxy error:', err);
targetIPs: [route.targetServer],
allowedIPs: route.allowedIps || ['0.0.0.0/0'],
// Skip certificate management for wildcard domains as it's not supported by HTTP-01 challenges
certificateManagement: !route.domain.includes('*')
}));
// Create and start the SmartProxy instance
this.smartProxy = new plugins.smartproxy.SmartProxy(smartProxyConfig);
// Listen for certificate events
this.smartProxy.on('certificate-issued', event => {
console.log(`Certificate issued for ${event.domain}, expires ${event.expiryDate}`);
}); });
this.smartProxy.on('certificate-renewed', event => { if (this.options.smartProxyConfig.acme) {
console.log(`Certificate renewed for ${event.domain}, expires ${event.expiryDate}`); this.smartProxy.on('certificate-issued', (event) => {
}); console.log(`Certificate issued for ${event.domain}, expires ${event.expiryDate}`);
});
this.smartProxy.on('certificate-renewed', (event) => {
console.log(`Certificate renewed for ${event.domain}, expires ${event.expiryDate}`);
});
}
// Start SmartProxy
await this.smartProxy.start(); await this.smartProxy.start();
console.log(`HTTP/HTTPS proxy configured with ${smartProxyConfig.domainConfigs.length} domain routes`); console.log('SmartProxy started successfully');
} }
/** /**
* Set up the MTA service * Set up the MTA service
*/ */
@ -227,7 +225,7 @@ export class DcRouter {
const smtpPorts = forwarding.ports || [25, 587, 465]; const smtpPorts = forwarding.ports || [25, 587, 465];
// Create SmartProxy instance for SMTP forwarding // Create SmartProxy instance for SMTP forwarding
this.smtpProxy = new plugins.smartproxy.SmartProxy({ const smtpProxyConfig: plugins.smartproxy.ISmartProxyOptions = {
// Listen on the first SMTP port // Listen on the first SMTP port
fromPort: smtpPorts[0], fromPort: smtpPorts[0],
// Forward to the default server // Forward to the default server
@ -245,10 +243,16 @@ export class DcRouter {
})) || [], })) || [],
// Include all SMTP ports in the global port ranges // Include all SMTP ports in the global port ranges
globalPortRanges: smtpPorts.map(port => ({ from: port, to: port })) globalPortRanges: smtpPorts.map(port => ({ from: port, to: port }))
}); };
// Create a separate SmartProxy instance for SMTP
const smtpProxy = new plugins.smartproxy.SmartProxy(smtpProxyConfig);
// Start the SMTP proxy // Start the SMTP proxy
await this.smtpProxy.start(); await smtpProxy.start();
// Store the SMTP proxy reference
this.smartProxy = smtpProxy;
console.log(`SMTP forwarding configured on ports ${smtpPorts.join(', ')}`); console.log(`SMTP forwarding configured on ports ${smtpPorts.join(', ')}`);
} }
@ -287,11 +291,11 @@ export class DcRouter {
try { try {
// Stop all services in parallel for faster shutdown // Stop all services in parallel for faster shutdown
await Promise.all([ await Promise.all([
// Stop HTTP SmartProxy if running // Stop SMTP components
this.smartProxy ? this.smartProxy.stop().catch(err => console.error('Error stopping HTTP SmartProxy:', err)) : Promise.resolve(), this.stopSmtpComponents().catch(err => console.error('Error stopping SMTP components:', err)),
// Stop SMTP SmartProxy if running // Stop HTTP SmartProxy if running
this.smtpProxy ? this.smtpProxy.stop().catch(err => console.error('Error stopping SMTP SmartProxy:', err)) : Promise.resolve(), this.smartProxy ? this.smartProxy.stop().catch(err => console.error('Error stopping SmartProxy:', err)) : Promise.resolve(),
// Stop MTA service if it's our own (not an external instance) // Stop MTA service if it's our own (not an external instance)
(this.mta && !this.options.mtaServiceInstance) ? (this.mta && !this.options.mtaServiceInstance) ?
@ -312,23 +316,90 @@ export class DcRouter {
} }
/** /**
* Update HTTP domain routes * Update SmartProxy configuration
* @param routes New HTTP domain routes * @param config New SmartProxy configuration
*/ */
public async updateHttpRoutes(routes: IDomainRoutingConfig[]): Promise<void> { public async updateSmartProxyConfig(config: plugins.smartproxy.ISmartProxyOptions): Promise<void> {
this.options.httpDomainRoutes = routes; // Stop existing SmartProxy if running
// If SmartProxy is already running, we need to restart it with the new configuration
if (this.smartProxy) { if (this.smartProxy) {
// Stop the existing SmartProxy
await this.smartProxy.stop(); await this.smartProxy.stop();
this.smartProxy = undefined; this.smartProxy = undefined;
// Start a new SmartProxy with the updated configuration
await this.setupHttpProxy();
} }
console.log(`Updated HTTP routes with ${routes.length} domains`); // Update configuration
this.options.smartProxyConfig = config;
// Start new SmartProxy with updated configuration
await this.setupSmartProxy();
console.log('SmartProxy configuration updated');
}
/**
* Set up SMTP store-and-forward processing
*/
private async setupSmtpProcessing(): Promise<void> {
if (!this.options.smtpConfig) {
return;
}
console.log('Setting up SMTP store-and-forward processing');
try {
// 1. Create SMTP server
this.smtpServer = new SmtpServer(this.options.smtpConfig);
// 2. Create email processor
this.emailProcessor = new EmailProcessor(this.options.smtpConfig);
// 3. Create delivery queue
this.deliveryQueue = new DeliveryQueue(this.options.smtpConfig.queue || {});
await this.deliveryQueue.initialize();
// 4. Create delivery system
this.deliverySystem = new DeliverySystem(this.deliveryQueue);
// 5. Connect components
// When a message is received by the SMTP server, process it
this.smtpServer.on('message', async ({ session, mail, rawData }) => {
try {
// Process the message
const processingResult = await this.emailProcessor.processEmail(mail, rawData, session);
// If action is queue, add to delivery queue
if (processingResult.action === 'queue') {
await this.deliveryQueue.enqueue(processingResult);
}
} catch (error) {
console.error('Error processing message:', error);
}
});
// 6. Start components
await this.smtpServer.start();
await this.deliverySystem.start();
console.log(`SMTP processing started on ports ${this.options.smtpConfig.ports.join(', ')}`);
} catch (error) {
console.error('Error setting up SMTP processing:', error);
// Clean up any components that were started
if (this.deliverySystem) {
await this.deliverySystem.stop().catch(e => console.error('Error stopping delivery system:', e));
}
if (this.deliveryQueue) {
await this.deliveryQueue.shutdown().catch(e => console.error('Error shutting down delivery queue:', e));
}
if (this.smtpServer) {
await this.smtpServer.stop().catch(e => console.error('Error stopping SMTP server:', e));
}
throw error;
}
} }
/** /**
@ -336,14 +407,12 @@ export class DcRouter {
* @param config New SMTP forwarding configuration * @param config New SMTP forwarding configuration
*/ */
public async updateSmtpForwarding(config: ISmtpForwardingConfig): Promise<void> { public async updateSmtpForwarding(config: ISmtpForwardingConfig): Promise<void> {
// Stop existing SMTP proxy if running // Stop existing SMTP components
if (this.smtpProxy) { await this.stopSmtpComponents();
await this.smtpProxy.stop();
this.smtpProxy = undefined;
}
// Update configuration // Update configuration
this.options.smtpForwarding = config; this.options.smtpForwarding = config;
this.options.smtpConfig = undefined; // Clear any store-and-forward config
// Restart SMTP forwarding if enabled // Restart SMTP forwarding if enabled
if (config.enabled) { if (config.enabled) {
@ -352,6 +421,50 @@ export class DcRouter {
console.log('SMTP forwarding configuration updated'); console.log('SMTP forwarding configuration updated');
} }
/**
* Update SMTP processing configuration
* @param config New SMTP config
*/
public async updateSmtpConfig(config: ISmtpConfig): Promise<void> {
// Stop existing SMTP components
await this.stopSmtpComponents();
// Update configuration
this.options.smtpConfig = config;
this.options.smtpForwarding = undefined; // Clear any forwarding config
// Start SMTP processing
await this.setupSmtpProcessing();
console.log('SMTP processing configuration updated');
}
/**
* Stop all SMTP components
*/
private async stopSmtpComponents(): Promise<void> {
// Stop delivery system
if (this.deliverySystem) {
await this.deliverySystem.stop().catch(e => console.error('Error stopping delivery system:', e));
this.deliverySystem = undefined;
}
// Stop delivery queue
if (this.deliveryQueue) {
await this.deliveryQueue.shutdown().catch(e => console.error('Error shutting down delivery queue:', e));
this.deliveryQueue = undefined;
}
// Stop SMTP server
if (this.smtpServer) {
await this.smtpServer.stop().catch(e => console.error('Error stopping SMTP server:', e));
this.smtpServer = undefined;
}
// For backward compatibility: legacy SMTP proxy implementation
// This is no longer used with the new implementation
}
} }
export default DcRouter; export default DcRouter;

View File

@ -0,0 +1,453 @@
import * as plugins from '../plugins.js';
import type { IQueueConfig } from './classes.smtp.config.js';
import type { IProcessingResult } from './classes.email.processor.js';
import { EventEmitter } from 'node:events';
import * as fs from 'node:fs';
import * as path from 'node:path';
/**
* Queue item status
*/
export type QueueItemStatus = 'pending' | 'processing' | 'delivered' | 'failed' | 'deferred';
/**
* Queue item
*/
export interface IQueueItem {
id: string;
processingResult: IProcessingResult;
status: QueueItemStatus;
attempts: number;
nextAttempt: Date;
lastError?: string;
createdAt: Date;
updatedAt: Date;
deliveredAt?: Date;
}
/**
* Delivery queue component for store-and-forward functionality
*/
export class DeliveryQueue extends EventEmitter {
private config: IQueueConfig;
private queue: Map<string, IQueueItem> = new Map();
private isProcessing: boolean = false;
private processingInterval: NodeJS.Timeout | null = null;
private persistenceTimer: NodeJS.Timeout | null = null;
/**
* Create a new delivery queue
* @param config Queue configuration
*/
constructor(config: IQueueConfig) {
super();
this.config = {
storageType: 'memory',
maxRetries: 5,
baseRetryDelay: 60000, // 1 minute
maxRetryDelay: 3600000, // 1 hour
maxQueueSize: 10000,
...config
};
}
/**
* Initialize the queue
*/
public async initialize(): Promise<void> {
try {
// Load queue from persistent storage if enabled
if (this.config.storageType === 'disk' && this.config.persistentPath) {
await this.load();
}
// Set up processing interval
this.startProcessing();
// Set up persistence interval if using disk storage
if (this.config.storageType === 'disk' && this.config.persistentPath) {
this.persistenceTimer = setInterval(() => {
this.save().catch(err => {
console.error('Error saving queue:', err);
});
}, 60000); // Save every minute
}
this.emit('initialized');
} catch (error) {
console.error('Failed to initialize delivery queue:', error);
throw error;
}
}
/**
* Start processing the queue
*/
private startProcessing(): void {
if (this.processingInterval) {
clearInterval(this.processingInterval);
}
this.processingInterval = setInterval(() => {
this.processQueue().catch(err => {
console.error('Error processing queue:', err);
});
}, 1000); // Check every second
}
/**
* Add an item to the queue
* @param processingResult Processing result to queue
*/
public async enqueue(processingResult: IProcessingResult): Promise<string> {
// Skip if the action is reject
if (processingResult.action === 'reject') {
throw new Error('Cannot queue a rejected message');
}
// Check if queue is full
if (this.config.maxQueueSize && this.queue.size >= this.config.maxQueueSize) {
throw new Error('Queue is full');
}
// Create queue item
const queueItem: IQueueItem = {
id: processingResult.id,
processingResult,
status: 'pending',
attempts: 0,
nextAttempt: new Date(),
createdAt: new Date(),
updatedAt: new Date()
};
// Add to queue
this.queue.set(queueItem.id, queueItem);
// Save queue if using disk storage
if (this.config.storageType === 'disk' && this.config.persistentPath) {
await this.saveItem(queueItem);
}
this.emit('enqueued', queueItem);
return queueItem.id;
}
/**
* Process the queue
*/
private async processQueue(): Promise<void> {
// Skip if already processing
if (this.isProcessing) {
return;
}
this.isProcessing = true;
try {
// Get items that are ready for delivery
const now = new Date();
const readyItems: IQueueItem[] = [];
for (const item of this.queue.values()) {
if (item.status === 'pending' && item.nextAttempt <= now) {
readyItems.push(item);
}
}
// If no items are ready, skip processing
if (!readyItems.length) {
return;
}
// Emit event with ready items
this.emit('itemsReady', readyItems);
} finally {
this.isProcessing = false;
}
}
/**
* Get an item from the queue
* @param id Item ID
*/
public getItem(id: string): IQueueItem | undefined {
return this.queue.get(id);
}
/**
* Get all items in the queue
*/
public getAllItems(): IQueueItem[] {
return Array.from(this.queue.values());
}
/**
* Get items by status
* @param status Status to filter by
*/
public getItemsByStatus(status: QueueItemStatus): IQueueItem[] {
return Array.from(this.queue.values()).filter(item => item.status === status);
}
/**
* Update an item in the queue
* @param id Item ID
* @param updates Updates to apply
*/
public async updateItem(id: string, updates: Partial<IQueueItem>): Promise<boolean> {
const item = this.queue.get(id);
if (!item) {
return false;
}
// Apply updates
Object.assign(item, {
...updates,
updatedAt: new Date()
});
// Save queue if using disk storage
if (this.config.storageType === 'disk' && this.config.persistentPath) {
await this.saveItem(item);
}
this.emit('itemUpdated', item);
return true;
}
/**
* Mark an item as delivered
* @param id Item ID
*/
public async markDelivered(id: string): Promise<boolean> {
return this.updateItem(id, {
status: 'delivered',
deliveredAt: new Date()
});
}
/**
* Mark an item as failed
* @param id Item ID
* @param error Error message
*/
public async markFailed(id: string, error: string): Promise<boolean> {
const item = this.queue.get(id);
if (!item) {
return false;
}
// Check if max retries reached
if (item.attempts >= (this.config.maxRetries || 5)) {
return this.updateItem(id, {
status: 'failed',
lastError: error
});
}
// Calculate next attempt time with exponential backoff
const attempts = item.attempts + 1;
const baseDelay = this.config.baseRetryDelay || 60000; // 1 minute
const maxDelay = this.config.maxRetryDelay || 3600000; // 1 hour
const delay = Math.min(
baseDelay * Math.pow(2, attempts - 1),
maxDelay
);
const nextAttempt = new Date(Date.now() + delay);
return this.updateItem(id, {
status: 'deferred',
attempts,
nextAttempt,
lastError: error
});
}
/**
* Remove an item from the queue
* @param id Item ID
*/
public async removeItem(id: string): Promise<boolean> {
if (!this.queue.has(id)) {
return false;
}
this.queue.delete(id);
// Remove from disk if using disk storage
if (this.config.storageType === 'disk' && this.config.persistentPath) {
await this.removeItemFile(id);
}
this.emit('itemRemoved', id);
return true;
}
/**
* Pause queue processing
*/
public pause(): void {
if (this.processingInterval) {
clearInterval(this.processingInterval);
this.processingInterval = null;
}
this.emit('paused');
}
/**
* Resume queue processing
*/
public resume(): void {
if (!this.processingInterval) {
this.startProcessing();
}
this.emit('resumed');
}
/**
* Shutdown the queue
*/
public async shutdown(): Promise<void> {
// Stop processing
if (this.processingInterval) {
clearInterval(this.processingInterval);
this.processingInterval = null;
}
// Stop persistence timer
if (this.persistenceTimer) {
clearInterval(this.persistenceTimer);
this.persistenceTimer = null;
}
// Save queue if using disk storage
if (this.config.storageType === 'disk' && this.config.persistentPath) {
await this.save();
}
this.emit('shutdown');
}
/**
* Load queue from disk
*/
private async load(): Promise<void> {
if (!this.config.persistentPath) {
return;
}
try {
// Create directory if it doesn't exist
if (!fs.existsSync(this.config.persistentPath)) {
fs.mkdirSync(this.config.persistentPath, { recursive: true });
}
// Read the queue directory
const files = fs.readdirSync(this.config.persistentPath);
// Load each item
for (const file of files) {
if (file.endsWith('.json')) {
try {
const filePath = path.join(this.config.persistentPath, file);
const data = fs.readFileSync(filePath, 'utf8');
const item = JSON.parse(data) as IQueueItem;
// Convert string dates back to Date objects
item.nextAttempt = new Date(item.nextAttempt);
item.createdAt = new Date(item.createdAt);
item.updatedAt = new Date(item.updatedAt);
if (item.deliveredAt) {
item.deliveredAt = new Date(item.deliveredAt);
}
// Add to queue
this.queue.set(item.id, item);
} catch (err) {
console.error(`Error loading queue item ${file}:`, err);
}
}
}
console.log(`Loaded ${this.queue.size} items from queue`);
} catch (error) {
console.error('Error loading queue:', error);
throw error;
}
}
/**
* Save queue to disk
*/
private async save(): Promise<void> {
if (!this.config.persistentPath) {
return;
}
try {
// Create directory if it doesn't exist
if (!fs.existsSync(this.config.persistentPath)) {
fs.mkdirSync(this.config.persistentPath, { recursive: true });
}
// Save each item
const savePromises = Array.from(this.queue.values()).map(item => this.saveItem(item));
await Promise.all(savePromises);
} catch (error) {
console.error('Error saving queue:', error);
throw error;
}
}
/**
* Save a single item to disk
* @param item Queue item to save
*/
private async saveItem(item: IQueueItem): Promise<void> {
if (!this.config.persistentPath) {
return;
}
try {
const filePath = path.join(this.config.persistentPath, `${item.id}.json`);
const data = JSON.stringify(item, null, 2);
await fs.promises.writeFile(filePath, data, 'utf8');
} catch (error) {
console.error(`Error saving queue item ${item.id}:`, error);
throw error;
}
}
/**
* Remove a single item file from disk
* @param id Item ID
*/
private async removeItemFile(id: string): Promise<void> {
if (!this.config.persistentPath) {
return;
}
try {
const filePath = path.join(this.config.persistentPath, `${id}.json`);
if (fs.existsSync(filePath)) {
await fs.promises.unlink(filePath);
}
} catch (error) {
console.error(`Error removing queue item file ${id}:`, error);
throw error;
}
}
}

View File

@ -0,0 +1,272 @@
import * as plugins from '../plugins.js';
import { DeliveryQueue } from './classes.delivery.queue.js';
import type { IQueueItem } from './classes.delivery.queue.js';
import type { IProcessingResult, IRoutingDecision } from './classes.email.processor.js';
import { EventEmitter } from 'node:events';
import { Readable } from 'node:stream';
/**
* Result of a delivery attempt
*/
export interface IDeliveryResult {
id: string;
success: boolean;
error?: string;
timestamp: Date;
destination: string;
messageId?: string;
}
/**
* Delivery system statistics
*/
export interface IDeliveryStats {
delivered: number;
failed: number;
pending: number;
inProgress: number;
totalAttempts: number;
}
/**
* Email delivery system with retry logic
*/
export class DeliverySystem extends EventEmitter {
private queue: DeliveryQueue;
private isRunning: boolean = false;
private stats: IDeliveryStats = {
delivered: 0,
failed: 0,
pending: 0,
inProgress: 0,
totalAttempts: 0
};
private connections: Map<string, any> = new Map();
private maxConcurrent: number = 5;
/**
* Create a new delivery system
* @param queue Delivery queue to process
* @param maxConcurrent Maximum concurrent deliveries
*/
constructor(queue: DeliveryQueue, maxConcurrent: number = 5) {
super();
this.queue = queue;
this.maxConcurrent = maxConcurrent;
// Listen for queue events
this.setupQueueListeners();
}
/**
* Set up queue event listeners
*/
private setupQueueListeners(): void {
// Listen for items ready to be delivered
this.queue.on('itemsReady', (items: IQueueItem[]) => {
if (this.isRunning) {
this.processItems(items).catch(err => {
console.error('Error processing queue items:', err);
});
}
});
}
/**
* Start the delivery system
*/
public async start(): Promise<void> {
this.isRunning = true;
this.emit('started');
// Update stats
this.updateStats();
}
/**
* Stop the delivery system
*/
public async stop(): Promise<void> {
this.isRunning = false;
// Close all connections
for (const connection of this.connections.values()) {
try {
if (connection.close) {
await connection.close();
}
} catch (error) {
console.error('Error closing connection:', error);
}
}
this.connections.clear();
this.emit('stopped');
}
/**
* Process items from the queue
* @param items Queue items to process
*/
private async processItems(items: IQueueItem[]): Promise<void> {
// Skip if not running
if (!this.isRunning) {
return;
}
// Count in-progress items
const inProgress = Array.from(this.queue.getAllItems()).filter(item =>
item.status === 'processing'
).length;
// Calculate how many items we can process concurrently
const availableSlots = Math.max(0, this.maxConcurrent - inProgress);
if (availableSlots === 0) {
return;
}
// Process up to availableSlots items
const itemsToProcess = items.slice(0, availableSlots);
// Process each item
for (const item of itemsToProcess) {
// Mark item as processing
await this.queue.updateItem(item.id, {
status: 'processing'
});
// Deliver the item
this.deliverItem(item).catch(error => {
console.error(`Error delivering item ${item.id}:`, error);
});
}
// Update stats
this.updateStats();
}
/**
* Deliver a single queue item
* @param item Queue item to deliver
*/
private async deliverItem(item: IQueueItem): Promise<void> {
try {
// Update stats
this.stats.inProgress++;
this.stats.totalAttempts++;
// Get processing result
const result = item.processingResult;
// Attempt delivery
const deliveryResult = await this.deliverEmail(result);
if (deliveryResult.success) {
// Mark as delivered
await this.queue.markDelivered(item.id);
// Update stats
this.stats.delivered++;
this.stats.inProgress--;
// Emit delivery event
this.emit('delivered', {
item,
result: deliveryResult
});
} else {
// Mark as failed (will retry if attempts < maxRetries)
await this.queue.markFailed(item.id, deliveryResult.error || 'Unknown error');
// Update stats
this.stats.inProgress--;
// Emit failure event
this.emit('deliveryFailed', {
item,
result: deliveryResult
});
}
// Update stats
this.updateStats();
} catch (error) {
console.error(`Error in deliverItem for ${item.id}:`, error);
// Mark as failed
await this.queue.markFailed(item.id, error.message || 'Internal error');
// Update stats
this.stats.inProgress--;
this.updateStats();
}
}
/**
* Deliver an email to its destination
* @param result Processing result containing the email to deliver
*/
private async deliverEmail(result: IProcessingResult): Promise<IDeliveryResult> {
const { routing, metadata, rawData } = result;
const { id, targetServer, port, useTls, authentication } = routing;
try {
// Create a transport for delivery
// In a real implementation, this would use nodemailer or a similar library
console.log(`Delivering email ${id} to ${targetServer}:${port} (TLS: ${useTls})`);
// Simulate delivery
await new Promise(resolve => setTimeout(resolve, 100));
// Simulate success
// In a real implementation, we would actually send the email
const success = Math.random() > 0.1; // 90% success rate for simulation
if (!success) {
throw new Error('Simulated delivery failure');
}
// Return success result
return {
id,
success: true,
timestamp: new Date(),
destination: `${targetServer}:${port}`,
messageId: `${id}@example.com`
};
} catch (error) {
console.error(`Delivery error for ${id}:`, error);
// Return failure result
return {
id,
success: false,
error: error.message || 'Unknown error',
timestamp: new Date(),
destination: `${targetServer}:${port}`
};
}
}
/**
* Update delivery system statistics
*/
private updateStats(): void {
// Get pending items
this.stats.pending = Array.from(this.queue.getAllItems()).filter(item =>
item.status === 'pending' || item.status === 'deferred'
).length;
// Emit stats update
this.emit('statsUpdated', this.getStats());
}
/**
* Get current delivery statistics
*/
public getStats(): IDeliveryStats {
return { ...this.stats };
}
}

View File

@ -0,0 +1,495 @@
import * as plugins from '../plugins.js';
import type { ISmtpConfig, IContentScannerConfig, ITransformationConfig } from './classes.smtp.config.js';
import type { ISmtpSession } from './classes.smtp.server.js';
import { EventEmitter } from 'node:events';
// Create standalone types to avoid interface compatibility issues
interface AddressObject {
address?: string;
name?: string;
[key: string]: any;
}
interface ExtendedAddressObject {
value: AddressObject | AddressObject[];
[key: string]: any;
}
// Don't extend ParsedMail directly to avoid type compatibility issues
interface ExtendedParsedMail {
// Basic properties from ParsedMail
subject?: string;
text?: string;
textAsHtml?: string;
html?: string;
attachments?: Array<any>;
headers?: Map<string, any>;
headerLines?: Array<{key: string; line: string}>;
messageId?: string;
date?: Date;
// Extended address objects
from?: ExtendedAddressObject;
to?: ExtendedAddressObject;
cc?: ExtendedAddressObject;
bcc?: ExtendedAddressObject;
// Add any other properties we need
[key: string]: any;
}
/**
* Email metadata extracted from parsed mail
*/
export interface IEmailMetadata {
id: string;
from: string;
fromDomain: string;
to: string[];
toDomains: string[];
subject?: string;
size: number;
hasAttachments: boolean;
receivedAt: Date;
clientIp: string;
authenticated: boolean;
authUser?: string;
}
/**
* Content scanning result
*/
export interface IScanResult {
id: string;
spamScore?: number;
hasVirus?: boolean;
blockedAttachments?: string[];
action: 'accept' | 'tag' | 'reject';
reason?: string;
}
/**
* Routing decision for an email
*/
export interface IRoutingDecision {
id: string;
targetServer: string;
port: number;
useTls: boolean;
authentication?: {
user?: string;
pass?: string;
};
headers?: Array<{
name: string;
value: string;
append?: boolean;
}>;
signDkim?: boolean;
dkimOptions?: {
domainName: string;
keySelector: string;
privateKey: string;
};
}
/**
* Complete processing result
*/
export interface IProcessingResult {
id: string;
metadata: IEmailMetadata;
scanResult: IScanResult;
routing: IRoutingDecision;
modifiedMessage?: ExtendedParsedMail;
originalMessage: ExtendedParsedMail;
rawData: string;
action: 'queue' | 'reject';
session: ISmtpSession;
}
/**
* Email Processor handles email processing pipeline
*/
export class EmailProcessor extends EventEmitter {
private config: ISmtpConfig;
private processingQueue: Map<string, IProcessingResult> = new Map();
/**
* Create a new email processor
* @param config SMTP configuration
*/
constructor(config: ISmtpConfig) {
super();
this.config = config;
}
/**
* Process an email message
* @param message Parsed email message
* @param rawData Raw email data
* @param session SMTP session
*/
public async processEmail(
message: ExtendedParsedMail,
rawData: string,
session: ISmtpSession
): Promise<IProcessingResult> {
try {
// Generate ID for this processing task
const id = plugins.uuid.v4();
// Extract metadata
const metadata = await this.extractMetadata(message, session, id);
// Scan content if enabled
const scanResult = await this.scanContent(message, metadata);
// If content scanning rejects the message, return early
if (scanResult.action === 'reject') {
const result: IProcessingResult = {
id,
metadata,
scanResult,
routing: {
id,
targetServer: '',
port: 0,
useTls: false
},
originalMessage: message,
rawData,
action: 'reject',
session
};
this.emit('rejected', result);
return result;
}
// Determine routing
const routing = await this.determineRouting(message, metadata);
// Apply transformations
const modifiedMessage = await this.applyTransformations(message, routing, scanResult);
// Create processing result
const result: IProcessingResult = {
id,
metadata,
scanResult,
routing,
modifiedMessage,
originalMessage: message,
rawData,
action: 'queue',
session
};
// Add to processing queue
this.processingQueue.set(id, result);
// Emit processed event
this.emit('processed', result);
return result;
} catch (error) {
console.error('Error processing email:', error);
throw error;
}
}
/**
* Extract metadata from email message
* @param message Parsed email
* @param session SMTP session
* @param id Processing ID
*/
private async extractMetadata(
message: ExtendedParsedMail,
session: ISmtpSession,
id: string
): Promise<IEmailMetadata> {
// Extract sender information
let from = '';
if (message.from && message.from.value) {
const fromValue = message.from.value;
if (Array.isArray(fromValue)) {
from = fromValue[0]?.address || '';
} else if (typeof fromValue === 'object' && 'address' in fromValue && fromValue.address) {
from = fromValue.address;
}
}
const fromDomain = from.split('@')[1] || '';
// Extract recipient information
let to: string[] = [];
if (message.to && message.to.value) {
const toValue = message.to.value;
if (Array.isArray(toValue)) {
to = toValue
.map(addr => (addr && 'address' in addr) ? addr.address || '' : '')
.filter(Boolean);
} else if (typeof toValue === 'object' && 'address' in toValue && toValue.address) {
to = [toValue.address];
}
}
const toDomains = to.map(addr => addr.split('@')[1] || '');
// Create metadata
return {
id,
from,
fromDomain,
to,
toDomains,
subject: message.subject,
size: Buffer.byteLength(message.html || message.textAsHtml || message.text || ''),
hasAttachments: message.attachments?.length > 0,
receivedAt: new Date(),
clientIp: session.remoteAddress,
authenticated: !!session.user,
authUser: session.user?.username
};
}
/**
* Scan email content
* @param message Parsed email
* @param metadata Email metadata
*/
private async scanContent(
message: ExtendedParsedMail,
metadata: IEmailMetadata
): Promise<IScanResult> {
// Skip if content scanning is disabled
if (!this.config.contentScanning || !this.config.scanners?.length) {
return {
id: metadata.id,
action: 'accept'
};
}
// Default result
const result: IScanResult = {
id: metadata.id,
action: 'accept'
};
// Placeholder for scanning results
let spamFound = false;
let virusFound = false;
const blockedAttachments: string[] = [];
// Apply each scanner
for (const scanner of this.config.scanners) {
switch (scanner.type) {
case 'spam':
// Placeholder for spam scanning
// In a real implementation, we would use a spam scanning library
const spamScore = Math.random() * 10; // Fake score between 0-10
result.spamScore = spamScore;
if (scanner.threshold && spamScore > scanner.threshold) {
spamFound = true;
if (scanner.action === 'reject') {
result.action = 'reject';
result.reason = `Spam score ${spamScore} exceeds threshold ${scanner.threshold}`;
} else if (scanner.action === 'tag') {
result.action = 'tag';
}
}
break;
case 'virus':
// Placeholder for virus scanning
// In a real implementation, we would use a virus scanning library
const hasVirus = false; // Fake result
result.hasVirus = hasVirus;
if (hasVirus) {
virusFound = true;
if (scanner.action === 'reject') {
result.action = 'reject';
result.reason = 'Message contains virus';
} else if (scanner.action === 'tag') {
result.action = 'tag';
}
}
break;
case 'attachment':
// Check attachments against blocked extensions
if (scanner.blockedExtensions && message.attachments?.length) {
for (const attachment of message.attachments) {
const filename = attachment.filename || '';
const extension = filename.substring(filename.lastIndexOf('.')).toLowerCase();
if (scanner.blockedExtensions.includes(extension)) {
blockedAttachments.push(filename);
if (scanner.action === 'reject') {
result.action = 'reject';
result.reason = `Blocked attachment type: ${extension}`;
} else if (scanner.action === 'tag') {
result.action = 'tag';
}
}
}
}
break;
}
// Set blocked attachments in result if any
if (blockedAttachments.length) {
result.blockedAttachments = blockedAttachments;
}
}
return result;
}
/**
* Determine routing for an email
* @param message Parsed email
* @param metadata Email metadata
*/
private async determineRouting(
message: ExtendedParsedMail,
metadata: IEmailMetadata
): Promise<IRoutingDecision> {
// Start with the default routing
const defaultRouting: IRoutingDecision = {
id: metadata.id,
targetServer: this.config.defaultServer,
port: this.config.defaultPort || 25,
useTls: this.config.useTls || false
};
// If no domain configs, use default routing
if (!this.config.domainConfigs?.length) {
return defaultRouting;
}
// Try to find matching domain config based on recipient domains
for (const domain of metadata.toDomains) {
for (const domainConfig of this.config.domainConfigs) {
// Check if domain matches any of the configured domains
if (domainConfig.domains.some(configDomain => this.domainMatches(domain, configDomain))) {
// Create routing from domain config
const routing: IRoutingDecision = {
id: metadata.id,
targetServer: domainConfig.targetIPs[0], // Use first target IP
port: domainConfig.port || 25,
useTls: domainConfig.useTls || false
};
// Add authentication if specified
if (domainConfig.authentication) {
routing.authentication = domainConfig.authentication;
}
// Add header modifications if specified
if (domainConfig.addHeaders && domainConfig.headerInfo?.length) {
routing.headers = domainConfig.headerInfo.map(h => ({
name: h.name,
value: h.value,
append: false
}));
}
// Add DKIM signing if specified
if (domainConfig.signDkim && domainConfig.dkimOptions) {
routing.signDkim = true;
routing.dkimOptions = domainConfig.dkimOptions;
}
return routing;
}
}
}
// No match found, use default routing
return defaultRouting;
}
/**
* Apply transformations to the email
* @param message Original parsed email
* @param routing Routing decision
* @param scanResult Scan result
*/
private async applyTransformations(
message: ExtendedParsedMail,
routing: IRoutingDecision,
scanResult: IScanResult
): Promise<ExtendedParsedMail> {
// Skip if no transformations configured
if (!this.config.transformations?.length) {
return message;
}
// Clone the message for modifications
// Note: In a real implementation, we would need to properly clone the message
const modifiedMessage = { ...message };
// Apply each transformation
for (const transformation of this.config.transformations) {
switch (transformation.type) {
case 'addHeader':
// Add a header to the message
if (transformation.header && transformation.value) {
// In a real implementation, we would modify the raw message headers
console.log(`Adding header ${transformation.header}: ${transformation.value}`);
}
break;
case 'dkimSign':
// Sign the message with DKIM
if (routing.signDkim && routing.dkimOptions) {
// In a real implementation, we would use mailauth.dkimSign
console.log(`Signing message with DKIM for domain ${routing.dkimOptions.domainName}`);
}
break;
}
}
return modifiedMessage;
}
/**
* Check if a domain matches a pattern (including wildcards)
* @param domain Domain to check
* @param pattern Pattern to match against
*/
private domainMatches(domain: string, pattern: string): boolean {
domain = domain.toLowerCase();
pattern = pattern.toLowerCase();
// Exact match
if (domain === pattern) {
return true;
}
// Wildcard match (*.example.com)
if (pattern.startsWith('*.')) {
const suffix = pattern.slice(2);
return domain.endsWith(suffix) && domain.length > suffix.length;
}
return false;
}
/**
* Update processor configuration
* @param config New configuration
*/
public updateConfig(config: Partial<ISmtpConfig>): void {
this.config = {
...this.config,
...config
};
this.emit('configUpdated', this.config);
}
}

View File

@ -0,0 +1,170 @@
import * as plugins from '../plugins.js';
/**
* Configuration for SMTP authentication
*/
export interface ISmtpAuthConfig {
/** Whether authentication is required */
required?: boolean;
/** Supported authentication methods */
methods?: ('PLAIN' | 'LOGIN' | 'OAUTH2')[];
/** Static user credentials */
users?: Array<{username: string, password: string}>;
/** LDAP URL for authentication */
ldapUrl?: string;
}
/**
* Configuration for TLS in SMTP connections
*/
export interface ISmtpTlsConfig {
/** Path to certificate file */
certPath?: string;
/** Path to key file */
keyPath?: string;
/** Path to CA certificate */
caPath?: string;
/** Minimum TLS version */
minVersion?: string;
/** Whether to use STARTTLS upgrade or implicit TLS */
useStartTls?: boolean;
/** Cipher suite for TLS */
ciphers?: string;
}
/**
* Configuration for content scanning
*/
export interface IContentScannerConfig {
/** Type of scanner */
type: 'spam' | 'virus' | 'attachment';
/** Threshold for spam detection */
threshold?: number;
/** Action to take when content matches scanning criteria */
action: 'tag' | 'reject';
/** File extensions to block (for attachment scanner) */
blockedExtensions?: string[];
}
/**
* Configuration for email transformations
*/
export interface ITransformationConfig {
/** Type of transformation */
type: string;
/** Header name for adding/modifying headers */
header?: string;
/** Header value for adding/modifying headers */
value?: string;
/** Domains for DKIM signing */
domains?: string[];
/** Whether to append to existing header or replace */
append?: boolean;
/** Additional transformation parameters */
[key: string]: any;
}
/**
* Configuration for DKIM signing
*/
export interface IDkimConfig {
/** Domain name for DKIM signature */
domainName: string;
/** Selector for DKIM */
keySelector: string;
/** Private key for DKIM signing */
privateKey: string;
}
/**
* Domain-specific routing configuration
*/
export interface ISmtpDomainConfig {
/** Domains this configuration applies to */
domains: string[];
/** Target SMTP servers for this domain */
targetIPs: string[];
/** Target port */
port?: number;
/** Whether to use TLS when connecting to target */
useTls?: boolean;
/** Authentication credentials for target server */
authentication?: {
user?: string;
pass?: string;
};
/** Allowed client IPs */
allowedIPs?: string[];
/** Rate limits for this domain */
rateLimits?: {
maxMessagesPerMinute?: number;
maxRecipientsPerMessage?: number;
};
/** Whether to add custom headers */
addHeaders?: boolean;
/** Headers to add */
headerInfo?: Array<{
name: string;
value: string;
}>;
/** Whether to sign emails with DKIM */
signDkim?: boolean;
/** DKIM configuration */
dkimOptions?: IDkimConfig;
}
/**
* Queue configuration
*/
export interface IQueueConfig {
/** Storage type for queue */
storageType?: 'memory' | 'disk';
/** Path for disk storage */
persistentPath?: string;
/** Maximum retry attempts */
maxRetries?: number;
/** Base delay between retries (ms) */
baseRetryDelay?: number;
/** Maximum delay between retries (ms) */
maxRetryDelay?: number;
/** Maximum queue size */
maxQueueSize?: number;
}
/**
* Complete SMTP configuration
*/
export interface ISmtpConfig {
/** SMTP ports to listen on */
ports: number[];
/** Hostname for SMTP server */
hostname: string;
/** Banner text for SMTP server */
banner?: string;
/** Maximum message size in bytes */
maxMessageSize?: number;
/** TLS configuration */
tls?: ISmtpTlsConfig;
/** Authentication configuration */
auth?: ISmtpAuthConfig;
/** Domain-specific configurations */
domainConfigs: ISmtpDomainConfig[];
/** Default routing */
defaultServer: string;
defaultPort?: number;
useTls?: boolean;
/** Content scanning configuration */
contentScanning?: boolean;
scanners?: IContentScannerConfig[];
/** Message transformations */
transformations?: ITransformationConfig[];
/** Queue configuration */
queue?: IQueueConfig;
}

View File

@ -0,0 +1,423 @@
import * as plugins from '../plugins.js';
import { Readable } from 'node:stream';
import type { ISmtpConfig, ISmtpAuthConfig } from './classes.smtp.config.js';
import { EventEmitter } from 'node:events';
/**
* Connection session information
*/
export interface ISmtpSession {
id: string;
remoteAddress: string;
remotePort: number;
clientHostname?: string;
secure: boolean;
transmissionType?: 'SMTP' | 'ESMTP';
user?: {
username: string;
[key: string]: any;
};
envelope?: {
mailFrom: {
address: string;
args: any;
};
rcptTo: Array<{
address: string;
args: any;
}>;
};
}
/**
* Authentication data
*/
export interface IAuthData {
method: string;
username: string;
password: string;
}
/**
* SMTP Server class for receiving emails
*/
export class SmtpServer extends EventEmitter {
private config: ISmtpConfig;
private server: any; // Will be SMTPServer from smtp-server once we add the dependency
private incomingConnections: Map<string, ISmtpSession> = new Map();
/**
* Create a new SMTP server
* @param config SMTP server configuration
*/
constructor(config: ISmtpConfig) {
super();
this.config = config;
}
/**
* Initialize and start the SMTP server
*/
public async start(): Promise<void> {
try {
// This is a placeholder for the actual server creation
// In the real implementation, we would use the smtp-server package
console.log(`Starting SMTP server on ports ${this.config.ports.join(', ')}`);
// Setup TLS options if provided
const tlsOptions = this.config.tls ? {
key: this.config.tls.keyPath ? await plugins.fs.promises.readFile(this.config.tls.keyPath, 'utf8') : undefined,
cert: this.config.tls.certPath ? await plugins.fs.promises.readFile(this.config.tls.certPath, 'utf8') : undefined,
ca: this.config.tls.caPath ? await plugins.fs.promises.readFile(this.config.tls.caPath, 'utf8') : undefined,
minVersion: this.config.tls.minVersion || 'TLSv1.2',
ciphers: this.config.tls.ciphers
} : undefined;
// Create the server
// Note: In the actual implementation, this would use SMTPServer from smtp-server
this.server = {
// Placeholder for server instance
async close() {
console.log('SMTP server closed');
}
};
// Set up event handlers
this.setupEventHandlers();
// Listen on all specified ports
for (const port of this.config.ports) {
// In actual implementation, this would call server.listen(port)
console.log(`SMTP server listening on port ${port}`);
}
this.emit('started');
} catch (error) {
console.error('Failed to start SMTP server:', error);
throw error;
}
}
/**
* Stop the SMTP server
*/
public async stop(): Promise<void> {
try {
if (this.server) {
// Close the server
await this.server.close();
this.server = null;
// Clear connection tracking
this.incomingConnections.clear();
this.emit('stopped');
}
} catch (error) {
console.error('Error stopping SMTP server:', error);
throw error;
}
}
/**
* Set up event handlers for the SMTP server
*/
private setupEventHandlers(): void {
// These would be connected to actual server events in implementation
// Connection handler
this.onConnect((session, callback) => {
// Store connection in tracking map
this.incomingConnections.set(session.id, session);
// Check if connection is allowed based on IP
if (!this.isIpAllowed(session.remoteAddress)) {
return callback(new Error('Connection refused'));
}
// Accept the connection
callback();
});
// Authentication handler
this.onAuth((auth, session, callback) => {
// Skip auth check if not required
if (!this.config.auth?.required) {
return callback(null, { user: auth.username });
}
// Check authentication
if (this.authenticateUser(auth)) {
return callback(null, { user: auth.username });
}
// Authentication failed
callback(new Error('Invalid credentials'));
});
// Sender validation
this.onMailFrom((address, session, callback) => {
// Validate sender address if needed
// Accept the sender
callback();
});
// Recipient validation
this.onRcptTo((address, session, callback) => {
// Validate recipient address
// Check if we handle this domain
if (!this.isDomainHandled(address.address.split('@')[1])) {
return callback(new Error('Domain not handled by this server'));
}
// Accept the recipient
callback();
});
// Message data handler
this.onData((stream, session, callback) => {
// Process the incoming message
this.processMessageData(stream, session)
.then(() => callback())
.catch(err => callback(err));
});
}
/**
* Process incoming message data
* @param stream Message data stream
* @param session SMTP session
*/
private async processMessageData(stream: Readable, session: ISmtpSession): Promise<void> {
return new Promise<void>((resolve, reject) => {
// Collect the message data
let messageData = '';
let messageSize = 0;
stream.on('data', (chunk) => {
messageData += chunk;
messageSize += chunk.length;
// Check size limits
if (this.config.maxMessageSize && messageSize > this.config.maxMessageSize) {
stream.unpipe();
return reject(new Error('Message size exceeds limit'));
}
});
stream.on('end', async () => {
try {
// Parse the email using mailparser
const parsedMail = await this.parseEmail(messageData);
// Emit message received event
this.emit('message', {
session,
mail: parsedMail,
rawData: messageData
});
resolve();
} catch (error) {
reject(error);
}
});
stream.on('error', (error) => {
reject(error);
});
});
}
/**
* Parse raw email data using mailparser
* @param rawData Raw email data
*/
private async parseEmail(rawData: string): Promise<any> {
// Use mailparser to parse the email
// We return 'any' here which will be treated as ExtendedParsedMail by consumers
return plugins.mailparser.simpleParser(rawData);
}
/**
* Check if an IP address is allowed to connect
* @param ip IP address
*/
private isIpAllowed(ip: string): boolean {
// Default to allowing all IPs if no restrictions
const defaultAllowed = ['0.0.0.0/0'];
// Check domain configs for IP restrictions
for (const domainConfig of this.config.domainConfigs) {
if (domainConfig.allowedIPs && domainConfig.allowedIPs.length > 0) {
// Check if IP matches any of the allowed IPs
for (const allowedIp of domainConfig.allowedIPs) {
if (this.ipMatchesRange(ip, allowedIp)) {
return true;
}
}
}
}
// Check against default allowed IPs
for (const allowedIp of defaultAllowed) {
if (this.ipMatchesRange(ip, allowedIp)) {
return true;
}
}
return false;
}
/**
* Check if an IP matches a range
* @param ip IP address to check
* @param range IP range in CIDR notation
*/
private ipMatchesRange(ip: string, range: string): boolean {
try {
// Use the 'ip' package to check if IP is in range
return plugins.ip.cidrSubnet(range).contains(ip);
} catch (error) {
console.error(`Invalid IP range: ${range}`, error);
return false;
}
}
/**
* Check if a domain is handled by this server
* @param domain Domain to check
*/
private isDomainHandled(domain: string): boolean {
// Check if the domain is configured in any domain config
for (const domainConfig of this.config.domainConfigs) {
for (const configDomain of domainConfig.domains) {
if (this.domainMatches(domain, configDomain)) {
return true;
}
}
}
return false;
}
/**
* Check if a domain matches a pattern (including wildcards)
* @param domain Domain to check
* @param pattern Pattern to match against
*/
private domainMatches(domain: string, pattern: string): boolean {
domain = domain.toLowerCase();
pattern = pattern.toLowerCase();
// Exact match
if (domain === pattern) {
return true;
}
// Wildcard match (*.example.com)
if (pattern.startsWith('*.')) {
const suffix = pattern.slice(2);
return domain.endsWith(suffix) && domain.length > suffix.length;
}
return false;
}
/**
* Authenticate a user
* @param auth Authentication data
*/
private authenticateUser(auth: IAuthData): boolean {
// Skip if no auth config
if (!this.config.auth) {
return true;
}
// Check if auth method is supported
if (this.config.auth.methods && !this.config.auth.methods.includes(auth.method as any)) {
return false;
}
// Check static user credentials
if (this.config.auth.users) {
const user = this.config.auth.users.find(u =>
u.username === auth.username && u.password === auth.password);
if (user) {
return true;
}
}
// LDAP authentication would go here
return false;
}
/**
* Event handler for connection
* @param handler Function to handle connection
*/
public onConnect(handler: (session: ISmtpSession, callback: (err?: Error) => void) => void): void {
// In actual implementation, this would connect to the server's 'connection' event
this.on('connect', handler);
}
/**
* Event handler for authentication
* @param handler Function to handle authentication
*/
public onAuth(handler: (auth: IAuthData, session: ISmtpSession, callback: (err?: Error, user?: any) => void) => void): void {
// In actual implementation, this would connect to the server's 'auth' event
this.on('auth', handler);
}
/**
* Event handler for MAIL FROM command
* @param handler Function to handle MAIL FROM
*/
public onMailFrom(handler: (address: { address: string; args: any }, session: ISmtpSession, callback: (err?: Error) => void) => void): void {
// In actual implementation, this would connect to the server's 'mail' event
this.on('mail', handler);
}
/**
* Event handler for RCPT TO command
* @param handler Function to handle RCPT TO
*/
public onRcptTo(handler: (address: { address: string; args: any }, session: ISmtpSession, callback: (err?: Error) => void) => void): void {
// In actual implementation, this would connect to the server's 'rcpt' event
this.on('rcpt', handler);
}
/**
* Event handler for DATA command
* @param handler Function to handle DATA
*/
public onData(handler: (stream: Readable, session: ISmtpSession, callback: (err?: Error) => void) => void): void {
// In actual implementation, this would connect to the server's 'data' event
this.on('dataReady', handler);
}
/**
* Update the server configuration
* @param config New configuration
*/
public updateConfig(config: Partial<ISmtpConfig>): void {
this.config = {
...this.config,
...config
};
// In a real implementation, this might require restarting the server
this.emit('configUpdated', this.config);
}
/**
* Get server statistics
*/
public getStats(): any {
return {
connections: this.incomingConnections.size,
// Additional stats would be included here
};
}
}

View File

@ -1,3 +1,11 @@
// Core DcRouter components
export * from './classes.dcrouter.js'; export * from './classes.dcrouter.js';
export * from './classes.smtp.portconfig.js'; export * from './classes.smtp.portconfig.js';
export * from './classes.email.domainrouter.js'; export * from './classes.email.domainrouter.js';
// SMTP Store-and-Forward components
export * from './classes.smtp.config.js';
export * from './classes.smtp.server.js';
export * from './classes.email.processor.js';
export * from './classes.delivery.queue.js';
export * from './classes.delivery.system.js';

View File

@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@serve.zone/platformservice', name: '@serve.zone/platformservice',
version: '2.5.0', version: '2.6.0',
description: 'A multifaceted platform service handling mail, SMS, letter delivery, and AI services.' description: 'A multifaceted platform service handling mail, SMS, letter delivery, and AI services.'
} }