platformservice/readme.plan.md

# PlatformService Roadmap

## Latest Changes

### API Type Safety Improvements
- [x] Create comprehensive TypeScript interfaces for all API methods
- [x] Replace any types with specific interfaces in EmailService and MtaConnector
- [x] Align interface types with their implementations
- [x] Document all interface properties with detailed JSDoc comments
- [x] Use type imports to ensure consistency between services
- [x] Fix TypeScript build errors from interface placements
- [x] Add proper method signatures for API endpoint handlers
- [x] Implement type-safe API responses

### Test Stability Improvements
- [x] Fix race conditions in SenderReputationMonitor tests
- [x] Disable filesystem operations during tests to prevent race conditions
- [x] Add proper cleanup of singleton instances and timeouts
- [x] Ensure all tests properly clean up shared resources
- [x] Set consistent test environment with NODE_ENV=test

### Error Handling Improvements
- [x] Add structured error types for better error reporting
- [x] Implement consistent error handling patterns across services
- [x] Add detailed error context for debugging
- [x] Create retry mechanisms for transient failures
- [x] Improve error logging with structured data

### Configuration Interface Standardization
- [x] Define consistent configuration interfaces across services  
- [x] Implement validation for all configuration objects
- [x] Add default values for optional configuration
- [x] Create documentation for all configuration options
- [x] Add migration helpers for configuration format changes

### Logging Enhancements
- [ ] Implement structured logging throughout the codebase
- [ ] Add context information to all log messages
- [ ] Create consistent log levels and usage patterns
- [ ] Add correlation IDs for request tracking
- [ ] Implement log filtering and sampling options

### Mailgun Removal
- [x] Remove Mailgun integration from keywords in package.json and npmextra.json
- [x] Update EmailService comments to remove mentions of Mailgun
- [x] Update error messages to reference MTA instead of generic email providers
- [x] Update the readme email example to use PlatformService reference instead of Mailgun API key

# DcRouter Consolidated Email Configuration Plan

## Overview
This plan outlines the consolidation of all email-related functionality in DcRouter under a unified `emailConfig` interface. This new approach combines MTA, SMTP forwarding, and store-and-forward processing into a single, pattern-based routing system that:

1. Uses glob patterns for domain matching (e.g., `*@task.vc` or `*@*.example.net`)
2. Shares ports across all email handling methods (25, 587, 465)
3. Allows different handling modes based on email domain patterns
4. Provides a flexible configuration interface for all email-related functionality

This consolidated approach simplifies configuration while enhancing flexibility, allowing domain-specific handling where, for example, `*@task.vc` emails are forwarded to another SMTP server while `*@lossless.digital` emails are processed by the MTA for programmatic use.

## 0. Configuration Approaches

### 0.1 Core SmartProxy Direct Configuration
DcRouter should leverage SmartProxy's configuration directly, exposing SmartProxy's full domain configuration options to give users maximum flexibility for all HTTP/HTTPS and TCP/SNI traffic:

```typescript
interface IDcRouterOptions {
  // Direct SmartProxy configuration for general HTTP/HTTPS and TCP/SNI traffic
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Consolidated email configuration
  emailConfig?: IEmailConfig;
  
  // Other DcRouter options...
}
```

This approach allows direct configuration of SmartProxy's powerful domain-based routing, giving full control over HTTP/HTTPS and SNI-based traffic:

```typescript
const dcRouter = new DcRouter({
  // Direct SmartProxy configuration for HTTP/HTTPS traffic
  smartProxyConfig: {
    fromPort: 443,
    toPort: 8080,
    targetIP: '10.0.0.10',
    sniEnabled: true,
    acme: {
      port: 80,
      enabled: true,
      autoRenew: true,
      useProduction: true,
      renewThresholdDays: 30,
      accountEmail: 'admin@example.com'
    },
    globalPortRanges: [
      { from: 80, to: 80 },
      { from: 443, to: 443 }
    ],
    
    // SmartProxy's full domain configuration flexibility
    domainConfigs: [
      {
        domains: ['example.com', 'www.example.com'],
        allowedIPs: ['0.0.0.0/0'],
        blockedIPs: ['1.2.3.4/32'],
        targetIPs: ['10.0.0.10', '10.0.0.11'],
        portRanges: [
          { from: 80, to: 80 },
          { from: 443, to: 443 }
        ],
        connectionTimeout: 60000,
        useNetworkProxy: true
      },
      // Additional domain configurations...
    ]
  },
  
  // Consolidated email configuration
  emailConfig: {
    // Email handling configuration...
  }
});
```

### 0.2 Consolidated Email Configuration
We'll implement a unified email configuration approach that combines MTA, SMTP forwarding, and store-and-forward processing into a single pattern-based routing system:

```typescript
interface IDcRouterOptions {
  // HTTP/HTTPS configuration
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Consolidated email handling
  emailConfig?: {
    // Global email server settings
    ports: number[];
    hostname: string;
    maxMessageSize?: number;
    
    // TLS configuration for all email services
    tls?: {
      certPath?: string;
      keyPath?: string;
      caPath?: string;
      minVersion?: string;
    };
    
    // Authentication for inbound connections
    auth?: {
      required?: boolean;
      methods?: ('PLAIN' | 'LOGIN' | 'OAUTH2')[];
      users?: Array<{username: string, password: string}>;
    };
    
    // Default routing for unmatched domains
    defaultMode: 'forward' | 'mta' | 'process';
    defaultServer?: string;
    defaultPort?: number;
    defaultTls?: boolean;
    
    // Domain-specific rules with glob pattern support
    domainRules: Array<{
      // Domain pattern (e.g., "*@example.com", "*@*.example.net")
      pattern: string;
      
      // Handling mode for this pattern
      mode: 'forward' | 'mta' | 'process';
      
      // Forward mode configuration
      target?: {
        server: string;
        port?: number;
        useTls?: boolean;
        authentication?: {
          user?: string;
          pass?: string;
        };
      };
      
      // MTA mode configuration
      mtaOptions?: {
        domain?: string;
        allowLocalDelivery?: boolean;
        localDeliveryPath?: string;
        dkimSign?: boolean;
        dkimOptions?: {
          domainName: string;
          keySelector: string;
          privateKey: string;
        };
      };
      
      // Process mode configuration
      contentScanning?: boolean;
      scanners?: Array<{
        type: 'spam' | 'virus' | 'attachment';
        threshold?: number;
        action: 'tag' | 'reject';
        blockedExtensions?: string[];
      }>;
      
      transformations?: Array<{
        type: string;
        [key: string]: any;
      }>;
      
      // Rate limits for this domain
      rateLimits?: {
        maxMessagesPerMinute?: number;
        maxRecipientsPerMessage?: number;
      };
    }>;
    
    // Queue configuration for all email processing
    queue?: {
      storageType?: 'memory' | 'disk';
      persistentPath?: string;
      maxRetries?: number;
      baseRetryDelay?: number;
      maxRetryDelay?: number;
    };
    
    // Advanced MTA settings
    mtaGlobalOptions?: {
      smtpBanner?: string;
      maxConnections?: number;
      connTimeout?: number;
      spoolDir?: string;
      dkimKeyPath?: string;
    };
  };
}

## 1. Core Architecture for Consolidated Email Processing

### 1.1 Unified Email Server
- [x] Create a unified email server component
  - Build on existing SmtpServer class but with enhanced routing capabilities
  - Configure to listen on standard ports (25, 587, 465) for all email handling
  - Implement TLS support (STARTTLS and implicit TLS)
  - Add support for authentication methods (PLAIN, LOGIN, OAUTH2)
  - Set up size limits and connection timeouts

### 1.2 Pattern-Based Domain Router
- [x] Create pattern matching system for email domains
  - Implement glob pattern matching for email addresses
  - Support patterns like `*@domain.com`, `*@*.domain.com`
  - Create priority-based matching system (most specific match wins)
  - Build rule evaluation engine to determine processing mode
  - Implement a fast lookup mechanism for incoming emails

### 1.3 Multi-Modal Processing System
- [x] Create a unified processing system with multiple modes
  - Forward mode: SMTP proxy functionality with enhanced routing
  - MTA mode: Programmatic email handling with local delivery options
  - Process mode: Full store-and-forward pipeline with content scanning
  - Implement mode-specific configuration and handlers
  - Create fallback handling for unmatched domains

### 1.4 Shared Infrastructure
- [x] Develop shared components across all email handling modes
  - [x] Create unified delivery queue for all outbound email
  - [x] Implement shared authentication system
  - [x] Build common TLS and certificate management
  - [x] Create uniform logging and metrics collection
  - [x] Develop shared rate limiting and throttling

## 2. Consolidated Email Processing Features

### 2.1 Pattern-Based Routing
- [x] Implement glob pattern-based email routing
  - Create glob pattern matching for both domains and full email addresses
  - Support wildcards for domains, subdomains, and local parts (e.g., `*@domain.com`, `user@*.domain.com`)
  - Add support for pattern matching priorities (most specific wins)
  - Implement cacheable pattern matching for performance
  - Create comprehensive test suite for pattern matching

### 2.2 Multi-Modal Processing
- [x] Develop multiple email handling modes
  - Forward mode: Simple SMTP forwarding to another server with enhanced routing
  - MTA mode: Process with the MTA for programmatic handling and local delivery
  - Process mode: Full store-and-forward processing with content scanning
  - Add mode-specific configuration validation
  - Implement seamless mode transitions based on patterns

### 2.3 Content Inspection and Transformation
- [x] Enhance content inspection for processing mode
  - [x] Improve MIME parsing and content extraction capabilities
  - [x] Enhance attachment scanning and filtering
  - [x] Add text analysis for spam and phishing detection
  - [x] Create more robust transformation framework
  - [x] Support content-based routing decisions

### 2.4 Unified Rate Limiting and Traffic Control
- [x] Build unified rate limiting across all modes
  - [x] Implement pattern-based rate limits
  - [x] Create hierarchical rate limiting (global, pattern, IP)
  - [x] Add real-time rate limit monitoring
  - [x] Develop traffic shaping capabilities
  - [x] Implement backpressure mechanisms for overload protection

## 3. DcRouter Integration

### 3.1 Unified Configuration Interface
- [x] Implement the consolidated emailConfig interface
  - Create the IEmailConfig interface with all required components
  - Replace existing SMTP, forwarding, and MTA configs with unified approach
  - Add backward compatibility layer for existing configurations
  - Provide comprehensive validation for the new configuration format
  - Add clear documentation and examples in code comments

### 3.2 Enhanced Management API
- [x] Develop enhanced management API for consolidated email handling
  - Create unified status reporting across all modes
  - Implement pattern-based rule management (add, update, remove)
  - Add comprehensive queue management across all modes
  - Create mode-specific monitoring endpoints
  - Implement enhanced configuration update methods

### 3.3 Unified Metrics and Logging
- [x] Create a unified metrics system for all email handling
  - Develop pattern-based metrics collection
  - Implement mode-specific performance metrics
  - Create pattern rule effectiveness measurements
  - Add real-time monitoring capabilities
  - Design comprehensive logging with correlation IDs

## 4. Detailed Component Specifications

### 4.0 DcRouter Configuration Extension

```typescript
export interface IDcRouterOptions {
  // Core configuration options
  
  // Direct SmartProxy configuration - gives full control over all TCP/SNI traffic
  // including HTTP, HTTPS, and any other TCP-based protocol
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Unified email configuration for all email handling modes
  emailConfig?: IEmailConfig;
  
  // Shared configurations
  tls?: {
    contactEmail: string;
    domain?: string;
    certPath?: string;
    keyPath?: string;
  };
  
  // Other DcRouter options
  dnsServerConfig?: plugins.smartdns.IDnsServerOptions;
}

/**
 * Consolidated email configuration interface
 */
export interface IEmailConfig {
  // Email server settings
  ports: number[];
  hostname: string;
  maxMessageSize?: number;
  
  // TLS configuration for email server
  tls?: {
    certPath?: string;
    keyPath?: string;
    caPath?: string;
    minVersion?: string;
  };
  
  // Authentication for inbound connections
  auth?: {
    required?: boolean;
    methods?: ('PLAIN' | 'LOGIN' | 'OAUTH2')[];
    users?: Array<{username: string, password: string}>;
  };
  
  // Default routing for unmatched domains
  defaultMode: EmailProcessingMode;
  defaultServer?: string;
  defaultPort?: number;
  defaultTls?: boolean;
  
  // Domain rules with glob pattern support
  domainRules: IDomainRule[];
  
  // Queue configuration for all email processing
  queue?: {
    storageType?: 'memory' | 'disk';
    persistentPath?: string;
    maxRetries?: number;
    baseRetryDelay?: number;
    maxRetryDelay?: number;
  };
  
  // Advanced MTA settings
  mtaGlobalOptions?: IMtaOptions;
}

/**
 * Email processing modes
 */
export type EmailProcessingMode = 'forward' | 'mta' | 'process';

/**
 * Domain rule interface for pattern-based routing
 */
export interface IDomainRule {
  // Domain pattern (e.g., "*@example.com", "*@*.example.net")
  pattern: string;
  
  // Handling mode for this pattern
  mode: EmailProcessingMode;
  
  // Forward mode configuration
  target?: {
    server: string;
    port?: number;
    useTls?: boolean;
    authentication?: {
      user?: string;
      pass?: string;
    };
  };
  
  // MTA mode configuration
  mtaOptions?: IMtaOptions;
  
  // Process mode configuration
  contentScanning?: boolean;
  scanners?: IContentScanner[];
  transformations?: ITransformation[];
  
  // Rate limits for this domain
  rateLimits?: {
    maxMessagesPerMinute?: number;
    maxRecipientsPerMessage?: number;
  };
}

/**
 * MTA options interface
 */
export interface IMtaOptions {
  domain?: string;
  allowLocalDelivery?: boolean;
  localDeliveryPath?: string;
  dkimSign?: boolean;
  dkimOptions?: {
    domainName: string;
    keySelector: string;
    privateKey: string;
  };
  smtpBanner?: string;
  maxConnections?: number;
  connTimeout?: number;
  spoolDir?: string;
}

/**
 * Content scanner interface
 */
export interface IContentScanner {
  type: 'spam' | 'virus' | 'attachment';
  threshold?: number;
  action: 'tag' | 'reject';
  blockedExtensions?: string[];
}

/**
 * Transformation interface
 */
export interface ITransformation {
  type: string;
  [key: string]: any;
}
```

### 4.1 UnifiedEmailServer Class

```typescript
/**
 * Options for the unified email server
 */
export interface IUnifiedEmailServerOptions {
  // Base server options
  ports: number[];
  hostname: string;
  banner?: string;
  
  // Authentication options
  auth?: {
    required?: boolean;
    methods?: ('PLAIN' | 'LOGIN' | 'OAUTH2')[];
    users?: Array<{username: string, password: string}>;
  };
  
  // TLS options
  tls?: {
    certPath?: string;
    keyPath?: string;
    caPath?: string;
    minVersion?: string;
    ciphers?: string;
  };
  
  // Limits
  maxMessageSize?: number;
  maxClients?: number;
  maxConnections?: number;
  
  // Connection options
  connectionTimeout?: number;
  socketTimeout?: number;
  
  // Domain rules
  domainRules: IDomainRule[];
  
  // Default handling for unmatched domains
  defaultMode: EmailProcessingMode;
  defaultServer?: string;
  defaultPort?: number;
  defaultTls?: boolean;
}

/**
 * Unified email server that handles all email traffic with pattern-based routing
 */
export class UnifiedEmailServer extends EventEmitter {
  constructor(options: IUnifiedEmailServerOptions);
  
  // Start and stop the server
  public start(): Promise<void>;
  public stop(): Promise<void>;
  
  // Core event handlers
  private onConnect(session: ISmtpSession, callback: (err?: Error) => void): void;
  private onAuth(auth: IAuthData, session: ISmtpSession, callback: (err?: Error, user?: any) => void): void;
  private onMailFrom(address: {address: string}, session: ISmtpSession, callback: (err?: Error) => void): void;
  private onRcptTo(address: {address: string}, session: ISmtpSession, callback: (err?: Error) => void): void;
  private onData(stream: Readable, session: ISmtpSession, callback: (err?: Error) => void): void;
  
  // Pattern matching and routing
  private matchDomainRule(address: string): IDomainRule | null;
  private determineProcessingMode(session: ISmtpSession): EmailProcessingMode;
  
  // Mode-specific processing
  private handleForwardMode(message: any, session: ISmtpSession, rule: IDomainRule): Promise<void>;
  private handleMtaMode(message: any, session: ISmtpSession, rule: IDomainRule): Promise<void>;
  private handleProcessMode(message: any, session: ISmtpSession, rule: IDomainRule): Promise<void>;
  
  // Helper methods
  private parseEmail(rawData: string): Promise<any>;
  private isIpAllowed(ip: string, rule: IDomainRule): boolean;
  private createDeliveryJob(message: any, rule: IDomainRule): IDeliveryJob;
  
  // Configuration updates
  public updateOptions(options: Partial<IUnifiedEmailServerOptions>): void;
  public updateDomainRules(rules: IDomainRule[]): void;
  
  // Server stats
  public getStats(): IServerStats;
}
```

### 4.2 DomainRouter Class

```typescript
/**
 * Options for the domain-based router
 */
export interface IDomainRouterOptions {
  // Domain rules with glob pattern matching
  domainRules: IDomainRule[];
  
  // Default handling for unmatched domains
  defaultMode: EmailProcessingMode;
  defaultServer?: string;
  defaultPort?: number;
  defaultTls?: boolean;
  
  // Pattern matching options
  caseSensitive?: boolean;
  priorityOrder?: 'most-specific' | 'first-match';
  
  // Cache settings for pattern matching
  enableCache?: boolean;
  cacheSize?: number;
}

/**
 * A pattern matching and routing class for email domains
 */
export class DomainRouter {
  constructor(options: IDomainRouterOptions);
  
  /**
   * Match an email address against defined rules
   * @param email Email address to match
   * @returns The matching rule or null if no match
   */
  public matchRule(email: string): IDomainRule | null;
  
  /**
   * Check if email matches a specific pattern
   * @param email Email address to check
   * @param pattern Pattern to check against
   * @returns True if matching, false otherwise
   */
  public matchesPattern(email: string, pattern: string): boolean;
  
  /**
   * Get all rules that match an email address
   * @param email Email address to match
   * @returns Array of matching rules
   */
  public getAllMatchingRules(email: string): IDomainRule[];
  
  /**
   * Add a new routing rule
   * @param rule Domain rule to add
   */
  public addRule(rule: IDomainRule): void;
  
  /**
   * Update an existing rule
   * @param pattern Pattern to update
   * @param updates Updates to apply
   */
  public updateRule(pattern: string, updates: Partial<IDomainRule>): boolean;
  
  /**
   * Remove a rule
   * @param pattern Pattern to remove
   */
  public removeRule(pattern: string): boolean;
  
  /**
   * Get rule by pattern
   * @param pattern Pattern to find
   */
  public getRule(pattern: string): IDomainRule | null;
  
  /**
   * Get all rules
   */
  public getRules(): IDomainRule[];
  
  /**
   * Update options
   * @param options New options
   */
  public updateOptions(options: Partial<IDomainRouterOptions>): void;
  
  /**
   * Clear pattern matching cache
   */
  public clearCache(): void;
}
```

### 4.3 MultiModeProcessor Class

```typescript
/**
 * Processing modes
 */
export type EmailProcessingMode = 'forward' | 'mta' | 'process';

/**
 * Processor options
 */
export interface IMultiModeProcessorOptions {
  // Processing options
  maxParallelProcessing?: number;
  processingTimeout?: number;
  
  // Mode handlers
  forwardHandler?: IForwardHandler;
  mtaHandler?: IMtaHandler;
  processHandler?: IProcessHandler;
  
  // Queue configuration
  queue?: {
    storageType?: 'memory' | 'disk';
    persistentPath?: string;
    maxRetries?: number;
    baseRetryDelay?: number;
    maxRetryDelay?: number;
  };
  
  // Shared services
  sharedServices?: {
    dkimSigner?: IDkimSigner;
    contentScanner?: IContentScanner;
    rateLimiter?: IRateLimiter;
  };
}

/**
 * Processes emails using different modes based on domain rules
 */
export class MultiModeProcessor extends EventEmitter {
  constructor(options: IMultiModeProcessorOptions);
  
  /**
   * Process an email using the appropriate mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   * @param mode Processing mode
   */
  public async processEmail(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule,
    mode: EmailProcessingMode
  ): Promise<IProcessingResult>;
  
  /**
   * Handle email in forward mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   */
  private async handleForwardMode(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule
  ): Promise<IProcessingResult>;
  
  /**
   * Handle email in MTA mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   */
  private async handleMtaMode(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule
  ): Promise<IProcessingResult>;
  
  /**
   * Handle email in process mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   */
  private async handleProcessMode(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule
  ): Promise<IProcessingResult>;
  
  /**
   * Update processor options
   * @param options New options
   */
  public updateOptions(options: Partial<IMultiModeProcessorOptions>): void;
  
  /**
   * Get processor statistics
   */
  public getStats(): IProcessorStats;
}
```

### 4.4 UnifiedDeliveryQueue Class

```typescript
/**
 * Queue item status
 */
export type QueueItemStatus = 'pending' | 'processing' | 'delivered' | 'failed' | 'deferred';

/**
 * Queue item interface
 */
export interface IQueueItem {
  id: string;
  processingMode: EmailProcessingMode;
  processingResult: any;
  rule: IDomainRule;
  status: QueueItemStatus;
  attempts: number;
  nextAttempt: Date;
  lastError?: string;
  createdAt: Date;
  updatedAt: Date;
  deliveredAt?: Date;
}

/**
 * Queue options interface
 */
export interface IQueueOptions {
  // Storage options
  storageType?: 'memory' | 'disk';
  persistentPath?: string;
  
  // Queue behavior
  checkInterval?: number;
  maxQueueSize?: number;
  maxPerDestination?: number;
  
  // Delivery attempts
  maxRetries?: number;
  baseRetryDelay?: number;
  maxRetryDelay?: number;
}

/**
 * A unified queue for all email modes
 */
export class UnifiedDeliveryQueue extends EventEmitter {
  constructor(options: IQueueOptions);
  
  /**
   * Initialize the queue
   */
  public async initialize(): Promise<void>;
  
  /**
   * Add an item to the queue
   * @param processingResult Processing result to queue
   * @param mode Processing mode
   * @param rule Domain rule
   */
  public async enqueue(processingResult: any, mode: EmailProcessingMode, rule: IDomainRule): Promise<string>;
  
  /**
   * Get an item from the queue
   * @param id Item ID
   */
  public getItem(id: string): IQueueItem | undefined;
  
  /**
   * Mark an item as delivered
   * @param id Item ID
   */
  public async markDelivered(id: string): Promise<boolean>;
  
  /**
   * Mark an item as failed
   * @param id Item ID
   * @param error Error message
   */
  public async markFailed(id: string, error: string): Promise<boolean>;
  
  /**
   * Get queue statistics
   */
  public getStats(): IQueueStats;
  
  /**
   * Pause queue processing
   */
  public pause(): void;
  
  /**
   * Resume queue processing
   */
  public resume(): void;
  
  /**
   * Shutdown the queue
   */
  public async shutdown(): Promise<void>;
}
```

### 4.5 MultiModeDeliverySystem Class

```typescript
/**
 * Delivery options
 */
export interface IMultiModeDeliveryOptions {
  // Connection options
  connectionPoolSize?: number;
  socketTimeout?: number;
  
  // Delivery behavior
  concurrentDeliveries?: number;
  sendTimeout?: number;
  
  // TLS options
  verifyCertificates?: boolean;
  tlsMinVersion?: string;
  
  // Mode-specific handlers
  forwardHandler?: IForwardDeliveryHandler;
  mtaHandler?: IMtaDeliveryHandler;
  processHandler?: IProcessDeliveryHandler;
  
  // Rate limiting
  globalRateLimit?: number;
  perPatternRateLimit?: Record<string, number>;
  
  // Event hooks
  onDeliveryStart?: (item: IQueueItem) => Promise<void>;
  onDeliverySuccess?: (item: IQueueItem, result: any) => Promise<void>;
  onDeliveryFailed?: (item: IQueueItem, error: string) => Promise<void>;
}

/**
 * Handles delivery for all email processing modes
 */
export class MultiModeDeliverySystem extends EventEmitter {
  constructor(queue: UnifiedDeliveryQueue, options: IMultiModeDeliveryOptions);
  
  /**
   * Start the delivery system
   */
  public async start(): Promise<void>;
  
  /**
   * Stop the delivery system
   */
  public async stop(): Promise<void>;
  
  /**
   * Deliver an item from the queue
   * @param item Queue item to deliver
   */
  private async deliverItem(item: IQueueItem): Promise<void>;
  
  /**
   * Handle delivery in forward mode
   * @param item Queue item
   */
  private async handleForwardDelivery(item: IQueueItem): Promise<any>;
  
  /**
   * Handle delivery in MTA mode
   * @param item Queue item
   */
  private async handleMtaDelivery(item: IQueueItem): Promise<any>;
  
  /**
   * Handle delivery in process mode
   * @param item Queue item
   */
  private async handleProcessDelivery(item: IQueueItem): Promise<any>;
  
  /**
   * Update delivery options
   * @param options New options
   */
  public updateOptions(options: Partial<IMultiModeDeliveryOptions>): void;
  
  /**
   * Get delivery statistics
   */
  public getStats(): IDeliveryStats;
}
```

### 4.6 DcRouter Integration with EmailConfig

```typescript
/**
 * DcRouter options with emailConfig
 */
export interface IDcRouterOptions {
  // Direct SmartProxy configuration for general HTTP/HTTPS and TCP/SNI traffic
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Consolidated email configuration
  emailConfig?: IEmailConfig;
  
  // Shared TLS configuration
  tls?: {
    contactEmail: string;
    domain?: string;
    certPath?: string;
    keyPath?: string;
  };
  
  // DNS server configuration
  dnsServerConfig?: plugins.smartdns.IDnsServerOptions;
}

/**
 * DcRouter with consolidated email handling
 */
export class DcRouter {
  // Core services
  public smartProxy?: plugins.smartproxy.SmartProxy;
  public dnsServer?: plugins.smartdns.DnsServer;
  
  // Unified email components
  public emailServer?: UnifiedEmailServer;
  public domainRouter?: DomainRouter;
  public multiModeProcessor?: MultiModeProcessor;
  public deliveryQueue?: UnifiedDeliveryQueue;
  public deliverySystem?: MultiModeDeliverySystem;
  
  constructor(options: IDcRouterOptions);
  
  /**
   * Start DcRouter services
   */
  public async start(): Promise<void>;
  
  /**
   * Stop DcRouter services
   */
  public async stop(): Promise<void>;
  
  /**
   * Set up email handling
   */
  private async setupEmailHandling(): Promise<void>;
  
  /**
   * Update email configuration
   * @param config New email configuration
   */
  public async updateEmailConfig(config: IEmailConfig): Promise<void>;
  
  /**
   * Update domain rules
   * @param rules New domain rules
   */
  public async updateDomainRules(rules: IDomainRule[]): Promise<void>;
  
  /**
   * Get DcRouter statistics
   */
  public getStats(): IDcRouterStats;
}
```

## 5. Implementation Phases

### Phase 1: Core Architecture and Pattern Matching
- [x] Create the UnifiedEmailServer class foundation
- [x] Implement the DomainRouter with glob pattern matching
- [x] Build pattern priority system (most specific match first)
- [x] Create pattern caching mechanism for performance
- [x] Implement validation for email patterns
- [x] Build test suite for pattern matching system

### Phase 2: Multi-Modal Processing Framework
- [x] Build the MultiModeProcessor class
- [x] Implement mode-specific handlers (forward, MTA, process)
- [x] Create processing pipeline for each mode
- [x] Implement content scanning for process mode
- [x] Build shared services infrastructure
- [x] Add validation for mode-specific configurations

### Phase 3: Unified Queue and Delivery System
- [x] Implement the UnifiedDeliveryQueue
- [x] Create persistent storage for all processing modes
- [x] Build the MultiModeDeliverySystem
- [x] Implement mode-specific delivery handlers
- [x] Create shared retry logic with exponential backoff
- [x] Add delivery tracking and notification

### Phase 4: DcRouter Integration
- [x] Implement the consolidated emailConfig interface
- [x] Integrate all components into DcRouter
- [x] Add configuration validation
- [x] Create management APIs for updating rules
- [x] Implement migration support for existing configurations
- [x] Build mode-specific metrics and logging

### Phase 5: Testing and Documentation
- [x] Create comprehensive unit tests for all components
- [x] Implement integration tests for all processing modes
- [x] Test pattern matching with complex scenarios
- [x] Create performance tests for high-volume scenarios
- [x] Build detailed documentation and examples
- [x] Identify and document legacy components to be deprecated (EmailDomainRouter)
- [x] Remove deprecated components (EmailDomainRouter)

## 6. Technical Requirements

### 6.1 Dependencies
- SMTP server library (smtp-server or similar)
- Email parsing library (mailparser or similar)
- MIME handling library
- DKIM signing library
- Queue management system (optional Redis support)
- Cryptographic libraries for TLS and authentication

### 6.2 Performance Targets
- Handle 1000+ concurrent SMTP connections
- Process 100+ messages per second on standard hardware
- Support message sizes up to 50MB
- Maintain delivery queue of 100,000+ messages
- Sub-second processing time for standard emails

### 6.3 Security Requirements
- Full TLS support with modern cipher configurations
- Authentication verification and rate limiting
- Input validation for all SMTP commands
- Secure storage of queued emails
- Proper error handling to prevent information leakage
- Access controls based on IP addresses and authentication

## 7. API Examples

### 7.1 Basic DcRouter SMTP Configuration

```typescript
const dcRouter = new DcRouter({
  // HTTP configuration...
  
  smtpConfig: {
    ports: [25, 587, 465],
    hostname: 'mail.example.com',
    maxMessageSize: 50 * 1024 * 1024, // 50MB
    
    // TLS configuration
    tls: {
      certPath: '/path/to/cert.pem',
      keyPath: '/path/to/key.pem'
    },
    
    // Domain routing
    domainConfigs: [
      {
        domains: ['example.com', '*.example.com'],
        targetIPs: ['mail1.example.com', 'mail2.example.com'],
        port: 25,
        useTls: true
      }
    ],
    
    // Default routing
    defaultServer: 'fallback-mail.example.com',
    defaultPort: 25,
    useTls: true,
    
    // Queue settings
    queueStorage: 'disk',
    persistentPath: '/var/mail/queue',
    maxRetries: 5
  }
});
```

### 7.2 Advanced Configuration with Processing

```typescript
const dcRouter = new DcRouter({
  // HTTP configuration...
  
  smtpConfig: {
    // Basic settings
    ports: [25, 587, 465],
    hostname: 'mail.example.com',
    
    // Domain routing with advanced features
    domainConfigs: [
      {
        domains: ['example.com', '*.example.com'],
        targetIPs: ['mail1.example.com', 'mail2.example.com'],
        port: 25,
        useTls: true,
        // Add custom headers
        addHeaders: true,
        headerInfo: [
          { name: 'X-Processed-By', value: 'gateway' },
          { name: 'X-Scanned', value: 'true' }
        ],
        // Sign with DKIM
        signDkim: true,
        dkimOptions: {
          domainName: 'example.com',
          keySelector: 'mail',
          privateKey: '...'
        },
        // Rate limiting
        rateLimits: {
          maxMessagesPerMinute: 100,
          maxRecipientsPerMessage: 50
        }
      }
    ],
    
    // Content scanning
    contentScanning: true,
    scanners: [
      {
        type: 'spam',
        threshold: 5.0,
        action: 'tag'
      },
      {
        type: 'virus',
        action: 'reject'
      },
      {
        type: 'attachment',
        blockedExtensions: ['.exe', '.bat', '.vbs'],
        action: 'reject'
      }
    ],
    
    // Transformations
    transformations: [
      {
        type: 'addHeader',
        header: 'X-Gateway',
        value: 'DcRouter 1.0'
      },
      {
        type: 'dkimSign',
        domains: ['example.com']
      }
    ]
  }
});
```

## 8. Extensibility Points

### 8.1 Plugin Architecture
- Custom content scanners
- Custom transformation handlers
- Routing rule extensions
- Authentication providers
- Queue storage backends

### 8.2 Event System
- Connection events (connect, disconnect, error)
- Message events (received, processed, queued, delivered)
- Error events (delivery failure, processing error)
- Performance events (queue size, processing time)
- Security events (authentication failure, policy violation)

## 9. Migration Plan

### 9.1 From Simple Proxy to Store-and-Forward
- [x] Create compatibility layer for existing configurations
- [x] Implement graceful transition from connection proxy to full processing
- [x] Add configuration validation to ensure smooth migration
- [x] Allow components to coexist for flexible deployment options
- [x] Provide documentation with comments for migrating existing deployments
- [x] Remove deprecated files after migration to consolidated approach
  - [x] Removed EmailDomainRouter class
  - [x] Updated imports and references to use the new DomainRouter

### 9.2 Backward Compatibility
- [x] Maintain support for basic proxy functionality
- [x] Allow MTA, SMTP forwarding, and store-and-forward to work together
- [x] Support multiple concurrent email handling approaches
- [x] Enable hybrid deployments with different services running simultaneously

### 9.3 Enhanced Coexistence Support
- [x] Modified the startup sequence to enable concurrent operation of multiple services:
  - MTA service can now run alongside SMTP forwarding and store-and-forward processing
  - Store-and-forward SMTP processing can run alongside MTA and SMTP forwarding
  - SMTP forwarding now uses its own dedicated SmartProxy instance (smtpProxy)
- [x] Updated component lifecycle management to properly start/stop all services
- [x] Added clear separation between service instances to avoid conflicts
- [x] Ensured configuration updates for one component don't affect others

## 10. SmartProxy Integration

### 10.1 SmartProxy Configuration Handling
- [x] Implement comprehensive support for SmartProxy configuration
  - Passed through all SmartProxy options directly in DcRouter's configuration
  - Added support for all SmartProxy domain configuration features
  - Implemented proper handling of SmartProxy events and callbacks
- [x] Added documentation on SmartProxy configuration:
  - Documented how all SmartProxy features are available through DcRouter
  - Added examples of different configuration approaches
  - Provided guidance in code comments

### 10.2 SMTP Integration with SmartProxy
- [x] Ensured store-and-forward SMTP works alongside SmartProxy
  - Handled SMTP ports separately from HTTP/HTTPS ports
  - Prevented port conflicts between SmartProxy and SMTP server
  - Created code structure showing SmartProxy and SMTP working together
- [x] Implemented combined usage model:
  - HTTP/HTTPS traffic using SmartProxy configuration
  - SMTP traffic using store-and-forward for advanced processing
  - Added support for multi-service environments

## 11. Documentation Requirements

### 11.1 Code Documentation
- [x] Comprehensive JSDoc comments for all classes and methods
- [x] Interface definitions with detailed parameter descriptions
- [x] Example code snippets for common operations
- [x] Architecture documentation with component diagrams
- [x] Decision logs for key design choices

### 11.2 User Documentation
- [x] Getting started guide with configuration approach selection guidance
- [x] Complete configuration reference for both approaches
- [x] Deployment scenarios and examples
- [x] Troubleshooting guide
- [x] Performance tuning recommendations
- [x] Security best practices

### 11.3 Direct SmartProxy Configuration Guide
- [x] Detailed guide on using SmartProxy's domain configuration capabilities
- [x] Examples of complex routing scenarios with SmartProxy
- [x] Performance optimization tips for SmartProxy configurations
- [x] Security settings for SmartProxy deployments