platformservice/readme.plan.md

DcRouter Consolidated Email Configuration Plan

Overview

This plan outlines the consolidation of all email-related functionality in DcRouter under a unified emailConfig interface. This new approach combines MTA, SMTP forwarding, and store-and-forward processing into a single, pattern-based routing system that:

1. Uses glob patterns for domain matching (e.g., *@task.vc or *@*.example.net)
2. Shares ports across all email handling methods (25, 587, 465)
3. Allows different handling modes based on email domain patterns
4. Provides a flexible configuration interface for all email-related functionality

This consolidated approach simplifies configuration while enhancing flexibility, allowing domain-specific handling where, for example, *@task.vc emails are forwarded to another SMTP server while *@lossless.digital emails are processed by the MTA for programmatic use.

0. Configuration Approaches

0.1 Core SmartProxy Direct Configuration

DcRouter should leverage SmartProxy's configuration directly, exposing SmartProxy's full domain configuration options to give users maximum flexibility for all HTTP/HTTPS and TCP/SNI traffic:

interface IDcRouterOptions {
  // Direct SmartProxy configuration for general HTTP/HTTPS and TCP/SNI traffic
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Consolidated email configuration
  emailConfig?: IEmailConfig;
  
  // Other DcRouter options...
}

This approach allows direct configuration of SmartProxy's powerful domain-based routing, giving full control over HTTP/HTTPS and SNI-based traffic:

const dcRouter = new DcRouter({
  // Direct SmartProxy configuration for HTTP/HTTPS traffic
  smartProxyConfig: {
    fromPort: 443,
    toPort: 8080,
    targetIP: '10.0.0.10',
    sniEnabled: true,
    acme: {
      port: 80,
      enabled: true,
      autoRenew: true,
      useProduction: true,
      renewThresholdDays: 30,
      accountEmail: 'admin@example.com'
    },
    globalPortRanges: [
      { from: 80, to: 80 },
      { from: 443, to: 443 }
    ],
    
    // SmartProxy's full domain configuration flexibility
    domainConfigs: [
      {
        domains: ['example.com', 'www.example.com'],
        allowedIPs: ['0.0.0.0/0'],
        blockedIPs: ['1.2.3.4/32'],
        targetIPs: ['10.0.0.10', '10.0.0.11'],
        portRanges: [
          { from: 80, to: 80 },
          { from: 443, to: 443 }
        ],
        connectionTimeout: 60000,
        useNetworkProxy: true
      },
      // Additional domain configurations...
    ]
  },
  
  // Consolidated email configuration
  emailConfig: {
    // Email handling configuration...
  }
});

0.2 Consolidated Email Configuration

We'll implement a unified email configuration approach that combines MTA, SMTP forwarding, and store-and-forward processing into a single pattern-based routing system:

interface IDcRouterOptions {
  // HTTP/HTTPS configuration
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Consolidated email handling
  emailConfig?: {
    // Global email server settings
    ports: number[];
    hostname: string;
    maxMessageSize?: number;
    
    // TLS configuration for all email services
    tls?: {
      certPath?: string;
      keyPath?: string;
      caPath?: string;
      minVersion?: string;
    };
    
    // Authentication for inbound connections
    auth?: {
      required?: boolean;
      methods?: ('PLAIN' | 'LOGIN' | 'OAUTH2')[];
      users?: Array<{username: string, password: string}>;
    };
    
    // Default routing for unmatched domains
    defaultMode: 'forward' | 'mta' | 'process';
    defaultServer?: string;
    defaultPort?: number;
    defaultTls?: boolean;
    
    // Domain-specific rules with glob pattern support
    domainRules: Array<{
      // Domain pattern (e.g., "*@example.com", "*@*.example.net")
      pattern: string;
      
      // Handling mode for this pattern
      mode: 'forward' | 'mta' | 'process';
      
      // Forward mode configuration
      target?: {
        server: string;
        port?: number;
        useTls?: boolean;
        authentication?: {
          user?: string;
          pass?: string;
        };
      };
      
      // MTA mode configuration
      mtaOptions?: {
        domain?: string;
        allowLocalDelivery?: boolean;
        localDeliveryPath?: string;
        dkimSign?: boolean;
        dkimOptions?: {
          domainName: string;
          keySelector: string;
          privateKey: string;
        };
      };
      
      // Process mode configuration
      contentScanning?: boolean;
      scanners?: Array<{
        type: 'spam' | 'virus' | 'attachment';
        threshold?: number;
        action: 'tag' | 'reject';
        blockedExtensions?: string[];
      }>;
      
      transformations?: Array<{
        type: string;
        [key: string]: any;
      }>;
      
      // Rate limits for this domain
      rateLimits?: {
        maxMessagesPerMinute?: number;
        maxRecipientsPerMessage?: number;
      };
    }>;
    
    // Queue configuration for all email processing
    queue?: {
      storageType?: 'memory' | 'disk';
      persistentPath?: string;
      maxRetries?: number;
      baseRetryDelay?: number;
      maxRetryDelay?: number;
    };
    
    // Advanced MTA settings
    mtaGlobalOptions?: {
      smtpBanner?: string;
      maxConnections?: number;
      connTimeout?: number;
      spoolDir?: string;
      dkimKeyPath?: string;
    };
  };
}

## 1. Core Architecture for Consolidated Email Processing

### 1.1 Unified Email Server
- [x] Create a unified email server component
  - Build on existing SmtpServer class but with enhanced routing capabilities
  - Configure to listen on standard ports (25, 587, 465) for all email handling
  - Implement TLS support (STARTTLS and implicit TLS)
  - Add support for authentication methods (PLAIN, LOGIN, OAUTH2)
  - Set up size limits and connection timeouts

### 1.2 Pattern-Based Domain Router
- [x] Create pattern matching system for email domains
  - Implement glob pattern matching for email addresses
  - Support patterns like `*@domain.com`, `*@*.domain.com`
  - Create priority-based matching system (most specific match wins)
  - Build rule evaluation engine to determine processing mode
  - Implement a fast lookup mechanism for incoming emails

### 1.3 Multi-Modal Processing System
- [x] Create a unified processing system with multiple modes
  - Forward mode: SMTP proxy functionality with enhanced routing
  - MTA mode: Programmatic email handling with local delivery options
  - Process mode: Full store-and-forward pipeline with content scanning
  - Implement mode-specific configuration and handlers
  - Create fallback handling for unmatched domains

### 1.4 Shared Infrastructure
- [x] Develop shared components across all email handling modes
  - [x] Create unified delivery queue for all outbound email
  - [x] Implement shared authentication system
  - [x] Build common TLS and certificate management
  - [x] Create uniform logging and metrics collection
  - [x] Develop shared rate limiting and throttling

## 2. Consolidated Email Processing Features

### 2.1 Pattern-Based Routing
- [x] Implement glob pattern-based email routing
  - Create glob pattern matching for both domains and full email addresses
  - Support wildcards for domains, subdomains, and local parts (e.g., `*@domain.com`, `user@*.domain.com`)
  - Add support for pattern matching priorities (most specific wins)
  - Implement cacheable pattern matching for performance
  - Create comprehensive test suite for pattern matching

### 2.2 Multi-Modal Processing
- [x] Develop multiple email handling modes
  - Forward mode: Simple SMTP forwarding to another server with enhanced routing
  - MTA mode: Process with the MTA for programmatic handling and local delivery
  - Process mode: Full store-and-forward processing with content scanning
  - Add mode-specific configuration validation
  - Implement seamless mode transitions based on patterns

### 2.3 Content Inspection and Transformation
- [x] Enhance content inspection for processing mode
  - [x] Improve MIME parsing and content extraction capabilities
  - [x] Enhance attachment scanning and filtering
  - [x] Add text analysis for spam and phishing detection
  - [x] Create more robust transformation framework
  - [x] Support content-based routing decisions

### 2.4 Unified Rate Limiting and Traffic Control
- [x] Build unified rate limiting across all modes
  - [x] Implement pattern-based rate limits
  - [x] Create hierarchical rate limiting (global, pattern, IP)
  - [x] Add real-time rate limit monitoring
  - [x] Develop traffic shaping capabilities
  - [x] Implement backpressure mechanisms for overload protection

## 3. DcRouter Integration

### 3.1 Unified Configuration Interface
- [x] Implement the consolidated emailConfig interface
  - Create the IEmailConfig interface with all required components
  - Replace existing SMTP, forwarding, and MTA configs with unified approach
  - Add backward compatibility layer for existing configurations
  - Provide comprehensive validation for the new configuration format
  - Add clear documentation and examples in code comments

### 3.2 Enhanced Management API
- [x] Develop enhanced management API for consolidated email handling
  - Create unified status reporting across all modes
  - Implement pattern-based rule management (add, update, remove)
  - Add comprehensive queue management across all modes
  - Create mode-specific monitoring endpoints
  - Implement enhanced configuration update methods

### 3.3 Unified Metrics and Logging
- [x] Create a unified metrics system for all email handling
  - Develop pattern-based metrics collection
  - Implement mode-specific performance metrics
  - Create pattern rule effectiveness measurements
  - Add real-time monitoring capabilities
  - Design comprehensive logging with correlation IDs

## 4. Detailed Component Specifications

### 4.0 DcRouter Configuration Extension

```typescript
export interface IDcRouterOptions {
  // Core configuration options
  
  // Direct SmartProxy configuration - gives full control over all TCP/SNI traffic
  // including HTTP, HTTPS, and any other TCP-based protocol
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Unified email configuration for all email handling modes
  emailConfig?: IEmailConfig;
  
  // Shared configurations
  tls?: {
    contactEmail: string;
    domain?: string;
    certPath?: string;
    keyPath?: string;
  };
  
  // Other DcRouter options
  dnsServerConfig?: plugins.smartdns.IDnsServerOptions;
}

/**
 * Consolidated email configuration interface
 */
export interface IEmailConfig {
  // Email server settings
  ports: number[];
  hostname: string;
  maxMessageSize?: number;
  
  // TLS configuration for email server
  tls?: {
    certPath?: string;
    keyPath?: string;
    caPath?: string;
    minVersion?: string;
  };
  
  // Authentication for inbound connections
  auth?: {
    required?: boolean;
    methods?: ('PLAIN' | 'LOGIN' | 'OAUTH2')[];
    users?: Array<{username: string, password: string}>;
  };
  
  // Default routing for unmatched domains
  defaultMode: EmailProcessingMode;
  defaultServer?: string;
  defaultPort?: number;
  defaultTls?: boolean;
  
  // Domain rules with glob pattern support
  domainRules: IDomainRule[];
  
  // Queue configuration for all email processing
  queue?: {
    storageType?: 'memory' | 'disk';
    persistentPath?: string;
    maxRetries?: number;
    baseRetryDelay?: number;
    maxRetryDelay?: number;
  };
  
  // Advanced MTA settings
  mtaGlobalOptions?: IMtaOptions;
}

/**
 * Email processing modes
 */
export type EmailProcessingMode = 'forward' | 'mta' | 'process';

/**
 * Domain rule interface for pattern-based routing
 */
export interface IDomainRule {
  // Domain pattern (e.g., "*@example.com", "*@*.example.net")
  pattern: string;
  
  // Handling mode for this pattern
  mode: EmailProcessingMode;
  
  // Forward mode configuration
  target?: {
    server: string;
    port?: number;
    useTls?: boolean;
    authentication?: {
      user?: string;
      pass?: string;
    };
  };
  
  // MTA mode configuration
  mtaOptions?: IMtaOptions;
  
  // Process mode configuration
  contentScanning?: boolean;
  scanners?: IContentScanner[];
  transformations?: ITransformation[];
  
  // Rate limits for this domain
  rateLimits?: {
    maxMessagesPerMinute?: number;
    maxRecipientsPerMessage?: number;
  };
}

/**
 * MTA options interface
 */
export interface IMtaOptions {
  domain?: string;
  allowLocalDelivery?: boolean;
  localDeliveryPath?: string;
  dkimSign?: boolean;
  dkimOptions?: {
    domainName: string;
    keySelector: string;
    privateKey: string;
  };
  smtpBanner?: string;
  maxConnections?: number;
  connTimeout?: number;
  spoolDir?: string;
}

/**
 * Content scanner interface
 */
export interface IContentScanner {
  type: 'spam' | 'virus' | 'attachment';
  threshold?: number;
  action: 'tag' | 'reject';
  blockedExtensions?: string[];
}

/**
 * Transformation interface
 */
export interface ITransformation {
  type: string;
  [key: string]: any;
}

4.1 UnifiedEmailServer Class

/**
 * Options for the unified email server
 */
export interface IUnifiedEmailServerOptions {
  // Base server options
  ports: number[];
  hostname: string;
  banner?: string;
  
  // Authentication options
  auth?: {
    required?: boolean;
    methods?: ('PLAIN' | 'LOGIN' | 'OAUTH2')[];
    users?: Array<{username: string, password: string}>;
  };
  
  // TLS options
  tls?: {
    certPath?: string;
    keyPath?: string;
    caPath?: string;
    minVersion?: string;
    ciphers?: string;
  };
  
  // Limits
  maxMessageSize?: number;
  maxClients?: number;
  maxConnections?: number;
  
  // Connection options
  connectionTimeout?: number;
  socketTimeout?: number;
  
  // Domain rules
  domainRules: IDomainRule[];
  
  // Default handling for unmatched domains
  defaultMode: EmailProcessingMode;
  defaultServer?: string;
  defaultPort?: number;
  defaultTls?: boolean;
}

/**
 * Unified email server that handles all email traffic with pattern-based routing
 */
export class UnifiedEmailServer extends EventEmitter {
  constructor(options: IUnifiedEmailServerOptions);
  
  // Start and stop the server
  public start(): Promise<void>;
  public stop(): Promise<void>;
  
  // Core event handlers
  private onConnect(session: ISmtpSession, callback: (err?: Error) => void): void;
  private onAuth(auth: IAuthData, session: ISmtpSession, callback: (err?: Error, user?: any) => void): void;
  private onMailFrom(address: {address: string}, session: ISmtpSession, callback: (err?: Error) => void): void;
  private onRcptTo(address: {address: string}, session: ISmtpSession, callback: (err?: Error) => void): void;
  private onData(stream: Readable, session: ISmtpSession, callback: (err?: Error) => void): void;
  
  // Pattern matching and routing
  private matchDomainRule(address: string): IDomainRule | null;
  private determineProcessingMode(session: ISmtpSession): EmailProcessingMode;
  
  // Mode-specific processing
  private handleForwardMode(message: any, session: ISmtpSession, rule: IDomainRule): Promise<void>;
  private handleMtaMode(message: any, session: ISmtpSession, rule: IDomainRule): Promise<void>;
  private handleProcessMode(message: any, session: ISmtpSession, rule: IDomainRule): Promise<void>;
  
  // Helper methods
  private parseEmail(rawData: string): Promise<any>;
  private isIpAllowed(ip: string, rule: IDomainRule): boolean;
  private createDeliveryJob(message: any, rule: IDomainRule): IDeliveryJob;
  
  // Configuration updates
  public updateOptions(options: Partial<IUnifiedEmailServerOptions>): void;
  public updateDomainRules(rules: IDomainRule[]): void;
  
  // Server stats
  public getStats(): IServerStats;
}

4.2 DomainRouter Class

/**
 * Options for the domain-based router
 */
export interface IDomainRouterOptions {
  // Domain rules with glob pattern matching
  domainRules: IDomainRule[];
  
  // Default handling for unmatched domains
  defaultMode: EmailProcessingMode;
  defaultServer?: string;
  defaultPort?: number;
  defaultTls?: boolean;
  
  // Pattern matching options
  caseSensitive?: boolean;
  priorityOrder?: 'most-specific' | 'first-match';
  
  // Cache settings for pattern matching
  enableCache?: boolean;
  cacheSize?: number;
}

/**
 * A pattern matching and routing class for email domains
 */
export class DomainRouter {
  constructor(options: IDomainRouterOptions);
  
  /**
   * Match an email address against defined rules
   * @param email Email address to match
   * @returns The matching rule or null if no match
   */
  public matchRule(email: string): IDomainRule | null;
  
  /**
   * Check if email matches a specific pattern
   * @param email Email address to check
   * @param pattern Pattern to check against
   * @returns True if matching, false otherwise
   */
  public matchesPattern(email: string, pattern: string): boolean;
  
  /**
   * Get all rules that match an email address
   * @param email Email address to match
   * @returns Array of matching rules
   */
  public getAllMatchingRules(email: string): IDomainRule[];
  
  /**
   * Add a new routing rule
   * @param rule Domain rule to add
   */
  public addRule(rule: IDomainRule): void;
  
  /**
   * Update an existing rule
   * @param pattern Pattern to update
   * @param updates Updates to apply
   */
  public updateRule(pattern: string, updates: Partial<IDomainRule>): boolean;
  
  /**
   * Remove a rule
   * @param pattern Pattern to remove
   */
  public removeRule(pattern: string): boolean;
  
  /**
   * Get rule by pattern
   * @param pattern Pattern to find
   */
  public getRule(pattern: string): IDomainRule | null;
  
  /**
   * Get all rules
   */
  public getRules(): IDomainRule[];
  
  /**
   * Update options
   * @param options New options
   */
  public updateOptions(options: Partial<IDomainRouterOptions>): void;
  
  /**
   * Clear pattern matching cache
   */
  public clearCache(): void;
}

4.3 MultiModeProcessor Class

/**
 * Processing modes
 */
export type EmailProcessingMode = 'forward' | 'mta' | 'process';

/**
 * Processor options
 */
export interface IMultiModeProcessorOptions {
  // Processing options
  maxParallelProcessing?: number;
  processingTimeout?: number;
  
  // Mode handlers
  forwardHandler?: IForwardHandler;
  mtaHandler?: IMtaHandler;
  processHandler?: IProcessHandler;
  
  // Queue configuration
  queue?: {
    storageType?: 'memory' | 'disk';
    persistentPath?: string;
    maxRetries?: number;
    baseRetryDelay?: number;
    maxRetryDelay?: number;
  };
  
  // Shared services
  sharedServices?: {
    dkimSigner?: IDkimSigner;
    contentScanner?: IContentScanner;
    rateLimiter?: IRateLimiter;
  };
}

/**
 * Processes emails using different modes based on domain rules
 */
export class MultiModeProcessor extends EventEmitter {
  constructor(options: IMultiModeProcessorOptions);
  
  /**
   * Process an email using the appropriate mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   * @param mode Processing mode
   */
  public async processEmail(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule,
    mode: EmailProcessingMode
  ): Promise<IProcessingResult>;
  
  /**
   * Handle email in forward mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   */
  private async handleForwardMode(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule
  ): Promise<IProcessingResult>;
  
  /**
   * Handle email in MTA mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   */
  private async handleMtaMode(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule
  ): Promise<IProcessingResult>;
  
  /**
   * Handle email in process mode
   * @param message Parsed email message
   * @param session SMTP session
   * @param rule Matching domain rule
   */
  private async handleProcessMode(
    message: any,
    session: ISmtpSession,
    rule: IDomainRule
  ): Promise<IProcessingResult>;
  
  /**
   * Update processor options
   * @param options New options
   */
  public updateOptions(options: Partial<IMultiModeProcessorOptions>): void;
  
  /**
   * Get processor statistics
   */
  public getStats(): IProcessorStats;
}

4.4 UnifiedDeliveryQueue Class

/**
 * Queue item status
 */
export type QueueItemStatus = 'pending' | 'processing' | 'delivered' | 'failed' | 'deferred';

/**
 * Queue item interface
 */
export interface IQueueItem {
  id: string;
  processingMode: EmailProcessingMode;
  processingResult: any;
  rule: IDomainRule;
  status: QueueItemStatus;
  attempts: number;
  nextAttempt: Date;
  lastError?: string;
  createdAt: Date;
  updatedAt: Date;
  deliveredAt?: Date;
}

/**
 * Queue options interface
 */
export interface IQueueOptions {
  // Storage options
  storageType?: 'memory' | 'disk';
  persistentPath?: string;
  
  // Queue behavior
  checkInterval?: number;
  maxQueueSize?: number;
  maxPerDestination?: number;
  
  // Delivery attempts
  maxRetries?: number;
  baseRetryDelay?: number;
  maxRetryDelay?: number;
}

/**
 * A unified queue for all email modes
 */
export class UnifiedDeliveryQueue extends EventEmitter {
  constructor(options: IQueueOptions);
  
  /**
   * Initialize the queue
   */
  public async initialize(): Promise<void>;
  
  /**
   * Add an item to the queue
   * @param processingResult Processing result to queue
   * @param mode Processing mode
   * @param rule Domain rule
   */
  public async enqueue(processingResult: any, mode: EmailProcessingMode, rule: IDomainRule): Promise<string>;
  
  /**
   * Get an item from the queue
   * @param id Item ID
   */
  public getItem(id: string): IQueueItem | undefined;
  
  /**
   * Mark an item as delivered
   * @param id Item ID
   */
  public async markDelivered(id: string): Promise<boolean>;
  
  /**
   * Mark an item as failed
   * @param id Item ID
   * @param error Error message
   */
  public async markFailed(id: string, error: string): Promise<boolean>;
  
  /**
   * Get queue statistics
   */
  public getStats(): IQueueStats;
  
  /**
   * Pause queue processing
   */
  public pause(): void;
  
  /**
   * Resume queue processing
   */
  public resume(): void;
  
  /**
   * Shutdown the queue
   */
  public async shutdown(): Promise<void>;
}

4.5 MultiModeDeliverySystem Class

/**
 * Delivery options
 */
export interface IMultiModeDeliveryOptions {
  // Connection options
  connectionPoolSize?: number;
  socketTimeout?: number;
  
  // Delivery behavior
  concurrentDeliveries?: number;
  sendTimeout?: number;
  
  // TLS options
  verifyCertificates?: boolean;
  tlsMinVersion?: string;
  
  // Mode-specific handlers
  forwardHandler?: IForwardDeliveryHandler;
  mtaHandler?: IMtaDeliveryHandler;
  processHandler?: IProcessDeliveryHandler;
  
  // Rate limiting
  globalRateLimit?: number;
  perPatternRateLimit?: Record<string, number>;
  
  // Event hooks
  onDeliveryStart?: (item: IQueueItem) => Promise<void>;
  onDeliverySuccess?: (item: IQueueItem, result: any) => Promise<void>;
  onDeliveryFailed?: (item: IQueueItem, error: string) => Promise<void>;
}

/**
 * Handles delivery for all email processing modes
 */
export class MultiModeDeliverySystem extends EventEmitter {
  constructor(queue: UnifiedDeliveryQueue, options: IMultiModeDeliveryOptions);
  
  /**
   * Start the delivery system
   */
  public async start(): Promise<void>;
  
  /**
   * Stop the delivery system
   */
  public async stop(): Promise<void>;
  
  /**
   * Deliver an item from the queue
   * @param item Queue item to deliver
   */
  private async deliverItem(item: IQueueItem): Promise<void>;
  
  /**
   * Handle delivery in forward mode
   * @param item Queue item
   */
  private async handleForwardDelivery(item: IQueueItem): Promise<any>;
  
  /**
   * Handle delivery in MTA mode
   * @param item Queue item
   */
  private async handleMtaDelivery(item: IQueueItem): Promise<any>;
  
  /**
   * Handle delivery in process mode
   * @param item Queue item
   */
  private async handleProcessDelivery(item: IQueueItem): Promise<any>;
  
  /**
   * Update delivery options
   * @param options New options
   */
  public updateOptions(options: Partial<IMultiModeDeliveryOptions>): void;
  
  /**
   * Get delivery statistics
   */
  public getStats(): IDeliveryStats;
}

4.6 DcRouter Integration with EmailConfig

/**
 * DcRouter options with emailConfig
 */
export interface IDcRouterOptions {
  // Direct SmartProxy configuration for general HTTP/HTTPS and TCP/SNI traffic
  smartProxyConfig?: plugins.smartproxy.ISmartProxyOptions;
  
  // Consolidated email configuration
  emailConfig?: IEmailConfig;
  
  // Shared TLS configuration
  tls?: {
    contactEmail: string;
    domain?: string;
    certPath?: string;
    keyPath?: string;
  };
  
  // DNS server configuration
  dnsServerConfig?: plugins.smartdns.IDnsServerOptions;
}

/**
 * DcRouter with consolidated email handling
 */
export class DcRouter {
  // Core services
  public smartProxy?: plugins.smartproxy.SmartProxy;
  public dnsServer?: plugins.smartdns.DnsServer;
  
  // Unified email components
  public emailServer?: UnifiedEmailServer;
  public domainRouter?: DomainRouter;
  public multiModeProcessor?: MultiModeProcessor;
  public deliveryQueue?: UnifiedDeliveryQueue;
  public deliverySystem?: MultiModeDeliverySystem;
  
  constructor(options: IDcRouterOptions);
  
  /**
   * Start DcRouter services
   */
  public async start(): Promise<void>;
  
  /**
   * Stop DcRouter services
   */
  public async stop(): Promise<void>;
  
  /**
   * Set up email handling
   */
  private async setupEmailHandling(): Promise<void>;
  
  /**
   * Update email configuration
   * @param config New email configuration
   */
  public async updateEmailConfig(config: IEmailConfig): Promise<void>;
  
  /**
   * Update domain rules
   * @param rules New domain rules
   */
  public async updateDomainRules(rules: IDomainRule[]): Promise<void>;
  
  /**
   * Get DcRouter statistics
   */
  public getStats(): IDcRouterStats;
}

5. Implementation Phases

Phase 1: Core Architecture and Pattern Matching

• Create the UnifiedEmailServer class foundation
• Implement the DomainRouter with glob pattern matching
• Build pattern priority system (most specific match first)
• Create pattern caching mechanism for performance
• Implement validation for email patterns
• Build test suite for pattern matching system

Phase 2: Multi-Modal Processing Framework

• Build the MultiModeProcessor class
• Implement mode-specific handlers (forward, MTA, process)
• Create processing pipeline for each mode
• Implement content scanning for process mode
• Build shared services infrastructure
• Add validation for mode-specific configurations

Phase 3: Unified Queue and Delivery System

• Implement the UnifiedDeliveryQueue
• Create persistent storage for all processing modes
• Build the MultiModeDeliverySystem
• Implement mode-specific delivery handlers
• Create shared retry logic with exponential backoff
• Add delivery tracking and notification

Phase 4: DcRouter Integration

• Implement the consolidated emailConfig interface
• Integrate all components into DcRouter
• Add configuration validation
• Create management APIs for updating rules
• Implement migration support for existing configurations
• Build mode-specific metrics and logging

Phase 5: Testing and Documentation

• Create comprehensive unit tests for all components
• Implement integration tests for all processing modes
• Test pattern matching with complex scenarios
• Create performance tests for high-volume scenarios
• Build detailed documentation and examples
• Identify and document legacy components to be deprecated (EmailDomainRouter)
• Remove deprecated components (EmailDomainRouter)

6. Technical Requirements

6.1 Dependencies

• SMTP server library (smtp-server or similar)
• Email parsing library (mailparser or similar)
• MIME handling library
• DKIM signing library
• Queue management system (optional Redis support)
• Cryptographic libraries for TLS and authentication

6.2 Performance Targets

• Handle 1000+ concurrent SMTP connections
• Process 100+ messages per second on standard hardware
• Support message sizes up to 50MB
• Maintain delivery queue of 100,000+ messages
• Sub-second processing time for standard emails

6.3 Security Requirements

• Full TLS support with modern cipher configurations
• Authentication verification and rate limiting
• Input validation for all SMTP commands
• Secure storage of queued emails
• Proper error handling to prevent information leakage
• Access controls based on IP addresses and authentication

7. API Examples

7.1 Basic DcRouter SMTP Configuration

const dcRouter = new DcRouter({
  // HTTP configuration...
  
  smtpConfig: {
    ports: [25, 587, 465],
    hostname: 'mail.example.com',
    maxMessageSize: 50 * 1024 * 1024, // 50MB
    
    // TLS configuration
    tls: {
      certPath: '/path/to/cert.pem',
      keyPath: '/path/to/key.pem'
    },
    
    // Domain routing
    domainConfigs: [
      {
        domains: ['example.com', '*.example.com'],
        targetIPs: ['mail1.example.com', 'mail2.example.com'],
        port: 25,
        useTls: true
      }
    ],
    
    // Default routing
    defaultServer: 'fallback-mail.example.com',
    defaultPort: 25,
    useTls: true,
    
    // Queue settings
    queueStorage: 'disk',
    persistentPath: '/var/mail/queue',
    maxRetries: 5
  }
});

7.2 Advanced Configuration with Processing

const dcRouter = new DcRouter({
  // HTTP configuration...
  
  smtpConfig: {
    // Basic settings
    ports: [25, 587, 465],
    hostname: 'mail.example.com',
    
    // Domain routing with advanced features
    domainConfigs: [
      {
        domains: ['example.com', '*.example.com'],
        targetIPs: ['mail1.example.com', 'mail2.example.com'],
        port: 25,
        useTls: true,
        // Add custom headers
        addHeaders: true,
        headerInfo: [
          { name: 'X-Processed-By', value: 'gateway' },
          { name: 'X-Scanned', value: 'true' }
        ],
        // Sign with DKIM
        signDkim: true,
        dkimOptions: {
          domainName: 'example.com',
          keySelector: 'mail',
          privateKey: '...'
        },
        // Rate limiting
        rateLimits: {
          maxMessagesPerMinute: 100,
          maxRecipientsPerMessage: 50
        }
      }
    ],
    
    // Content scanning
    contentScanning: true,
    scanners: [
      {
        type: 'spam',
        threshold: 5.0,
        action: 'tag'
      },
      {
        type: 'virus',
        action: 'reject'
      },
      {
        type: 'attachment',
        blockedExtensions: ['.exe', '.bat', '.vbs'],
        action: 'reject'
      }
    ],
    
    // Transformations
    transformations: [
      {
        type: 'addHeader',
        header: 'X-Gateway',
        value: 'DcRouter 1.0'
      },
      {
        type: 'dkimSign',
        domains: ['example.com']
      }
    ]
  }
});

8. Extensibility Points

8.1 Plugin Architecture

• Custom content scanners
• Custom transformation handlers
• Routing rule extensions
• Authentication providers
• Queue storage backends

8.2 Event System

• Connection events (connect, disconnect, error)
• Message events (received, processed, queued, delivered)
• Error events (delivery failure, processing error)
• Performance events (queue size, processing time)
• Security events (authentication failure, policy violation)

9. Migration Plan

9.1 From Simple Proxy to Store-and-Forward

• Create compatibility layer for existing configurations
• Implement graceful transition from connection proxy to full processing
• Add configuration validation to ensure smooth migration
• Allow components to coexist for flexible deployment options
• Provide documentation with comments for migrating existing deployments
• Remove deprecated files after migration to consolidated approach
  • Removed EmailDomainRouter class
  • Updated imports and references to use the new DomainRouter

9.2 Backward Compatibility

• Maintain support for basic proxy functionality
• Allow MTA, SMTP forwarding, and store-and-forward to work together
• Support multiple concurrent email handling approaches
• Enable hybrid deployments with different services running simultaneously

9.3 Enhanced Coexistence Support

• Modified the startup sequence to enable concurrent operation of multiple services:
  • MTA service can now run alongside SMTP forwarding and store-and-forward processing
  • Store-and-forward SMTP processing can run alongside MTA and SMTP forwarding
  • SMTP forwarding now uses its own dedicated SmartProxy instance (smtpProxy)
• Updated component lifecycle management to properly start/stop all services
• Added clear separation between service instances to avoid conflicts
• Ensured configuration updates for one component don't affect others

10. SmartProxy Integration

10.1 SmartProxy Configuration Handling

• Implement comprehensive support for SmartProxy configuration
  • Passed through all SmartProxy options directly in DcRouter's configuration
  • Added support for all SmartProxy domain configuration features
  • Implemented proper handling of SmartProxy events and callbacks
• Added documentation on SmartProxy configuration:
  • Documented how all SmartProxy features are available through DcRouter
  • Added examples of different configuration approaches
  • Provided guidance in code comments

10.2 SMTP Integration with SmartProxy

• Ensured store-and-forward SMTP works alongside SmartProxy
  • Handled SMTP ports separately from HTTP/HTTPS ports
  • Prevented port conflicts between SmartProxy and SMTP server
  • Created code structure showing SmartProxy and SMTP working together
• Implemented combined usage model:
  • HTTP/HTTPS traffic using SmartProxy configuration
  • SMTP traffic using store-and-forward for advanced processing
  • Added support for multi-service environments

11. Documentation Requirements

11.1 Code Documentation

• Comprehensive JSDoc comments for all classes and methods
• Interface definitions with detailed parameter descriptions
• Example code snippets for common operations
• Architecture documentation with component diagrams
• Decision logs for key design choices

11.2 User Documentation

• Getting started guide with configuration approach selection guidance
• Complete configuration reference for both approaches
• Deployment scenarios and examples
• Troubleshooting guide
• Performance tuning recommendations
• Security best practices

11.3 Direct SmartProxy Configuration Guide

• Detailed guide on using SmartProxy's domain configuration capabilities
• Examples of complex routing scenarios with SmartProxy
• Performance optimization tips for SmartProxy configurations
• Security settings for SmartProxy deployments