dcrouter/readme.md

# dcrouter

**dcrouter: a traffic router intended to be gating your datacenter.**

A comprehensive traffic routing solution that provides unified gateway capabilities for HTTP/HTTPS, TCP/SNI, email (SMTP), and DNS protocols. Designed for enterprises requiring robust traffic management, automatic certificate provisioning, and enterprise-grade email infrastructure.

## Table of Contents

- [Features](#features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Architecture](#architecture)
- [Configuration](#configuration)
- [Email System](#email-system)
- [SmartProxy Routing](#smartproxy-routing)
- [Security Features](#security-features)
- [API Reference](#api-reference)
- [Examples](#examples)
- [Troubleshooting](#troubleshooting)

## Features

### 🌐 **Universal Traffic Router**
- **HTTP/HTTPS routing** with pattern matching and virtual hosts
- **TCP/SNI proxy** for any protocol with TLS termination/passthrough
- **DNS server** with authoritative and dynamic record management
- **Multi-protocol support** on the same infrastructure

### 🔒 **Enterprise Security**
- **Automatic TLS certificates** via ACME with DNS-01 challenges
- **IP reputation checking** and real-time threat detection
- **Content scanning** for spam, viruses, and malicious content
- **Comprehensive security logging** with correlation tracking

### 📧 **Complete Email Infrastructure**
- **Multi-domain SMTP server** on standard ports (25, 587, 465)
- **Pattern-based email routing** with three processing modes
- **DKIM, SPF, DMARC** authentication and verification
- **Enterprise deliverability** with IP warmup and reputation management

### ⚡ **High Performance**
- **Connection pooling** and efficient resource management
- **Load balancing** with automatic failover
- **Rate limiting** at multiple levels
- **Real-time metrics** and monitoring

## Installation

```bash
npm install @serve.zone/dcrouter --save
```

### Prerequisites

- Node.js 18+ with ES modules support
- Valid domain with DNS control (for ACME certificates)
- Cloudflare API token (for DNS challenges)

## Quick Start

### Basic HTTP/HTTPS Router

```typescript
import { DcRouter } from '@serve.zone/dcrouter';

const router = new DcRouter({
  smartProxyConfig: {
    routes: [
      {
        name: 'web-service',
        match: { domains: ['example.com'], ports: [443] },
        action: {
          type: 'forward',
          target: { host: '192.168.1.10', port: 8080 },
          tls: { mode: 'terminate', certificate: 'auto' }
        }
      }
    ],
    acme: {
      email: 'admin@example.com',
      enabled: true,
      useProduction: true
    }
  }
});

await router.start();
console.log('DcRouter started successfully');
```

### Basic Email Router

```typescript
import { DcRouter } from '@serve.zone/dcrouter';

const router = new DcRouter({
  emailConfig: {
    ports: [25, 587, 465],
    hostname: 'mail.example.com',
    routes: [
      {
        name: 'local-mail',
        match: { recipients: '*@example.com' },
        action: {
          type: 'process',
          process: {
            scan: true,
            dkim: true,
            queue: 'normal'
          }
        }
      }
    ],
    tls: {
      keyPath: './certs/key.pem',
      certPath: './certs/cert.pem'
    }
  }
});

await router.start();
```

## Architecture

### System Overview

```mermaid
graph TB
    subgraph "External Traffic"
        HTTP[HTTP/HTTPS Clients]
        SMTP[SMTP Clients]
        TCP[TCP Clients]
        DNS[DNS Queries]
    end
    
    subgraph "DcRouter Core"
        DcRouter[DcRouter Orchestrator]
        SmartProxy[SmartProxy Engine]
        EmailServer[Unified Email Server]
        DnsServer[DNS Server]
        CertManager[Certificate Manager]
    end
    
    subgraph "Backend Services"
        WebServices[Web Services]
        MailServers[Mail Servers]
        Databases[Databases]
        APIs[Internal APIs]
    end
    
    HTTP --> SmartProxy
    TCP --> SmartProxy
    SMTP --> EmailServer
    DNS --> DnsServer
    
    DcRouter --> SmartProxy
    DcRouter --> EmailServer
    DcRouter --> DnsServer
    DcRouter --> CertManager
    
    SmartProxy --> WebServices
    SmartProxy --> APIs
    EmailServer --> MailServers
    EmailServer --> Databases
    
    CertManager -.-> SmartProxy
    CertManager -.-> EmailServer
```

### Core Components

#### **DcRouter Orchestrator**
Central coordination engine that manages all services and provides unified configuration.

#### **SmartProxy Engine**
High-performance HTTP/HTTPS and TCP/SNI proxy with:
- Pattern-based routing
- TLS termination/passthrough
- Load balancing
- Connection pooling

#### **Unified Email Server**
Enterprise-grade SMTP server with:
- Multi-domain support
- Pattern-based routing
- Three processing modes
- Complete authentication stack

#### **Certificate Manager**
Automatic TLS certificate provisioning via ACME with DNS-01 challenges.

## Configuration

### Complete Configuration Interface

```typescript
interface IDcRouterOptions {
  // SmartProxy configuration for HTTP/HTTPS/TCP routing
  smartProxyConfig?: {
    routes: IRouteConfig[];
    acme?: IAcmeConfig;
    allowSessionTicket?: boolean;
  };
  
  // Email system configuration
  emailConfig?: {
    ports: number[];
    hostname: string;
    routes: IEmailRoute[];          // Route-based configuration
    auth?: IAuthConfig;
    tls?: ITlsConfig;
    maxMessageSize?: number;
    rateLimits?: IRateLimitConfig;
    useSocketHandler?: boolean;     // Enable socket-handler mode (no port binding)
  };
  
  // DNS server configuration
  dnsServerConfig?: {
    port?: number;
    authoritative?: boolean;
    records?: IDnsRecord[];
  };
  
  // DNS domain for automatic DNS-over-HTTPS setup
  dnsDomain?: string;               // e.g., 'dns.example.com'
  
  // TLS and certificate configuration
  tls?: {
    contactEmail: string;
    domain: string;
  };
  
  // DNS challenge configuration
  dnsChallenge?: {
    cloudflareApiKey: string;
  };
}
```

### Route Configuration

```typescript
interface IRouteConfig {
  name: string;
  priority?: number;
  match: {
    domains?: string[];
    ports?: number | number[] | { from: number; to: number }[];
  };
  action: {
    type: 'forward' | 'redirect' | 'serve';
    target?: {
      host: string;
      port: number | 'preserve' | ((context: any) => number);
    };
    tls?: {
      mode: 'terminate' | 'passthrough';
      certificate?: 'auto' | string;
    };
    security?: {
      ipAllowList?: string[];
      ipBlockList?: string[];
    };
  };
}
```

## Socket-Handler Mode

DcRouter supports an advanced socket-handler mode that eliminates internal port binding for both DNS and email services. Instead of services listening on internal ports, SmartProxy passes sockets directly to the services.

### DNS Socket-Handler

When `dnsDomain` is configured, DcRouter automatically:
- Sets up DNS server for UDP on port 53
- Creates SmartProxy routes for DNS-over-HTTPS (DoH) on the specified domain
- Uses socket-handler for HTTPS/DoH traffic (no HTTPS port binding)

```typescript
const router = new DcRouter({
  dnsDomain: 'dns.example.com',    // Enables DNS with DoH
  smartProxyConfig: {
    // DNS routes are automatically created
  }
});
```

This creates:
- UDP DNS service on port 53 (standard DNS queries)
- HTTPS routes for `dns.example.com/dns-query` and `dns.example.com/resolve`
- Automatic TLS certificates via Let's Encrypt

### Email Socket-Handler

When `useSocketHandler` is enabled in email config:
- Email server doesn't bind to any ports
- SmartProxy passes sockets directly to email handlers
- Reduces latency and resource usage

```typescript
const router = new DcRouter({
  emailConfig: {
    ports: [25, 587, 465],
    hostname: 'mail.example.com',
    useSocketHandler: true,         // Enable socket-handler mode
    routes: [/* email routes */]
  }
});
```

### Benefits of Socket-Handler Mode

1. **Performance**: Eliminates internal port forwarding overhead
2. **Security**: No exposed internal ports
3. **Resource Efficiency**: Fewer open ports and listeners
4. **Simplified Networking**: Direct socket passing
5. **Automatic Configuration**: Routes created automatically

### Traditional vs Socket-Handler Mode

**Traditional Mode (default):**
```
External Port → SmartProxy → Internal Port → Service
    25             →           10025          → Email
```

**Socket-Handler Mode:**
```
External Port → SmartProxy → Socket Handler → Service
    25             →         (direct socket)  → Email
```

## Email System

### Email Route Actions

#### **Forward Action**
Routes emails to external SMTP servers.

```typescript
{
  name: 'forward-to-internal',
  match: { recipients: '*@company.com' },
  action: {
    type: 'forward',
    forward: {
      host: 'internal-mail.company.com',
      port: 25,
      auth: {
        username: 'relay-user',
        password: 'relay-pass'
      },
      addHeaders: {
        'X-Forwarded-By': 'dcrouter'
      }
    }
  }
}
```

#### **Process Action**
Full Mail Transfer Agent functionality with scanning and delivery queues.

```typescript
{
  name: 'process-notifications',
  match: { recipients: '*@notifications.company.com' },
  action: {
    type: 'process',
    process: {
      scan: true,
      dkim: true,
      queue: 'priority'
    }
  }
}
```

#### **Deliver Action**
Local delivery for mailbox storage.

```typescript
{
  name: 'deliver-local',
  match: { recipients: '*@marketing.company.com' },
  action: {
    type: 'deliver'
  }
}
```

#### **Reject Action**
Reject emails with custom SMTP responses.

```typescript
{
  name: 'reject-spam',
  match: { 
    senders: '*@spam-domain.com',
    sizeRange: { min: 1000000 } // > 1MB
  },
  action: {
    type: 'reject',
    reject: {
      code: 550,
      message: 'Message rejected due to policy'
    }
  }
}
```

### Common Email Routing Patterns

#### **IP-Based Relay**
Allow internal networks to relay through the server:
```typescript
{
  name: 'office-relay',
  priority: 100,
  match: { clientIp: ['192.168.0.0/16', '10.0.0.0/8'] },
  action: {
    type: 'forward',
    forward: { host: 'internal-mail.company.com', port: 25 }
  }
}
```

#### **Domain-Based Routing**
Route different domains to different servers:
```typescript
{
  name: 'partner-domain',
  match: { recipients: '*@partner.com' },
  action: {
    type: 'forward', 
    forward: { host: 'partner-mail.com', port: 587 }
  }
}
```

#### **Authentication-Based Processing**
Different handling for authenticated vs unauthenticated senders:
```typescript
{
  name: 'authenticated-users',
  match: { authenticated: true },
  action: {
    type: 'process',
    process: { scan: false, dkim: true, queue: 'priority' }
  }
},
{
  name: 'unauthenticated-reject',
  match: { authenticated: false },
  action: {
    type: 'reject',
    reject: { code: 550, message: 'Authentication required' }
  }
}
```

#### **Content-Based Filtering**
Filter based on size, subject, or headers:
```typescript
{
  name: 'large-email-reject',
  match: { sizeRange: { min: 25000000 } }, // > 25MB
  action: {
    type: 'reject',
    reject: { code: 552, message: 'Message too large' }
  }
},
{
  name: 'priority-emails',
  match: { 
    headers: { 'X-Priority': 'high' },
    subject: /urgent|emergency/i
  },
  action: {
    type: 'process',
    process: { queue: 'priority' }
  }
}
```

### Email Security Features

#### **Route Matching Patterns**

**Glob Pattern Matching**
```typescript
// Email address patterns
match: { recipients: '*@example.com' }        // All addresses at domain
match: { recipients: 'admin@*' }             // Admin at any domain
match: { senders: ['*@trusted.com', '*@partner.com'] } // Multiple patterns

// CIDR IP matching
match: { clientIp: '192.168.0.0/16' }        // Private subnet
match: { clientIp: ['10.0.0.0/8', '172.16.0.0/12'] } // Multiple ranges

// Header matching
match: { 
  headers: {
    'X-Priority': 'high',
    'Subject': /urgent|emergency/i
  }
}

// Size and content matching
match: {
  sizeRange: { min: 1000, max: 5000000 },    // 1KB to 5MB
  hasAttachments: true,
  subject: /invoice|receipt/i
}
```

#### **Content Scanning**
```typescript
const scanners = [
  {
    type: 'spam',
    threshold: 5.0,
    action: 'tag',
    headers: ['X-Spam-Score', 'X-Spam-Status']
  },
  {
    type: 'virus',
    action: 'reject',
    quarantine: true
  },
  {
    type: 'attachment',
    blockedExtensions: ['.exe', '.bat', '.scr'],
    maxSize: 25 * 1024 * 1024 // 25MB
  }
];
```

## SmartProxy Routing

### HTTP/HTTPS Routing

```typescript
const routes = [
  // API routing with path-based forwarding
  {
    name: 'api-gateway',
    match: {
      domains: ['api.example.com'],
      ports: [443]
    },
    action: {
      type: 'forward',
      target: {
        host: '192.168.1.20',
        port: (context) => {
          // Route based on path
          if (context.path.startsWith('/v1/')) return 8080;
          if (context.path.startsWith('/v2/')) return 8081;
          return 8080;
        }
      },
      tls: {
        mode: 'terminate',
        certificate: 'auto'
      }
    }
  },
  
  // Static file serving
  {
    name: 'static-assets',
    match: {
      domains: ['cdn.example.com'],
      ports: [443]
    },
    action: {
      type: 'serve',
      root: '/var/www/static',
      tls: {
        mode: 'terminate',
        certificate: 'auto'
      }
    }
  }
];
```

### TCP/SNI Routing

```typescript
const tcpRoutes = [
  // Database connection routing
  {
    name: 'database-cluster',
    match: {
      ports: [{ from: 5432, to: 5439 }]
    },
    action: {
      type: 'forward',
      target: {
        host: '192.168.1.30',
        port: 'preserve'
      },
      security: {
        ipAllowList: ['192.168.1.0/24']
      }
    }
  },
  
  // SNI-based routing for TLS services
  {
    name: 'secure-service',
    match: {
      domains: ['secure.example.com'],
      ports: [8443]
    },
    action: {
      type: 'forward',
      target: {
        host: '192.168.1.40',
        port: 8443
      },
      tls: {
        mode: 'passthrough'
      }
    }
  }
];
```

## Security Features

### IP Reputation Checking

```typescript
import { IpReputationChecker } from '@serve.zone/dcrouter';

const ipChecker = new IpReputationChecker({
  providers: ['spamhaus', 'barracuda', 'surbl'],
  cacheTimeout: 3600000, // 1 hour
  threshold: 0.7
});

// Check IP reputation
const result = await ipChecker.checkIp('192.0.2.1');
if (result.isBlocked) {
  console.log(`IP blocked: ${result.reason}`);
}
```

### Content Security Scanner

```typescript
import { ContentScanner } from '@serve.zone/dcrouter';

const scanner = new ContentScanner({
  spamThreshold: 5.0,
  virusScanning: true,
  attachmentFiltering: {
    maxSize: 25 * 1024 * 1024,
    blockedTypes: ['.exe', '.bat', '.scr']
  }
});

const scanResult = await scanner.scanEmail(email);
```

## API Reference

### DcRouter Class

#### Constructor
```typescript
constructor(options: IDcRouterOptions)
```

#### Methods

##### `start(): Promise<void>`
Starts all configured services (SmartProxy, email server, DNS server).

##### `stop(): Promise<void>`
Gracefully stops all services.

##### `updateRoutes(routes: IRouteConfig[]): Promise<void>`
Updates SmartProxy routes dynamically.

##### `updateDomainRules(rules: IDomainRule[]): Promise<void>`
Updates email domain routing rules.

##### `getStats(): IStatsResponse`
Returns real-time statistics for all services.

### Email Service API

#### `sendEmail(options: IEmailOptions): Promise<string>`
```typescript
const emailId = await router.emailService.sendEmail({
  from: 'sender@example.com',
  to: 'recipient@example.com',
  subject: 'Test Email',
  html: '<p>Hello World</p>',
  attachments: []
});
```

#### `getEmailStatus(emailId: string): IEmailStatus`
```typescript
const status = router.emailService.getEmailStatus(emailId);
console.log(status.status); // 'pending', 'sent', 'delivered', 'bounced'
```

#### `getDeliveryReport(emailId: string): IDeliveryReport`
Detailed delivery information including bounce reasons and tracking data.

## Examples

### Complete Enterprise Setup

```typescript
import { DcRouter } from '@serve.zone/dcrouter';

const router = new DcRouter({
  // HTTP/HTTPS routing
  smartProxyConfig: {
    routes: [
      // Main website
      {
        name: 'website',
        priority: 100,
        match: { domains: ['example.com', 'www.example.com'], ports: [443] },
        action: {
          type: 'forward',
          target: { host: '192.168.1.10', port: 80 },
          tls: { mode: 'terminate', certificate: 'auto' }
        }
      },
      
      // API services
      {
        name: 'api',
        priority: 110,
        match: { domains: ['api.example.com'], ports: [443] },
        action: {
          type: 'forward',
          target: { host: '192.168.1.20', port: 8080 },
          tls: { mode: 'terminate', certificate: 'auto' }
        }
      },
      
      // Internal services
      {
        name: 'internal',
        match: { ports: [{ from: 8000, to: 8999 }] },
        action: {
          type: 'forward',
          target: { host: '192.168.1.30', port: 'preserve' },
          security: { ipAllowList: ['192.168.0.0/16'] }
        }
      }
    ],
    
    // ACME certificate automation
    acme: {
      email: 'ssl@example.com',
      enabled: true,
      useProduction: true,
      autoRenew: true
    }
  },
  
  // Enterprise email system
  emailConfig: {
    ports: [25, 587, 465],
    hostname: 'mail.example.com',
    
    // Authentication configuration
    auth: {
      required: true,
      methods: ['PLAIN', 'LOGIN']
    },
    
    // TLS configuration
    tls: {
      keyPath: './certs/mail-key.pem',
      certPath: './certs/mail-cert.pem'
    },
    
    // Email routing rules
    routes: [
      // Relay from office network
      {
        name: 'office-relay',
        priority: 100,
        match: { clientIp: '192.168.0.0/16' },
        action: {
          type: 'forward',
          forward: {
            host: 'internal-mail.example.com',
            port: 25
          }
        }
      },
      
      // Transactional emails via processing
      {
        name: 'notifications',
        priority: 50,
        match: { recipients: '*@notifications.example.com' },
        action: {
          type: 'process',
          process: {
            scan: true,
            dkim: true,
            queue: 'priority'
          }
        }
      },
      
      // Internal emails forwarded to Exchange
      {
        name: 'internal-mail',
        priority: 25,
        match: { recipients: '*@example.com' },
        action: {
          type: 'forward',
          forward: {
            host: 'exchange.internal.example.com',
            port: 25
          }
        }
      },
      
      // Default reject
      {
        name: 'default-reject',
        match: { recipients: '*' },
        action: {
          type: 'reject',
          reject: {
            code: 550,
            message: 'Relay denied'
          }
        }
      }
    ]
  },
  
  // DNS server for ACME challenges
  dnsServerConfig: {
    port: 53,
    authoritative: true
  },
  
  // Cloudflare DNS challenges
  dnsChallenge: {
    cloudflareApiKey: process.env.CLOUDFLARE_API_KEY
  }
});

// Start the router
await router.start();
console.log('Enterprise DcRouter started');

// Monitor statistics
setInterval(() => {
  const stats = router.getStats();
  console.log('Active connections:', stats.activeConnections);
  console.log('Emails processed:', stats.emailsProcessed);
}, 60000);
```

### Email Template System

```typescript
import { EmailService, TemplateManager } from '@serve.zone/dcrouter';

// Setup email templates
const templateManager = new TemplateManager();
templateManager.addTemplate('welcome', {
  subject: 'Welcome to {{company}}!',
  html: `
    <h1>Welcome {{name}}!</h1>
    <p>Thank you for joining {{company}}.</p>
    <p>Your account: {{email}}</p>
  `,
  text: 'Welcome {{name}}! Thank you for joining {{company}}.'
});

// Send templated email
const emailService = new EmailService(router);
await emailService.sendTemplatedEmail('welcome', {
  to: 'user@example.com',
  templateData: {
    name: 'John Doe',
    company: 'Example Corp',
    email: 'user@example.com'
  }
});
```

## Troubleshooting

### Common Issues

#### Certificate Issues
```bash
# Check certificate status
curl -I https://your-domain.com

# Verify ACME challenge accessibility
curl http://your-domain.com/.well-known/acme-challenge/test
```

#### Email Delivery Issues
```bash
# Test SMTP connectivity
telnet your-server.com 25

# Check DKIM record
dig TXT mail._domainkey.your-domain.com

# Verify SPF record
dig TXT your-domain.com
```

#### Email Routing Issues

**Route Not Matching**
- Check route priority order (higher priority = evaluated first)
- Verify glob patterns: `*@example.com` matches domain, `admin@*` matches user
- Test CIDR notation: `192.168.0.0/16` includes all 192.168.x.x addresses
- Confirm authentication state matches your expectations

**Common Route Patterns**
```typescript
// Debug route to log all traffic
{
  name: 'debug-all',
  priority: 1000,
  match: { recipients: '*' },
  action: { type: 'process', process: { scan: false } }
}

// Catch-all reject (should be lowest priority)
{
  name: 'default-reject',
  priority: 0,
  match: { recipients: '*' },
  action: { type: 'reject', reject: { code: 550, message: 'No route' } }
}
```

#### DNS Issues
```bash
# Test DNS server
dig @your-server.com your-domain.com

# Check DNS propagation
dig your-domain.com @8.8.8.8
```

### Logging and Monitoring

```typescript
import { SmartLog } from '@push.rocks/smartlog';

// Configure logging
const logger = new SmartLog({
  level: 'info',
  transport: 'console'
});

// Monitor email events
router.emailServer.on('emailReceived', (email) => {
  logger.log('info', `Email received: ${email.from} -> ${email.to}`);
});

router.emailServer.on('emailSent', (result) => {
  logger.log('info', `Email sent: ${result.messageId} (${result.status})`);
});

// Monitor proxy events
router.smartProxy.on('connectionEstablished', (connection) => {
  logger.log('info', `Connection: ${connection.clientIp} -> ${connection.target}`);
});
```

### Performance Tuning

```typescript
const performanceConfig = {
  // Connection limits
  maxConnections: 1000,
  connectionTimeout: 30000,
  
  // Email queue settings
  emailQueue: {
    concurrency: 10,
    maxRetries: 3,
    retryDelay: 300000
  },
  
  // Cache settings
  cache: {
    ipReputation: { ttl: 3600000 }, // 1 hour
    dns: { ttl: 300000 }, // 5 minutes
    certificates: { ttl: 86400000 } // 24 hours
  }
};
```

## License

MIT License - see LICENSE file for details.

## Testing

### Comprehensive Test Suite

DcRouter includes a comprehensive test suite covering all aspects of the system:

#### SMTP Protocol Tests
- **Commands**: EHLO, HELO, MAIL FROM, RCPT TO, DATA, RSET, NOOP, QUIT, VRFY, EXPN, HELP
- **Extensions**: SIZE, PIPELINING, STARTTLS
- **Connection Management**: TLS/plain connections, timeouts, limits, rejection handling
- **Error Handling**: Syntax errors, invalid sequences, temporary/permanent failures
- **Email Processing**: Basic sending, multiple recipients, large emails, invalid addresses
- **Security**: Authentication, rate limiting
- **Performance**: Throughput testing
- **Edge Cases**: Very large emails, special characters

#### Running Tests

```bash
# Run all tests
pnpm test

# Run specific test categories
tsx test/suite/commands/test.ehlo-command.ts
tsx test/suite/connection/test.tls-connection.ts
tsx test/suite/email-processing/test.basic-email.ts

# Run with verbose output
tstest test/suite/security/test.authentication.ts --verbose
```

### Test Infrastructure

The test suite uses a self-contained pattern where each test:
1. Starts its own SMTP server instance
2. Runs comprehensive test scenarios
3. Cleans up all resources
4. Provides detailed logging for debugging

This ensures tests are isolated, reliable, and can run in parallel.

## Support

- Documentation: [https://docs.serve.zone/dcrouter](https://docs.serve.zone/dcrouter)
- Issues: [https://github.com/serve-zone/dcrouter/issues](https://github.com/serve-zone/dcrouter/issues)
- Community: [https://community.serve.zone](https://community.serve.zone)