fix(plan): create plan for improving email setup.
This commit is contained in:
812
readme.plan.md
812
readme.plan.md
@ -1,303 +1,563 @@
|
||||
# DcRouter Socket-Handler Integration Plan
|
||||
|
||||
First line: Remember to reread CLAUDE.md file for guidelines.
|
||||
# DcRouter Storage Manager and Email DNS Configuration Plan
|
||||
|
||||
## Overview
|
||||
Integrate socket-handler support for both DNS and Mail services in DcRouter, allowing smartproxy to pass sockets directly instead of servers listening on ports.
|
||||
Implement a central storage manager for DcRouter that supports multiple storage backends (filesystem, custom functions, or memory) and enhance email configuration with unified domain management that cleanly separates infrastructure setup from routing rules.
|
||||
|
||||
## Core Logic
|
||||
- **DNS**: If dnsDomain is set, DnsServer is instantiated with socket-handler for HTTPS/DoH
|
||||
- **Mail**: If useSocketHandler is enabled in emailConfig, mail servers accept sockets from smartproxy
|
||||
- **Automatic Routing**: Setting these options automatically configures smartproxy routes
|
||||
### Key Concept: Infrastructure vs Routing
|
||||
- **Email Config (domains)** = Infrastructure setup (which domains, DNS handling, DKIM, rate limits)
|
||||
- **Route Config (routes)** = ALL email handling logic (forward, process, deliver, reject)
|
||||
- **Default behavior**: If no route matches, log the email and show warning
|
||||
|
||||
## Core Requirements
|
||||
|
||||
### Storage Manager
|
||||
- Flexible storage backend support
|
||||
- Filesystem storage via `fsPath`
|
||||
- Custom storage via read/write functions
|
||||
- Memory storage as fallback with console warning
|
||||
- Key-value storage interface
|
||||
- Used by all components that need persistence
|
||||
|
||||
### Email DNS Configuration
|
||||
- **Infrastructure vs Routing Separation**:
|
||||
- **Email Config (domains)** = Infrastructure setup (which domains, DNS handling, defaults)
|
||||
- **Route Config (routes)** = Individual email routing (pattern matching, specific actions)
|
||||
- **DKIM Always Enabled** - All domains get DKIM keys automatically
|
||||
- **Per-Domain DNS Modes** - Each email domain can have different DNS handling
|
||||
- **Forward Mode** - Simple forwarding to another server (no local processing)
|
||||
- **Internal DNS Mode** - Use the built-in DNS server with NS delegation
|
||||
- **External DNS Mode** - Use external DNS with validation and setup instructions
|
||||
- Storage required for DNS records, DKIM keys, and email routing data
|
||||
|
||||
## Architecture
|
||||
|
||||
### DNS Architecture
|
||||
- **HTTPS/DoH Traffic**: smartproxy → socket-handler → smartdns (no port listening)
|
||||
- **UDP Traffic**: Direct to smartdns DnsServer on VM IP:53 (bypasses smartproxy)
|
||||
- **Automatic Setup**: Setting dnsDomain triggers all DNS infrastructure
|
||||
### Storage Manager Design
|
||||
```typescript
|
||||
interface IStorageConfig {
|
||||
fsPath?: string;
|
||||
readFunction?: (key: string) => Promise<string>;
|
||||
writeFunction?: (key: string, value: string) => Promise<void>;
|
||||
}
|
||||
|
||||
### Mail Architecture
|
||||
- **Socket-Handler Mode**: smartproxy → socket-handler → mail server (no port listening)
|
||||
- **Traditional Mode**: smartproxy → forward → mail server (listening on ports)
|
||||
- **Configurable**: useSocketHandler flag determines the mode
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### 1. Analyze Current Architecture
|
||||
- [x] Examine how DcRouter currently integrates services
|
||||
- [x] Study email server socket handling in SmtpServer
|
||||
- [x] Check existing smartproxy socket-handler patterns
|
||||
- [x] Review UnifiedEmailServer connection handling
|
||||
|
||||
### 2. DNS Socket-Handler Implementation
|
||||
|
||||
#### 2.1 Create DNS Socket Handler
|
||||
- [x] Create a custom socket handler that processes DNS-over-HTTPS requests
|
||||
- [x] Handler should use smartdns without port binding
|
||||
- [x] Use built-in handleHttpsSocket method from smartdns
|
||||
- [x] Set manualHttpsMode: true in DnsServer options
|
||||
|
||||
#### 2.2 Configure smartdns DnsServer
|
||||
- [x] Create DnsServer instance with UDP only on VM IP interface
|
||||
- [x] Set manualHttpsMode: true to disable HTTPS port binding
|
||||
- [x] Configure to listen on port 53 for UDP traffic only
|
||||
- [x] Use handleHttpsSocket method for DoH sockets
|
||||
|
||||
### 3. Mail Socket-Handler Implementation
|
||||
|
||||
#### 3.1 Modify SmtpServer
|
||||
- [x] ConnectionManager already has handleConnection method
|
||||
- [x] Connection handling already works with provided sockets
|
||||
- [x] Session management works with socket-handler mode
|
||||
- [x] Backward compatibility maintained
|
||||
|
||||
#### 3.2 Update UnifiedEmailServer
|
||||
- [x] Add useSocketHandler flag to IUnifiedEmailServerOptions
|
||||
- [x] Modify start() to skip server creation when useSocketHandler is true
|
||||
- [x] Add handleSocket method to accept sockets from smartproxy
|
||||
- [x] All email processing works in socket-handler mode
|
||||
|
||||
#### 3.3 Create Mail Socket Handler
|
||||
- [x] Create socket handler for SMTP/submission/SMTPS
|
||||
- [x] Handle TLS for port 465 (immediate TLS wrapping)
|
||||
- [x] Pass sockets to UnifiedEmailServer.handleSocket
|
||||
|
||||
### 4. DcRouter Integration
|
||||
|
||||
#### 4.1 Configuration Updates
|
||||
- [x] Add `dnsDomain` property to DcRouter config
|
||||
- [x] Add `useSocketHandler` to email config options
|
||||
- [x] Update interface definitions
|
||||
|
||||
#### 4.2 Route Generation Updates
|
||||
- [x] Modify generateEmailRoutes to create socket-handler actions when enabled
|
||||
- [x] Create DNS routes with socket-handler actions
|
||||
- [x] Ensure proper domain and path matching
|
||||
|
||||
#### 4.3 Service Lifecycle
|
||||
- [x] Update setupSmartProxy to handle socket-handler routes
|
||||
- [x] Services start correctly without port binding
|
||||
- [x] Socket cleanup handled by connection managers
|
||||
|
||||
### 5. SmartProxy Route Configuration
|
||||
- [x] DNS routes: match dnsDomain with paths /dns-query and /resolve
|
||||
- [x] Mail routes: match ports 25, 587, 465 with socket-handler actions
|
||||
- [x] Configure TLS handling for each protocol appropriately
|
||||
- [x] Automatic Let's Encrypt via smartproxy certificate: 'auto'
|
||||
|
||||
### 6. Testing
|
||||
|
||||
#### 6.1 DNS Testing
|
||||
- [x] Test that DNS server is NOT instantiated when dnsDomain is not set
|
||||
- [x] Test that DNS server IS instantiated when dnsDomain is set
|
||||
- [x] Test DNS route generation with correct configuration
|
||||
- [x] Test DNS socket handler creation and functionality
|
||||
- [x] Verify routes use socket-handler action type
|
||||
|
||||
#### 6.2 Mail Testing
|
||||
- [x] Test traditional port-based mail delivery (forward action)
|
||||
- [x] Test socket-handler mail delivery (socket-handler action)
|
||||
- [x] Test email route generation for different ports
|
||||
- [x] Test SMTPS (port 465) uses terminate TLS mode
|
||||
- [x] Test SMTP/Submission (ports 25/587) use passthrough TLS mode
|
||||
|
||||
#### 6.3 Integration Testing
|
||||
- [x] Test both DNS and Mail with socket-handlers simultaneously
|
||||
- [x] Test mixed configuration (DNS socket-handler, email traditional)
|
||||
- [x] Test socket handler function creation
|
||||
- [x] Unit tests pass without port conflicts
|
||||
- [x] All route generation logic tested
|
||||
|
||||
## Technical Details
|
||||
class StorageManager {
|
||||
// Unified interface for all storage operations
|
||||
async get(key: string): Promise<string | null>
|
||||
async set(key: string, value: string): Promise<void>
|
||||
async delete(key: string): Promise<void>
|
||||
async list(prefix?: string): Promise<string[]>
|
||||
async exists(key: string): Promise<boolean>
|
||||
}
|
||||
```
|
||||
|
||||
### DcRouter Configuration
|
||||
```typescript
|
||||
// DcRouter config with socket-handler support
|
||||
const dcRouterConfig = {
|
||||
// ... other config
|
||||
dnsDomain: 'example.com', // Optional - if set, DNS server is enabled
|
||||
interface IDcRouterOptions {
|
||||
// ... existing options
|
||||
|
||||
storage?: {
|
||||
fsPath?: string;
|
||||
readFunction?: (key: string) => Promise<string>;
|
||||
writeFunction?: (key: string, value: string) => Promise<void>;
|
||||
};
|
||||
|
||||
emailConfig?: IUnifiedEmailServerOptions;
|
||||
}
|
||||
|
||||
// Updated UnifiedEmailServerOptions with backward-compatible domains
|
||||
interface IUnifiedEmailServerOptions {
|
||||
ports: number[];
|
||||
hostname: string;
|
||||
|
||||
// Backward compatible - can be strings or full config objects
|
||||
domains: (string | IEmailDomainConfig)[];
|
||||
|
||||
// Pattern-based routing rules (evaluated after domain matching)
|
||||
routes: IEmailRoute[];
|
||||
|
||||
// Global defaults for all domains
|
||||
defaults?: {
|
||||
dnsMode?: 'forward' | 'internal-dns' | 'external-dns';
|
||||
dkim?: IEmailDomainConfig['dkim'];
|
||||
rateLimits?: IEmailDomainConfig['rateLimits'];
|
||||
};
|
||||
|
||||
// ... existing options (auth, tls, limits, etc.)
|
||||
}
|
||||
|
||||
interface IEmailDomainConfig {
|
||||
// Domain name
|
||||
domain: string;
|
||||
|
||||
// DNS handling mode
|
||||
dnsMode: 'forward' | 'internal-dns' | 'external-dns';
|
||||
|
||||
// DNS configuration based on mode
|
||||
dns?: {
|
||||
// For 'forward' mode
|
||||
forward?: {
|
||||
skipDnsValidation?: boolean;
|
||||
targetDomain?: string;
|
||||
};
|
||||
|
||||
// For 'internal-dns' mode (requires dnsDomain in DcRouter)
|
||||
internal?: {
|
||||
/** TTL for DNS records in seconds (default: 3600) */
|
||||
ttl?: number;
|
||||
/** MX record priority (default: 10) */
|
||||
mxPriority?: number;
|
||||
};
|
||||
|
||||
// For 'external-dns' mode
|
||||
external?: {
|
||||
/**
|
||||
* Custom DNS servers to query (rarely needed)
|
||||
* Default: Follow standard DNS resolution chain
|
||||
* Only use for special cases like internal corporate DNS
|
||||
*/
|
||||
servers?: string[];
|
||||
/**
|
||||
* Which DNS records to validate
|
||||
* Default: ['MX', 'SPF', 'DKIM', 'DMARC']
|
||||
* Validation always runs; setup instructions always shown if missing
|
||||
*/
|
||||
requiredRecords?: ('MX' | 'SPF' | 'DKIM' | 'DMARC')[];
|
||||
};
|
||||
};
|
||||
|
||||
|
||||
// Per-domain DKIM settings
|
||||
// DKIM is always enabled; use this to override defaults
|
||||
dkim?: {
|
||||
/** DKIM selector (default: 'default') */
|
||||
selector?: string;
|
||||
/** Key size in bits (default: 2048) */
|
||||
keySize?: number;
|
||||
/** Automatically rotate keys (default: false) */
|
||||
rotateKeys?: boolean;
|
||||
/** Days between key rotations (default: 90) */
|
||||
rotationInterval?: number;
|
||||
};
|
||||
|
||||
// Per-domain rate limits
|
||||
rateLimits?: {
|
||||
outbound?: {
|
||||
messagesPerMinute?: number;
|
||||
messagesPerHour?: number;
|
||||
messagesPerDay?: number;
|
||||
};
|
||||
inbound?: {
|
||||
messagesPerMinute?: number;
|
||||
connectionsPerIp?: number;
|
||||
recipientsPerMessage?: number;
|
||||
};
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Validation and Warnings
|
||||
|
||||
### No Route Match Warning
|
||||
When an email arrives and no route matches:
|
||||
```typescript
|
||||
console.warn(
|
||||
'⚠️ WARNING: No route matched for email:\n' +
|
||||
` From: ${email.from}\n` +
|
||||
` To: ${email.to}\n` +
|
||||
` Subject: ${email.subject}\n` +
|
||||
' Email will be logged but not processed.\n' +
|
||||
' Add a route to handle this email pattern.'
|
||||
);
|
||||
// Email is stored in logs but no further action taken
|
||||
```
|
||||
|
||||
### Internal DNS Mode Validation
|
||||
When a domain is configured with `dnsMode: 'internal-dns'`:
|
||||
```typescript
|
||||
// Check if dnsDomain is configured
|
||||
if (!dcRouter.dnsDomain) {
|
||||
console.error(
|
||||
'❌ ERROR: Domain "mail.hosted.com" is configured to use internal DNS,\n' +
|
||||
' but dnsDomain is not set in DcRouter configuration.\n' +
|
||||
' Please configure dnsDomain to enable the DNS server.\n' +
|
||||
' Example: dnsDomain: "ns.myservice.com"'
|
||||
);
|
||||
throw new Error('Internal DNS mode requires dnsDomain to be configured');
|
||||
}
|
||||
|
||||
// Check if NS delegation exists
|
||||
const nsRecords = await dns.resolveNs(domainConfig.domain);
|
||||
const isDelegated = nsRecords.includes(dcRouter.dnsDomain);
|
||||
|
||||
if (!isDelegated) {
|
||||
console.log(
|
||||
'📋 DNS Delegation Required for ' + domainConfig.domain + ':\n' +
|
||||
'━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n' +
|
||||
'Please add this NS record at your domain registrar:\n' +
|
||||
` ${domainConfig.domain}. NS ${dcRouter.dnsDomain}.\n` +
|
||||
'━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n' +
|
||||
'This delegation is required for internal DNS mode to work.'
|
||||
);
|
||||
} else {
|
||||
console.log(
|
||||
'✅ NS delegation verified: ' + domainConfig.domain + ' -> ' + dcRouter.dnsDomain
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### External DNS Mode Validation
|
||||
When a domain is configured with `dnsMode: 'external-dns'`:
|
||||
```typescript
|
||||
// Check current DNS records
|
||||
const currentRecords = await checkDnsRecords('mail.external.com');
|
||||
const requiredChanges = [];
|
||||
|
||||
if (!currentRecords.mx) {
|
||||
requiredChanges.push('Add MX record: external.com -> mail.external.com (priority 10)');
|
||||
}
|
||||
|
||||
if (!currentRecords.spf) {
|
||||
requiredChanges.push('Add TXT record: external.com -> "v=spf1 a mx ~all"');
|
||||
}
|
||||
|
||||
// DKIM is always enabled, so always check for DKIM record
|
||||
if (!currentRecords.dkim) {
|
||||
const dkimPublicKey = await storage.get(`/email/dkim/${domain}/public.key`);
|
||||
requiredChanges.push(
|
||||
`Add TXT record: ${domainConfig.dkim?.selector || 'default'}._domainkey.external.com -> "${dkimPublicKey}"`
|
||||
);
|
||||
}
|
||||
|
||||
if (requiredChanges.length > 0) {
|
||||
console.log(
|
||||
'📋 DNS Configuration Required for mail.external.com:\n' +
|
||||
'━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n' +
|
||||
requiredChanges.map((change, i) => `${i + 1}. ${change}`).join('\n') +
|
||||
'\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━'
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Forward Mode Validation
|
||||
When a domain is configured with `dnsMode: 'forward'`:
|
||||
- No DNS validation needed (pure forwarding)
|
||||
- DKIM keys are still generated and stored (for consistency)
|
||||
- DKIM signing only happens if routes process the email (not pure forward)
|
||||
- Note: Actual forwarding behavior must be defined in routes
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Storage Manager Implementation
|
||||
|
||||
#### 1.1 Create Storage Manager Core
|
||||
- [ ] Create `ts/storage/classes.storagemanager.ts`
|
||||
- [ ] Implement base StorageManager class
|
||||
- [ ] Add storage backend detection logic
|
||||
- [ ] Implement key namespacing for different components
|
||||
|
||||
#### 1.2 Implement Storage Backends
|
||||
- [ ] Filesystem backend using `fsPath`
|
||||
- [ ] Custom function backend using provided read/write functions
|
||||
- [ ] Memory backend with Map<string, string>
|
||||
- [ ] Add console warning for memory backend
|
||||
|
||||
#### 1.3 Storage Interface Methods
|
||||
- [ ] Implement get/set/delete/list/exists methods
|
||||
- [ ] Add JSON serialization helpers
|
||||
- [ ] Add atomic write support for filesystem
|
||||
- [ ] Add key validation and sanitization
|
||||
|
||||
#### 1.4 Integration Points
|
||||
- [ ] Add StorageManager instance to DcRouter
|
||||
- [ ] Pass storage to components that need it
|
||||
- [ ] Update component constructors to accept storage
|
||||
|
||||
### Phase 2: Email DNS Configuration Implementation
|
||||
|
||||
#### 2.1 Update Email Configuration
|
||||
- [ ] Update IUnifiedEmailServerOptions to support string | IEmailDomainConfig
|
||||
- [ ] Implement backward compatibility for string domains
|
||||
- [ ] Add domain configuration validation
|
||||
- [ ] Implement infrastructure vs routing separation
|
||||
|
||||
#### 2.2 Domain Configuration Processing
|
||||
- [ ] Parse domains array (string vs IEmailDomainConfig)
|
||||
- [ ] Apply global defaults to domain configs
|
||||
- [ ] Validate each domain's DNS mode configuration
|
||||
- [ ] Create domain registry for quick lookups
|
||||
|
||||
#### 2.3 DNS Mode Implementations
|
||||
- [ ] **Forward Mode**: Skip DNS handling, validate target reachability
|
||||
- [ ] **Internal DNS Mode**:
|
||||
- [ ] Validate dnsDomain is set in DcRouter config
|
||||
- [ ] Check NS delegation exists (query for NS records)
|
||||
- [ ] Show instructions if NS delegation is missing
|
||||
- [ ] Log success if NS delegation is properly configured
|
||||
- [ ] Automatically create MX, SPF, DKIM, DMARC records in internal DNS
|
||||
- [ ] Apply TTL (default: 3600) and MX priority (default: 10)
|
||||
- [ ] Store records via StorageManager
|
||||
- [ ] Register domains with DnsServer
|
||||
- [ ] **External DNS Mode**:
|
||||
- [ ] Use standard DNS resolution (or custom servers if specified)
|
||||
- [ ] Always validate required records (default: MX, SPF, DKIM, DMARC)
|
||||
- [ ] Always show setup instructions if records are missing
|
||||
- [ ] Cache DNS query results in storage
|
||||
|
||||
#### 2.4 Per-Domain Features
|
||||
- [ ] Implement per-domain DKIM key management
|
||||
- [ ] Apply per-domain rate limits
|
||||
- [ ] Handle per-domain email processing rules
|
||||
- [ ] Automatic DKIM key rotation based on domain config
|
||||
|
||||
### Phase 3: Storage Usage Implementation
|
||||
|
||||
#### 3.1 Email Component Storage
|
||||
- [ ] DKIM keys storage
|
||||
- [ ] Email routing rules storage
|
||||
- [ ] Bounce/complaint tracking
|
||||
- [ ] Reputation data persistence
|
||||
|
||||
#### 3.2 DNS Component Storage
|
||||
- [ ] DNS records storage
|
||||
- [ ] DNSSEC keys storage
|
||||
- [ ] Zone data persistence
|
||||
- [ ] Cache storage
|
||||
|
||||
#### 3.3 Certificate Storage
|
||||
- [ ] Let's Encrypt certificates
|
||||
- [ ] Certificate renewal data
|
||||
- [ ] ACME account keys
|
||||
|
||||
### Phase 4: Testing
|
||||
|
||||
#### 4.1 Storage Manager Tests
|
||||
- [ ] Test filesystem backend
|
||||
- [ ] Test custom function backend
|
||||
- [ ] Test memory backend with warning
|
||||
- [ ] Test backend switching
|
||||
- [ ] Test concurrent access
|
||||
|
||||
#### 4.2 Email DNS Mode Tests
|
||||
- [ ] Test external DNS lookups
|
||||
- [ ] Test integrated DNS mode
|
||||
- [ ] Test DNS record creation
|
||||
- [ ] Test DKIM key generation
|
||||
|
||||
#### 4.3 Integration Tests
|
||||
- [ ] Test storage persistence across restarts
|
||||
- [ ] Test DNS mode switching
|
||||
- [ ] Test data migration between backends
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Storage Key Structure
|
||||
```
|
||||
/email/dkim/{domain}/private.key
|
||||
/email/dkim/{domain}/public.key
|
||||
/email/routes/{routeId}.json
|
||||
/email/reputation/{domain}.json
|
||||
/dns/records/{domain}/{type}/{id}.json
|
||||
/dns/zones/{domain}.json
|
||||
/certificates/{domain}/cert.pem
|
||||
/certificates/{domain}/key.pem
|
||||
```
|
||||
|
||||
### Memory Storage Warning
|
||||
```typescript
|
||||
console.warn(
|
||||
'<27> WARNING: StorageManager is using in-memory storage.\n' +
|
||||
' Data will be lost when the process restarts.\n' +
|
||||
' Configure storage.fsPath or storage functions for persistence.'
|
||||
);
|
||||
```
|
||||
|
||||
### Configuration Examples
|
||||
|
||||
#### Complete Email Domain Configuration
|
||||
```typescript
|
||||
{
|
||||
dnsDomain: 'ns.myservice.com', // Required for internal-dns mode
|
||||
emailConfig: {
|
||||
ports: [25, 587, 465],
|
||||
domains: ['mail.example.com'],
|
||||
useSocketHandler: true, // Enable socket-handler mode for mail
|
||||
routes: [/* email routing rules */]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### DNS Implementation
|
||||
|
||||
#### Conditional DNS Instantiation
|
||||
```typescript
|
||||
// In DcRouter startup logic
|
||||
if (config.dnsDomain) {
|
||||
// Create DNS server instance
|
||||
this.dnsServer = new DnsServer({
|
||||
udpPort: 53,
|
||||
udpBindAddress: vmIpAddress,
|
||||
httpsPort: undefined, // No HTTPS listening
|
||||
dnssecZone: config.dnsDomain
|
||||
});
|
||||
|
||||
// Create smartproxy route for DoH
|
||||
const dnsRoute = {
|
||||
match: {
|
||||
domain: config.dnsDomain,
|
||||
path: ['/dns-query', '/resolve']
|
||||
},
|
||||
action: {
|
||||
type: 'socket-handler',
|
||||
handler: this.createDnsSocketHandler()
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
#### DNS Socket Handler
|
||||
```typescript
|
||||
const createDnsSocketHandler = () => {
|
||||
return async (socket: net.Socket) => {
|
||||
// Handle HTTP/2 for DoH
|
||||
const session = http2.createSession(socket);
|
||||
hostname: 'mail.myservice.com',
|
||||
|
||||
session.on('stream', async (stream, headers) => {
|
||||
if (headers[':path'] === '/dns-query') {
|
||||
const dnsQuery = await parseDnsQuery(stream);
|
||||
const response = await dnsServer.resolveQuery(dnsQuery);
|
||||
stream.respond({ ':status': 200, 'content-type': 'application/dns-message' });
|
||||
stream.end(response);
|
||||
}
|
||||
});
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Mail Implementation
|
||||
|
||||
#### Email Route Generation with Socket-Handler
|
||||
```typescript
|
||||
private generateEmailRoutes(emailConfig: IUnifiedEmailServerOptions): IRouteConfig[] {
|
||||
const routes: IRouteConfig[] = [];
|
||||
|
||||
for (const port of emailConfig.ports) {
|
||||
const action = emailConfig.useSocketHandler
|
||||
? {
|
||||
type: 'socket-handler',
|
||||
handler: this.createMailSocketHandler(port)
|
||||
// Mix of simple strings and full configs (backward compatible)
|
||||
domains: [
|
||||
// Simple domain (uses all defaults including DKIM)
|
||||
'simple.com',
|
||||
|
||||
{
|
||||
// Forward-only domain (no local DNS needed)
|
||||
domain: 'forwarded.com',
|
||||
dnsMode: 'forward',
|
||||
dns: {
|
||||
forward: {
|
||||
skipDnsValidation: false
|
||||
}
|
||||
}
|
||||
: {
|
||||
// DKIM uses defaults (always enabled)
|
||||
// Handling is defined in routes, not here!
|
||||
},
|
||||
{
|
||||
// Using internal DNS server
|
||||
domain: 'mail.hosted.com',
|
||||
dnsMode: 'internal-dns',
|
||||
dns: {
|
||||
internal: {
|
||||
mxPriority: 10 // Records are always auto-created for internal DNS
|
||||
}
|
||||
},
|
||||
dkim: {
|
||||
// Override DKIM defaults
|
||||
keySize: 2048 // Using 2048 explicitly (same as default)
|
||||
}
|
||||
// Handling is defined in routes, not here!
|
||||
},
|
||||
{
|
||||
// Using external DNS
|
||||
domain: 'mail.external.com',
|
||||
dnsMode: 'external-dns',
|
||||
dns: {
|
||||
external: {
|
||||
// Validation runs automatically
|
||||
// Only override requiredRecords if you want fewer checks
|
||||
requiredRecords: ['MX', 'SPF', 'DKIM'] // Skipping DMARC check
|
||||
}
|
||||
},
|
||||
// No dkim object needed - uses all defaults
|
||||
rateLimits: {
|
||||
outbound: {
|
||||
messagesPerMinute: 100
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
|
||||
// Pattern-based routing (ALL email handling happens here)
|
||||
routes: [
|
||||
// Forward all mail for forwarded.com
|
||||
{
|
||||
name: 'forward-domain',
|
||||
match: { recipients: '*@forwarded.com' },
|
||||
action: {
|
||||
type: 'forward',
|
||||
target: { host: 'localhost', port: mapPort(port) }
|
||||
};
|
||||
|
||||
routes.push({
|
||||
match: { ports: [port] },
|
||||
action
|
||||
});
|
||||
forward: {
|
||||
host: '192.168.1.10',
|
||||
port: 25,
|
||||
preserveHeaders: true
|
||||
}
|
||||
}
|
||||
},
|
||||
// Process mail for hosted domains
|
||||
{
|
||||
name: 'process-hosted',
|
||||
match: { recipients: ['*@mail.hosted.com', '*@mail.external.com'] },
|
||||
action: {
|
||||
type: 'process',
|
||||
process: {
|
||||
scan: true,
|
||||
dkim: true,
|
||||
spf: true,
|
||||
queue: 'normal'
|
||||
}
|
||||
}
|
||||
},
|
||||
// VIP bypass scanning
|
||||
{
|
||||
name: 'vip-bypass',
|
||||
match: { recipients: 'ceo@*' },
|
||||
action: {
|
||||
type: 'process',
|
||||
process: { scan: false, queue: 'priority' }
|
||||
}
|
||||
},
|
||||
// Reject large messages
|
||||
{
|
||||
name: 'large-reject',
|
||||
match: {
|
||||
recipients: '*@*',
|
||||
sizeRange: { min: 50 * 1024 * 1024 }
|
||||
},
|
||||
action: {
|
||||
type: 'reject',
|
||||
reject: {
|
||||
code: 552,
|
||||
message: 'Message too large'
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
return routes;
|
||||
}
|
||||
```
|
||||
|
||||
#### Mail Socket Handler
|
||||
```typescript
|
||||
const createMailSocketHandler = (port: number) => {
|
||||
return async (socket: net.Socket) => {
|
||||
// Determine protocol based on port
|
||||
const isSecure = port === 465;
|
||||
|
||||
if (isSecure) {
|
||||
// For SMTPS, handle TLS immediately
|
||||
const tlsSocket = new tls.TLSSocket(socket, {
|
||||
isServer: true,
|
||||
cert: tlsCert,
|
||||
key: tlsKey
|
||||
});
|
||||
await this.emailServer.handleSocket(tlsSocket, port);
|
||||
} else {
|
||||
// For SMTP/Submission, pass raw socket
|
||||
await this.emailServer.handleSocket(socket, port);
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
## Benefits
|
||||
|
||||
#### UnifiedEmailServer Socket Handling
|
||||
```typescript
|
||||
// In UnifiedEmailServer class
|
||||
public async handleSocket(socket: net.Socket, port: number): Promise<void> {
|
||||
// Create session for this socket
|
||||
const session = this.sessionManager.createSession(socket);
|
||||
|
||||
// Handle connection based on port
|
||||
switch (port) {
|
||||
case 25: // SMTP
|
||||
case 587: // Submission (STARTTLS)
|
||||
await this.connectionManager.handleConnection(socket, session);
|
||||
break;
|
||||
case 465: // SMTPS (already TLS)
|
||||
await this.connectionManager.handleSecureConnection(socket, session);
|
||||
break;
|
||||
}
|
||||
}
|
||||
### Storage Manager Benefits
|
||||
- **Flexibility**: Multiple storage backend options
|
||||
- **Portability**: Easy to switch storage backends
|
||||
- **Testing**: Memory backend for tests
|
||||
- **Cloud-Ready**: Custom functions can use cloud storage
|
||||
|
||||
## Dependencies
|
||||
- @push.rocks/smartproxy (already integrated)
|
||||
- @push.rocks/smartdns (to be added)
|
||||
### Email DNS Benefits
|
||||
- **Unified Configuration**: Infrastructure and routing cleanly separated
|
||||
- **Backward Compatible**: Existing string domain configs continue to work
|
||||
- **Flexible DNS Modes**: Choose per-domain how DNS is handled
|
||||
- **External Mode**: Works with existing DNS infrastructure
|
||||
- **Internal Mode**: Self-contained email system with automatic record creation
|
||||
- **Forward Mode**: Simple mail forwarding without DNS complexity
|
||||
- **Automatic Setup**: DNS records created automatically for internal mode
|
||||
- **DKIM Management**: Keys generated and rotated automatically per domain
|
||||
- **Per-Domain Settings**: Rate limits, DKIM, and handling customizable per domain
|
||||
|
||||
## Implementation Phases
|
||||
## Migration Path
|
||||
|
||||
### Phase 1: DNS Socket-Handler (Priority)
|
||||
1. Add smartdns dependency
|
||||
2. Implement DNS socket-handler
|
||||
3. Add dnsDomain configuration
|
||||
4. Test DoH functionality
|
||||
|
||||
### Phase 2: Mail Socket-Handler
|
||||
1. Refactor SmtpServer for socket handling
|
||||
2. Update UnifiedEmailServer
|
||||
3. Implement mail socket-handlers
|
||||
4. Add useSocketHandler configuration
|
||||
5. Test all mail protocols
|
||||
|
||||
### Phase 3: Integration & Testing
|
||||
1. Full integration testing
|
||||
2. Performance benchmarking
|
||||
3. Documentation updates
|
||||
4. Migration guide for existing users
|
||||
|
||||
## Implementation Status: COMPLETE ✅
|
||||
|
||||
All planned features have been successfully implemented and tested:
|
||||
- **DNS Socket-Handler**: Automatic DNS-over-HTTPS setup with `dnsDomain` configuration
|
||||
- **Email Socket-Handler**: Direct socket passing with `useSocketHandler` flag
|
||||
- **SmartProxy Integration**: Automatic route generation for both services
|
||||
- **Full Test Coverage**: Unit tests for all functionality
|
||||
- **Documentation**: Updated readme.md and changelog.md
|
||||
1. Start with memory storage (with warning)
|
||||
2. Configure filesystem path for persistence
|
||||
3. Migrate to custom functions for cloud deployment
|
||||
4. Switch DNS modes based on infrastructure
|
||||
|
||||
## Notes
|
||||
|
||||
### DNS Notes
|
||||
- UDP traffic bypasses proxy entirely (kernel-level routing)
|
||||
- DoH provides encrypted DNS over HTTPS through proxy
|
||||
- Socket handler allows DNS processing without port binding
|
||||
- DNS functionality is entirely optional - only enabled when dnsDomain is configured
|
||||
- Setting dnsDomain triggers automatic setup of all DNS infrastructure
|
||||
- Automatic Let's Encrypt certificate provisioning for configured dnsDomain
|
||||
- Storage manager is singleton within DcRouter instance
|
||||
- All storage operations are async
|
||||
- Keys are hierarchical (path-like)
|
||||
- Values are strings (JSON for complex data)
|
||||
- Internal DNS mode requires `dnsDomain` to be set in DcRouter
|
||||
- Domain configuration is backward compatible (strings still work)
|
||||
- Clean separation: domains = infrastructure, routes = handling
|
||||
- Domain config only defines: which domains, DNS mode, DKIM settings, rate limits
|
||||
- DKIM is always enabled for all domains (use dkim object to override defaults)
|
||||
- ALL email handling logic lives in routes
|
||||
- If no route matches, email is logged with warning
|
||||
- This eliminates duplication between domain and route configs
|
||||
|
||||
### Mail Notes
|
||||
- Socket-handler mode eliminates internal port binding for mail services
|
||||
- Traditional port forwarding mode remains available for compatibility
|
||||
- STARTTLS negotiation handled within socket-handler for ports 25/587
|
||||
- Port 465 (SMTPS) requires immediate TLS handshake in socket-handler
|
||||
- Connection pooling and session management work in both modes
|
||||
- Socket-handler reduces latency by eliminating internal forwarding
|
||||
### Default Values
|
||||
- **DKIM** (always enabled for all domains):
|
||||
- `selector`: 'default'
|
||||
- `keySize`: 2048 bits
|
||||
- `rotateKeys`: false
|
||||
- `rotationInterval`: 90 days
|
||||
- **Internal DNS Mode**:
|
||||
- Records are always automatically created (MX, SPF, DKIM, DMARC)
|
||||
- `ttl`: 3600 seconds (1 hour)
|
||||
- `mxPriority`: 10
|
||||
- **External DNS Mode**:
|
||||
- `servers`: Standard DNS resolution chain (rarely needs override)
|
||||
- `requiredRecords`: ['MX', 'SPF', 'DKIM', 'DMARC']
|
||||
- Validation always runs on startup
|
||||
- Setup instructions always shown if validation fails
|
||||
|
||||
### General Benefits
|
||||
- Reduced resource usage (no internal port binding)
|
||||
- Better performance (direct socket passing)
|
||||
- Simplified configuration (automatic route creation)
|
||||
- Enhanced security (no exposed internal ports)
|
||||
- Unified approach for all services through smartproxy
|
||||
### DNS Delegation for Internal Mode
|
||||
When using internal DNS mode, you must manually configure NS delegation at your domain registrar:
|
||||
|
||||
**Example Setup:**
|
||||
- DcRouter config: `dnsDomain: 'ns.myservice.com'`
|
||||
- Email domain: `mail.example.com`
|
||||
- **Manual step at registrar**: Add NS record: `mail.example.com. NS ns.myservice.com.`
|
||||
|
||||
This delegation tells the internet that your DcRouter DNS server (`ns.myservice.com`) is authoritative for `mail.example.com`. Without this manual step, internal DNS mode won't work.
|
||||
|
||||
**The system automatically checks NS delegation:**
|
||||
- ✅ If properly delegated: `✅ NS delegation verified: mail.example.com -> ns.myservice.com`
|
||||
- ❌ If not delegated: Shows instructions to add the NS record at your registrar
|
||||
|
||||
This validation ensures your internal DNS is properly configured before attempting to handle email.
|
||||
Reference in New Issue
Block a user