docs(readme): update documentation with StorageManager and domain configuration features

- Add flexible storage system section with backend examples
- Document email domain configuration with DNS modes
- Update configuration interfaces with storage options
- Add examples for filesystem, custom, and memory storage
- Include data migration examples between backends
- Document storage usage patterns and key structure
- Update test suite documentation with new test categories
This commit is contained in:
Philipp Kunz 2025-05-30 07:06:00 +00:00
parent 53b64025f3
commit e6251ab655

191
readme.md
View File

@ -44,6 +44,12 @@ A comprehensive traffic routing solution that provides unified gateway capabilit
- **Rate limiting** at multiple levels
- **Real-time metrics** and monitoring
### 💾 **Flexible Storage System**
- **Multiple storage backends**: filesystem, custom functions, or memory
- **Unified storage interface** for all components
- **Automatic data migration** between backends
- **Persistent configuration** for domains, routes, and security data
## Installation
```bash
@ -207,12 +213,18 @@ interface IDcRouterOptions {
emailConfig?: {
ports: number[];
hostname: string;
routes: IEmailRoute[]; // Route-based configuration
domains?: IEmailDomainConfig[]; // Domain infrastructure setup
routes: IEmailRoute[]; // Route-based email handling
auth?: IAuthConfig;
tls?: ITlsConfig;
maxMessageSize?: number;
rateLimits?: IRateLimitConfig;
useSocketHandler?: boolean; // Enable socket-handler mode (no port binding)
defaults?: { // Global defaults for all domains
dnsMode?: 'forward' | 'internal-dns' | 'external-dns';
dkim?: IDkimConfig;
rateLimits?: IRateLimitConfig;
};
};
// DNS server configuration
@ -235,6 +247,13 @@ interface IDcRouterOptions {
dnsChallenge?: {
cloudflareApiKey: string;
};
// Storage configuration
storage?: {
fsPath?: string; // Filesystem storage path
readFunction?: (key: string) => Promise<string>; // Custom read function
writeFunction?: (key: string, value: string) => Promise<void>; // Custom write function
};
}
```
@ -333,6 +352,73 @@ External Port → SmartProxy → Socket Handler → Service
## Email System
### Email Domain Configuration
DcRouter separates email infrastructure (which domains to handle) from routing logic (how to handle emails):
#### **DNS Modes**
**Forward Mode** - Simple mail forwarding without local DNS:
```typescript
{
domain: 'forwarded.com',
dnsMode: 'forward',
dns: {
forward: {
skipDnsValidation: true,
targetDomain: 'mail.target.com'
}
}
}
```
**Internal DNS Mode** - Use built-in DNS server (requires `dnsDomain` in DcRouter config):
```typescript
{
domain: 'mail.example.com',
dnsMode: 'internal-dns',
dns: {
internal: {
mxPriority: 10,
ttl: 3600
}
},
dkim: {
selector: 'mail2024',
keySize: 2048,
rotateKeys: true,
rotationInterval: 90
}
}
```
**External DNS Mode** - Use existing DNS infrastructure:
```typescript
{
domain: 'mail.external.com',
dnsMode: 'external-dns',
dns: {
external: {
requiredRecords: ['MX', 'SPF', 'DKIM', 'DMARC']
}
},
rateLimits: {
inbound: {
messagesPerMinute: 100,
connectionsPerIp: 10
}
}
}
```
#### **DKIM Management**
DKIM is always enabled for all domains. Keys are automatically:
- Generated on first use
- Stored persistently via StorageManager
- Rotated based on configuration
- Cleaned up after grace period
### Email Route Actions
#### **Forward Action**
@ -630,6 +716,78 @@ const tcpRoutes = [
];
```
## Storage System
### StorageManager
DcRouter includes a flexible storage system that supports multiple backends:
#### **Filesystem Storage**
```typescript
const router = new DcRouter({
storage: {
fsPath: '/var/lib/dcrouter/data'
}
});
```
#### **Custom Storage Backend**
```typescript
const router = new DcRouter({
storage: {
readFunction: async (key) => {
// Read from Redis, S3, etc.
return await myDatabase.get(key);
},
writeFunction: async (key, value) => {
// Write to Redis, S3, etc.
await myDatabase.set(key, value);
}
}
});
```
#### **Memory Storage (Development)**
```typescript
const router = new DcRouter({
// No storage config = memory storage with warning
});
```
### Storage Usage
The storage system is used for:
- **DKIM Keys**: `/email/dkim/{domain}/private.key`, `/email/dkim/{domain}/public.key`
- **Email Routes**: `/email/routes/{routeId}.json`
- **Bounce Lists**: `/email/bounces/suppression.json`
- **IP Reputation**: `/security/ip-reputation/{ip}.json`
- **Domain Configs**: `/email/domains/{domain}.json`
### Data Migration
Migrate data between storage backends:
```typescript
import { StorageManager } from '@serve.zone/dcrouter';
// Export from filesystem
const fsStorage = new StorageManager({ fsPath: './data' });
const keys = await fsStorage.list('/');
const data = {};
for (const key of keys) {
data[key] = await fsStorage.get(key);
}
// Import to cloud storage
const cloudStorage = new StorageManager({
readFunction: cloudRead,
writeFunction: cloudWrite
});
for (const [key, value] of Object.entries(data)) {
await cloudStorage.set(key, value);
}
```
## Security Features
### IP Reputation Checking
@ -776,6 +934,25 @@ const router = new DcRouter({
ports: [25, 587, 465],
hostname: 'mail.example.com',
// Domain configuration
domains: [
{
domain: 'example.com',
dnsMode: 'external-dns',
dkim: {
selector: 'mail',
rotateKeys: true
}
},
{
domain: 'notifications.example.com',
dnsMode: 'internal-dns',
rateLimits: {
outbound: { messagesPerHour: 10000 }
}
}
],
// Authentication configuration
auth: {
required: true,
@ -857,6 +1034,11 @@ const router = new DcRouter({
// Cloudflare DNS challenges
dnsChallenge: {
cloudflareApiKey: process.env.CLOUDFLARE_API_KEY
},
// Persistent storage
storage: {
fsPath: '/var/lib/dcrouter/data'
}
});
@ -1032,6 +1214,13 @@ DcRouter includes a comprehensive test suite covering all aspects of the system:
- **Performance**: Throughput testing
- **Edge Cases**: Very large emails, special characters
#### Storage and Configuration Tests
- **Storage Manager**: All backend types (filesystem, custom, memory)
- **Integration**: Component storage usage and persistence
- **DNS Validation**: All DNS modes (forward, internal, external)
- **DNS Mode Switching**: Dynamic configuration changes
- **Data Migration**: Moving data between storage backends
#### Running Tests
```bash