docs(readme): update documentation with StorageManager and domain configuration features
- Add flexible storage system section with backend examples - Document email domain configuration with DNS modes - Update configuration interfaces with storage options - Add examples for filesystem, custom, and memory storage - Include data migration examples between backends - Document storage usage patterns and key structure - Update test suite documentation with new test categories
This commit is contained in:
parent
53b64025f3
commit
e6251ab655
191
readme.md
191
readme.md
@ -44,6 +44,12 @@ A comprehensive traffic routing solution that provides unified gateway capabilit
|
|||||||
- **Rate limiting** at multiple levels
|
- **Rate limiting** at multiple levels
|
||||||
- **Real-time metrics** and monitoring
|
- **Real-time metrics** and monitoring
|
||||||
|
|
||||||
|
### 💾 **Flexible Storage System**
|
||||||
|
- **Multiple storage backends**: filesystem, custom functions, or memory
|
||||||
|
- **Unified storage interface** for all components
|
||||||
|
- **Automatic data migration** between backends
|
||||||
|
- **Persistent configuration** for domains, routes, and security data
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@ -207,12 +213,18 @@ interface IDcRouterOptions {
|
|||||||
emailConfig?: {
|
emailConfig?: {
|
||||||
ports: number[];
|
ports: number[];
|
||||||
hostname: string;
|
hostname: string;
|
||||||
routes: IEmailRoute[]; // Route-based configuration
|
domains?: IEmailDomainConfig[]; // Domain infrastructure setup
|
||||||
|
routes: IEmailRoute[]; // Route-based email handling
|
||||||
auth?: IAuthConfig;
|
auth?: IAuthConfig;
|
||||||
tls?: ITlsConfig;
|
tls?: ITlsConfig;
|
||||||
maxMessageSize?: number;
|
maxMessageSize?: number;
|
||||||
rateLimits?: IRateLimitConfig;
|
rateLimits?: IRateLimitConfig;
|
||||||
useSocketHandler?: boolean; // Enable socket-handler mode (no port binding)
|
useSocketHandler?: boolean; // Enable socket-handler mode (no port binding)
|
||||||
|
defaults?: { // Global defaults for all domains
|
||||||
|
dnsMode?: 'forward' | 'internal-dns' | 'external-dns';
|
||||||
|
dkim?: IDkimConfig;
|
||||||
|
rateLimits?: IRateLimitConfig;
|
||||||
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
// DNS server configuration
|
// DNS server configuration
|
||||||
@ -235,6 +247,13 @@ interface IDcRouterOptions {
|
|||||||
dnsChallenge?: {
|
dnsChallenge?: {
|
||||||
cloudflareApiKey: string;
|
cloudflareApiKey: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// Storage configuration
|
||||||
|
storage?: {
|
||||||
|
fsPath?: string; // Filesystem storage path
|
||||||
|
readFunction?: (key: string) => Promise<string>; // Custom read function
|
||||||
|
writeFunction?: (key: string, value: string) => Promise<void>; // Custom write function
|
||||||
|
};
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -333,6 +352,73 @@ External Port → SmartProxy → Socket Handler → Service
|
|||||||
|
|
||||||
## Email System
|
## Email System
|
||||||
|
|
||||||
|
### Email Domain Configuration
|
||||||
|
|
||||||
|
DcRouter separates email infrastructure (which domains to handle) from routing logic (how to handle emails):
|
||||||
|
|
||||||
|
#### **DNS Modes**
|
||||||
|
|
||||||
|
**Forward Mode** - Simple mail forwarding without local DNS:
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
domain: 'forwarded.com',
|
||||||
|
dnsMode: 'forward',
|
||||||
|
dns: {
|
||||||
|
forward: {
|
||||||
|
skipDnsValidation: true,
|
||||||
|
targetDomain: 'mail.target.com'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Internal DNS Mode** - Use built-in DNS server (requires `dnsDomain` in DcRouter config):
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
domain: 'mail.example.com',
|
||||||
|
dnsMode: 'internal-dns',
|
||||||
|
dns: {
|
||||||
|
internal: {
|
||||||
|
mxPriority: 10,
|
||||||
|
ttl: 3600
|
||||||
|
}
|
||||||
|
},
|
||||||
|
dkim: {
|
||||||
|
selector: 'mail2024',
|
||||||
|
keySize: 2048,
|
||||||
|
rotateKeys: true,
|
||||||
|
rotationInterval: 90
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**External DNS Mode** - Use existing DNS infrastructure:
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
domain: 'mail.external.com',
|
||||||
|
dnsMode: 'external-dns',
|
||||||
|
dns: {
|
||||||
|
external: {
|
||||||
|
requiredRecords: ['MX', 'SPF', 'DKIM', 'DMARC']
|
||||||
|
}
|
||||||
|
},
|
||||||
|
rateLimits: {
|
||||||
|
inbound: {
|
||||||
|
messagesPerMinute: 100,
|
||||||
|
connectionsPerIp: 10
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### **DKIM Management**
|
||||||
|
|
||||||
|
DKIM is always enabled for all domains. Keys are automatically:
|
||||||
|
- Generated on first use
|
||||||
|
- Stored persistently via StorageManager
|
||||||
|
- Rotated based on configuration
|
||||||
|
- Cleaned up after grace period
|
||||||
|
|
||||||
### Email Route Actions
|
### Email Route Actions
|
||||||
|
|
||||||
#### **Forward Action**
|
#### **Forward Action**
|
||||||
@ -630,6 +716,78 @@ const tcpRoutes = [
|
|||||||
];
|
];
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Storage System
|
||||||
|
|
||||||
|
### StorageManager
|
||||||
|
|
||||||
|
DcRouter includes a flexible storage system that supports multiple backends:
|
||||||
|
|
||||||
|
#### **Filesystem Storage**
|
||||||
|
```typescript
|
||||||
|
const router = new DcRouter({
|
||||||
|
storage: {
|
||||||
|
fsPath: '/var/lib/dcrouter/data'
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
#### **Custom Storage Backend**
|
||||||
|
```typescript
|
||||||
|
const router = new DcRouter({
|
||||||
|
storage: {
|
||||||
|
readFunction: async (key) => {
|
||||||
|
// Read from Redis, S3, etc.
|
||||||
|
return await myDatabase.get(key);
|
||||||
|
},
|
||||||
|
writeFunction: async (key, value) => {
|
||||||
|
// Write to Redis, S3, etc.
|
||||||
|
await myDatabase.set(key, value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
#### **Memory Storage (Development)**
|
||||||
|
```typescript
|
||||||
|
const router = new DcRouter({
|
||||||
|
// No storage config = memory storage with warning
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Storage Usage
|
||||||
|
|
||||||
|
The storage system is used for:
|
||||||
|
- **DKIM Keys**: `/email/dkim/{domain}/private.key`, `/email/dkim/{domain}/public.key`
|
||||||
|
- **Email Routes**: `/email/routes/{routeId}.json`
|
||||||
|
- **Bounce Lists**: `/email/bounces/suppression.json`
|
||||||
|
- **IP Reputation**: `/security/ip-reputation/{ip}.json`
|
||||||
|
- **Domain Configs**: `/email/domains/{domain}.json`
|
||||||
|
|
||||||
|
### Data Migration
|
||||||
|
|
||||||
|
Migrate data between storage backends:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { StorageManager } from '@serve.zone/dcrouter';
|
||||||
|
|
||||||
|
// Export from filesystem
|
||||||
|
const fsStorage = new StorageManager({ fsPath: './data' });
|
||||||
|
const keys = await fsStorage.list('/');
|
||||||
|
const data = {};
|
||||||
|
for (const key of keys) {
|
||||||
|
data[key] = await fsStorage.get(key);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Import to cloud storage
|
||||||
|
const cloudStorage = new StorageManager({
|
||||||
|
readFunction: cloudRead,
|
||||||
|
writeFunction: cloudWrite
|
||||||
|
});
|
||||||
|
for (const [key, value] of Object.entries(data)) {
|
||||||
|
await cloudStorage.set(key, value);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
## Security Features
|
## Security Features
|
||||||
|
|
||||||
### IP Reputation Checking
|
### IP Reputation Checking
|
||||||
@ -776,6 +934,25 @@ const router = new DcRouter({
|
|||||||
ports: [25, 587, 465],
|
ports: [25, 587, 465],
|
||||||
hostname: 'mail.example.com',
|
hostname: 'mail.example.com',
|
||||||
|
|
||||||
|
// Domain configuration
|
||||||
|
domains: [
|
||||||
|
{
|
||||||
|
domain: 'example.com',
|
||||||
|
dnsMode: 'external-dns',
|
||||||
|
dkim: {
|
||||||
|
selector: 'mail',
|
||||||
|
rotateKeys: true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
domain: 'notifications.example.com',
|
||||||
|
dnsMode: 'internal-dns',
|
||||||
|
rateLimits: {
|
||||||
|
outbound: { messagesPerHour: 10000 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
|
||||||
// Authentication configuration
|
// Authentication configuration
|
||||||
auth: {
|
auth: {
|
||||||
required: true,
|
required: true,
|
||||||
@ -857,6 +1034,11 @@ const router = new DcRouter({
|
|||||||
// Cloudflare DNS challenges
|
// Cloudflare DNS challenges
|
||||||
dnsChallenge: {
|
dnsChallenge: {
|
||||||
cloudflareApiKey: process.env.CLOUDFLARE_API_KEY
|
cloudflareApiKey: process.env.CLOUDFLARE_API_KEY
|
||||||
|
},
|
||||||
|
|
||||||
|
// Persistent storage
|
||||||
|
storage: {
|
||||||
|
fsPath: '/var/lib/dcrouter/data'
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
@ -1032,6 +1214,13 @@ DcRouter includes a comprehensive test suite covering all aspects of the system:
|
|||||||
- **Performance**: Throughput testing
|
- **Performance**: Throughput testing
|
||||||
- **Edge Cases**: Very large emails, special characters
|
- **Edge Cases**: Very large emails, special characters
|
||||||
|
|
||||||
|
#### Storage and Configuration Tests
|
||||||
|
- **Storage Manager**: All backend types (filesystem, custom, memory)
|
||||||
|
- **Integration**: Component storage usage and persistence
|
||||||
|
- **DNS Validation**: All DNS modes (forward, internal, external)
|
||||||
|
- **DNS Mode Switching**: Dynamic configuration changes
|
||||||
|
- **Data Migration**: Moving data between storage backends
|
||||||
|
|
||||||
#### Running Tests
|
#### Running Tests
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
Loading…
x
Reference in New Issue
Block a user