Files
dcrouter/readme.hints.md

773 lines
27 KiB
Markdown
Raw Normal View History

# Implementation Hints and Learnings
## smartmta Migration (2026-02-11)
### Overview
dcrouter's custom MTA code (~27,149 lines / 68 files in `ts/mail/` + `ts/deliverability/`) has been replaced with `@push.rocks/smartmta` v5.2.1, a TypeScript+Rust hybrid MTA. dcrouter is now an orchestrator that wires together SmartProxy, smartmta, smartdns, smartradius, and OpsServer.
### Architecture
- **No socket-handler mode** — smartmta's Rust SMTP server binds its own ports directly
- **SmartProxy forward mode only** — external email ports forwarded to internal ports where smartmta listens
- Email traffic flow: External Port → SmartProxy → Internal Port → smartmta UnifiedEmailServer
### Key API Differences (smartmta vs old custom MTA)
- `updateEmailRoutes()` instead of `updateRoutes()`
- `dkimCreator` is public (no need for `(this.emailServer as any).dkimCreator`)
- `bounceManager` is private, but exposed via public methods:
- `emailServer.getSuppressionList()`
- `emailServer.getHardBouncedAddresses()`
- `emailServer.getBounceHistory(email)`
- `emailServer.removeFromSuppressionList(email)`
- `Email` class imported from `@push.rocks/smartmta`
- `IAttachment` type accessed via `Core` namespace: `import { type Core } from '@push.rocks/smartmta'; type IAttachment = Core.IAttachment;`
### Deleted Directories
- `ts/mail/` (60 files) — replaced by smartmta
- `ts/deliverability/` (3 files) — IPWarmupManager/SenderReputationMonitor will move to smartmta
- `ts/errors/email.errors.ts`, `ts/errors/mta.errors.ts` — smartmta has its own errors
- `ts/cache/documents/classes.cached.bounce.ts`, `classes.cached.suppression.ts`, `classes.cached.dkim.ts` — smartmta handles its own persistence
### Remaining Cache Documents
- `CachedEmail` — kept (dcrouter-level queue persistence)
- `CachedIPReputation` — kept (dcrouter-level IP reputation caching)
### Dependencies Removed
mailauth, mailparser, @types/mailparser, ip, @push.rocks/smartmail, @push.rocks/smartrule, node-forge
### Pre-existing Test Failures (not caused by migration)
- `test/test.jwt-auth.ts``response.text is not a function` (webrequest compatibility issue)
- `test/test.opsserver-api.ts` — same webrequest issue, timeouts
### smartmta Location
Source at `../../push.rocks/smartmta`, release with `gitzone commit -ypbrt`
## Dependency Upgrade (2026-02-11)
### SmartProxy v23.1.2 Route Validation
- SmartProxy 23.1.2 enforces stricter route validation
- Forward actions MUST use `targets` (array) instead of `target` (singular)
- Test configurations that call `DcRouter.start()` need `cacheConfig: { enabled: false }` to avoid `/etc/dcrouter` permission errors
```typescript
// WRONG - will fail validation
action: { type: 'forward', target: { host: 'localhost', port: 10025 } }
// CORRECT
action: { type: 'forward', targets: [{ host: 'localhost', port: 10025 }] }
```
**Files Fixed:**
- `ts/classes.dcrouter.ts` - `generateEmailRoutes()` method
- `test/test.dcrouter.email.ts` - Updated assertions and added `cacheConfig: { enabled: false }`
## Dependency Upgrade (2026-02-10)
### SmartProxy v23.1.0 Upgrade
- `@push.rocks/smartproxy`: 22.4.2 → 23.1.0
**Key Changes:**
- Rust-based proxy components for improved performance
- Rust binary runs as separate process via IPC
- `getStatistics()` now returns `Promise<any>` (was synchronous)
- nftables-proxy removed (not used by dcrouter)
**Code Changes Required:**
```typescript
// Old (synchronous)
const proxyStats = this.dcRouter.smartProxy.getStatistics();
// New (async)
const proxyStats = await this.dcRouter.smartProxy.getStatistics();
```
**Files Modified:**
- `ts/monitoring/classes.metricsmanager.ts` - Added `await` to `getStatistics()` call
## Dependency Upgrade (2026-02-01)
### Major Upgrades Completed
- `@api.global/typedserver`: 3.0.80 → 8.3.0
- `@api.global/typedsocket`: 3.1.1 → 4.1.0
- `@apiclient.xyz/cloudflare`: 6.4.3 → 7.1.0
- `@design.estate/dees-catalog`: 1.12.4 → 3.41.4
- `@push.rocks/smartpath`: 5.1.0 → 6.0.0
- `@push.rocks/smartproxy`: 19.6.17 → 22.4.2
- `@push.rocks/smartrequest`: 2.1.0 → 5.0.1
- `uuid`: 11.1.0 → 13.0.0
### Breaking Changes Fixed
1. **SmartProxy v22**: `target``targets` (array)
```typescript
// Old
action: { type: 'forward', target: { host: 'x', port: 25 } }
// New
action: { type: 'forward', targets: [{ host: 'x', port: 25 }] }
```
2. **SmartRequest v5**: `SmartRequestClient``SmartRequest`, `.body``.json()`
```typescript
// Old
const resp = await plugins.smartrequest.SmartRequestClient.create()...post();
const json = resp.body;
// New
const resp = await plugins.smartrequest.SmartRequest.create()...post();
const json = await resp.json();
```
3. **dees-catalog v3**: Icon naming changed to library-prefixed format
```typescript
// Old (deprecated but supported)
<dees-icon iconFA="check"></dees-icon>
// New
<dees-icon icon="fa:check"></dees-icon>
<dees-icon icon="lucide:menu"></dees-icon>
```
### TC39 Decorators
- ts_web components updated to use `accessor` keyword for `@state()` decorators
- Required for TC39 standard decorator support
### tswatch Configuration
The project now uses tswatch for development:
```bash
pnpm run watch
```
Configuration in `npmextra.json`:
```json
{
"@git.zone/tswatch": {
"watchers": [{
"name": "dcrouter-dev",
"watch": ["ts/**/*.ts", "ts_*/**/*.ts", "test_watch/devserver.ts"],
"command": "pnpm run build && tsrun test_watch/devserver.ts",
"restart": true,
"debounce": 500,
"runOnStart": true
}]
}
}
```
## RADIUS Server Integration (2026-02-01)
### Overview
DcRouter now supports RADIUS server functionality for network authentication via `@push.rocks/smartradius`.
### Key Features
- **MAC Authentication Bypass (MAB)** - Authenticate network devices based on MAC address
- **VLAN Assignment** - Assign VLANs based on MAC address or OUI patterns
- **RADIUS Accounting** - Track sessions, data usage, and billing
### Configuration Example
```typescript
const dcRouter = new DcRouter({
radiusConfig: {
authPort: 1812, // Authentication port (default)
acctPort: 1813, // Accounting port (default)
clients: [
{
name: 'switch-1',
ipRange: '192.168.1.0/24',
secret: 'shared-secret',
enabled: true
}
],
vlanAssignment: {
defaultVlan: 100, // VLAN for unknown MACs
allowUnknownMacs: true,
mappings: [
{ mac: '00:11:22:33:44:55', vlan: 10, enabled: true },
{ mac: '00:11:22', vlan: 20, enabled: true } // OUI pattern
]
},
accounting: {
enabled: true,
retentionDays: 30
}
}
});
```
### Components
- `RadiusServer` - Main server wrapping smartradius
- `VlanManager` - MAC-to-VLAN mapping with OUI pattern support
- `AccountingManager` - Session tracking and billing data
### OpsServer API Endpoints
- `getRadiusClients` / `setRadiusClient` / `removeRadiusClient` - Client management
- `getVlanMappings` / `setVlanMapping` / `removeVlanMapping` - VLAN mappings
- `testVlanAssignment` - Test what VLAN a MAC would get
- `getRadiusSessions` / `disconnectRadiusSession` - Session management
- `getRadiusStatistics` / `getRadiusAccountingSummary` - Statistics
### Files
- `ts/radius/` - RADIUS module
- `ts/opsserver/handlers/radius.handler.ts` - OpsServer handler
- `ts_interfaces/requests/radius.ts` - TypedRequest interfaces
## Test Fix: test.dcrouter.email.ts (2026-02-01)
### Issue
The test `DcRouter class - Custom email storage path` was failing with "domainConfigs is not iterable".
### Root Cause
The test was using outdated email config properties:
- Used `domainRules: []` (non-existent property)
- Used `defaultMode` (non-existent property)
- Missing required `domains: []` property
- Missing required `routes: []` property
- Referenced `router.unifiedEmailServer` instead of `router.emailServer`
### Fix
Updated the test to use the correct `IUnifiedEmailServerOptions` interface properties:
```typescript
const emailConfig: IEmailConfig = {
ports: [2525],
hostname: 'mail.example.com',
domains: [], // Required: domain configurations
routes: [] // Required: email routing rules
};
```
And fixed the property name:
```typescript
expect(router.emailServer).toBeTruthy(); // Not unifiedEmailServer
```
### Key Learning
When using `IUnifiedEmailServerOptions` (aliased as `IEmailConfig` in some tests):
- `domains: IEmailDomainConfig[]` is required (array of domain configs)
- `routes: IEmailRoute[]` is required (email routing rules)
- Access the email server via `dcRouter.emailServer` not `dcRouter.unifiedEmailServer`
2025-06-23 00:19:47 +00:00
## Network Metrics Implementation (2025-06-23)
### SmartProxy Metrics API Integration
- Updated to use new SmartProxy metrics API (v19.6.7)
- Use `getMetrics()` for detailed metrics with grouped methods:
```typescript
const metrics = smartProxy.getMetrics();
metrics.connections.active() // Current active connections
metrics.throughput.instant() // Real-time throughput {in, out}
metrics.connections.topIPs(10) // Top 10 IPs by connection count
```
- Use `getStatistics()` for basic stats
### Network Traffic Display
- All throughput values shown in bits per second (kbit/s, Mbit/s, Gbit/s)
- Conversion: `bytesPerSecond * 8 / 1000000` for Mbps
- Network graph shows separate lines for inbound (green) and outbound (purple)
- Throughput tiles and graph use same data source for consistency
### Requests/sec vs Connections
- Requests/sec shows HTTP request counts (derived from connections)
- Single connection can handle multiple requests
- Current implementation tracks connections, not individual requests
- Trend line shows historical request counts, not throughput
## DKIM Implementation Status (2025-05-30)
**Note:** DKIM is now handled by `@push.rocks/smartmta`. The `dkimCreator` is a public property on `UnifiedEmailServer`.
## SmartProxy Usage
### New Route-Based Architecture (v18+)
- SmartProxy now uses a route-based configuration system
- Routes define match criteria and actions instead of simple port-to-port forwarding
- All traffic types (HTTP, HTTPS, TCP, WebSocket) are configured through routes
```typescript
// NEW: Route-based SmartProxy configuration
const smartProxy = new plugins.smartproxy.SmartProxy({
routes: [
{
name: 'https-traffic',
match: {
ports: 443,
domains: ['example.com', '*.example.com']
},
action: {
type: 'forward',
target: {
host: 'backend.server.com',
port: 8080
}
},
tls: {
mode: 'terminate',
certificate: 'auto'
}
}
],
defaults: {
target: {
host: 'fallback.server.com',
port: 8080
}
},
acme: {
accountEmail: 'admin@example.com',
enabled: true,
useProduction: true
}
});
```
### Migration from Old to New
```typescript
// OLD configuration style (deprecated)
{
fromPort: 443,
toPort: 8080,
targetIP: 'backend.server.com',
domainConfigs: [...]
}
// NEW route-based style
{
routes: [{
name: 'main-route',
match: { ports: 443 },
action: {
type: 'forward',
target: { host: 'backend.server.com', port: 8080 }
}
}]
}
```
### Direct Component Usage
- Use SmartProxy components directly instead of creating your own wrappers
- SmartProxy already includes Port80Handler and NetworkProxy functionality
- When using SmartProxy, configure it directly rather than instantiating Port80Handler or NetworkProxy separately
### Certificate Management
- SmartProxy has built-in ACME certificate management
- Configure it in the `acme` property of SmartProxy options
- Use `accountEmail` (not `email`) for the ACME contact email
- SmartProxy handles both HTTP-01 challenges and certificate application automatically
## qenv Usage
### Direct Usage
- Use qenv directly instead of creating environment variable wrappers
- Instantiate qenv with appropriate basePath and nogitPath:
```typescript
const qenv = new plugins.qenv.Qenv('./', '.nogit/');
const value = await qenv.getEnvVarOnDemand('ENV_VAR_NAME');
```
## TypeScript Interfaces
### SmartProxy Interfaces
- Always check the interfaces from the node_modules to ensure correct property names
- Important interfaces for the new architecture:
- `ISmartProxyOptions`: Main configuration with `routes` array
- `IRouteConfig`: Individual route configuration
- `IRouteMatch`: Match criteria for routes
- `IRouteTarget`: Target configuration for forwarding
- `IAcmeOptions`: ACME certificate configuration
- `TTlsMode`: TLS handling modes ('passthrough' | 'terminate' | 'terminate-and-reencrypt')
### New Route Configuration
```typescript
interface IRouteConfig {
name: string;
match: {
ports: number | number[];
domains?: string | string[];
path?: string;
headers?: Record<string, string | RegExp>;
};
action: {
type: 'forward' | 'redirect' | 'block' | 'static';
target?: {
host: string | string[] | ((context) => string);
port: number | 'preserve' | ((context) => number);
};
};
tls?: {
mode: TTlsMode;
certificate?: 'auto' | { key: string; cert: string; };
};
security?: {
authentication?: IRouteAuthentication;
rateLimit?: IRouteRateLimit;
ipAllowList?: string[];
ipBlockList?: string[];
};
}
```
### Required Properties
- For `ISmartProxyOptions`, `routes` array is the main configuration
- For `IAcmeOptions`, use `accountEmail` for the contact email
- Routes must have `name`, `match`, and `action` properties
## Testing
### Test Structure
- Follow the project's test structure, using `@push.rocks/tapbundle`
- Use `expect(value).toEqual(expected)` for equality checks
- Use `expect(value).toBeTruthy()` for boolean assertions
```typescript
tap.test('test description', async () => {
const result = someFunction();
expect(result.property).toEqual('expected value');
expect(result.valid).toBeTruthy();
});
```
### Cleanup
- Include a cleanup test to ensure proper test resource handling
- Add a `stop` test to forcefully end the test when needed:
```typescript
tap.test('stop', async () => {
await tap.stopForcefully();
});
```
## Architecture Principles
### Simplicity
- Prefer direct usage of libraries instead of creating wrappers
- Don't reinvent functionality that already exists in dependencies
- Keep interfaces clean and focused, avoiding unnecessary abstraction layers
### Component Integration
- Leverage built-in integrations between components (like SmartProxy's ACME handling)
- Use parallel operations for performance (like in the `stop()` method)
- Separate concerns clearly (HTTP handling vs. SMTP handling)
## Email Integration with SmartProxy
### Architecture (Post-Migration)
- Email traffic is routed through SmartProxy using automatic route generation
- smartmta's UnifiedEmailServer runs on internal ports and receives forwarded traffic from SmartProxy
- SmartProxy handles external ports (25, 587, 465) and forwards to internal ports
- smartmta's Rust SMTP bridge handles SMTP protocol processing
### Port Mapping
- External port 25 → Internal port 10025 (SMTP)
- External port 587 → Internal port 10587 (Submission)
- External port 465 → Internal port 10465 (SMTPS)
### TLS Handling
- Ports 25 and 587: Use 'passthrough' mode (STARTTLS handled by smartmta)
- Port 465: Use 'terminate' mode (SmartProxy handles TLS termination)
## SmartMetrics Integration (2025-06-12) - COMPLETED
### Overview
Fixed the UI metrics display to show accurate CPU and memory data from SmartMetrics.
### Key Findings
1. **CPU Metrics:**
- SmartMetrics provides `cpuUsageText` as a string percentage
- MetricsManager parses it as `cpuUsage.user` (system is always 0)
- UI was incorrectly dividing by 2, showing half the actual CPU usage
2. **Memory Metrics:**
- SmartMetrics calculates `maxMemoryMB` as minimum of:
- V8 heap size limit
- System total memory
- Docker memory limit (if available)
- Provides `memoryUsageBytes` (total process memory including children)
- Provides `memoryPercentage` (pre-calculated percentage)
- UI was only showing heap usage, missing actual memory constraints
### Changes Made
1. **MetricsManager Enhanced:**
- Added `maxMemoryMB` from SmartMetrics instance
- Added `actualUsageBytes` from SmartMetrics data
- Added `actualUsagePercentage` from SmartMetrics data
- Kept existing memory fields for compatibility
2. **Interface Updated:**
- Added optional fields to `IServerStats.memoryUsage`
- Fields are optional to maintain backward compatibility
3. **UI Fixed:**
- Removed incorrect CPU division by 2
- Uses `actualUsagePercentage` when available (falls back to heap percentage)
- Shows actual memory usage vs max memory limit (not just heap)
### Result
- CPU now shows accurate usage percentage
- Memory shows percentage of actual constraints (Docker/system/V8 limits)
- Better monitoring for containerized environments
## Network UI Implementation (2025-06-20) - COMPLETED
### Overview
Revamped the Network UI to display real network data from SmartProxy instead of mock data.
### Architecture
1. **MetricsManager Integration:**
- Already integrates with SmartProxy via `dcRouter.smartProxy.getStats()`
- Extended with `getNetworkStats()` method to expose unused metrics:
- `getConnectionsByIP()` - Connection counts by IP address
- `getThroughputRate()` - Real-time bandwidth rates (bytes/second)
- `getTopIPs()` - Top connecting IPs sorted by connection count
- Note: SmartProxy base interface doesn't include all methods, manual implementation required
2. **Existing Infrastructure Leveraged:**
- `getActiveConnections` endpoint already exists in security.handler.ts
- Enhanced to include real SmartProxy data via MetricsManager
- IConnectionInfo interface already supports network data structures
3. **State Management:**
- Added `INetworkState` interface following existing patterns
- Created `networkStatePart` with connections, throughput, and IP data
- Integrated with existing auto-refresh mechanism
4. **UI Changes (Minimal):**
- Removed `generateMockData()` method and all mock generation
- Connected to real `networkStatePart` state
- Added `renderTopIPs()` section to display top connected IPs
- Updated traffic chart to show real request data
- Kept all existing UI components (DeesTable, DeesChartArea)
### Implementation Details
1. **Data Transformation:**
- Converts IConnectionInfo[] to INetworkRequest[] for table display
- Calculates traffic buckets based on selected time range
- Maps connection data to chart-compatible format
2. **Real Metrics Displayed:**
- Active connections count (from server stats)
- Requests per second (calculated from recent connections)
- Throughput rates (currently showing 0 until SmartProxy exposes rates)
- Top IPs with connection counts and percentages
3. **TypeScript Fixes:**
- SmartProxy methods like `getThroughputRate()` not in base interface
- Implemented manual fallbacks for missing methods
- Fixed `publicIpv4``publicIp` property name
### Result
- Network view now shows real connection activity
- Auto-refreshes with other stats every second
- Displays actual IPs and connection counts
- No more mock/demo data
- Minimal code changes (streamlined approach)
### Throughput Data Fix (2025-06-20)
The throughput was showing 0 because:
1. MetricsManager was hardcoding throughputRate to 0, assuming the method didn't exist
2. SmartProxy's `getStats()` returns `IProxyStats` interface, but the actual object (`MetricsCollector`) implements `IProxyStatsExtended`
3. `getThroughputRate()` only exists in the extended interface
**Solution implemented:**
1. Updated MetricsManager to check if methods exist at runtime and call them
2. Added property name mapping (`bytesInPerSec``bytesInPerSecond`)
3. Created new `getNetworkStats` endpoint in security.handler.ts
4. Updated frontend to call the new endpoint for complete network metrics
The throughput data now flows correctly from SmartProxy → MetricsManager → API → UI.
## Email Operations Dashboard (2026-02-01)
### Overview
Replaced mock data in the email UI with real backend data from the delivery queue and security logger.
### New Files Created
- `ts_interfaces/requests/email-ops.ts` - TypedRequest interfaces for email operations
- `ts/opsserver/handlers/email-ops.handler.ts` - Backend handler for email operations
### Key Interfaces
- `IReq_GetQueuedEmails` - Fetch emails from delivery queue by status
- `IReq_GetSentEmails` - Fetch delivered emails
- `IReq_GetFailedEmails` - Fetch failed emails
- `IReq_ResendEmail` - Re-queue a failed email for retry
- `IReq_GetSecurityIncidents` - Fetch security events from SecurityLogger
- `IReq_GetBounceRecords` - Fetch bounce records and suppression list
- `IReq_RemoveFromSuppressionList` - Remove email from suppression list
### UI Changes (ops-view-emails.ts)
- Replaced mock folders (inbox/sent/draft/trash) with operations views:
- **Queued**: Emails pending delivery
- **Sent**: Successfully delivered emails
- **Failed**: Failed emails with resend capability
- **Security**: Security incidents from SecurityLogger
- Removed `generateMockEmails()` method
- Added state management via `emailOpsStatePart` in appstate.ts
- Added resend button for failed emails
- Added security incident detail view
### Data Flow
```
UnifiedDeliveryQueue → EmailOpsHandler → TypedRequest → Frontend State → UI
SecurityLogger → EmailOpsHandler → TypedRequest → Frontend State → UI
BounceManager → EmailOpsHandler → TypedRequest → Frontend State → UI
```
### Backend Data Access
The handler accesses data from:
- `dcRouter.emailServer.deliveryQueue` - Email queue items (IQueueItem)
- `SecurityLogger.getInstance()` - Security events (ISecurityEvent)
- `emailServer.bounceManager` - Bounce records and suppression list
## OpsServer UI Fixes (2026-02-02)
### Configuration Page Fix
The configuration page had field name mismatches between frontend and backend:
- Frontend expected `server` and `storage` sections
- Backend returns `proxy` section (not `server`)
- Backend has no `storage` section
**Fix**: Updated `ops-view-config.ts` to use correct section names:
- `proxy` instead of `server`
- Removed non-existent `storage` section
- Added optional chaining (`?.`) for safety
### Auth Persistence Fix
Login state was using `'soft'` mode in Smartstate which is memory-only:
- User login was lost on page refresh
- State reset to logged out after browser restart
**Changes**:
1. `ts_web/appstate.ts`: Changed loginStatePart from `'soft'` to `'persistent'`
- Now uses IndexedDB to persist across browser sessions
2. `ts/opsserver/handlers/admin.handler.ts`: JWT expiry changed from 7 days to 24 hours
3. `ts_web/elements/ops-dashboard.ts`: Added JWT expiry check on session restore
- Validates stored JWT hasn't expired before auto-logging in
- Clears expired sessions and shows login form
## Config UI Read-Only Conversion (2026-02-03)
### Overview
The configuration UI has been converted from an editable interface to a read-only display. DcRouter is configured through code or remotely, not through the UI.
### Changes Made
1. **Backend (`ts/opsserver/handlers/config.handler.ts`)**:
- Removed `updateConfiguration` handler
- Removed `updateConfiguration()` private method
- Kept `getConfiguration` handler (read-only)
2. **Interfaces (`ts_interfaces/requests/config.ts`)**:
- Removed `IReq_UpdateConfiguration` interface
- Kept `IReq_GetConfiguration` interface
3. **Frontend (`ts_web/elements/ops-view-config.ts`)**:
- Removed `editingSection` and `editedConfig` state properties
- Removed `startEdit()`, `cancelEdit()`, `saveConfig()` methods
- Removed Edit/Save/Cancel buttons
- Removed warning banner about immediate changes
- Enhanced read-only display with:
- Status badges for boolean values (enabled/disabled)
- Array display as pills/tags with counts
- Section icons (mail, globe, network, shield)
- Better formatting for numbers and byte sizes
- Empty state handling ("Not configured", "None configured")
- Info note explaining configuration is read-only
4. **State Management (`ts_web/appstate.ts`)**:
- Removed `updateConfigurationAction`
- Kept `fetchConfigurationAction` (read-only)
5. **Tests (`test/test.protected-endpoint.ts`)**:
- Replaced `updateConfiguration` tests with `verifyIdentity` tests
- Added test for read-only config access
- Kept auth flow testing with different protected endpoint
6. **Documentation**:
- `readme.md`: Updated API endpoints to show config as read-only
- `ts_web/readme.md`: Removed `updateConfigurationAction` from actions list
- `ts_interfaces/readme.md`: Removed `IReq_UpdateConfiguration` from table
### Visual Display Features
- Boolean values shown as colored badges (green=enabled, red=disabled)
- Arrays displayed as pills with count summaries
- Section headers with relevant Lucide icons
- Numbers formatted with locale separators
- Byte sizes auto-formatted (B, KB, MB, GB)
- Time values shown with "seconds" suffix
- Nested objects with visual indentation
## Smartdata Cache System (2026-02-03)
### Overview
DcRouter now uses smartdata + LocalTsmDb for persistent caching. Data is stored at `/etc/dcrouter/tsmdb`.
### Technology Stack
| Layer | Package | Purpose |
|-------|---------|---------|
| ORM | `@push.rocks/smartdata` | Document classes, decorators, queries |
| Database | `@push.rocks/smartmongo` (LocalTsmDb) | Embedded TsmDB via Unix socket |
### TC39 Decorators
The project uses TC39 Stage 3 decorators (not experimental decorators). The tsconfig was updated:
- Removed `experimentalDecorators: true`
- Removed `emitDecoratorMetadata: true`
This is required for smartdata v7+ compatibility.
### Cache Document Classes
Located in `ts/cache/documents/`:
| Class | Purpose | Default TTL |
|-------|---------|-------------|
| `CachedEmail` | Email queue items | 30 days |
| `CachedIPReputation` | IP reputation lookups | 24 hours |
Note: CachedBounce, CachedSuppression, and CachedDKIMKey were removed in the smartmta migration (smartmta handles its own persistence for those).
### Usage Pattern
```typescript
// Document classes use smartdata decorators
@plugins.smartdata.Collection(() => getDb())
export class CachedEmail extends CachedDocument<CachedEmail> {
@plugins.smartdata.svDb()
public createdAt: Date = new Date();
@plugins.smartdata.svDb()
public expiresAt: Date = new Date(Date.now() + TTL.DAYS_30);
@plugins.smartdata.unI()
@plugins.smartdata.svDb()
public id: string;
// ...
}
// Query examples
const email = await CachedEmail.getInstance({ id: 'abc123' });
const pending = await CachedEmail.getInstances({ status: 'pending' });
await email.save();
await email.delete();
```
### Configuration
```typescript
const dcRouter = new DcRouter({
cacheConfig: {
enabled: true,
storagePath: '/etc/dcrouter/tsmdb',
dbName: 'dcrouter',
cleanupIntervalHours: 1,
ttlConfig: {
emails: 30, // days
ipReputation: 1, // days
bounces: 30, // days
dkimKeys: 90, // days
suppression: 30 // days
}
}
});
```
### Cache Cleaner
- Runs hourly by default (configurable via `cleanupIntervalHours`)
- Finds and deletes documents where `expiresAt < now()`
- Uses smartdata's `getInstances()` + `delete()` pattern
### Key Files
- `ts/cache/classes.cachedb.ts` - CacheDb singleton wrapper
- `ts/cache/classes.cached.document.ts` - Base class with TTL support
- `ts/cache/classes.cache.cleaner.ts` - Periodic cleanup service
- `ts/cache/documents/*.ts` - Document class definitions