feat(security): migrate content scanning and bounce detection to Rust security bridge; add scanContent IPC command and Rust content scanner with tests; update TS RustSecurityBridge and callers, and adjust CI package references
This commit is contained in:
212
readme.plan.md
212
readme.plan.md
@@ -1,198 +1,24 @@
|
||||
# Mailer Implementation Plan & Progress
|
||||
# Rust Migration Plan
|
||||
|
||||
## Project Goals
|
||||
## Completed Phases
|
||||
|
||||
Build a Deno-based mail server package (`@serve.zone/mailer`) with:
|
||||
1. CLI interface similar to nupst/spark
|
||||
2. SMTP server and client (ported from dcrouter)
|
||||
3. HTTP REST API (Mailgun-compatible)
|
||||
4. Automatic DNS management via Cloudflare
|
||||
5. Systemd daemon service
|
||||
6. Binary distribution via npm
|
||||
### Phase 3: Rust Primary Backend (DKIM/SPF/DMARC/IP Reputation)
|
||||
- Rust is the mandatory security backend — no TS fallbacks
|
||||
- All DKIM signing/verification, SPF, DMARC, IP reputation through Rust bridge
|
||||
|
||||
## Completed Work
|
||||
### Phase 5: BounceManager + ContentScanner
|
||||
- BounceManager bounce detection delegated to Rust `detectBounce` IPC command
|
||||
- ContentScanner pattern matching delegated to new Rust `scanContent` IPC command
|
||||
- New module: `rust/crates/mailer-security/src/content_scanner.rs` (10 Rust tests)
|
||||
- ~215 lines removed from BounceManager, ~350 lines removed from ContentScanner
|
||||
- Binary attachment scanning (PE headers, VBA macros) stays in TS
|
||||
- Custom rules (runtime-configured) stay in TS
|
||||
- Net change: ~-560 TS lines, +265 Rust lines
|
||||
|
||||
### ✅ Phase 1: Project Structure
|
||||
- [x] Created Deno-based project structure (deno.json, package.json)
|
||||
- [x] Set up bin/ wrappers for npm binary distribution
|
||||
- [x] Created compilation scripts (compile-all.sh)
|
||||
- [x] Set up install scripts (install-binary.js)
|
||||
- [x] Created TypeScript source directory structure
|
||||
## Deferred
|
||||
|
||||
### ✅ Phase 2: Mail Implementation (Ported from dcrouter)
|
||||
- [x] Copied and adapted mail/core/ (Email, EmailValidator, BounceManager, TemplateManager)
|
||||
- [x] Copied and adapted mail/delivery/ (SMTP client, SMTP server, queues, rate limiting)
|
||||
- [x] Copied and adapted mail/routing/ (EmailRouter, DomainRegistry, DnsManager)
|
||||
- [x] Copied and adapted mail/security/ (DKIM, SPF, DMARC)
|
||||
- [x] Fixed all imports from .js to .ts extensions
|
||||
- [x] Created stub modules for dcrouter dependencies (storage, security, deliverability, errors)
|
||||
|
||||
### ✅ Phase 3: Supporting Modules
|
||||
- [x] Created logger module (simple console logging)
|
||||
- [x] Created paths module (project paths)
|
||||
- [x] Created plugins.ts (Deno dependencies + Node.js compatibility)
|
||||
- [x] Added required npm dependencies (lru-cache, mailaddress-validator, cloudflare)
|
||||
|
||||
### ✅ Phase 4: DNS Management
|
||||
- [x] Created DnsManager class with DNS record generation
|
||||
- [x] Created CloudflareClient for automatic DNS setup
|
||||
- [x] Added DNS validation functionality
|
||||
|
||||
### ✅ Phase 5: HTTP API
|
||||
- [x] Created ApiServer class with basic routing
|
||||
- [x] Implemented Mailgun-compatible endpoint structure
|
||||
- [x] Added authentication and rate limiting stubs
|
||||
|
||||
### ✅ Phase 6: Configuration Management
|
||||
- [x] Created ConfigManager for JSON-based config storage
|
||||
- [x] Added domain configuration support
|
||||
- [x] Implemented config load/save functionality
|
||||
|
||||
### ✅ Phase 7: Daemon Service
|
||||
- [x] Created DaemonManager to coordinate SMTP server and API server
|
||||
- [x] Added start/stop functionality
|
||||
- [x] Integrated with ConfigManager
|
||||
|
||||
### ✅ Phase 8: CLI Interface
|
||||
- [x] Created MailerCli class with command routing
|
||||
- [x] Implemented service commands (start/stop/restart/status/enable/disable)
|
||||
- [x] Implemented domain commands (add/remove/list)
|
||||
- [x] Implemented DNS commands (setup/validate/show)
|
||||
- [x] Implemented send command
|
||||
- [x] Implemented config commands (show/set)
|
||||
- [x] Added help and version commands
|
||||
|
||||
### ✅ Phase 9: Documentation
|
||||
- [x] Created comprehensive README.md
|
||||
- [x] Documented all CLI commands
|
||||
- [x] Documented HTTP API endpoints
|
||||
- [x] Provided configuration examples
|
||||
- [x] Documented DNS requirements
|
||||
- [x] Created changelog
|
||||
|
||||
## Next Steps (Remaining Work)
|
||||
|
||||
### Testing & Debugging
|
||||
1. Fix remaining import/dependency issues
|
||||
2. Test compilation with `deno compile`
|
||||
3. Test CLI commands end-to-end
|
||||
4. Test SMTP sending/receiving
|
||||
5. Test HTTP API endpoints
|
||||
6. Write unit tests
|
||||
|
||||
### Systemd Integration
|
||||
1. Create systemd service file
|
||||
2. Implement service enable/disable
|
||||
3. Add service status checking
|
||||
4. Test daemon auto-restart
|
||||
|
||||
### Cloudflare Integration
|
||||
1. Test actual Cloudflare API calls
|
||||
2. Handle Cloudflare errors gracefully
|
||||
3. Add zone detection
|
||||
4. Verify DNS record creation
|
||||
|
||||
### Production Readiness
|
||||
1. Add proper error handling throughout
|
||||
2. Implement logging to files
|
||||
3. Add rate limiting implementation
|
||||
4. Implement API key authentication
|
||||
5. Add TLS certificate management
|
||||
6. Implement email queue persistence
|
||||
|
||||
### Advanced Features
|
||||
1. Webhook support for incoming emails
|
||||
2. Email template system
|
||||
3. Analytics and reporting
|
||||
4. SMTP credential management
|
||||
5. Email event tracking
|
||||
6. Bounce handling
|
||||
|
||||
## Known Issues
|
||||
|
||||
1. Some npm dependencies may need version adjustments
|
||||
2. Deno crypto APIs may need adaptation for DKIM signing
|
||||
3. Buffer vs Uint8Array conversions may be needed
|
||||
4. Some dcrouter-specific code may need further adaptation
|
||||
|
||||
## File Structure Overview
|
||||
|
||||
```
|
||||
mailer/
|
||||
├── README.md ✅ Complete
|
||||
├── license ✅ Complete
|
||||
├── changelog.md ✅ Complete
|
||||
├── deno.json ✅ Complete
|
||||
├── package.json ✅ Complete
|
||||
├── mod.ts ✅ Complete
|
||||
│
|
||||
├── bin/
|
||||
│ └── mailer-wrapper.js ✅ Complete
|
||||
│
|
||||
├── scripts/
|
||||
│ ├── compile-all.sh ✅ Complete
|
||||
│ └── install-binary.js ✅ Complete
|
||||
│
|
||||
└── ts/
|
||||
├── 00_commitinfo_data.ts ✅ Complete
|
||||
├── index.ts ✅ Complete
|
||||
├── cli.ts ✅ Complete
|
||||
├── plugins.ts ✅ Complete
|
||||
├── logger.ts ✅ Complete
|
||||
├── paths.ts ✅ Complete
|
||||
├── classes.mailer.ts ✅ Complete
|
||||
│
|
||||
├── cli/
|
||||
│ ├── index.ts ✅ Complete
|
||||
│ └── mailer-cli.ts ✅ Complete
|
||||
│
|
||||
├── api/
|
||||
│ ├── index.ts ✅ Complete
|
||||
│ ├── api-server.ts ✅ Complete
|
||||
│ └── routes/ ✅ Structure ready
|
||||
│
|
||||
├── dns/
|
||||
│ ├── index.ts ✅ Complete
|
||||
│ ├── dns-manager.ts ✅ Complete
|
||||
│ └── cloudflare-client.ts ✅ Complete
|
||||
│
|
||||
├── daemon/
|
||||
│ ├── index.ts ✅ Complete
|
||||
│ └── daemon-manager.ts ✅ Complete
|
||||
│
|
||||
├── config/
|
||||
│ ├── index.ts ✅ Complete
|
||||
│ └── config-manager.ts ✅ Complete
|
||||
│
|
||||
├── storage/
|
||||
│ └── index.ts ✅ Stub complete
|
||||
│
|
||||
├── security/
|
||||
│ └── index.ts ✅ Stub complete
|
||||
│
|
||||
├── deliverability/
|
||||
│ └── index.ts ✅ Stub complete
|
||||
│
|
||||
├── errors/
|
||||
│ └── index.ts ✅ Stub complete
|
||||
│
|
||||
└── mail/ ✅ Ported from dcrouter
|
||||
├── core/ ✅ Complete
|
||||
├── delivery/ ✅ Complete
|
||||
├── routing/ ✅ Complete
|
||||
└── security/ ✅ Complete
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
The mailer package structure is **95% complete**. All major components have been implemented:
|
||||
- Project structure and build system ✅
|
||||
- Mail implementation ported from dcrouter ✅
|
||||
- CLI interface ✅
|
||||
- DNS management ✅
|
||||
- HTTP API ✅
|
||||
- Configuration system ✅
|
||||
- Daemon management ✅
|
||||
- Documentation ✅
|
||||
|
||||
**Remaining work**: Testing, debugging dependency issues, systemd integration, and production hardening.
|
||||
| Component | Rationale |
|
||||
|-----------|-----------|
|
||||
| EmailValidator | Already thin; uses smartmail; minimal gain |
|
||||
| DNS record generation | Pure string building; zero benefit from Rust |
|
||||
| MIME building (`toRFC822String`) | Sync in TS, async via IPC; too much blast radius |
|
||||
|
||||
Reference in New Issue
Block a user