platformservice/readme.plan.md

DcRouter Implementation Plan

This document outlines the plan for developing DcRouter, a central routing component for the platform service. DcRouter will provide:

• HTTP/HTTPS reverse proxy with TLS (ACME) support
• TCP/SNI-based proxying
• Mail Transfer Agent (MTA) for inbound/outbound SMTP
• Rule-based routing for web and mail traffic
• DNS-based dynamic routing
• Configuration, persistence, logging, and monitoring

1. Read and Summarize Dependencies

We will leverage existing modules; key points from their READMEs:

• @push.rocks/smartproxy: high-performance HTTP/HTTPS reverse proxy, TLS termination, ACME HTTP-01, WebSockets, low-level port forwarding (nftables), TCP/SNI proxy with dynamic routing and IP filtering.
• @push.rocks/smartmail: unified mail data structures and utilities.
• mailparser: parse raw SMTP messages into structured objects.
• mailauth: SPF/DKIM/DMARC validation and signing (DKIM).
• @api.global/typedserver / typedsocket: typed HTTP and TCP server abstractions.
• @push.rocks/smartrule: rule engine for dynamic decision-making on traffic.
• @push.rocks/smartdns: DNS resolution and caching for routing decisions.
• @push.rocks/smartdata: persistent key/value or document store for configs.
• @push.rocks/smartlog and @push.rocks/smartrx: logging and metrics/event tracking.
• @push.rocks/qenv: environment-based configuration loading.
• ts/mta: built-in MTA implementation (MtaService, SMTPServer, EmailSendJob, DKIMCreator/Verifier, DNSManager, ApiManager, Email class).

2. High-Level Architecture

DcRouter will instantiate and coordinate:

1. HTTP(S) Proxy Layer
   • Use NetworkProxy and Port80Handler from smartproxy
   • Manage virtual hosts, certificates, redirects
2. TCP/SNI Proxy Layer (SmartProxy)
   • Use SmartProxy from @push.rocks/smartproxy to:
     • Forward TCP connections by port or port ranges to backends (nftables-like port forwarding).
     • Route TLS connections based on SNI to different target IPs/ports (SNI bridge).
     • Integrate HTTP reverse proxy on select ports via NetworkProxyBridge.
     • Manage ACME certificates with built-in Port80Handler and CertProvisioner.
     • Apply IP filtering, connection rate limiting, timeouts, and detailed logging.
3. SMTP/MTA Layer
   • Leverage MtaService from ts/mta, coordinating subcomponents:
     • SMTPServer (classes.smtpserver): SMTP listener on ports 25/587 with STARTTLS/TLS
     • Email and EmailSendJob: queue-based outbound sending with retries and rate limiting
     • DKIMCreator / DKIMVerifier: automatic key management, signing, and verification
     • DNSManager: generate and update DNS records for DKIM/SPF/MTA
     • ApiManager: certificate provisioning and renewal via typedrequest/typedsocket
   • Integrate inbound processing (mailparser, SPF/DKIM validation) and routing rules
4. Rule Engine
   • Use smartrule to define routing rules for HTTP domains and mail addresses
   • Support wildcard matching, priority, and dynamic updates
5. DNS Integration
   • Use smartdns to resolve backend targets by name, with caching
6. Persistence & Config
   • Load environment config via qenv
   • Store dynamic state (rules, domains, MTA settings) in smartdata
7. Logging & Metrics
   • Instrument all layers with smartlog and smartrx
8. Management API / CLI
   • Expose REST/CLI commands via typedserver for: add/remove domains, rules, view status, reload config, etc.

3. Feature Breakdown and Milestones

• A. Core Router scaffolding (DcRouter class)
• B. HTTP/HTTPS proxy integration
• C. SmartProxy Integration
  • C1. Define IPortProxySettings with fromPort, toPort, domainConfigs, and global port ranges
  • C2. Configure domainConfigs to mix port forwarding, SNI routing, and useNetworkProxy flags
  • C3. Initialize and start SmartProxy, including Port80Handler for ACME and CertProvisioner
  • C4. Wire SmartProxy events (e.g., certificate issuance, connection logging) into DcRouter
  • C5. Support runtime updates to domainConfigs for dynamic routing changes
• D. MTA Integration (ts/mta)
• D1. Scaffold integration of MtaService into DcRouter
• D2. Configure and start SMTPServer with TLS/STARTTLS on ports 25/587
• D3. Wire inbound email handling via processIncomingEmail and rule engine
• D4. Implement outbound queue processing (EmailSendJob), retries, rate limiting
• D5. Manage DKIM keys and DNS records (DKIMCreator, DNSManager)
• D6. Provision and auto-renew certificates (ApiManager)
• E. Rule engine wiring for HTTP and SMTP
• F. DNS-based dynamic resolution
• G. Persistence layer for configs and state
• H. Logging/metrics instrumentation
• I. CLI and REST API endpoints
• J. Automated tests for each layer
• K. Documentation and examples in README

4. Next Steps

1. Review and refine this plan.
2. Begin implementation starting with the core scaffolding (A).
3. Iterate through features in milestone order.

Last updated: 2025-05-04