diff --git a/changelog.md b/changelog.md index 7be976c..4cbb32c 100644 --- a/changelog.md +++ b/changelog.md @@ -1,5 +1,16 @@ # Changelog +## 2026-02-10 - 2.0.1 - fix(docs/readme) +update README: clarify APIs, document RustSecurityBridge, update examples and architecture diagram + +- Documented RustSecurityBridge: startup/shutdown, automatic delegation, compound verifyEmail API, and individual operations +- Clarified verification APIs: SpfVerifier.verify() and DmarcVerifier.verify() examples now take an Email object as the first argument +- Updated example method names/usages: scanEmail, createEmail, evaluateRoutes, checkMessageLimit, isEmailSuppressed, DKIMCreator rotation and output formatting +- Reformatted architecture diagram and added Rust Security Bridge and expanded Rust Acceleration details +- Rate limiter example updated: renamed/standardized config keys (maxMessagesPerMinute, domains) and added additional limits (maxRecipientsPerMessage, maxConnectionsPerIP, etc.) +- DNS management documentation reorganized: UnifiedEmailServer now handles DNS record setup automatically; DNSManager usage clarified for standalone checks +- Minor wording/formatting tweaks throughout README (arrow styles, headings, test counts) + ## 2026-02-10 - 2.0.0 - BREAKING CHANGE(smartmta) Rebrand package to @push.rocks/smartmta, add consolidated email security verification and IPC handler diff --git a/readme.md b/readme.md index e58858c..502f62c 100644 --- a/readme.md +++ b/readme.md @@ -1,6 +1,6 @@ # @push.rocks/smartmta -A high-performance, enterprise-grade Mail Transfer Agent (MTA) built from scratch in TypeScript with Rust acceleration β no nodemailer, no shortcuts. π +A high-performance, enterprise-grade Mail Transfer Agent (MTA) built from scratch in TypeScript with Rust acceleration β no nodemailer, no shortcuts. ## Issue Reporting and Security @@ -18,7 +18,7 @@ npm install @push.rocks/smartmta `@push.rocks/smartmta` is a **complete mail server solution** β SMTP server, SMTP client, email security, content scanning, and delivery management β all built with a custom SMTP implementation. No wrappers around nodemailer. No half-measures. -### β¨ What's Inside +### What's Inside | Module | What It Does | |---|---| @@ -31,38 +31,42 @@ npm install @push.rocks/smartmta | **Bounce Manager** | Automatic bounce detection, classification (hard/soft), and tracking | | **Content Scanner** | Spam, phishing, malware, XSS, and suspicious link detection | | **IP Reputation** | DNSBL checks, proxy/TOR/VPN detection, risk scoring | -| **Rate Limiter** | Hierarchical rate limiting (global, per-domain, per-sender) | +| **Rate Limiter** | Hierarchical rate limiting (global, per-domain, per-IP) | | **Delivery Queue** | Persistent queue with exponential backoff retry | | **Template Engine** | Email templates with variable substitution | | **Domain Registry** | Multi-domain management with per-domain configuration | | **DNS Manager** | Automatic DNS record management with Cloudflare API integration | | **Rust Accelerator** | Performance-critical operations (DKIM, MIME, validation) in Rust via IPC | +| **Rust Security Bridge** | Compound email security verification (DKIM+SPF+DMARC) via Rust binary | -### ποΈ Architecture +### Architecture ``` -βββββββββββββββββββββββββββββββββββββββββββββββββββββββ -β UnifiedEmailServer β -β (orchestrates all components, emits events) β -ββββββββββββ¬βββββββββββ¬ββββββββββββ¬ββββββββββββββββββββ€ -β SMTP β Email β Security β Delivery β -β Server β Router β Stack β System β -β βββββββ β βββββββ β ββββββββ β βββββββββββββββ β -β β TLS β β βMatchβ β β DKIM β β β Queue β β -β β Authβ β βRouteβ β β SPF β β β Rate Limit β β -β β Cmd β β β Act β β βDMARC β β β SMTP Client β β -β β Dataβ β β β β βIPRep β β β Retry Logic β β -β βββββββ β βββββββ β βScan β β βββββββββββββββ β -β β β ββββββββ β β -ββββββββββββ΄βββββββββββ΄ββββββββββββ΄ββββββββββββββββββββ€ -β Rust Acceleration Layer β -β (mailer-core, mailer-security via smartrust IPC) β -βββββββββββββββββββββββββββββββββββββββββββββββββββββββ +βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ +β UnifiedEmailServer β +β (orchestrates all components, emits events) β +ββββββββββββ¬βββββββββββ¬βββββββββββββ¬βββββββββββββββββββββββ€ +β SMTP β Email β Security β Delivery β +β Server β Router β Stack β System β +β βββββββ β βββββββ β βββββββββ β ββββββββββββββββββ β +β β TLS β β βMatchβ β β DKIM β β β Queue β β +β β Authβ β βRouteβ β β SPF β β β Rate Limit β β +β β Cmd β β β Act β β β DMARC β β β SMTP Client β β +β β Dataβ β β β β β IPRep β β β Retry Logic β β +β βββββββ β βββββββ β β Scan β β ββββββββββββββββββ β +β β β βββββββββ β β +ββββββββββββ΄βββββββββββ΄βββββββββββββ΄βββββββββββββββββββββββ€ +β Rust Security Bridge β +β (RustSecurityBridge singleton via smartrust IPC) β +βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€ +β Rust Acceleration Layer β +β (mailer-core, mailer-security, mailer-bin) β +βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ ``` ## Usage -### π§ Setting Up the Email Server +### Setting Up the Email Server The central entry point is `UnifiedEmailServer`, which orchestrates SMTP, routing, security, and delivery: @@ -134,7 +138,7 @@ const emailServer = new UnifiedEmailServer(dcRouterRef, { await emailServer.start(); ``` -### π§ Sending Emails with the SMTP Client +### Sending Emails with the SMTP Client Create and send emails using the built-in SMTP client with connection pooling: @@ -177,9 +181,9 @@ const result = await client.sendMail(email); console.log(`Message sent: ${result.messageId}`); ``` -### π DKIM Signing +### DKIM Signing -Automatic DKIM key generation, storage, and signing per domain: +DKIM key management is handled by `DKIMCreator`, which generates, stores, and rotates keys per domain. Signing is performed automatically by `UnifiedEmailServer` during outbound delivery β there is no standalone `signEmail()` call: ```typescript import { DKIMCreator } from '@push.rocks/smartmta'; @@ -192,36 +196,45 @@ await dkimCreator.handleDKIMKeysForDomain('example.com'); // Get the DNS record you need to publish const dnsRecord = await dkimCreator.getDNSRecordForDomain('example.com'); console.log(dnsRecord); -// β { type: 'TXT', name: 'default._domainkey.example.com', value: 'v=DKIM1; k=rsa; p=...' } +// -> { type: 'TXT', name: 'default._domainkey.example.com', value: 'v=DKIM1; k=rsa; p=...' } -// Sign an email -const signedEmail = await dkimCreator.signEmail(email); +// Check if keys need rotation +const needsRotation = await dkimCreator.needsRotation('example.com', 'default', 90); +if (needsRotation) { + const newSelector = await dkimCreator.rotateDkimKeys('example.com', 'default', 2048); + console.log(`Rotated to selector: ${newSelector}`); +} ``` -### π‘οΈ Email Authentication (SPF, DKIM, DMARC) +When `UnifiedEmailServer.start()` is called, DKIM signing is applied to all outbound mail automatically using the keys managed by `DKIMCreator`. The `RustSecurityBridge` can also perform DKIM signing via its `signDkim()` method for high-performance scenarios. -Verify incoming emails against all three authentication standards: +### Email Authentication (SPF, DKIM, DMARC) + +Verify incoming emails against all three authentication standards. Note that the first argument to `SpfVerifier.verify()` and `DmarcVerifier.verify()` is an `Email` object: ```typescript import { DKIMVerifier, SpfVerifier, DmarcVerifier } from '@push.rocks/smartmta'; -// SPF verification +// SPF verification β first arg is an Email object const spfVerifier = new SpfVerifier(); -const spfResult = await spfVerifier.verify(senderIP, senderDomain, ehloHostname); -// β { result: 'pass' | 'fail' | 'softfail' | 'neutral' | 'none' | 'temperror' | 'permerror' } +const spfResult = await spfVerifier.verify(email, senderIP, heloDomain); +// -> { result: 'pass' | 'fail' | 'softfail' | 'neutral' | 'none' | 'temperror' | 'permerror', +// domain: string, ip: string } // DKIM verification const dkimVerifier = new DKIMVerifier(); const dkimResult = await dkimVerifier.verify(rawEmailContent); -// DMARC verification +// DMARC verification β first arg is an Email object const dmarcVerifier = new DmarcVerifier(); -const dmarcResult = await dmarcVerifier.verify(fromDomain, spfResult, dkimResult); +const dmarcResult = await dmarcVerifier.verify(email, spfResult, dkimResult); +// -> { action: 'pass' | 'quarantine' | 'reject', hasDmarc: boolean, +// spfDomainAligned: boolean, dkimDomainAligned: boolean, ... } ``` -### π Email Routing +### Email Routing -Pattern-based routing engine with priority ordering and flexible match criteria: +Pattern-based routing engine with priority ordering and flexible match criteria. Routes are evaluated by priority (highest first) using `evaluateRoutes()`: ```typescript import { EmailRouter } from '@push.rocks/smartmta'; @@ -271,13 +284,13 @@ const router = new EmailRouter([ }, ]); -// Routes are evaluated by priority (highest first) -const matchedRoute = router.route(email, context); +// Evaluate routes against an email context +const matchedRoute = await router.evaluateRoutes(emailContext); ``` -### π΅οΈ Content Scanning +### Content Scanning -Built-in content scanner for detecting spam, phishing, malware, and other threats: +Built-in content scanner for detecting spam, phishing, malware, and other threats. Use the `scanEmail()` method: ```typescript import { ContentScanner } from '@push.rocks/smartmta'; @@ -300,11 +313,11 @@ const scanner = new ContentScanner({ ], }); -const result = await scanner.scan(email); -// β { isClean: false, threatScore: 85, threatType: 'phishing', scannedElements: [...] } +const result = await scanner.scanEmail(email); +// -> { isClean: false, threatScore: 85, threatType: 'phishing', scannedElements: [...] } ``` -### π IP Reputation Checking +### IP Reputation Checking Check sender IP addresses against DNSBL blacklists and classify IP types: @@ -318,37 +331,52 @@ const ipChecker = new IPReputationChecker({ }); const reputation = await ipChecker.checkReputation('192.168.1.1'); -// β { score: 85, isSpam: false, isProxy: false, isTor: false, blacklists: [] } +// -> { score: 85, isSpam: false, isProxy: false, isTor: false, blacklists: [] } ``` -### β±οΈ Rate Limiting +When the `RustSecurityBridge` is running, `IPReputationChecker` automatically delegates DNSBL lookups to the Rust binary for improved performance. -Hierarchical rate limiting to protect your server and maintain deliverability: +### Rate Limiting + +Hierarchical rate limiting to protect your server and maintain deliverability. Configuration uses `maxMessagesPerMinute` and organizes domain-level limits under the `domains` key: ```typescript import { UnifiedRateLimiter } from '@push.rocks/smartmta'; const rateLimiter = new UnifiedRateLimiter({ global: { - maxPerMinute: 1000, - maxPerHour: 10000, + maxMessagesPerMinute: 1000, + maxRecipientsPerMessage: 500, + maxConnectionsPerIP: 20, + maxErrorsPerIP: 10, + maxAuthFailuresPerIP: 5, + blockDuration: 600000, // 10 minutes }, - perDomain: { + domains: { 'example.com': { - maxPerMinute: 100, - maxPerHour: 1000, + maxMessagesPerMinute: 100, + maxRecipientsPerMessage: 50, }, }, - perSender: { - maxPerMinute: 20, - maxPerHour: 200, - }, }); + +// Check before sending +const allowed = rateLimiter.checkMessageLimit( + 'sender@example.com', + '192.168.1.1', + recipientCount, + undefined, + 'example.com' +); + +if (!allowed.allowed) { + console.log(`Rate limited: ${allowed.reason}`); +} ``` -### π¬ Bounce Management +### Bounce Management -Automatic bounce detection, classification, and tracking: +Automatic bounce detection, classification, and suppression tracking. Use `isEmailSuppressed()` to check if an address should be suppressed: ```typescript import { BounceManager } from '@push.rocks/smartmta'; @@ -361,22 +389,26 @@ const bounce = await bounceManager.processSmtpFailure( '550 5.1.1 User unknown', { originalEmailId: 'msg-123' } ); -// β { bounceType: 'invalid_recipient', bounceCategory: 'hard', ... } +// -> { bounceType: 'invalid_recipient', bounceCategory: 'hard', ... } -// Check if an address is known to bounce -const shouldSuppress = bounceManager.shouldSuppressDelivery('recipient@example.com'); +// Check if an address is suppressed due to bounces +const suppressed = bounceManager.isEmailSuppressed('recipient@example.com'); + +// Manually manage the suppression list +bounceManager.addToSuppressionList('bad@example.com', 'repeated hard bounces'); +bounceManager.removeFromSuppressionList('recovered@example.com'); ``` -### π Email Templates +### Email Templates -Template engine with variable substitution for transactional and notification emails: +Template engine with variable substitution for transactional and notification emails. Use `createEmail()` to produce a ready-to-send `Email` from a registered template: ```typescript import { TemplateManager } from '@push.rocks/smartmta'; const templates = new TemplateManager({ from: 'noreply@example.com', - footerHtml: '
Β© 2026 Example Corp
', + footerHtml: '2026 Example Corp
', }); // Register a template @@ -391,52 +423,107 @@ templates.registerTemplate({ category: 'transactional', }); -// Render and send -const email = templates.renderTemplate('welcome', { +// Create an Email object from the template +const email = await templates.createEmail('welcome', { to: 'newuser@example.com', variables: { name: 'Alice' }, }); ``` -### π DNS Management with Cloudflare +### DNS Management -Automatic DNS record setup for MX, SPF, DKIM, and DMARC via the Cloudflare API: +DNS record management for email authentication is handled internally by `UnifiedEmailServer`. The `DnsManager` is not instantiated directly β it receives its configuration from the `dcRouter` reference and automatically ensures MX, SPF, DKIM, and DMARC records are in place for all configured domains: ```typescript -import { DnsManager } from '@push.rocks/smartmta'; +// DNS management is automatic when using UnifiedEmailServer. +// When the server starts, it calls ensureDnsRecords() internally +// for all configured domains, setting up: +// - MX records pointing to your mail server +// - SPF TXT records authorizing your server IP +// - DKIM TXT records with public keys from DKIMCreator +// - DMARC TXT records with your policy -const dnsManager = new DnsManager({ +const emailServer = new UnifiedEmailServer(dcRouterRef, { + hostname: 'mail.example.com', domains: [ { domain: 'example.com', dnsMode: 'external-dns', // managed via Cloudflare API }, ], + // ... other config }); -// Auto-configure all required DNS records -await dnsManager.setupDnsForDomain('example.com', { - serverIp: '203.0.113.10', - mxHostname: 'mail.example.com', -}); +// DNS records are set up automatically on start +await emailServer.start(); ``` -## π¦ Rust Acceleration +For DNS lookups and record verification outside of the server lifecycle, the `DNSManager` class (note the capital N) can be used directly: -Performance-critical operations are implemented in Rust and communicate with the TypeScript runtime via `@push.rocks/smartrust` (JSON-over-stdin/stdout IPC): +```typescript +import { DNSManager, DKIMCreator } from '@push.rocks/smartmta'; -- **mailer-core**: Email type validation, MIME building, bounce detection -- **mailer-security**: DKIM signing/verification, SPF checks, DMARC policy, IP reputation/DNSBL +const dkimCreator = new DKIMCreator('/path/to/keys'); +const dnsManager = new DNSManager(dkimCreator); + +// Verify all email authentication records for a domain +const results = await dnsManager.verifyEmailAuthRecords('example.com', 'default'); +console.log(results.spf); // { valid: boolean, record: string, ... } +console.log(results.dkim); // { valid: boolean, record: string, ... } +console.log(results.dmarc); // { valid: boolean, record: string, ... } + +// Generate recommended DNS records +const records = await dnsManager.generateAllRecommendedRecords('example.com'); +``` + +## Rust Acceleration + +Performance-critical operations are implemented in Rust and communicate with the TypeScript runtime via `@push.rocks/smartrust` (JSON-over-stdin/stdout IPC). + +### Rust Crates The Rust workspace is at `rust/` with five crates: | Crate | Status | Purpose | |---|---|---| -| `mailer-core` | β Complete | Email types, validation, MIME, bounce detection | -| `mailer-security` | β Complete | DKIM, SPF, DMARC, IP reputation | -| `mailer-bin` | β Complete | CLI + smartrust IPC bridge | -| `mailer-smtp` | π Phase 2 | SMTP protocol in Rust | -| `mailer-napi` | π Phase 2 | Native Node.js addon | +| `mailer-core` | Complete (26 tests) | Email types, validation, MIME building, bounce detection | +| `mailer-security` | Complete (12 tests) | DKIM signing/verification, SPF checks, DMARC policy, IP reputation/DNSBL | +| `mailer-bin` | Complete | CLI + smartrust IPC bridge (handles `verifyEmail` compound method) | +| `mailer-smtp` | Planned (Phase 3) | SMTP protocol in Rust | +| `mailer-napi` | Planned (Phase 3) | Native Node.js addon | + +### RustSecurityBridge + +The `RustSecurityBridge` is a singleton that manages the Rust binary process and provides high-performance security verification. It is automatically started and stopped with `UnifiedEmailServer`: + +```typescript +import { RustSecurityBridge } from '@push.rocks/smartmta'; + +const bridge = RustSecurityBridge.getInstance(); +await bridge.start(); + +// Compound verification: DKIM + SPF + DMARC in a single IPC call +const securityResult = await bridge.verifyEmail({ + rawMessage: rawEmailString, + ip: '203.0.113.10', + heloDomain: 'sender.example.com', + mailFrom: 'user@example.com', +}); +// -> { dkim: [...], spf: { result, explanation }, dmarc: { result, policy } } + +// Individual operations +const dkimResults = await bridge.verifyDkim(rawEmailString); +const spfResult = await bridge.checkSpf({ + ip: '203.0.113.10', + heloDomain: 'sender.example.com', + mailFrom: 'user@example.com', +}); +const reputationResult = await bridge.checkIpReputation('203.0.113.10'); + +await bridge.stop(); +``` + +When the bridge is running, the TypeScript security components (`SpfVerifier`, `DKIMVerifier`, `IPReputationChecker`) automatically delegate to the Rust binary. If the binary is unavailable, the system falls back gracefully to TypeScript-only verification. ## Project Structure @@ -450,10 +537,10 @@ smartmta/ β β β βββ smtpserver/ # SMTP server with TLS, auth, pipelining β β βββ routing/ # UnifiedEmailServer, EmailRouter, DomainRegistry, DnsManager β β βββ security/ # DKIMCreator, DKIMVerifier, SpfVerifier, DmarcVerifier -β βββ security/ # ContentScanner, IPReputationChecker, SecurityLogger +β βββ security/ # ContentScanner, IPReputationChecker, RustSecurityBridge βββ rust/ # Rust workspace β βββ crates/ # mailer-core, mailer-security, mailer-bin, mailer-smtp, mailer-napi -βββ test/ # Comprehensive test suite (RFC compliance, security, performance, edge cases) +βββ test/ # Comprehensive test suite βββ dist_ts/ # Compiled output ``` diff --git a/ts/00_commitinfo_data.ts b/ts/00_commitinfo_data.ts index 3cb90df..8e3a27c 100644 --- a/ts/00_commitinfo_data.ts +++ b/ts/00_commitinfo_data.ts @@ -3,6 +3,6 @@ */ export const commitinfo = { name: '@push.rocks/smartmta', - version: '2.0.0', + version: '2.0.1', description: 'A high-performance, enterprise-grade Mail Transfer Agent (MTA) built from scratch in TypeScript with Rust acceleration.' }