Files

Juergen Kunz 248bfcfe78 feat(docs): document Rust-side in-process security pipeline and update README to reflect SMTP server behavior and crate/test counts

2026-02-10 22:43:50 +00:00

24 KiB

Raw Blame History

@push.rocks/smartmta

A high-performance, enterprise-grade Mail Transfer Agent (MTA) built from scratch in TypeScript with a Rust-powered SMTP engine — no nodemailer, no shortcuts. 🚀

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install

pnpm install @push.rocks/smartmta
# or
npm install @push.rocks/smartmta

After installation, run pnpm build to compile the Rust binary (mailer-bin). The Rust binary is required — smartmta will not start without it.

Overview

@push.rocks/smartmta is a complete mail server solution — SMTP server, SMTP client, email security, content scanning, and delivery management — all built with a custom SMTP implementation. The SMTP server itself runs as a Rust binary for maximum performance, communicating with the TypeScript orchestration layer via IPC.

⚡ What's Inside

Module	What It Does
Rust SMTP Server	High-performance SMTP engine written in Rust — TCP/TLS listener, STARTTLS, AUTH, pipelining, per-connection rate limiting
SMTP Client	Outbound delivery with connection pooling, retry logic, TLS negotiation
DKIM	Key generation, signing, and verification — per domain, with automatic rotation
SPF	Full SPF record validation via Rust
DMARC	Policy enforcement and verification
Email Router	Pattern-based routing with priority, forward/deliver/reject/process actions
Bounce Manager	Automatic bounce detection via Rust, classification (hard/soft), and suppression tracking
Content Scanner	Spam, phishing, malware, XSS, and suspicious link detection — powered by Rust
IP Reputation	DNSBL checks, proxy/TOR/VPN detection, risk scoring via Rust
Rate Limiter	Hierarchical rate limiting (global, per-domain, per-IP)
Delivery Queue	Persistent queue with exponential backoff retry
Template Engine	Email templates with variable substitution
Domain Registry	Multi-domain management with per-domain configuration
DNS Manager	Automatic DNS record management with Cloudflare API integration
Rust Security Bridge	All security ops (DKIM+SPF+DMARC+DNSBL+content scanning) run in Rust via IPC

🏗️ Architecture

┌──────────────────────────────────────────────────────────────┐
│                    UnifiedEmailServer                         │
│     (orchestrates all components, emits events)               │
├───────────┬───────────┬──────────────┬───────────────────────┤
│   Email   │  Security │  Delivery    │  Configuration        │
│   Router  │  Stack    │  System      │                       │
│  ┌──────┐ │ ┌───────┐ │ ┌──────────┐ │  ┌────────────────┐  │
│  │Match │ │ │ DKIM  │ │ │ Queue    │ │  │ DomainRegistry │  │
│  │Route │ │ │ SPF   │ │ │ Rate Lim │ │  │ DnsManager     │  │
│  │ Act  │ │ │ DMARC │ │ │ SMTP Cli │ │  │ DKIMCreator    │  │
│  └──────┘ │ │ IPRep │ │ │ Retry    │ │  │ Templates      │  │
│           │ │ Scan  │ │ └──────────┘ │  └────────────────┘  │
│           │ └───────┘ │              │                       │
├───────────┴───────────┴──────────────┴───────────────────────┤
│              Rust Security Bridge (smartrust IPC)             │
├──────────────────────────────────────────────────────────────┤
│                   Rust Acceleration Layer                      │
│  ┌──────────────┐  ┌───────────────┐  ┌──────────────────┐  │
│  │ mailer-smtp  │  │mailer-security│  │   mailer-core    │  │
│  │ SMTP Server  │  │DKIM/SPF/DMARC │  │ Types/Validation │  │
│  │ TLS/AUTH     │  │IP Rep/Content │  │ MIME/Bounce      │  │
│  └──────────────┘  └───────────────┘  └──────────────────┘  │
└──────────────────────────────────────────────────────────────┘

Data flow for inbound mail:

Rust SMTP server accepts the connection and handles the full SMTP protocol
On DATA completion, Rust runs the security pipeline in-process (DKIM/SPF/DMARC verification, content scanning, IP reputation check) — zero IPC round-trips
Rust emits an emailReceived event via IPC with pre-computed security results attached
TypeScript processes the email (routing decisions using the pre-computed results, delivery)
Rust sends the final SMTP response to the client

Usage

🚀 Setting Up the Email Server

The central entry point is UnifiedEmailServer, which orchestrates the Rust SMTP server, routing, security, and delivery:

import { UnifiedEmailServer } from '@push.rocks/smartmta';

const emailServer = new UnifiedEmailServer(dcRouterRef, {
  // Ports to listen on (465 = implicit TLS, 25/587 = STARTTLS)
  ports: [25, 587, 465],
  hostname: 'mail.example.com',

  // Multi-domain configuration
  domains: [
    {
      domain: 'example.com',
      dnsMode: 'external-dns',
      dkim: {
        selector: 'default',
        keySize: 2048,
        rotateKeys: true,
        rotationInterval: 90,
      },
      rateLimits: {
        outbound: { messagesPerMinute: 100 },
        inbound: { messagesPerMinute: 200, connectionsPerIp: 20 },
      },
    },
  ],

  // Routing rules (evaluated by priority, highest first)
  routes: [
    {
      name: 'catch-all-forward',
      priority: 10,
      match: {
        recipients: '*@example.com',
      },
      action: {
        type: 'forward',
        forward: {
          host: 'internal-mail.example.com',
          port: 25,
        },
      },
    },
    {
      name: 'reject-spam-senders',
      priority: 100,
      match: {
        senders: '*@spamdomain.com',
      },
      action: {
        type: 'reject',
        reject: {
          code: 550,
          message: 'Sender rejected by policy',
        },
      },
    },
  ],

  // Authentication settings for the SMTP server
  auth: {
    required: false,
    methods: ['PLAIN', 'LOGIN'],
    users: [{ username: 'outbound', password: 'secret' }],
  },

  // TLS certificates
  tls: {
    certPath: '/etc/ssl/mail.crt',
    keyPath: '/etc/ssl/mail.key',
  },

  maxMessageSize: 25 * 1024 * 1024, // 25 MB
  maxClients: 500,
});

// start() boots the Rust SMTP server, security bridge, DNS records, and delivery queue
await emailServer.start();

🔒 Note: start() will throw if the Rust binary is not compiled. Run pnpm build first.

📧 Sending Emails with the SMTP Client

Create and send emails using the built-in SMTP client with connection pooling:

import { Email, Delivery } from '@push.rocks/smartmta';

// Create a client with connection pooling
const client = Delivery.smtpClientMod.createSmtpClient({
  host: 'smtp.example.com',
  port: 587,
  secure: false,     // will upgrade via STARTTLS
  pool: true,
  maxConnections: 5,
  auth: {
    user: 'sender@example.com',
    pass: 'your-password',
  },
});

// Build an email
const email = new Email({
  from: 'sender@example.com',
  to: ['recipient@example.com'],
  cc: ['cc@example.com'],
  subject: 'Hello from smartmta!',
  text: 'Plain text body',
  html: '<h1>Hello!</h1><p>HTML body with <strong>formatting</strong></p>',
  priority: 'high',
  attachments: [
    {
      filename: 'report.pdf',
      content: pdfBuffer,
      contentType: 'application/pdf',
    },
  ],
});

// Send it
const result = await client.sendMail(email);
console.log(`Message sent: ${result.messageId}`);

Additional client factories are available:

// Pooled client for high-throughput scenarios
const pooled = Delivery.smtpClientMod.createPooledSmtpClient({ /* ... */ });

// Optimized for bulk sending
const bulk = Delivery.smtpClientMod.createBulkSmtpClient({ /* ... */ });

// Optimized for transactional emails
const transactional = Delivery.smtpClientMod.createTransactionalSmtpClient({ /* ... */ });

🔑 DKIM Signing

DKIM key management is handled by DKIMCreator, which generates, stores, and rotates keys per domain. Signing is performed automatically by UnifiedEmailServer during outbound delivery:

import { DKIMCreator } from '@push.rocks/smartmta';

const dkimCreator = new DKIMCreator('/path/to/keys');

// Auto-generate keys if they don't exist
await dkimCreator.handleDKIMKeysForDomain('example.com');

// Get the DNS record you need to publish
const dnsRecord = await dkimCreator.getDNSRecordForDomain('example.com');
console.log(dnsRecord);
// -> { type: 'TXT', name: 'default._domainkey.example.com', value: 'v=DKIM1; k=rsa; p=...' }

// Check if keys need rotation
const needsRotation = await dkimCreator.needsRotation('example.com', 'default', 90);
if (needsRotation) {
  const newSelector = await dkimCreator.rotateDkimKeys('example.com', 'default', 2048);
  console.log(`Rotated to selector: ${newSelector}`);
}

When UnifiedEmailServer.start() is called, DKIM signing is applied to all outbound mail automatically using the Rust security bridge's signDkim() method for maximum performance.

🛡️ Email Authentication (SPF, DKIM, DMARC)

Verify incoming emails against all three authentication standards. All verification is powered by the Rust binary:

import { DKIMVerifier, SpfVerifier, DmarcVerifier } from '@push.rocks/smartmta';

// SPF verification — first arg is an Email object
const spfVerifier = new SpfVerifier();
const spfResult = await spfVerifier.verify(email, senderIP, heloDomain);
// -> { result: 'pass' | 'fail' | 'softfail' | 'neutral' | 'none' | 'temperror' | 'permerror',
//      domain: string, ip: string }

// DKIM verification — takes raw email content
const dkimVerifier = new DKIMVerifier();
const dkimResult = await dkimVerifier.verify(rawEmailContent);

// DMARC verification — first arg is an Email object
const dmarcVerifier = new DmarcVerifier();
const dmarcResult = await dmarcVerifier.verify(email, spfResult, dkimResult);
// -> { action: 'pass' | 'quarantine' | 'reject', hasDmarc: boolean,
//      spfDomainAligned: boolean, dkimDomainAligned: boolean, ... }

🔀 Email Routing

Pattern-based routing engine with priority ordering and flexible match criteria. Routes are evaluated by priority (highest first):

import { EmailRouter } from '@push.rocks/smartmta';

const router = new EmailRouter([
  {
    name: 'admin-mail',
    priority: 100,
    match: {
      recipients: 'admin@example.com',
      authenticated: true,
    },
    action: {
      type: 'deliver',
    },
  },
  {
    name: 'external-forward',
    priority: 50,
    match: {
      recipients: '*@example.com',
      sizeRange: { max: 10 * 1024 * 1024 },  // under 10MB
    },
    action: {
      type: 'forward',
      forward: {
        host: 'backend-mail.internal',
        port: 25,
        preserveHeaders: true,
      },
    },
  },
  {
    name: 'process-with-scanning',
    priority: 10,
    match: {
      recipients: '*@*',
    },
    action: {
      type: 'process',
      process: {
        scan: true,
        dkim: true,
        queue: 'normal',
      },
    },
  },
]);

// Evaluate routes against an email context
const matchedRoute = await router.evaluateRoutes(emailContext);

Match criteria available:

Criterion	Description
`recipients`	Glob patterns for recipient addresses (`*@example.com`)
`senders`	Glob patterns for sender addresses
`clientIp`	IP addresses or CIDR ranges
`authenticated`	Require authentication status
`headers`	Match specific headers (string or RegExp)
`sizeRange`	Message size constraints (`{ min?, max? }`)
`subject`	Subject line pattern (string or RegExp)
`hasAttachments`	Filter by attachment presence

🔍 Content Scanning

Built-in content scanner for detecting spam, phishing, malware, and other threats. Text pattern scanning runs in Rust for performance; binary attachment scanning (PE headers, VBA macros) runs in TypeScript:

import { ContentScanner } from '@push.rocks/smartmta';

const scanner = new ContentScanner({
  scanSubject: true,
  scanBody: true,
  scanAttachments: true,
  blockExecutables: true,
  blockMacros: true,
  minThreatScore: 30,
  highThreatScore: 70,
  customRules: [
    {
      pattern: /bitcoin.*wallet/i,
      type: 'scam',
      score: 80,
      description: 'Cryptocurrency scam pattern',
    },
  ],
});

const result = await scanner.scanEmail(email);
// -> { isClean: false, threatScore: 85, threatType: 'phishing', scannedElements: [...] }

🌐 IP Reputation Checking

Check sender IP addresses against DNSBL blacklists and classify IP types. DNSBL lookups run in Rust:

import { IPReputationChecker } from '@push.rocks/smartmta';

const ipChecker = IPReputationChecker.getInstance({
  enableDNSBL: true,
  dnsblServers: ['zen.spamhaus.org', 'bl.spamcop.net'],
  cacheTTL: 24 * 60 * 60 * 1000,  // 24 hours
});

const reputation = await ipChecker.checkReputation('192.168.1.1');
// -> { score: 85, isSpam: false, isProxy: false, isTor: false, blacklists: [] }

⏱️ Rate Limiting

Hierarchical rate limiting to protect your server and maintain deliverability:

import { Delivery } from '@push.rocks/smartmta';
const { UnifiedRateLimiter } = Delivery;

const rateLimiter = new UnifiedRateLimiter({
  global: {
    maxMessagesPerMinute: 1000,
    maxRecipientsPerMessage: 500,
    maxConnectionsPerIP: 20,
    maxErrorsPerIP: 10,
    maxAuthFailuresPerIP: 5,
    blockDuration: 600000, // 10 minutes
  },
  domains: {
    'example.com': {
      maxMessagesPerMinute: 100,
      maxRecipientsPerMessage: 50,
    },
  },
});

// Check before sending
const allowed = rateLimiter.checkMessageLimit(
  'sender@example.com',
  '192.168.1.1',
  recipientCount,
  undefined,
  'example.com'
);

if (!allowed.allowed) {
  console.log(`Rate limited: ${allowed.reason}`);
}

📬 Bounce Management

Automatic bounce detection (via Rust), classification, and suppression tracking:

import { Core } from '@push.rocks/smartmta';
const { BounceManager } = Core;

const bounceManager = new BounceManager();

// Process an SMTP failure
const bounce = await bounceManager.processSmtpFailure(
  'recipient@example.com',
  '550 5.1.1 User unknown',
  { originalEmailId: 'msg-123' }
);
// -> { bounceType: 'invalid_recipient', bounceCategory: 'hard', ... }

// Check if an address is suppressed due to bounces
const suppressed = bounceManager.isEmailSuppressed('recipient@example.com');

// Manually manage the suppression list
bounceManager.addToSuppressionList('bad@example.com', 'repeated hard bounces');
bounceManager.removeFromSuppressionList('recovered@example.com');

📝 Email Templates

Template engine with variable substitution for transactional and notification emails:

import { Core } from '@push.rocks/smartmta';
const { TemplateManager } = Core;

const templates = new TemplateManager({
  from: 'noreply@example.com',
  footerHtml: '<p>&copy; 2026 Example Corp</p>',
});

// Register a template
templates.registerTemplate({
  id: 'welcome',
  name: 'Welcome Email',
  description: 'Sent to new users',
  from: 'welcome@example.com',
  subject: 'Welcome, {{name}}!',
  bodyHtml: '<h1>Welcome, {{name}}!</h1><p>Your account is ready.</p>',
  bodyText: 'Welcome, {{name}}! Your account is ready.',
  category: 'transactional',
});

// Create an Email object from the template
const email = await templates.createEmail('welcome', {
  to: 'newuser@example.com',
  variables: { name: 'Alice' },
});

🌍 DNS Management

DNS record management for email authentication is handled automatically by UnifiedEmailServer. When the server starts, it ensures MX, SPF, DKIM, and DMARC records are in place for all configured domains via the Cloudflare API:

const emailServer = new UnifiedEmailServer(dcRouterRef, {
  hostname: 'mail.example.com',
  domains: [
    {
      domain: 'example.com',
      dnsMode: 'external-dns',  // managed via Cloudflare API
    },
  ],
  // ... other config
});

// DNS records are set up automatically on start:
//   - MX records pointing to your mail server
//   - SPF TXT records authorizing your server IP
//   - DKIM TXT records with public keys from DKIMCreator
//   - DMARC TXT records with your policy
await emailServer.start();

🦀 RustSecurityBridge

The RustSecurityBridge is the singleton that manages the Rust binary process. It handles security verification, content scanning, bounce detection, and the SMTP server lifecycle — all via @push.rocks/smartrust IPC:

import { RustSecurityBridge } from '@push.rocks/smartmta';

const bridge = RustSecurityBridge.getInstance();
await bridge.start();

// Compound verification: DKIM + SPF + DMARC in a single IPC call
const securityResult = await bridge.verifyEmail({
  rawMessage: rawEmailString,
  ip: '203.0.113.10',
  heloDomain: 'sender.example.com',
  mailFrom: 'user@example.com',
});
// -> { dkim: [...], spf: { result, explanation }, dmarc: { result, policy } }

// Individual security operations
const dkimResults = await bridge.verifyDkim(rawEmailString);
const spfResult = await bridge.checkSpf({
  ip: '203.0.113.10',
  heloDomain: 'sender.example.com',
  mailFrom: 'user@example.com',
});
const reputationResult = await bridge.checkIpReputation('203.0.113.10');

// DKIM signing
const signed = await bridge.signDkim({
  email: rawEmailString,
  domain: 'example.com',
  selector: 'default',
  privateKeyPem: privateKey,
});

// Content scanning
const scanResult = await bridge.scanContent({
  subject: 'Win a free iPhone!!!',
  body: '<a href="http://phishing.example.com">Click here</a>',
  from: 'scammer@evil.com',
});

// Bounce detection
const bounceResult = await bridge.detectBounce({
  subject: 'Delivery Status Notification (Failure)',
  body: '550 5.1.1 User unknown',
  from: 'mailer-daemon@example.com',
});

await bridge.stop();

⚠️ Important: The Rust bridge is mandatory. There are no TypeScript fallbacks. If the Rust binary is unavailable, UnifiedEmailServer.start() will throw an error.

🦀 Rust Acceleration Layer

Performance-critical operations are implemented in Rust and communicate with the TypeScript runtime via @push.rocks/smartrust (JSON-over-stdin/stdout IPC). The Rust workspace lives at rust/ with five crates:

Crate	Status	Purpose
`mailer-core`	✅ Complete (26 tests)	Email types, validation, MIME building, bounce detection
`mailer-security`	✅ Complete (22 tests)	DKIM sign/verify, SPF, DMARC, IP reputation/DNSBL, content scanning
`mailer-smtp`	✅ Complete (77 tests)	Full SMTP protocol engine — TCP/TLS server, STARTTLS, AUTH, pipelining, in-process security pipeline, rate limiting
`mailer-bin`	✅ Complete	CLI + smartrust IPC bridge — security, content scanning, SMTP server lifecycle
`mailer-napi`	🔜 Planned	Native Node.js addon (N-API)

What Runs in Rust

Operation	Runs In	Why
SMTP server (port listening, protocol, TLS)	Rust	Performance, memory safety, zero-copy parsing
DKIM signing & verification	Rust	Crypto-heavy, benefits from native speed
SPF validation	Rust	DNS lookups with async resolver
DMARC policy checking	Rust	Integrates with SPF/DKIM results
IP reputation / DNSBL	Rust	Parallel DNS queries
Content scanning (text patterns)	Rust	Regex engine performance
Bounce detection (pattern matching)	Rust	Regex engine performance
Email validation & MIME building	Rust	Parsing performance
Binary attachment scanning	TypeScript	Buffer data too large for IPC
Email routing & orchestration	TypeScript	Business logic, flexibility
Delivery queue & retry	TypeScript	State management, persistence
Template rendering	TypeScript	String interpolation

Project Structure

smartmta/
├── ts/                              # TypeScript source
│   ├── mail/
│   │   ├── core/                    # Email, EmailValidator, BounceManager, TemplateManager
│   │   ├── delivery/                # DeliverySystem, Queue, RateLimiter
│   │   │   └── smtpclient/          # SMTP client with connection pooling
│   │   ├── routing/                 # UnifiedEmailServer, EmailRouter, DomainRegistry, DnsManager
│   │   └── security/                # DKIMCreator, DKIMVerifier, SpfVerifier, DmarcVerifier
│   └── security/                    # ContentScanner, IPReputationChecker, RustSecurityBridge
├── rust/                            # Rust workspace
│   └── crates/
│       ├── mailer-core/             # Email types, validation, MIME, bounce detection
│       ├── mailer-security/         # DKIM, SPF, DMARC, IP reputation, content scanning
│       ├── mailer-smtp/             # Full SMTP server (TCP/TLS, state machine, rate limiting)
│       ├── mailer-bin/              # CLI + smartrust IPC bridge
│       └── mailer-napi/             # N-API addon (planned)
├── test/                            # Test suite
├── dist_ts/                         # Compiled TypeScript output
└── dist_rust/                       # Compiled Rust binaries

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the LICENSE file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

24 KiB Raw Blame History