feat(remote-ingress): add Remote Ingress hub integration, OpsServer UI, APIs, and docs

2026-02-18 18:47:18 +00:00
parent 86e6c4f600
commit dce1de8c4b
13 changed files with 456 additions and 43 deletions
--- a/readme.md
+++ b/readme.md
@@ -4,7 +4,7 @@

 **dcrouter: The all-in-one gateway for your datacenter.** 🚀

-A comprehensive traffic routing solution that provides unified gateway capabilities for HTTP/HTTPS, TCP/SNI, email (SMTP), DNS, and RADIUS protocols. Designed for enterprises requiring robust traffic management, automatic TLS certificate provisioning, and enterprise-grade email infrastructure — all from a single process.
+A comprehensive traffic routing solution that provides unified gateway capabilities for HTTP/HTTPS, TCP/SNI, email (SMTP), DNS, RADIUS, and remote edge ingress — all from a single process. Designed for enterprises requiring robust traffic management, automatic TLS certificate provisioning, distributed edge networking, and enterprise-grade email infrastructure.

 ## Issue Reporting and Security

@@ -21,6 +21,7 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
 - [Email System](#email-system)
 - [DNS Server](#dns-server)
 - [RADIUS Server](#radius-server)
+- [Remote Ingress](#remote-ingress)
 - [Certificate Management](#certificate-management)
 - [Storage & Caching](#storage--caching)
 - [Security Features](#security-features)
@@ -60,6 +61,14 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
 - **RADIUS accounting** for session tracking, traffic metering, and billing
 - **Real-time management** via OpsServer API

+### 🌍 Remote Ingress (powered by [remoteingress](https://code.foss.global/serve.zone/remoteingress))
+- **Distributed edge networking** — accept traffic at remote edge nodes and tunnel it to the hub
+- **Edge registration CRUD** with secret-based authentication
+- **Auto-derived ports** — edges automatically pick up ports from routes tagged with `remoteIngress.enabled`
+- **Connection tokens** — generate a single opaque base64url token containing hubHost, hubPort, edgeId, and secret for easy edge provisioning
+- **Real-time status monitoring** — connected/disconnected state, public IP, active tunnels, heartbeat tracking
+- **OpsServer dashboard** with enable/disable, edit, secret regeneration, token copy, and delete actions
+
 ### ⚡ High Performance
 - **Rust-powered proxy engine** via SmartProxy for maximum throughput
 - **Rust-powered MTA engine** via smartmta (TypeScript + Rust hybrid) for reliable email delivery
@@ -76,8 +85,9 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
 ### 🖥️ OpsServer Dashboard
 - **Web-based management interface** with real-time monitoring
 - **JWT authentication** with session persistence
- **Live views** for connections, email queues, DNS queries, RADIUS sessions, certificates, and security events
+- **Live views** for connections, email queues, DNS queries, RADIUS sessions, certificates, remote ingress edges, and security events
 - **Domain-centric certificate overview** with backoff status and one-click reprovisioning
+- **Remote ingress management** with connection token generation and one-click copy
 - **Read-only configuration display** — DcRouter is configured through code

 ## Installation
@@ -219,6 +229,13 @@ const router = new DcRouter({
    accounting: { enabled: true, retentionDays: 30 }
  },

+  // Remote Ingress — edge nodes tunnel traffic to this hub
+  remoteIngressConfig: {
+    enabled: true,
+    tunnelPort: 8443,
+    hubDomain: 'hub.example.com',
+  },
+
  // Persistent storage
  storage: { fsPath: '/var/lib/dcrouter/data' },

@@ -246,6 +263,7 @@ graph TB
        TCP[TCP Clients]
        DNS[DNS Queries]
        RAD[RADIUS Clients]
+        EDGE[Edge Nodes]
    end

    subgraph "DcRouter Core"
@@ -254,6 +272,7 @@ graph TB
        ES[smartmta Email Server<br/><i>TypeScript + Rust</i>]
        DS[SmartDNS Server<br/><i>Rust-powered</i>]
        RS[SmartRadius Server]
+        RI[RemoteIngress Hub<br/><i>Rust data plane</i>]
        CM[Certificate Manager<br/><i>smartacme v9</i>]
        OS[OpsServer Dashboard]
        MM[Metrics Manager]
@@ -273,11 +292,13 @@ graph TB
    SMTP --> ES
    DNS --> DS
    RAD --> RS
+    EDGE --> RI

    DC --> SP
    DC --> ES
    DC --> DS
    DC --> RS
+    DC --> RI
    DC --> CM
    DC --> OS
    DC --> MM
@@ -288,6 +309,7 @@ graph TB
    SP --> API
    ES --> MAIL
    ES --> DB
+    RI --> SP

    CM -.-> SP
    CM -.-> ES
@@ -303,6 +325,7 @@ graph TB
 | **DNS Server** | `@push.rocks/smartdns` | Authoritative DNS with dynamic records and DKIM TXT auto-generation (Rust engine) |
 | **SmartAcme** | `@push.rocks/smartacme` | ACME certificate management with per-domain dedup, concurrency control, and rate limiting |
 | **RADIUS Server** | `@push.rocks/smartradius` | Network authentication with MAB, VLAN assignment, and accounting |
+| **RemoteIngress** | `@serve.zone/remoteingress` | Distributed edge tunneling with Rust data plane and TS management |
 | **OpsServer** | `@api.global/typedserver` | Web dashboard + TypedRequest API for monitoring and management |
 | **MetricsManager** | `@push.rocks/smartmetrics` | Real-time metrics collection (CPU, memory, email, DNS, security) |
 | **StorageManager** | built-in | Pluggable key-value storage (filesystem, custom, or in-memory) |
@@ -312,19 +335,20 @@ graph TB

 DcRouter acts purely as an **orchestrator** — it doesn't implement protocols itself. Instead, it wires together best-in-class packages for each protocol:

-1. **On `start()`**: DcRouter initializes OpsServer (port 3000), then spins up SmartProxy, smartmta, SmartDNS, and SmartRadius based on which configs are provided.
-2. **During operation**: Each service handles its own protocol independently. SmartProxy uses a Rust-powered engine for maximum throughput. smartmta uses a hybrid TypeScript + Rust architecture for reliable email delivery. SmartAcme v9 handles all certificate operations with built-in concurrency control and rate limiting.
+1. **On `start()`**: DcRouter initializes OpsServer (port 3000), then spins up SmartProxy, smartmta, SmartDNS, SmartRadius, and RemoteIngress based on which configs are provided.
+2. **During operation**: Each service handles its own protocol independently. SmartProxy uses a Rust-powered engine for maximum throughput. smartmta uses a hybrid TypeScript + Rust architecture for reliable email delivery. RemoteIngress runs a Rust data plane for edge tunnel networking. SmartAcme v9 handles all certificate operations with built-in concurrency control and rate limiting.
 3. **On `stop()`**: All services are gracefully shut down in parallel, including cleanup of HTTP agents and DNS clients.

 ### Rust-Powered Architecture

-DcRouter itself is a pure TypeScript orchestrator, but three of its core sub-components ship with **compiled Rust binaries** for performance-critical paths. At runtime each package detects the platform, unpacks the correct binary, and communicates with TypeScript over IPC/FFI — so you get the ergonomics of TypeScript with the throughput of native code.
+DcRouter itself is a pure TypeScript orchestrator, but several of its core sub-components ship with **compiled Rust binaries** for performance-critical paths. At runtime each package detects the platform, unpacks the correct binary, and communicates with TypeScript over IPC/FFI — so you get the ergonomics of TypeScript with the throughput of native code.

 | Component | Rust Binary | What It Handles |
 |-----------|-------------|-----------------|
 | **SmartProxy** | `smartproxy-bin` | All TCP/TLS/HTTP proxy networking, NFTables integration, connection metrics |
 | **smartmta** | `mailer-bin` | SMTP server + client, DKIM/SPF/DMARC, content scanning, IP reputation |
 | **SmartDNS** | `smartdns-bin` | DNS server (UDP + DNS-over-HTTPS), DNSSEC, DNS client resolution |
+| **RemoteIngress** | `remoteingress-bin` | Edge tunnel data plane, multiplexed streams, heartbeat management |
 | **SmartRadius** | — | Pure TypeScript (no Rust component) |

 ## Configuration Reference
@@ -333,6 +357,10 @@ DcRouter itself is a pure TypeScript orchestrator, but three of its core sub-com

 ```typescript
 interface IDcRouterOptions {
+  // ── Base ───────────────────────────────────────────────────────
+  /** Base directory for all dcrouter data. Defaults to ~/.serve.zone/dcrouter */
+  baseDir?: string;
+
  // ── Traffic Routing ────────────────────────────────────────────
  /** SmartProxy config for HTTP/HTTPS and TCP/SNI routing */
  smartProxyConfig?: ISmartProxyOptions;
@@ -376,6 +404,18 @@ interface IDcRouterOptions {
    accounting?: { enabled: boolean; retentionDays?: number };
  };

+  // ── Remote Ingress ─────────────────────────────────────────────
+  /** Remote Ingress hub for edge tunnel connections */
+  remoteIngressConfig?: {
+    enabled?: boolean;                   // default: false
+    tunnelPort?: number;                 // default: 8443
+    hubDomain?: string;                  // External hostname for connection tokens
+    tls?: {
+      certPath?: string;
+      keyPath?: string;
+    };
+  };
+
  // ── TLS & Certificates ────────────────────────────────────────
  tls?: {
    contactEmail: string;
@@ -701,6 +741,107 @@ RADIUS is fully manageable at runtime via the OpsServer API:
 - Session monitoring and forced disconnects
 - Accounting summaries and statistics

+## Remote Ingress
+
+DcRouter can act as a **hub** for distributed edge nodes using [`@serve.zone/remoteingress`](https://code.foss.global/serve.zone/remoteingress). Edge nodes accept incoming traffic at remote locations and tunnel it back to the hub over a single multiplexed connection. This is ideal for scenarios where you need to accept traffic at multiple geographic locations but process it centrally.
+
+### Enabling Remote Ingress
+
+```typescript
+const router = new DcRouter({
+  remoteIngressConfig: {
+    enabled: true,
+    tunnelPort: 8443,
+    hubDomain: 'hub.example.com',  // Embedded in connection tokens
+  },
+  // Routes tagged with remoteIngress are auto-derived to edge listen ports
+  smartProxyConfig: {
+    routes: [
+      {
+        name: 'web-via-edge',
+        match: { domains: ['app.example.com'], ports: [443] },
+        action: {
+          type: 'forward',
+          targets: [{ host: '192.168.1.10', port: 8080 }],
+          tls: { mode: 'terminate', certificate: 'auto' }
+        },
+        remoteIngress: { enabled: true }  // Edges will listen on port 443
+      }
+    ]
+  }
+});
+
+await router.start();
+```
+
+### Edge Registration
+
+Edges are registered via the OpsServer API (or dashboard UI). Each edge gets a unique ID and secret:
+
+```typescript
+// Via TypedRequest API
+const createReq = new TypedRequest<IReq_CreateRemoteIngress>(
+  'https://hub:3000/typedrequest', 'createRemoteIngress'
+);
+const { edge } = await createReq.fire({
+  identity,
+  name: 'edge-nyc-01',
+  autoDerivePorts: true,
+  tags: ['us-east'],
+});
+// edge.secret is returned only on creation — save it!
+```
+
+### Connection Tokens 🔑
+
+Instead of configuring edges with four separate values (hubHost, hubPort, edgeId, secret), DcRouter can generate a single **connection token** — an opaque base64url string that encodes everything:
+
+```typescript
+// Via TypedRequest API
+const tokenReq = new TypedRequest<IReq_GetRemoteIngressConnectionToken>(
+  'https://hub:3000/typedrequest', 'getRemoteIngressConnectionToken'
+);
+const { token } = await tokenReq.fire({ identity, edgeId: 'edge-uuid' });
+// token = "eyJoIjoiaHViLmV4YW1wbGUuY29tIiwicCI6ODQ0MywiZSI6I..."
+
+// On the edge side, just pass the token:
+const edge = new RemoteIngressEdge({ token });
+await edge.start();
+```
+
+The token is generated using `remoteingress.encodeConnectionToken()` and contains `{ hubHost, hubPort, edgeId, secret }`. The `hubHost` comes from `remoteIngressConfig.hubDomain` (or can be overridden per-request).
+
+In the OpsServer dashboard, click **"Copy Token"** on any edge row to copy the connection token to your clipboard.
+
+### Auto-Derived Ports
+
+When routes have `remoteIngress: { enabled: true }`, edges with `autoDerivePorts: true` (default) automatically pick up those routes' ports. You can also use `edgeFilter` to restrict which edges get which ports:
+
+```typescript
+{
+  name: 'web-route',
+  match: { ports: [443] },
+  action: { /* ... */ },
+  remoteIngress: {
+    enabled: true,
+    edgeFilter: ['us-east', 'edge-uuid-123']  // Only edges with matching id or tags
+  }
+}
+```
+
+### Dashboard Actions
+
+The OpsServer Remote Ingress view provides:
+
+| Action | Description |
+|--------|-------------|
+| **Create Edge Node** | Register a new edge with name, ports, tags |
+| **Enable / Disable** | Toggle an edge on or off |
+| **Edit** | Modify name, manual ports, auto-derive setting, tags |
+| **Regenerate Secret** | Issue a new secret (invalidates the old one) |
+| **Copy Token** | Generate and copy a base64url connection token to clipboard |
+| **Delete** | Remove the edge registration |
+
 ## Certificate Management

 DcRouter uses [`@push.rocks/smartacme`](https://code.foss.global/push.rocks/smartacme) v9 for ACME certificate provisioning. smartacme v9 brings significant improvements over previous versions:
@@ -767,6 +908,7 @@ The OpsServer includes a **Certificates** view showing:
 - Expiry dates and issuer information
 - Backoff status for failed domains
 - One-click reprovisioning per domain
+- Certificate import and export

 ## Storage & Caching

@@ -788,7 +930,7 @@ storage: {
 // Simply omit the storage config
 ```

-Used for: TLS certificates, DKIM keys, email routes, bounce/suppression lists, IP reputation data, domain configs, cert backoff state.
+Used for: TLS certificates, DKIM keys, email routes, bounce/suppression lists, IP reputation data, domain configs, cert backoff state, remote ingress edge registrations.

 ### Cache Database

@@ -874,7 +1016,8 @@ The OpsServer provides a web-based management interface served on port 3000. It'
 | 📊 **Overview** | Real-time server stats, CPU/memory, connection counts, email throughput |
 | 🌐 **Network** | Active connections, top IPs, throughput rates, SmartProxy metrics |
 | 📧 **Email** | Queue monitoring (queued/sent/failed), bounce records, security incidents |
-| 🔐 **Certificates** | Domain-centric certificate overview, status, backoff info, reprovisioning |
+| 🔐 **Certificates** | Domain-centric certificate overview, status, backoff info, reprovisioning, import/export |
+| 🌍 **RemoteIngress** | Edge node management, connection status, token generation, enable/disable |
 | 📜 **Logs** | Real-time log viewer with level filtering and search |
 | ⚙️ **Configuration** | Read-only view of current system configuration |
 | 🛡️ **Security** | IP reputation, rate limit status, blocked connections |
@@ -906,6 +1049,18 @@ All management is done via TypedRequest over HTTP POST to `/typedrequest`:
 'getCertificateOverview'               // Domain-centric certificate status
 'reprovisionCertificate'               // Reprovision by route name (legacy)
 'reprovisionCertificateDomain'         // Reprovision by domain (preferred)
+'importCertificate'                    // Import a certificate
+'exportCertificate'                    // Export a certificate
+'deleteCertificate'                    // Delete a certificate
+
+// Remote Ingress
+'getRemoteIngresses'                   // List all edge registrations
+'createRemoteIngress'                  // Register a new edge
+'updateRemoteIngress'                  // Update edge settings
+'deleteRemoteIngress'                  // Remove an edge
+'regenerateRemoteIngressSecret'        // Issue a new secret
+'getRemoteIngressStatus'               // Runtime status of all edges
+'getRemoteIngressConnectionToken'      // Generate a connection token for an edge

 // Configuration (read-only)
 'getConfiguration'                     // Current system config
@@ -957,6 +1112,8 @@ const router = new DcRouter(options: IDcRouterOptions);
 | `emailServer` | `UnifiedEmailServer` | Email server instance (from smartmta) |
 | `dnsServer` | `DnsServer` | DNS server instance |
 | `radiusServer` | `RadiusServer` | RADIUS server instance |
+| `remoteIngressManager` | `RemoteIngressManager` | Edge registration CRUD manager |
+| `tunnelManager` | `TunnelManager` | Tunnel lifecycle and status manager |
 | `storageManager` | `StorageManager` | Storage backend |
 | `opsServer` | `OpsServer` | OpsServer/dashboard instance |
 | `metricsManager` | `MetricsManager` | Metrics collector |
@@ -1000,7 +1157,7 @@ import { data, requests } from '@serve.zone/dcrouter/interfaces';
 DcRouter includes a comprehensive test suite covering all system components:

 ```bash
-# Run all tests (10 files, 73 tests)
+# Run all tests
 pnpm test

 # Run a specific test file