feat(CertificateManager): Implement on-demand certificate retrieval for missing SNI certificates. When no certificate is found for a TLS ClientHello, the system now automatically registers the domain with the Port80Handler to trigger ACME issuance and immediately falls back to using the default certificate to complete the handshake. Additionally, HTTP requests on port 80 for unrecognized domains now return a 503 indicating that certificate issuance is in progress.

changelog.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## 2025-05-05 - 10.2.0 - feat(CertificateManager)
+Implement on-demand certificate retrieval for missing SNI certificates. When no certificate is found for a TLS ClientHello, the system now automatically registers the domain with the Port80Handler to trigger ACME issuance and immediately falls back to using the default certificate to complete the handshake. Additionally, HTTP requests on port 80 for unrecognized domains now return a 503 indicating that certificate issuance is in progress.
+
+- In CertificateManager.handleSNI, if no certificate is cached, call port80Handler.addDomain to trigger on-demand provisioning.
+- Update Port80Handler.handleRequest to register unknown domains and return a 503 for ACME HTTP-01 challenge requests.
+- Emit observability events (e.g. certificateRequested) so dynamic certificate requests can be tracked.
+- Fallback to default SSL context to allow TLS handshake while certificate issuance is performed.
+- Update and extend unit and integration tests to verify the new on-demand certificate flow.
+
 ## 2025-05-05 - 10.1.0 - feat(smartproxy)
 Implement fallback to NetworkProxy on missing SNI and rename certProvider to certProvisionFunction in CertProvisioner
 

readme.plan.md

@@ -1,18 +1,42 @@
-# Plan: Fallback to NetworkProxy on Missing SNI
+# Plan: On-Demand Certificate Retrieval in NetworkProxy
 
-When a TLS ClientHello arrives without an SNI extension, we currently send a TLS alert and close the connection. Instead, if NetworkProxy is in use, we want to forward the connection to the network proxy first, and only issue the TLS error if proxying is not possible.
+When a TLS connection arrives with an SNI for a domain that has no certificate yet, we want to automatically kick off certificate issuance (ACME HTTP-01 or DNS-01) so the domain is provisioned on the fly without prior manual configuration.
 
 ## Goals
-- Allow TLS connections with no SNI to be forwarded to NetworkProxy when configured for any domain.
-- Only send a TLS unrecognized_name alert if proxy forwarding is unavailable or fails.
+- Automatically initiate certificate issuance upon first TLS handshake for an unprovisioned domain.
+- Use Port80Handler (HTTP-01) or custom `certProvisionFunction` (e.g., DNS-01) to retrieve the certificate.
+- Continue the TLS handshake immediately using the default certificate, then swap to the new certificate on subsequent connections.
+- For HTTP traffic on port 80, register the domain for ACME and return a 503 until the challenge is complete.
 
 ## Plan
-- [ ] In `ts/smartproxy/classes.pp.connectionhandler.ts`, locate the SNI-block branch in `handleStandardConnection` that checks `allowSessionTicket === false && isClientHello && !serverName`.
-- [ ] Replace the direct TLS alert/error logic with:
-  - If NetworkProxy is enabled (global `useNetworkProxy` setting or an active NetworkProxy instance), call `this.handleNetworkProxyConnection(socket, record)` (passing the buffered ClientHello) before issuing a TLS alert.
-  - Supply an error callback to `forwardToNetworkProxy`; if proxying fails or no NetworkProxy is available, fall back to the original TLS alert sequence.
-- [ ] Ensure existing metrics and cleanup (`record.incomingTerminationReason`, termination stats) are correctly tracked in the forwarding error path.
-- [ ] Add or update unit tests to simulate a TLS ClientHello without SNI on port 443 and verify:
-  - When NetworkProxy is enabled, the connection is forwarded to the proxy.
-  - If proxy forwarding fails or the domain is not configured, a TLS alert is sent and the socket is closed.
-- [ ] Run `pnpm test` to confirm no regressions and that the new behavior is correctly covered.
+1. Detect missing certificate in SNI callback:
+   - In `ts/networkproxy/classes.np.networkproxy.ts` (or within `CertificateManager.handleSNI`), after looking up `certificateCache`, if no cert is found:
+     - Call `port80Handler.addDomain({ domainName, sslRedirect: false, acmeMaintenance: true })` to trigger dynamic provisioning.
+     - Emit a `certificateRequested` event for observability.
+     - Immediately call `cb(null, defaultSecureContext)` so the handshake uses the default cert.
+
+2. HTTP-01 fallback on port 80:
+   - In `ts/port80handler/classes.port80handler.ts``, in `handleRequest()`, when a request arrives for a new domain not in `domainCertificates`:
+     - Call `addDomain({ domainName, sslRedirect: false, acmeMaintenance: true })`.
+     - Return HTTP 503 with a message like “Certificate issuance in progress.”
+
+3. CertProvisioner & events:
+   - Ensure `CertProvisioner` is subscribed to `Port80Handler` for newly added domains.
+   - After certificate issuance completes, `Port80Handler` emits `CERTIFICATE_ISSUED`, `CertificateManager` caches and writes disk, and future SNI callbacks will serve the new cert.
+
+4. Metrics and cleanup:
+   - Track dynamic requests count via a `certificateRequested` event or metric.
+   - Handle error paths: if ACME/DNS fails, emit `CERTIFICATE_FAILED` and continue serving default cert.
+
+5. Tests:
+   - Simulate a TLS ClientHello for an unconfigured domain:
+     • Verify `port80Handler.addDomain` is called and `certificateRequested` event emitted.
+     • Confirm handshake completes with default cert context.
+   - Simulate HTTP-01 challenge flow for a new domain:
+     • Verify on first HTTP request, `addDomain` is invoked and 503 returned.
+     • After manually injecting a challenge in `Http01MemoryHandler`, verify 200 with key authorization.
+   - Simulate successful ACME response and ensure SNI now returns the real cert.
+
+6. Final validation:
+   - Run `pnpm test` to ensure all existing tests pass.
+   - Add new unit/integration tests for the dynamic provisioning flow.

test/test.certprovisioner.unit.ts

@@ -2,7 +2,7 @@ import { tap, expect } from '@push.rocks/tapbundle';
 import * as plugins from '../ts/plugins.js';
 import { CertProvisioner } from '../ts/smartproxy/classes.pp.certprovisioner.js';
 import type { IDomainConfig, ISmartProxyCertProvisionObject } from '../ts/smartproxy/classes.pp.interfaces.js';
-import type { ICertificateData } from '../ts/port80handler/classes.port80handler.js';
+import type { ICertificateData } from '../ts/common/types.js';
 
 // Fake Port80Handler stub
 class FakePort80Handler extends plugins.EventEmitter {
@@ -36,7 +36,10 @@ tap.test('CertProvisioner handles static provisioning', async () => {
       domainName: domain,
       publicKey: 'CERT',
       privateKey: 'KEY',
-      validUntil: Date.now() + 3600 * 1000
+      validUntil: Date.now() + 3600 * 1000,
+      created: Date.now(),
+      csr: 'CSR',
+      id: 'ID',
     };
   };
   const prov = new CertProvisioner(
@@ -117,7 +120,10 @@ tap.test('CertProvisioner on-demand static provisioning', async () => {
     domainName: domain,
     publicKey: 'PKEY',
     privateKey: 'PRIV',
-    validUntil: Date.now() + 1000
+    validUntil: Date.now() + 1000,
+    created: Date.now(),
+    csr: 'CSR',
+    id: 'ID',
   });
   const prov = new CertProvisioner(
     domainConfigs,

test/test.router.ts

@@ -31,10 +31,10 @@ function createProxyConfig(
 ): tsclass.network.IReverseProxyConfig {
   return {
     hostName: hostname,
-    destinationIp,
-    destinationPort: destinationPort.toString(), // Convert to string for IReverseProxyConfig
     publicKey: 'mock-cert',
-    privateKey: 'mock-key'
+    privateKey: 'mock-key',
+    destinationIps: [destinationIp],
+    destinationPorts: [destinationPort],
   } as tsclass.network.IReverseProxyConfig;
 }
 

ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartproxy',
-  version: '10.1.0',
+  version: '10.2.0',
   description: 'A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, dynamic routing with authentication options, and automatic ACME certificate management.'
 }

ts/networkproxy/classes.np.certificatemanager.ts

@@ -183,7 +183,6 @@ export class CertificateManager {
     
     // Check if we have a certificate for this domain
     const certs = this.certificateCache.get(domain);
-    
     if (certs) {
       try {
         // Create TLS context with the cached certificate
@@ -191,7 +190,6 @@ export class CertificateManager {
           key: certs.key,
           cert: certs.cert
         });
-        
         this.logger.debug(`Using cached certificate for ${domain}`);
         cb(null, context);
         return;
@@ -199,6 +197,19 @@ export class CertificateManager {
         this.logger.error(`Error creating secure context for ${domain}:`, err);
       }
     }
+    // No existing certificate: trigger dynamic provisioning via Port80Handler
+    if (this.port80Handler) {
+      try {
+        this.logger.info(`Triggering on-demand certificate retrieval for ${domain}`);
+        this.port80Handler.addDomain({
+          domainName: domain,
+          sslRedirect: false,
+          acmeMaintenance: true
+        });
+      } catch (err) {
+        this.logger.error(`Error registering domain for on-demand certificate: ${domain}`, err);
+      }
+    }
     
     // Check if we should trigger certificate issuance
     if (this.options.acme?.enabled && this.port80Handler && !domain.includes('*')) {

ts/port80handler/classes.port80handler.ts

@@ -247,66 +247,6 @@ export class Port80Handler extends plugins.EventEmitter {
     }
   }
   
-  /**
-   * Sets a certificate for a domain directly (for externally obtained certificates)
-   * @param domain The domain for the certificate
-   * @param certificate The certificate (PEM format)
-   * @param privateKey The private key (PEM format)
-   * @param expiryDate Optional expiry date
-   */
-  public setCertificate(domain: string, certificate: string, privateKey: string, expiryDate?: Date): void {
-    if (!domain || !certificate || !privateKey) {
-      throw new Port80HandlerError('Domain, certificate and privateKey are required');
-    }
-    
-    // Don't allow setting certificates for glob patterns
-    if (this.isGlobPattern(domain)) {
-      throw new Port80HandlerError('Cannot set certificate for glob pattern domains');
-    }
-    
-    let domainInfo = this.domainCertificates.get(domain);
-    
-    if (!domainInfo) {
-      // Create default domain options if not already configured
-      const defaultOptions: IDomainOptions = {
-        domainName: domain,
-        sslRedirect: true,
-        acmeMaintenance: true
-      };
-      
-      domainInfo = { 
-        options: defaultOptions, 
-        certObtained: false, 
-        obtainingInProgress: false 
-      };
-      this.domainCertificates.set(domain, domainInfo);
-    }
-    
-    domainInfo.certificate = certificate;
-    domainInfo.privateKey = privateKey;
-    domainInfo.certObtained = true;
-    domainInfo.obtainingInProgress = false;
-    
-    if (expiryDate) {
-      domainInfo.expiryDate = expiryDate;
-    } else {
-      // Extract expiry date from certificate
-      domainInfo.expiryDate = this.extractExpiryDateFromCertificate(certificate, domain);
-    }
-    
-    console.log(`Certificate set for ${domain}`);
-    
-    // (Persistence of certificates moved to CertProvisioner)
-    
-    // Emit certificate event
-    this.emitCertificateEvent(Port80HandlerEvents.CERTIFICATE_ISSUED, {
-      domain,
-      certificate,
-      privateKey,
-      expiryDate: domainInfo.expiryDate || this.getDefaultExpiryDate()
-    });
-  }
-  
   /**
    * Gets the certificate for a domain if it exists
    * @param domain The domain to get the certificate for
@@ -409,9 +349,19 @@ export class Port80Handler extends plugins.EventEmitter {
     // Extract domain (ignoring any port in the Host header)
     const domain = hostHeader.split(':')[0];
 
+    // Dynamic provisioning: if domain not yet managed, register for ACME and return 503
+    if (!this.domainCertificates.has(domain)) {
+      try {
+        this.addDomain({ domainName: domain, sslRedirect: false, acmeMaintenance: true });
+      } catch (err) {
+        console.error(`Error registering domain for on-demand provisioning: ${err}`);
+      }
+      res.statusCode = 503;
+      res.end('Certificate issuance in progress');
+      return;
+    }
     // Get domain config, using glob pattern matching if needed
     const domainMatch = this.getDomainInfoForRequest(domain);
-    
     if (!domainMatch) {
       res.statusCode = 404;
       res.end('Domain not configured');
@@ -715,36 +665,6 @@ export class Port80Handler extends plugins.EventEmitter {
     return result;
   }
   
-  /**
-   * Gets information about managed domains
-   * @returns Array of domain information
-   */
-  public getManagedDomains(): Array<{
-    domain: string;
-    isGlobPattern: boolean;
-    hasCertificate: boolean;
-    hasForwarding: boolean;
-    sslRedirect: boolean;
-    acmeMaintenance: boolean;
-  }> {
-    return Array.from(this.domainCertificates.entries()).map(([domain, info]) => ({
-      domain,
-      isGlobPattern: this.isGlobPattern(domain),
-      hasCertificate: info.certObtained,
-      hasForwarding: !!info.options.forward,
-      sslRedirect: info.options.sslRedirect,
-      acmeMaintenance: info.options.acmeMaintenance
-    }));
-  }
-  
-  /**
-   * Gets configuration details
-   * @returns Current configuration
-   */
-  public getConfig(): Required<IAcmeOptions> {
-    return { ...this.options };
-  }
-  
   /**
    * Request a certificate renewal for a specific domain.
    * @param domain The domain to renew.