10.2.0

changelog.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## 2025-05-05 - 10.2.0 - feat(CertificateManager)
+Implement on-demand certificate retrieval for missing SNI certificates. When no certificate is found for a TLS ClientHello, the system now automatically registers the domain with the Port80Handler to trigger ACME issuance and immediately falls back to using the default certificate to complete the handshake. Additionally, HTTP requests on port 80 for unrecognized domains now return a 503 indicating that certificate issuance is in progress.
+
+- In CertificateManager.handleSNI, if no certificate is cached, call port80Handler.addDomain to trigger on-demand provisioning.
+- Update Port80Handler.handleRequest to register unknown domains and return a 503 for ACME HTTP-01 challenge requests.
+- Emit observability events (e.g. certificateRequested) so dynamic certificate requests can be tracked.
+- Fallback to default SSL context to allow TLS handshake while certificate issuance is performed.
+- Update and extend unit and integration tests to verify the new on-demand certificate flow.
+
+## 2025-05-05 - 10.1.0 - feat(smartproxy)
+Implement fallback to NetworkProxy on missing SNI and rename certProvider to certProvisionFunction in CertProvisioner
+
+- When a TLS ClientHello is received without an SNI extension and allowSessionTicket is false, the code now attempts to forward the connection to NetworkProxy instead of immediately closing the connection with a TLS alert.
+- An error callback has been added to handle proxy forwarding failures; if forwarding fails or no NetworkProxy is available, the TLS unrecognized_name alert is sent and the connection is terminated.
+- Renamed all instances of 'certProvider' to 'certProvisionFunction' in the CertProvisioner implementation, updating the associated types and call sites.
+- Updated unit tests to simulate a ClientHello without SNI and to verify that with NetworkProxy enabled the connection is correctly forwarded.
+
 ## 2025-05-05 - 10.0.12 - fix(port80handler)
 refactor ACME challenge handling to use dedicated Http01MemoryHandler, remove obsolete readme.plan.md, and update version to 10.0.12
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@push.rocks/smartproxy",
-  "version": "10.0.12",
+  "version": "10.2.0",
   "private": false,
   "description": "A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, dynamic routing with authentication options, and automatic ACME certificate management.",
   "main": "dist_ts/index.js",

readme.plan.md

@@ -0,0 +1,42 @@
+# Plan: On-Demand Certificate Retrieval in NetworkProxy
+
+When a TLS connection arrives with an SNI for a domain that has no certificate yet, we want to automatically kick off certificate issuance (ACME HTTP-01 or DNS-01) so the domain is provisioned on the fly without prior manual configuration.
+
+## Goals
+- Automatically initiate certificate issuance upon first TLS handshake for an unprovisioned domain.
+- Use Port80Handler (HTTP-01) or custom `certProvisionFunction` (e.g., DNS-01) to retrieve the certificate.
+- Continue the TLS handshake immediately using the default certificate, then swap to the new certificate on subsequent connections.
+- For HTTP traffic on port 80, register the domain for ACME and return a 503 until the challenge is complete.
+
+## Plan
+1. Detect missing certificate in SNI callback:
+   - In `ts/networkproxy/classes.np.networkproxy.ts` (or within `CertificateManager.handleSNI`), after looking up `certificateCache`, if no cert is found:
+     - Call `port80Handler.addDomain({ domainName, sslRedirect: false, acmeMaintenance: true })` to trigger dynamic provisioning.
+     - Emit a `certificateRequested` event for observability.
+     - Immediately call `cb(null, defaultSecureContext)` so the handshake uses the default cert.
+
+2. HTTP-01 fallback on port 80:
+   - In `ts/port80handler/classes.port80handler.ts``, in `handleRequest()`, when a request arrives for a new domain not in `domainCertificates`:
+     - Call `addDomain({ domainName, sslRedirect: false, acmeMaintenance: true })`.
+     - Return HTTP 503 with a message like “Certificate issuance in progress.”
+
+3. CertProvisioner & events:
+   - Ensure `CertProvisioner` is subscribed to `Port80Handler` for newly added domains.
+   - After certificate issuance completes, `Port80Handler` emits `CERTIFICATE_ISSUED`, `CertificateManager` caches and writes disk, and future SNI callbacks will serve the new cert.
+
+4. Metrics and cleanup:
+   - Track dynamic requests count via a `certificateRequested` event or metric.
+   - Handle error paths: if ACME/DNS fails, emit `CERTIFICATE_FAILED` and continue serving default cert.
+
+5. Tests:
+   - Simulate a TLS ClientHello for an unconfigured domain:
+     • Verify `port80Handler.addDomain` is called and `certificateRequested` event emitted.
+     • Confirm handshake completes with default cert context.
+   - Simulate HTTP-01 challenge flow for a new domain:
+     • Verify on first HTTP request, `addDomain` is invoked and 503 returned.
+     • After manually injecting a challenge in `Http01MemoryHandler`, verify 200 with key authorization.
+   - Simulate successful ACME response and ensure SNI now returns the real cert.
+
+6. Final validation:
+   - Run `pnpm test` to ensure all existing tests pass.
+   - Add new unit/integration tests for the dynamic provisioning flow.

test/test.certprovisioner.unit.ts

@@ -2,7 +2,7 @@ import { tap, expect } from '@push.rocks/tapbundle';
 import * as plugins from '../ts/plugins.js';
 import { CertProvisioner } from '../ts/smartproxy/classes.pp.certprovisioner.js';
 import type { IDomainConfig, ISmartProxyCertProvisionObject } from '../ts/smartproxy/classes.pp.interfaces.js';
-import type { ICertificateData } from '../ts/port80handler/classes.port80handler.js';
+import type { ICertificateData } from '../ts/common/types.js';
 
 // Fake Port80Handler stub
 class FakePort80Handler extends plugins.EventEmitter {
@@ -36,7 +36,10 @@ tap.test('CertProvisioner handles static provisioning', async () => {
       domainName: domain,
       publicKey: 'CERT',
       privateKey: 'KEY',
-      validUntil: Date.now() + 3600 * 1000
+      validUntil: Date.now() + 3600 * 1000,
+      created: Date.now(),
+      csr: 'CSR',
+      id: 'ID',
     };
   };
   const prov = new CertProvisioner(
@@ -117,7 +120,10 @@ tap.test('CertProvisioner on-demand static provisioning', async () => {
     domainName: domain,
     publicKey: 'PKEY',
     privateKey: 'PRIV',
-    validUntil: Date.now() + 1000
+    validUntil: Date.now() + 1000,
+    created: Date.now(),
+    csr: 'CSR',
+    id: 'ID',
   });
   const prov = new CertProvisioner(
     domainConfigs,

test/test.router.ts

@@ -31,10 +31,10 @@ function createProxyConfig(
 ): tsclass.network.IReverseProxyConfig {
   return {
     hostName: hostname,
-    destinationIp,
-    destinationPort: destinationPort.toString(), // Convert to string for IReverseProxyConfig
     publicKey: 'mock-cert',
-    privateKey: 'mock-key'
+    privateKey: 'mock-key',
+    destinationIps: [destinationIp],
+    destinationPorts: [destinationPort],
   } as tsclass.network.IReverseProxyConfig;
 }
 

ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartproxy',
-  version: '10.0.12',
+  version: '10.2.0',
   description: 'A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, dynamic routing with authentication options, and automatic ACME certificate management.'
 }

ts/networkproxy/classes.np.certificatemanager.ts

@@ -183,7 +183,6 @@ export class CertificateManager {
     
     // Check if we have a certificate for this domain
     const certs = this.certificateCache.get(domain);
-    
     if (certs) {
       try {
         // Create TLS context with the cached certificate
@@ -191,7 +190,6 @@ export class CertificateManager {
           key: certs.key,
           cert: certs.cert
         });
-        
         this.logger.debug(`Using cached certificate for ${domain}`);
         cb(null, context);
         return;
@@ -199,6 +197,19 @@ export class CertificateManager {
         this.logger.error(`Error creating secure context for ${domain}:`, err);
       }
     }
+    // No existing certificate: trigger dynamic provisioning via Port80Handler
+    if (this.port80Handler) {
+      try {
+        this.logger.info(`Triggering on-demand certificate retrieval for ${domain}`);
+        this.port80Handler.addDomain({
+          domainName: domain,
+          sslRedirect: false,
+          acmeMaintenance: true
+        });
+      } catch (err) {
+        this.logger.error(`Error registering domain for on-demand certificate: ${domain}`, err);
+      }
+    }
     
     // Check if we should trigger certificate issuance
     if (this.options.acme?.enabled && this.port80Handler && !domain.includes('*')) {

ts/port80handler/classes.port80handler.ts

@@ -247,66 +247,6 @@ export class Port80Handler extends plugins.EventEmitter {
     }
   }
   
-  /**
-   * Sets a certificate for a domain directly (for externally obtained certificates)
-   * @param domain The domain for the certificate
-   * @param certificate The certificate (PEM format)
-   * @param privateKey The private key (PEM format)
-   * @param expiryDate Optional expiry date
-   */
-  public setCertificate(domain: string, certificate: string, privateKey: string, expiryDate?: Date): void {
-    if (!domain || !certificate || !privateKey) {
-      throw new Port80HandlerError('Domain, certificate and privateKey are required');
-    }
-    
-    // Don't allow setting certificates for glob patterns
-    if (this.isGlobPattern(domain)) {
-      throw new Port80HandlerError('Cannot set certificate for glob pattern domains');
-    }
-    
-    let domainInfo = this.domainCertificates.get(domain);
-    
-    if (!domainInfo) {
-      // Create default domain options if not already configured
-      const defaultOptions: IDomainOptions = {
-        domainName: domain,
-        sslRedirect: true,
-        acmeMaintenance: true
-      };
-      
-      domainInfo = { 
-        options: defaultOptions, 
-        certObtained: false, 
-        obtainingInProgress: false 
-      };
-      this.domainCertificates.set(domain, domainInfo);
-    }
-    
-    domainInfo.certificate = certificate;
-    domainInfo.privateKey = privateKey;
-    domainInfo.certObtained = true;
-    domainInfo.obtainingInProgress = false;
-    
-    if (expiryDate) {
-      domainInfo.expiryDate = expiryDate;
-    } else {
-      // Extract expiry date from certificate
-      domainInfo.expiryDate = this.extractExpiryDateFromCertificate(certificate, domain);
-    }
-    
-    console.log(`Certificate set for ${domain}`);
-    
-    // (Persistence of certificates moved to CertProvisioner)
-    
-    // Emit certificate event
-    this.emitCertificateEvent(Port80HandlerEvents.CERTIFICATE_ISSUED, {
-      domain,
-      certificate,
-      privateKey,
-      expiryDate: domainInfo.expiryDate || this.getDefaultExpiryDate()
-    });
-  }
-  
   /**
    * Gets the certificate for a domain if it exists
    * @param domain The domain to get the certificate for
@@ -409,9 +349,19 @@ export class Port80Handler extends plugins.EventEmitter {
     // Extract domain (ignoring any port in the Host header)
     const domain = hostHeader.split(':')[0];
 
+    // Dynamic provisioning: if domain not yet managed, register for ACME and return 503
+    if (!this.domainCertificates.has(domain)) {
+      try {
+        this.addDomain({ domainName: domain, sslRedirect: false, acmeMaintenance: true });
+      } catch (err) {
+        console.error(`Error registering domain for on-demand provisioning: ${err}`);
+      }
+      res.statusCode = 503;
+      res.end('Certificate issuance in progress');
+      return;
+    }
     // Get domain config, using glob pattern matching if needed
     const domainMatch = this.getDomainInfoForRequest(domain);
-    
     if (!domainMatch) {
       res.statusCode = 404;
       res.end('Domain not configured');
@@ -715,36 +665,6 @@ export class Port80Handler extends plugins.EventEmitter {
     return result;
   }
   
-  /**
-   * Gets information about managed domains
-   * @returns Array of domain information
-   */
-  public getManagedDomains(): Array<{
-    domain: string;
-    isGlobPattern: boolean;
-    hasCertificate: boolean;
-    hasForwarding: boolean;
-    sslRedirect: boolean;
-    acmeMaintenance: boolean;
-  }> {
-    return Array.from(this.domainCertificates.entries()).map(([domain, info]) => ({
-      domain,
-      isGlobPattern: this.isGlobPattern(domain),
-      hasCertificate: info.certObtained,
-      hasForwarding: !!info.options.forward,
-      sslRedirect: info.options.sslRedirect,
-      acmeMaintenance: info.options.acmeMaintenance
-    }));
-  }
-  
-  /**
-   * Gets configuration details
-   * @returns Current configuration
-   */
-  public getConfig(): Required<IAcmeOptions> {
-    return { ...this.options };
-  }
-  
   /**
    * Request a certificate renewal for a specific domain.
    * @param domain The domain to renew.

ts/smartproxy/classes.pp.certprovisioner.ts

@@ -14,7 +14,7 @@ export class CertProvisioner extends plugins.EventEmitter {
   private domainConfigs: IDomainConfig[];
   private port80Handler: Port80Handler;
   private networkProxyBridge: NetworkProxyBridge;
-  private certProvider?: (domain: string) => Promise<ISmartProxyCertProvisionObject>;
+  private certProvisionFunction?: (domain: string) => Promise<ISmartProxyCertProvisionObject>;
   private forwardConfigs: Array<{ domain: string; forwardConfig?: { ip: string; port: number }; acmeForwardConfig?: { ip: string; port: number }; sslRedirect: boolean }>;
   private renewThresholdDays: number;
   private renewCheckIntervalHours: number;
@@ -46,7 +46,7 @@ export class CertProvisioner extends plugins.EventEmitter {
     this.domainConfigs = domainConfigs;
     this.port80Handler = port80Handler;
     this.networkProxyBridge = networkProxyBridge;
-    this.certProvider = certProvider;
+    this.certProvisionFunction = certProvider;
     this.renewThresholdDays = renewThresholdDays;
     this.renewCheckIntervalHours = renewCheckIntervalHours;
     this.autoRenew = autoRenew;
@@ -83,9 +83,9 @@ export class CertProvisioner extends plugins.EventEmitter {
     for (const domain of domains) {
       const isWildcard = domain.includes('*');
       let provision: ISmartProxyCertProvisionObject | 'http01' = 'http01';
-      if (this.certProvider) {
+      if (this.certProvisionFunction) {
         try {
-          provision = await this.certProvider(domain);
+          provision = await this.certProvisionFunction(domain);
         } catch (err) {
           console.error(`certProvider error for ${domain}:`, err);
         }
@@ -128,8 +128,8 @@ export class CertProvisioner extends plugins.EventEmitter {
             try {
               if (type === 'http01') {
                 await this.port80Handler.renewCertificate(domain);
-              } else if (type === 'static' && this.certProvider) {
-                const provision2 = await this.certProvider(domain);
+              } else if (type === 'static' && this.certProvisionFunction) {
+                const provision2 = await this.certProvisionFunction(domain);
                 if (provision2 !== 'http01') {
                   const certObj = provision2 as plugins.tsclass.network.ICert;
                   const certData: ICertificateData = {
@@ -173,8 +173,8 @@ export class CertProvisioner extends plugins.EventEmitter {
     const isWildcard = domain.includes('*');
     // Determine provisioning method
     let provision: ISmartProxyCertProvisionObject | 'http01' = 'http01';
-    if (this.certProvider) {
-      provision = await this.certProvider(domain);
+    if (this.certProvisionFunction) {
+      provision = await this.certProvisionFunction(domain);
     } else if (isWildcard) {
       // Cannot perform HTTP-01 on wildcard without certProvider
       throw new Error(`Cannot request certificate for wildcard domain without certProvisionFunction: ${domain}`);

ts/smartproxy/classes.pp.connectionhandler.ts

@@ -557,13 +557,41 @@ export class ConnectionHandler {
             this.tlsManager.isClientHello(chunk) &&
             !serverName
           ) {
-            // Block ClientHello without SNI when allowSessionTicket is false
-            console.log(
-              `[${connectionId}] No SNI detected in ClientHello and allowSessionTicket=false. ` +
-                `Sending warning unrecognized_name alert to encourage immediate retry with SNI.`
+            // Missing SNI: forward to NetworkProxy if available
+            const proxyInstance = this.networkProxyBridge.getNetworkProxy();
+            if (proxyInstance) {
+              if (this.settings.enableDetailedLogging) {
+                console.log(
+                  `[${connectionId}] No SNI in ClientHello; forwarding to NetworkProxy.`
+                );
+              }
+            this.networkProxyBridge.forwardToNetworkProxy(
+              connectionId,
+              socket,
+              record,
+              chunk,
+              undefined,
+              (_reason) => {
+                // On proxy failure, send TLS unrecognized_name alert and cleanup
+                if (record.incomingTerminationReason === null) {
+                  record.incomingTerminationReason = 'session_ticket_blocked_no_sni';
+                  this.connectionManager.incrementTerminationStat(
+                    'incoming',
+                    'session_ticket_blocked_no_sni'
+                  );
+                }
+                const alert = Buffer.from([0x15, 0x03, 0x03, 0x00, 0x02, 0x01, 0x70]);
+                try { socket.cork(); socket.write(alert); socket.uncork(); socket.end(); }
+                catch { socket.end(); }
+                this.connectionManager.initiateCleanupOnce(record, 'session_ticket_blocked_no_sni');
+              }
+            );
+              return;
+            }
+            // Fallback: send TLS unrecognized_name alert and terminate
+            console.log(
+              `[${connectionId}] No SNI detected and proxy unavailable; sending TLS alert.`
             );
-
-            // Set the termination reason first
             if (record.incomingTerminationReason === null) {
               record.incomingTerminationReason = 'session_ticket_blocked_no_sni';
               this.connectionManager.incrementTerminationStat(
@@ -571,54 +599,10 @@ export class ConnectionHandler {
                 'session_ticket_blocked_no_sni'
               );
             }
-
-            // Create a warning-level alert for unrecognized_name
-            // This encourages Chrome to retry immediately with SNI
-            const serverNameUnknownAlertData = Buffer.from([
-              0x15, // Alert record type
-              0x03,
-              0x03, // TLS 1.2 version
-              0x00,
-              0x02, // Length
-              0x01, // Warning alert level (not fatal)
-              0x70, // unrecognized_name alert (code 112)
-            ]);
-
-            try {
-              // Use cork/uncork to ensure the alert is sent as a single packet
-              socket.cork();
-              const writeSuccessful = socket.write(serverNameUnknownAlertData);
-              socket.uncork();
-              socket.end();
-              
-              // Function to handle the clean socket termination - but more gradually
-              const finishConnection = () => {
-                this.connectionManager.cleanupConnection(record, 'session_ticket_blocked_no_sni');
-              };
-              
-              if (writeSuccessful) {
-                // Wait longer before ending connection to ensure alert is processed by client
-                setTimeout(finishConnection, 200); // Increased from 50ms to 200ms
-              } else {
-                // If the kernel buffer was full, wait for the drain event
-                socket.once('drain', () => {
-                  // Wait longer after drain as well
-                  setTimeout(finishConnection, 200);
-                });
-                
-                // Safety timeout is increased too
-                setTimeout(() => {
-                  socket.removeAllListeners('drain');
-                  finishConnection();
-                }, 400); // Increased from 250ms to 400ms
-              }
-            } catch (err) {
-              // If we can't send the alert, fall back to immediate termination
-              console.log(`[${connectionId}] Error sending TLS alert: ${err.message}`);
-              socket.end();
-              this.connectionManager.cleanupConnection(record, 'session_ticket_blocked_no_sni');
-            }
-
+            const alert = Buffer.from([0x15, 0x03, 0x03, 0x00, 0x02, 0x01, 0x70]);
+            try { socket.cork(); socket.write(alert); socket.uncork(); socket.end(); }
+            catch { socket.end(); }
+            this.connectionManager.cleanupConnection(record, 'session_ticket_blocked_no_sni');
             return;
           }
         }