fix(network-proxy, route-utils, route-manager): Normalize IPv6-mapped IPv4 addresses in IP matching functions and remove deprecated legacy configuration methods in NetworkProxy. Update route-utils and route-manager to compare both canonical and IPv6-mapped IP forms, adjust tests accordingly, and clean up legacy exports.

readme.plan.md

@@ -1,139 +1,103 @@
-# Enhanced NetworkProxy with Native Route-Based Configuration
+# SmartProxy Configuration Troubleshooting
 
-## Project Goal
-Transform NetworkProxy to natively use route-based configurations (`IRouteConfig`) as its primary configuration format, completely eliminating translation layers while maintaining backward compatibility through adapter methods for existing code.
+## IPv6/IPv4 Mapping Issue
 
-## Current Status
+### Problem Identified
+The SmartProxy is failing to match connections for wildcard domains (like `*.lossless.digital`) when IP restrictions are in place. After extensive debugging, the root cause has been identified:
 
-The current implementation uses:
-- SmartProxy has a rich `IRouteConfig` format with match/action pattern
-- NetworkProxy uses a simpler `IReverseProxyConfig` focused on hostname and destination
-- `NetworkProxyBridge` translates between these formats, losing information
-- Dynamic function-based hosts and ports aren't supported in NetworkProxy
-- Duplicate configuration logic exists across components
+When a connection comes in from an IPv4 address (e.g., `212.95.99.130`), the Node.js server receives it as an IPv6-mapped IPv4 address with the format `::ffff:212.95.99.130`. However, the route configuration is expecting the exact string `212.95.99.130`, causing a mismatch.
 
-## Planned Enhancements
+From the debug logs:
+```
+[DEBUG] Route rejected: clientIp mismatch. Request: ::ffff:212.95.99.130, Route patterns: ["212.95.99.130"]
+```
 
-### Phase 1: Convert NetworkProxy to Native Route Configuration
-- [x] 1.1 Refactor NetworkProxy to use `IRouteConfig` as its primary internal format
-- [x] 1.3 Update all internal processing to work directly with route configs
-- [x] 1.4 Add a type-safe context object matching SmartProxy's
-- [x] 1.5 Ensure backward compatibility for all existing NetworkProxy methods
-- [x] 1.6 Remove `IReverseProxyConfig` usage in NetworkProxy
+### Solution
 
-### Phase 2: Native Route Configuration Processing
-- [x] 2.1 Make `updateRouteConfigs(routes: IRouteConfig[])` the primary configuration method
-- [x] 2.3 Implement a full RouteManager in NetworkProxy (reusing code from SmartProxy if possible)
-- [x] 2.4 Support all route matching criteria (domains, paths, headers, clientIp)
-- [x] 2.5 Handle priority-based route matching and conflict resolution
-- [x] 2.6 Update certificate management to work with routes directly
+To fix this issue, update the route configurations to include both formats of the IP address. Here's how to modify the affected route:
 
-### Phase 3: Simplify NetworkProxyBridge
-- [x] 3.1 Update NetworkProxyBridge to directly pass route configs to NetworkProxy
-- [x] 3.2 Remove all translation/conversion logic in the bridge
-- [x] 3.3 Simplify domain registration from routes to Port80Handler
-- [x] 3.4 Make the bridge a lightweight pass-through component
-- [x] 3.5 Add comprehensive logging for route synchronization
-- [x] 3.6 Streamline certificate handling between components
+```typescript
+// Wildcard domain route for *.lossless.digital
+{
+  match: {
+    ports: 443,
+    domains: ['*.lossless.digital'],
+    clientIp: ['212.95.99.130', '::ffff:212.95.99.130'],  // Include both formats
+  },
+  action: {
+    type: 'forward',
+    target: {
+      host: '212.95.99.130',
+      port: 443
+    },
+    tls: {
+      mode: 'passthrough'
+    },
+    security: {
+      allowedIps: ['212.95.99.130', '::ffff:212.95.99.130']  // Include both formats
+    }
+  },
+  name: 'Wildcard lossless.digital route (IP restricted)'
+}
+```
 
-### Phase 4: Native Function-Based Target Support
-- [x] 4.1 Implement IRouteContext creation in NetworkProxy's request handler
-- [x] 4.2 Add direct support for function-based host evaluation
-- [x] 4.3 Add direct support for function-based port evaluation
-- [x] 4.4 Implement caching for function results to improve performance
-- [x] 4.5 Add comprehensive error handling for function execution
-- [x] 4.6 Share context object implementation with SmartProxy
+### Alternative Long-Term Fix
 
-### Phase 5: Enhanced HTTP Features Using Route Logic
-- [x] 5.1 Implement full route-based header manipulation
-- [x] 5.2 Add support for URL rewriting using route context
-- [x] 5.3 Support template variable resolution for strings
-- [x] 5.4 Implement route security features (IP filtering, rate limiting)
-- [x] 5.5 Add context-aware CORS handling
-- [x] 5.6 Enable route-based WebSocket upgrades
+A more robust solution would be to modify the SmartProxy codebase to automatically handle IPv6-mapped IPv4 addresses by normalizing them before comparison. This would involve:
 
-### Phase 6: Testing, Documentation and Code Sharing
-- [x] 6.1 Create comprehensive tests for native route configuration
-- [x] 6.2 Add specific tests for function-based targets
-- [x] 6.3 Document NetworkProxy's native route capabilities
-- [x] 6.4 Create shared utilities between SmartProxy and NetworkProxy
-- [x] 6.5 Provide migration guide for direct NetworkProxy users
-- [ ] 6.6 Benchmark performance improvements
+1. Modifying the `matchIpPattern` function in `route-manager.ts` to normalize IPv6-mapped IPv4 addresses:
 
-### Phase 7: Unify Component Architecture
-- [x] 7.1 Implement a shared RouteManager used by both SmartProxy and NetworkProxy
-- [x] 7.2 Extract common route matching logic to a shared utility module
-- [x] 7.3 Consolidate duplicate security management code
-- [x] 7.4 Remove all legacy NetworkProxyBridge conversion code
-- [x] 7.5 Make the NetworkProxyBridge a pure proxy pass-through component
-- [x] 7.6 Standardize event naming and handling across components
+```typescript
+private matchIpPattern(pattern: string, ip: string): boolean {
+  // Normalize IPv6-mapped IPv4 addresses
+  const normalizedIp = ip.startsWith('::ffff:') ? ip.substring(7) : ip;
+  const normalizedPattern = pattern.startsWith('::ffff:') ? pattern.substring(7) : pattern;
+  
+  // Handle exact match with normalized addresses
+  if (normalizedPattern === normalizedIp) {
+    return true;
+  }
+  
+  // Rest of the existing function...
+}
+```
 
-### Phase 8: Certificate Management Consolidation
-- [x] 8.1 Create a unified CertificateManager component
-- [x] 8.2 Centralize certificate storage and renewal logic
-- [x] 8.3 Simplify ACME challenge handling across proxies
-- [x] 8.4 Implement shared certificate events for all components
-- [x] 8.5 Remove redundant certificate synchronization logic
-- [x] 8.6 Standardize SNI handling between different proxies
+2. Making similar modifications to other IP-related functions in the codebase.
 
-### Phase 9: Context and Configuration Standardization
-- [x] 9.1 Implement a single shared IRouteContext class
-- [x] 9.2 Remove all duplicate context creation logic
-- [x] 9.3 Standardize option interfaces across components
-- [x] 9.4 Create shared default configurations
-- [x] 9.5 Implement a unified configuration validation system
-- [x] 9.6 Add runtime type checking for configurations
+## Wild Card Domain Matching Issue
 
-### Phase 10: Component Consolidation
-- [x] 10.1 Merge SmartProxy and NetworkProxy functionality where appropriate
-- [x] 10.2 Create a unified connection pool management system
-- [x] 10.3 Standardize timeout handling across components
-- [x] 10.4 Implement shared logging and monitoring
-- [x] 10.5 Remove all deprecated methods and legacy compatibility
-- [x] 10.6 Reduce API surface area to essentials
+### Explanation
 
-### Phase 11: Performance Optimization & Advanced Features
-- [ ] 11.1 Conduct comprehensive performance benchmarking
-- [ ] 11.2 Optimize memory usage in high-connection scenarios
-- [ ] 11.3 Implement connection pooling for backend targets
-- [ ] 11.4 Add support for HTTP/3 and QUIC protocols
-- [ ] 11.5 Enhance WebSocket support with compression and multiplexing
-- [ ] 11.6 Add advanced observability through metrics and tracing integration
+The wildcard domain matching in SmartProxy works as follows:
 
-## Benefits of Simplified Architecture
+1. When a pattern like `*.lossless.digital` is specified, it's converted to a regex: `/^.*\.lossless\.digital$/i`
+2. This correctly matches any subdomain like `my.lossless.digital`, `api.lossless.digital`, etc.
+3. However, it does NOT match the apex domain `lossless.digital` (without a subdomain)
 
-1. **Reduced Duplication**:
-   - Shared route processing logic
-   - Single certificate management system
-   - Unified context objects
+If you need to match both the apex domain and subdomains, use a list:
+```typescript
+domains: ['lossless.digital', '*.lossless.digital']
+```
 
-2. **Simplified Codebase**:
-   - Fewer managers with cleaner responsibilities
-   - Consistent APIs across components
-   - Reduced complexity in bridge components
+## Debugging SmartProxy
 
-3. **Improved Maintainability**:
-   - Easier to understand component relationships
-   - Consolidated logic for critical operations
-   - Clearer separation of concerns
+To debug routing issues in SmartProxy:
 
-4. **Enhanced Performance**:
-   - Less overhead in communication between components
-   - Reduced memory usage through shared objects
-   - More efficient request processing
+1. Add detailed logging to the `route-manager.js` file in the `dist_ts` directory:
+   - `findMatchingRoute` method - to see what criteria are being checked
+   - `matchRouteDomain` method - to see domain matching logic
+   - `matchDomain` method - to see pattern matching
+   - `matchIpPattern` method - to see IP matching logic
 
-5. **Better Developer Experience**:
-   - Consistent conceptual model across system
-   - More intuitive configuration interface
-   - Simplified debugging and troubleshooting
+2. Run the proxy with debugging enabled:
+   ```
+   pnpm run startNew
+   ```
 
-## Implementation Approach
+3. Monitor the logs for detailed information about the routing process and identify where matches are failing.
 
-The implementation of phases 7-10 will focus on gradually consolidating duplicate functionality:
+## Priority and Route Order
 
-1. First, implement shared managers and utilities to be used by both proxies
-2. Then consolidate certificate management to simplify ACME handling
-3. Create standardized context objects and configurations
-4. Finally, merge overlapping functionality between proxy components
+Remember that routes are evaluated in priority order (higher priority first). If multiple routes could match the same request, ensure that the more specific routes have higher priority.
 
-This approach will maintain compatibility with existing code while progressively simplifying the architecture to reduce complexity and improve performance.
+When routes have the same priority (or none specified), they're evaluated in the order they're defined in the configuration.