fix(routing): unify route based architecture

readme.plan.md

@@ -1,168 +1,139 @@
-# SmartProxy Complete Route-Based Implementation Plan
+# Enhanced NetworkProxy with Native Route-Based Configuration
 
 ## Project Goal
-Complete the refactoring of SmartProxy to a pure route-based configuration approach by:
-1. Removing all remaining domain-based configuration code with no backward compatibility
-2. Updating internal components to work directly and exclusively with route configurations
-3. Eliminating all conversion functions and domain-based interfaces
-4. Cleaning up deprecated methods and interfaces completely
-5. Focusing entirely on route-based helper functions for the best developer experience
+Transform NetworkProxy to natively use route-based configurations (`IRouteConfig`) as its primary configuration format, completely eliminating translation layers while maintaining backward compatibility through adapter methods for existing code.
 
 ## Current Status
-The major refactoring to route-based configuration has been successfully completed:
-- SmartProxy now works exclusively with route-based configurations in its public API
-- All test files have been updated to use route-based configurations
-- Documentation has been updated to explain the route-based approach
-- Helper functions have been implemented for creating route configurations
-- All features are working correctly with the new approach
 
-### Completed Phases:
-1. ✅ **Phase 1:** CertProvisioner has been fully refactored to work natively with routes
-2. ✅ **Phase 2:** NetworkProxyBridge now works directly with route configurations
-3. ✅ **Phase 3:** Legacy domain configuration code has been removed
-4. ✅ **Phase 4:** Route helpers and configuration experience have been enhanced
-5. ✅ **Phase 5:** Tests and validation have been completed
+The current implementation uses:
+- SmartProxy has a rich `IRouteConfig` format with match/action pattern
+- NetworkProxy uses a simpler `IReverseProxyConfig` focused on hostname and destination
+- `NetworkProxyBridge` translates between these formats, losing information
+- Dynamic function-based hosts and ports aren't supported in NetworkProxy
+- Duplicate configuration logic exists across components
 
-### Project Status:
-✅ COMPLETED (May 10, 2025): SmartProxy has been fully refactored to a pure route-based configuration approach with no backward compatibility for domain-based configurations.
+## Planned Enhancements
 
-## Implementation Checklist
+### Phase 1: Convert NetworkProxy to Native Route Configuration
+- [x] 1.1 Refactor NetworkProxy to use `IRouteConfig` as its primary internal format
+- [x] 1.3 Update all internal processing to work directly with route configs
+- [x] 1.4 Add a type-safe context object matching SmartProxy's
+- [x] 1.5 Ensure backward compatibility for all existing NetworkProxy methods
+- [x] 1.6 Remove `IReverseProxyConfig` usage in NetworkProxy
 
-### Phase 1: Refactor CertProvisioner for Native Route Support ✅
-- [x] 1.1 Update CertProvisioner constructor to store routeConfigs directly
-- [x] 1.2 Remove extractDomainsFromRoutes() method and domainConfigs array
-- [x] 1.3 Create extractCertificateRoutesFromRoutes() method to find routes needing certificates
-- [x] 1.4 Update provisionAllDomains() to work with route configurations
-- [x] 1.5 Update provisionDomain() to handle route configs
-- [x] 1.6 Modify renewal tracking to use routes instead of domains
-- [x] 1.7 Update renewals scheduling to use route-based approach
-- [x] 1.8 Refactor requestCertificate() method to use routes
-- [x] 1.9 Update ICertificateData interface to include route references
-- [x] 1.10 Update certificate event handling to include route information
-- [x] 1.11 Add unit tests for route-based certificate provisioning
-- [x] 1.12 Add tests for wildcard domain handling with routes
-- [x] 1.13 Test certificate renewal with route configurations
-- [x] 1.14 Update certificate-types.ts to remove domain-based types
+### Phase 2: Native Route Configuration Processing
+- [x] 2.1 Make `updateRouteConfigs(routes: IRouteConfig[])` the primary configuration method
+- [x] 2.3 Implement a full RouteManager in NetworkProxy (reusing code from SmartProxy if possible)
+- [x] 2.4 Support all route matching criteria (domains, paths, headers, clientIp)
+- [x] 2.5 Handle priority-based route matching and conflict resolution
+- [x] 2.6 Update certificate management to work with routes directly
 
-### Phase 2: Refactor NetworkProxyBridge for Direct Route Processing ✅
-- [x] 2.1 Update NetworkProxyBridge constructor to work directly with routes
-- [x] 2.2 Refactor syncRoutesToNetworkProxy() to eliminate domain conversion
-- [x] 2.3 Rename convertRoutesToNetworkProxyConfigs() to mapRoutesToNetworkProxyConfigs()
-- [x] 2.4 Maintain syncDomainConfigsToNetworkProxy() as deprecated wrapper
-- [x] 2.5 Implement direct mapping from routes to NetworkProxy configs
-- [x] 2.6 Update handleCertificateEvent() to work with routes
-- [x] 2.7 Update applyExternalCertificate() to use route information
-- [x] 2.8 Update registerDomainsWithPort80Handler() to extract domains from routes
-- [x] 2.9 Update certificate request flow to track route references
-- [x] 2.10 Test NetworkProxyBridge with pure route configurations
-- [x] 2.11 Successfully build and run all tests
+### Phase 3: Simplify NetworkProxyBridge
+- [x] 3.1 Update NetworkProxyBridge to directly pass route configs to NetworkProxy
+- [x] 3.2 Remove all translation/conversion logic in the bridge
+- [x] 3.3 Simplify domain registration from routes to Port80Handler
+- [x] 3.4 Make the bridge a lightweight pass-through component
+- [x] 3.5 Add comprehensive logging for route synchronization
+- [x] 3.6 Streamline certificate handling between components
 
-### Phase 3: Remove Legacy Domain Configuration Code
-- [x] 3.1 Identify all imports of domain-config.ts and update them
-- [x] 3.2 Create route-based alternatives for any remaining domain-config usage
-- [x] 3.3 Delete domain-config.ts
-- [x] 3.4 Identify all imports of domain-manager.ts and update them
-- [x] 3.5 Delete domain-manager.ts
-- [x] 3.6 Update forwarding-types.ts (route-based only)
-- [x] 3.7 Add route-based domain support to Port80Handler
-- [x] 3.8 Create IPort80RouteOptions and extractPort80RoutesFromRoutes utility
-- [x] 3.9 Update SmartProxy.ts to use route-based domain management
-- [x] 3.10 Provide compatibility layer for domain-based interfaces
-- [x] 3.11 Update IDomainForwardConfig to IRouteForwardConfig
-- [x] 3.12 Update JSDoc comments to reference routes instead of domains
-- [x] 3.13 Run build to find any remaining type errors
-- [x] 3.14 Fix all type errors to ensure successful build
-- [x] 3.15 Update tests to use route-based approach instead of domain-based
-- [x] 3.16 Fix all failing tests
-- [x] 3.17 Verify build and test suite pass successfully
+### Phase 4: Native Function-Based Target Support
+- [x] 4.1 Implement IRouteContext creation in NetworkProxy's request handler
+- [x] 4.2 Add direct support for function-based host evaluation
+- [x] 4.3 Add direct support for function-based port evaluation
+- [x] 4.4 Implement caching for function results to improve performance
+- [x] 4.5 Add comprehensive error handling for function execution
+- [x] 4.6 Share context object implementation with SmartProxy
 
-### Phase 4: Enhance Route Helpers and Configuration Experience ✅
-- [x] 4.1 Create route-validators.ts with validation functions
-- [x] 4.2 Add validateRouteConfig() function for configuration validation
-- [x] 4.3 Add mergeRouteConfigs() utility function
-- [x] 4.4 Add findMatchingRoutes() helper function
-- [x] 4.5 Expand createStaticFileRoute() with more options
-- [x] 4.6 Add createApiRoute() helper for API gateway patterns
-- [x] 4.7 Add createAuthRoute() for authentication configurations
-- [x] 4.8 Add createWebSocketRoute() helper for WebSocket support
-- [x] 4.9 Create routePatterns.ts with common route patterns
-- [x] 4.10 Update utils/index.ts to export all helpers
-- [x] 4.11 Add schema validation for route configurations
-- [x] 4.12 Create utils for route pattern testing
-- [x] 4.13 Update docs with pure route-based examples
-- [x] 4.14 Remove any legacy code examples from documentation
+### Phase 5: Enhanced HTTP Features Using Route Logic
+- [x] 5.1 Implement full route-based header manipulation
+- [x] 5.2 Add support for URL rewriting using route context
+- [x] 5.3 Support template variable resolution for strings
+- [x] 5.4 Implement route security features (IP filtering, rate limiting)
+- [x] 5.5 Add context-aware CORS handling
+- [x] 5.6 Enable route-based WebSocket upgrades
 
-### Phase 5: Testing and Validation ✅
-- [x] 5.1 Update all tests to use pure route-based components
-- [x] 5.2 Create test cases for potential edge cases
-- [x] 5.3 Create a test for domain wildcard handling
-- [x] 5.4 Test all helper functions
-- [x] 5.5 Test certificate provisioning with routes
-- [x] 5.6 Test NetworkProxy integration with routes
-- [x] 5.7 Benchmark route matching performance
-- [x] 5.8 Compare memory usage before and after changes
-- [x] 5.9 Optimize route operations for large configurations
-- [x] 5.10 Verify public API matches documentation
-- [x] 5.11 Check for any backward compatibility issues
-- [x] 5.12 Ensure all examples in README work correctly
-- [x] 5.13 Run full test suite with new implementation
-- [x] 5.14 Create a final PR with all changes
+### Phase 6: Testing, Documentation and Code Sharing
+- [x] 6.1 Create comprehensive tests for native route configuration
+- [x] 6.2 Add specific tests for function-based targets
+- [x] 6.3 Document NetworkProxy's native route capabilities
+- [x] 6.4 Create shared utilities between SmartProxy and NetworkProxy
+- [x] 6.5 Provide migration guide for direct NetworkProxy users
+- [ ] 6.6 Benchmark performance improvements
 
-## Clean Break Approach
+### Phase 7: Unify Component Architecture
+- [x] 7.1 Implement a shared RouteManager used by both SmartProxy and NetworkProxy
+- [x] 7.2 Extract common route matching logic to a shared utility module
+- [x] 7.3 Consolidate duplicate security management code
+- [x] 7.4 Remove all legacy NetworkProxyBridge conversion code
+- [x] 7.5 Make the NetworkProxyBridge a pure proxy pass-through component
+- [x] 7.6 Standardize event naming and handling across components
 
-To keep our codebase as clean as possible, we are taking a clean break approach with NO migration or compatibility support for domain-based configuration. We will:
+### Phase 8: Certificate Management Consolidation
+- [x] 8.1 Create a unified CertificateManager component
+- [x] 8.2 Centralize certificate storage and renewal logic
+- [x] 8.3 Simplify ACME challenge handling across proxies
+- [x] 8.4 Implement shared certificate events for all components
+- [x] 8.5 Remove redundant certificate synchronization logic
+- [x] 8.6 Standardize SNI handling between different proxies
 
-1. Completely remove all domain-based code
-2. Not provide any migration utilities in the codebase
-3. Focus solely on the route-based approach
-4. Document the route-based API as the only supported method
+### Phase 9: Context and Configuration Standardization
+- [x] 9.1 Implement a single shared IRouteContext class
+- [x] 9.2 Remove all duplicate context creation logic
+- [x] 9.3 Standardize option interfaces across components
+- [x] 9.4 Create shared default configurations
+- [x] 9.5 Implement a unified configuration validation system
+- [x] 9.6 Add runtime type checking for configurations
 
-This approach prioritizes codebase clarity over backward compatibility, which is appropriate since we've already made a clean break in the public API with v14.0.0.
+### Phase 10: Component Consolidation
+- [x] 10.1 Merge SmartProxy and NetworkProxy functionality where appropriate
+- [x] 10.2 Create a unified connection pool management system
+- [x] 10.3 Standardize timeout handling across components
+- [x] 10.4 Implement shared logging and monitoring
+- [x] 10.5 Remove all deprecated methods and legacy compatibility
+- [x] 10.6 Reduce API surface area to essentials
 
-## File Changes
+### Phase 11: Performance Optimization & Advanced Features
+- [ ] 11.1 Conduct comprehensive performance benchmarking
+- [ ] 11.2 Optimize memory usage in high-connection scenarios
+- [ ] 11.3 Implement connection pooling for backend targets
+- [ ] 11.4 Add support for HTTP/3 and QUIC protocols
+- [ ] 11.5 Enhance WebSocket support with compression and multiplexing
+- [ ] 11.6 Add advanced observability through metrics and tracing integration
 
-### Files to Delete (Remove Completely)
-- [x] `/ts/forwarding/config/domain-config.ts` - Deleted with no replacement
-- [x] `/ts/forwarding/config/domain-manager.ts` - Deleted with no replacement
-- [x] `/ts/forwarding/config/forwarding-types.ts` - Updated with pure route-based types
-- [x] Any domain-config related tests have been updated to use route-based approach
+## Benefits of Simplified Architecture
 
-### Files to Modify (Remove All Domain References)
-- [x] `/ts/certificate/providers/cert-provisioner.ts` - Complete rewrite to use routes only ✅
-- [x] `/ts/proxies/smart-proxy/network-proxy-bridge.ts` - Direct route processing implementation ✅
-- [x] `/ts/certificate/models/certificate-types.ts` - Updated with route-based interfaces ✅
-- [x] `/ts/certificate/index.ts` - Cleaned up domain-related types and exports
-- [x] `/ts/http/port80/port80-handler.ts` - Updated to work exclusively with routes
-- [x] `/ts/proxies/smart-proxy/smart-proxy.ts` - Removed domain references
-- [x] `test/test.forwarding.ts` - Updated to use route-based approach
-- [x] `test/test.forwarding.unit.ts` - Updated to use route-based approach
+1. **Reduced Duplication**:
+   - Shared route processing logic
+   - Single certificate management system
+   - Unified context objects
 
-### New Files to Create (Route-Focused)
-- [x] `/ts/proxies/smart-proxy/utils/route-helpers.ts` - Created with helper functions for common route configurations
-- [x] `/ts/proxies/smart-proxy/utils/route-migration-utils.ts` - Added migration utilities from domains to routes
-- [x] `/ts/proxies/smart-proxy/utils/route-validators.ts` - Validation utilities for route configurations
-- [x] `/ts/proxies/smart-proxy/utils/route-utils.ts` - Additional route utility functions
-- [x] `/ts/proxies/smart-proxy/utils/route-patterns.ts` - Common route patterns for easy configuration
-- [x] `/ts/proxies/smart-proxy/utils/index.ts` - Central export point for all route utilities
+2. **Simplified Codebase**:
+   - Fewer managers with cleaner responsibilities
+   - Consistent APIs across components
+   - Reduced complexity in bridge components
 
-## Benefits of Complete Refactoring
+3. **Improved Maintainability**:
+   - Easier to understand component relationships
+   - Consolidated logic for critical operations
+   - Clearer separation of concerns
 
-1. **Codebase Simplicity**:
-   - No dual implementation or conversion logic
-   - Simplified mental model for developers
-   - Easier to maintain and extend
+4. **Enhanced Performance**:
+   - Less overhead in communication between components
+   - Reduced memory usage through shared objects
+   - More efficient request processing
 
-2. **Performance Improvements**:
-   - Remove conversion overhead
-   - More efficient route matching
-   - Reduced memory footprint
+5. **Better Developer Experience**:
+   - Consistent conceptual model across system
+   - More intuitive configuration interface
+   - Simplified debugging and troubleshooting
 
-3. **Better Developer Experience**:
-   - Consistent API throughout
-   - Cleaner documentation
-   - More intuitive configuration patterns
+## Implementation Approach
 
-4. **Future-Proof Design**:
-   - Clear foundation for new features
-   - Easier to implement advanced routing capabilities
-   - Better integration with modern web patterns
+The implementation of phases 7-10 will focus on gradually consolidating duplicate functionality:
+
+1. First, implement shared managers and utilities to be used by both proxies
+2. Then consolidate certificate management to simplify ACME handling
+3. Create standardized context objects and configurations
+4. Finally, merge overlapping functionality between proxy components
+
+This approach will maintain compatibility with existing code while progressively simplifying the architecture to reduce complexity and improve performance.