smartproxy/readme.plan.md

SmartProxy Project Restructuring Plan

Project Goal

Reorganize the SmartProxy codebase to improve maintainability, readability, and developer experience through:

1. Standardized naming conventions
2. Consistent directory structure
3. Modern TypeScript patterns
4. Clear separation of concerns

Current Architecture Analysis

Based on code analysis, SmartProxy has several well-defined but inconsistently named modules:

1. SmartProxy - Primary TCP/SNI-based proxy with configurable routing
2. NetworkProxy - HTTP/HTTPS reverse proxy with TLS termination
3. Port80Handler - HTTP port 80 handling for ACME and redirects
4. NfTablesProxy - Low-level port forwarding via nftables
5. Forwarding System - New unified configuration for all forwarding types

The codebase employs several strong design patterns:

• Factory Pattern for creating forwarding handlers
• Strategy Pattern for implementing different forwarding methods
• Manager Pattern for encapsulating domain, connection, and security logic
• Event-Driven Architecture for loose coupling between components

Target Directory Structure

/ts
├── /core                     # Core functionality
│   ├── /models               # Data models and interfaces
│   ├── /utils                # Shared utilities (IP validation, logging, etc.)
│   └── /events               # Common event definitions
├── /certificate              # Certificate management
│   ├── /acme                 # ACME-specific functionality
│   ├── /providers            # Certificate providers (static, ACME)
│   └── /storage              # Certificate storage mechanisms
├── /forwarding               # Forwarding system
│   ├── /handlers             # Various forwarding handlers
│   │   ├── base-handler.ts   # Abstract base handler
│   │   ├── http-handler.ts   # HTTP-only handler
│   │   └── ...               # Other handlers
│   ├── /config               # Configuration models
│   │   ├── forwarding-types.ts     # Type definitions
│   │   ├── domain-config.ts        # Domain config utilities
│   │   └── domain-manager.ts       # Domain routing manager
│   └── /factory              # Factory for creating handlers
├── /proxies                  # Different proxy implementations
│   ├── /smart-proxy          # SmartProxy implementation
│   │   ├── /models           # SmartProxy-specific interfaces
│   │   ├── smart-proxy.ts    # Main SmartProxy class
│   │   └── ...               # Supporting classes
│   ├── /network-proxy        # NetworkProxy implementation
│   │   ├── /models           # NetworkProxy-specific interfaces
│   │   ├── network-proxy.ts  # Main NetworkProxy class
│   │   └── ...               # Supporting classes
│   └── /nftables-proxy       # NfTablesProxy implementation
├── /tls                      # TLS-specific functionality
│   ├── /sni                  # SNI handling components
│   └── /alerts               # TLS alerts system
└── /http                     # HTTP-specific functionality
    ├── /port80               # Port80Handler components
    ├── /router               # HTTP routing system
    └── /redirects            # Redirect handlers

Implementation Plan

Phase 1: Project Setup & Core Structure (Week 1)

• Create new directory structure

  • Create core subdirectories within ts directory
  • Set up barrel files (index.ts) in each directory

• Migrate core utilities

  • Move ts/plugins.ts → ts/core/utils/plugins.ts
  • Move ts/common/types.ts → ts/core/models/common-types.ts
  • Move ts/common/eventUtils.ts → ts/core/utils/event-utils.ts
  • Extract ValidationUtils → ts/core/utils/validation-utils.ts
  • Extract IpUtils → ts/core/utils/ip-utils.ts

• Update build and test scripts

  • Modify package.json build script for new structure
  • Create parallel test structure

Phase 2: Forwarding System Migration (Weeks 1-2)

This component has the cleanest design, so we'll start migration here:

• Migrate forwarding types and interfaces

  • Move ts/smartproxy/types/forwarding.types.ts → ts/forwarding/config/forwarding-types.ts
  • Normalize interface names (remove 'I' prefix where appropriate)

• Migrate domain configuration

  • Move ts/smartproxy/forwarding/domain-config.ts → ts/forwarding/config/domain-config.ts
  • Move ts/smartproxy/forwarding/domain-manager.ts → ts/forwarding/config/domain-manager.ts

• Migrate handler implementations

  • Move base handler: forwarding.handler.ts → ts/forwarding/handlers/base-handler.ts
  • Move HTTP handler: http.handler.ts → ts/forwarding/handlers/http-handler.ts
  • Move passthrough handler: https-passthrough.handler.ts → ts/forwarding/handlers/https-passthrough-handler.ts
  • Move TLS termination handlers to respective files in ts/forwarding/handlers/
  • Move factory: forwarding.factory.ts → ts/forwarding/factory/forwarding-factory.ts

• Create proper forwarding system exports

  • Update all imports in forwarding components using relative paths
  • Create comprehensive barrel file in ts/forwarding/index.ts
  • Test forwarding system in isolation

Phase 3: Certificate Management Migration (Week 2)

• Create certificate management structure

  • Create ts/certificate/models/certificate-types.ts for interfaces
  • Extract certificate events to ts/certificate/events/

• Migrate certificate providers

  • Move ts/classes.pp.certprovisioner.ts → ts/certificate/providers/cert-provisioner.ts
  • Move ts/common/acmeFactory.ts → ts/certificate/acme/acme-factory.ts
  • Extract ACME challenge handling to ts/certificate/acme/challenge-handler.ts

• Update certificate utilities

  • Move ts/helpers.certificates.ts → ts/certificate/utils/certificate-helpers.ts
  • Create proper exports in ts/certificate/index.ts

Phase 4: TLS & SNI Handling Migration (Week 2-3)

• Migrate TLS alert system

  • Move ts/smartproxy/classes.pp.tlsalert.ts → ts/tls/alerts/tls-alert.ts
  • Extract common TLS utilities to ts/tls/utils/tls-utils.ts

• Migrate SNI handling

  • Move ts/smartproxy/classes.pp.snihandler.ts → ts/tls/sni/sni-handler.ts
  • Extract SNI extraction to ts/tls/sni/sni-extraction.ts
  • Extract ClientHello parsing to ts/tls/sni/client-hello-parser.ts

Phase 5: HTTP Component Migration (Week 3)

• Migrate Port80Handler

  • Move ts/port80handler/classes.port80handler.ts → ts/http/port80/port80-handler.ts
  • Extract ACME challenge handling to ts/http/port80/challenge-responder.ts

• Migrate redirect handlers

  • Move ts/redirect/classes.redirect.ts → ts/http/redirects/redirect-handler.ts
  • Create ts/http/redirects/ssl-redirect.ts for specialized redirects

• Migrate router components

  • Move ts/classes.router.ts → ts/http/router/proxy-router.ts
  • Extract route matching to ts/http/router/route-matcher.ts

Phase 6: Proxy Implementation Migration (Weeks 3-4)

• Migrate SmartProxy components

  • First, migrate interfaces to ts/proxies/smart-proxy/models/
  • Move core class: ts/smartproxy/classes.smartproxy.ts → ts/proxies/smart-proxy/smart-proxy.ts
  • Move supporting classes using consistent naming
  • Normalize interface names (SmartProxyOptions instead of IPortProxySettings)

• Migrate NetworkProxy components

  • First, migrate interfaces to ts/proxies/network-proxy/models/
  • Move core class: ts/networkproxy/classes.np.networkproxy.ts → ts/proxies/network-proxy/network-proxy.ts
  • Move supporting classes using consistent naming

• Migrate NfTablesProxy

  • Move ts/nfttablesproxy/classes.nftablesproxy.ts → ts/proxies/nftables-proxy/nftables-proxy.ts

Phase 7: Integration & Main Module (Week 4-5)

• Create main entry points

  • Update ts/index.ts with all public exports
  • Ensure backward compatibility with type aliases
  • Implement proper namespace exports

• Update module dependencies

  • Update relative import paths in all modules
  • Resolve circular dependencies if found
  • Test cross-module integration

Phase 8: Interface Normalization (Week 5)

• Standardize interface naming

  • Rename IPortProxySettings → SmartProxyOptions
  • Rename IDomainConfig → DomainConfig
  • Rename IConnectionRecord → ConnectionRecord
  • Rename INetworkProxyOptions → NetworkProxyOptions
  • Rename other interfaces for consistency

• Provide backward compatibility

  • Add type aliases for renamed interfaces
  • Ensure all exports are compatible with existing code

Phase 9: Testing & Validation (Weeks 5-6)

• Reorganize test structure

  • Create test directories matching source structure
  • Move tests to appropriate directories
  • Update test imports and references

• Add test coverage for new components

  • Create unit tests for extracted utilities
  • Ensure integration tests cover all scenarios
  • Validate backward compatibility

Phase 10: Documentation (Weeks 6-7)

• Update core documentation

  • Update README.md with new structure and examples
  • Create architecture diagram showing component relationships
  • Document import patterns and best practices

• Create specialized documentation

  • ARCHITECTURE.md for system overview
  • FORWARDING.md for forwarding system specifics
  • CERTIFICATE.md for certificate management details
  • DEVELOPMENT.md for contributor guidelines

• Update example files

  • Update existing examples to use new structure
  • Add new examples demonstrating key scenarios

Phase 11: Release & Migration Guide (Week 8)

• Prepare for release

  • Final testing and validation
  • Performance comparison with previous version
  • Create detailed changelog

• Create migration guide

  • Document breaking changes
  • Provide upgrade instructions
  • Include code examples for common scenarios

Detailed File Migration Table

Current File	New File	Status
Core/Common Files
ts/common/types.ts	ts/core/models/common-types.ts	❌
ts/common/eventUtils.ts	ts/core/utils/event-utils.ts	❌
ts/common/acmeFactory.ts	ts/certificate/acme/acme-factory.ts	❌
ts/plugins.ts	ts/core/utils/plugins.ts	❌
ts/00_commitinfo_data.ts	ts/core/utils/commit-info.ts	❌
Certificate Management
ts/helpers.certificates.ts	ts/certificate/utils/certificate-helpers.ts	❌
ts/classes.pp.certprovisioner.ts	ts/certificate/providers/cert-provisioner.ts	❌
TLS and SNI Handling
ts/smartproxy/classes.pp.tlsalert.ts	ts/tls/alerts/tls-alert.ts	❌
ts/smartproxy/classes.pp.snihandler.ts	ts/tls/sni/sni-handler.ts	❌
HTTP Components
ts/port80handler/classes.port80handler.ts	ts/http/port80/port80-handler.ts	❌
ts/redirect/classes.redirect.ts	ts/http/redirects/redirect-handler.ts	❌
ts/classes.router.ts	ts/http/router/proxy-router.ts	❌
SmartProxy Components
ts/smartproxy/classes.smartproxy.ts	ts/proxies/smart-proxy/smart-proxy.ts	❌
ts/smartproxy/classes.pp.interfaces.ts	ts/proxies/smart-proxy/models/interfaces.ts	❌
ts/smartproxy/classes.pp.connectionhandler.ts	ts/proxies/smart-proxy/connection-handler.ts	❌
ts/smartproxy/classes.pp.connectionmanager.ts	ts/proxies/smart-proxy/connection-manager.ts	❌
ts/smartproxy/classes.pp.domainconfigmanager.ts	ts/proxies/smart-proxy/domain-config-manager.ts	❌
ts/smartproxy/classes.pp.portrangemanager.ts	ts/proxies/smart-proxy/port-range-manager.ts	❌
ts/smartproxy/classes.pp.securitymanager.ts	ts/proxies/smart-proxy/security-manager.ts	❌
ts/smartproxy/classes.pp.timeoutmanager.ts	ts/proxies/smart-proxy/timeout-manager.ts	❌
ts/smartproxy/classes.pp.networkproxybridge.ts	ts/proxies/smart-proxy/network-proxy-bridge.ts	❌
NetworkProxy Components
ts/networkproxy/classes.np.networkproxy.ts	ts/proxies/network-proxy/network-proxy.ts	❌
ts/networkproxy/classes.np.certificatemanager.ts	ts/proxies/network-proxy/certificate-manager.ts	❌
ts/networkproxy/classes.np.connectionpool.ts	ts/proxies/network-proxy/connection-pool.ts	❌
ts/networkproxy/classes.np.requesthandler.ts	ts/proxies/network-proxy/request-handler.ts	❌
ts/networkproxy/classes.np.websockethandler.ts	ts/proxies/network-proxy/websocket-handler.ts	❌
ts/networkproxy/classes.np.types.ts	ts/proxies/network-proxy/models/types.ts	❌
NFTablesProxy Components
ts/nfttablesproxy/classes.nftablesproxy.ts	ts/proxies/nftables-proxy/nftables-proxy.ts	❌
Forwarding System
ts/smartproxy/types/forwarding.types.ts	ts/forwarding/config/forwarding-types.ts	❌
ts/smartproxy/forwarding/domain-config.ts	ts/forwarding/config/domain-config.ts	❌
ts/smartproxy/forwarding/domain-manager.ts	ts/forwarding/config/domain-manager.ts	❌
ts/smartproxy/forwarding/forwarding.handler.ts	ts/forwarding/handlers/base-handler.ts	❌
ts/smartproxy/forwarding/http.handler.ts	ts/forwarding/handlers/http-handler.ts	❌
ts/smartproxy/forwarding/https-passthrough.handler.ts	ts/forwarding/handlers/https-passthrough-handler.ts	❌
ts/smartproxy/forwarding/https-terminate-to-http.handler.ts	ts/forwarding/handlers/https-terminate-to-http-handler.ts	❌
ts/smartproxy/forwarding/https-terminate-to-https.handler.ts	ts/forwarding/handlers/https-terminate-to-https-handler.ts	❌
ts/smartproxy/forwarding/forwarding.factory.ts	ts/forwarding/factory/forwarding-factory.ts	❌
ts/smartproxy/forwarding/index.ts	ts/forwarding/index.ts	❌
Examples and Entry Points
ts/examples/forwarding-example.ts	ts/examples/forwarding-example.ts	❌
ts/index.ts	ts/index.ts (updated)	❌

Import Strategy

Since path aliases will not be used, we'll maintain standard relative imports throughout the codebase:

1. Import Strategy for Deeply Nested Files

   // Example: Importing from another component in a nested directory
   // From ts/forwarding/handlers/http-handler.ts to ts/core/utils/validation-utils.ts
   import { validateConfig } from '../../../core/utils/validation-utils.js';
   

2. Barrel Files for Convenience

   // ts/forwarding/index.ts
   export * from './config/forwarding-types.js';
   export * from './handlers/base-handler.js';
   // ... other exports
   
   // Then in consuming code:
   import { ForwardingHandler, httpOnly } from '../../forwarding/index.js';
   

3. Flattened Imports Where Sensible

   // Avoid excessive nesting with targeted exports
   // ts/index.ts will export key components for external use
   import { SmartProxy, NetworkProxy } from '../index.js';
   

Expected Outcomes

Improved Code Organization

• Related code will be grouped together in domain-specific directories
• Consistent naming conventions will make code navigation intuitive
• Clear module boundaries will prevent unintended dependencies

Enhanced Developer Experience

• Standardized interface naming will improve type clarity
• Better documentation will help new contributors get started
• Clear and predictable file locations

Maintainability Benefits

• Smaller, focused files with clear responsibilities
• Unified patterns for common operations
• Improved separation of concerns between components
• Better test organization matching source structure

Performance and Compatibility

• No performance regression from structural changes
• Backward compatibility through type aliases and consistent exports
• Clear migration path for dependent projects

Migration Strategy

To ensure a smooth transition, we'll follow this approach for each component:

1. Create the new file structure first
2. Migrate code while updating relative imports
3. Test each component as it's migrated
4. Only remove old files once all dependencies are updated
5. Use a phased approach to allow parallel work

This approach ensures the codebase remains functional throughout the restructuring process while progressively adopting the new organization.

Measuring Success

We'll measure the success of this restructuring by:

1. Reduced complexity in the directory structure
2. Improved code coverage through better test organization
3. Faster onboarding time for new developers
4. Less time spent navigating the codebase
5. Cleaner git blame output showing cohesive component changes

Special Considerations

• We'll maintain backward compatibility for all public APIs
• We'll provide detailed upgrade guides for any breaking changes
• We'll ensure the build process produces compatible output
• We'll preserve commit history using git move operations where possible