smartproxy/readme.plan.md

# SmartProxy Project Restructuring Plan

## Project Goal
Reorganize the SmartProxy codebase to improve maintainability, readability, and developer experience through:
1. Standardized naming conventions
2. Consistent directory structure
3. Modern TypeScript patterns
4. Clear separation of concerns

## Current Architecture Analysis

Based on code analysis, SmartProxy has several well-defined but inconsistently named modules:

1. **SmartProxy** - Primary TCP/SNI-based proxy with configurable routing
2. **NetworkProxy** - HTTP/HTTPS reverse proxy with TLS termination
3. **Port80Handler** - HTTP port 80 handling for ACME and redirects
4. **NfTablesProxy** - Low-level port forwarding via nftables
5. **Forwarding System** - New unified configuration for all forwarding types

The codebase employs several strong design patterns:
- **Factory Pattern** for creating forwarding handlers
- **Strategy Pattern** for implementing different forwarding methods
- **Manager Pattern** for encapsulating domain, connection, and security logic
- **Event-Driven Architecture** for loose coupling between components

## Target Directory Structure

```
/ts
├── /core                     # Core functionality
│   ├── /models               # Data models and interfaces
│   ├── /utils                # Shared utilities (IP validation, logging, etc.)
│   └── /events               # Common event definitions
├── /certificate              # Certificate management
│   ├── /acme                 # ACME-specific functionality
│   ├── /providers            # Certificate providers (static, ACME)
│   └── /storage              # Certificate storage mechanisms
├── /forwarding               # Forwarding system
│   ├── /handlers             # Various forwarding handlers
│   │   ├── base-handler.ts   # Abstract base handler
│   │   ├── http-handler.ts   # HTTP-only handler
│   │   └── ...               # Other handlers
│   ├── /config               # Configuration models
│   │   ├── forwarding-types.ts     # Type definitions
│   │   ├── domain-config.ts        # Domain config utilities
│   │   └── domain-manager.ts       # Domain routing manager
│   └── /factory              # Factory for creating handlers
├── /proxies                  # Different proxy implementations
│   ├── /smart-proxy          # SmartProxy implementation
│   │   ├── /models           # SmartProxy-specific interfaces
│   │   ├── smart-proxy.ts    # Main SmartProxy class
│   │   └── ...               # Supporting classes
│   ├── /network-proxy        # NetworkProxy implementation
│   │   ├── /models           # NetworkProxy-specific interfaces
│   │   ├── network-proxy.ts  # Main NetworkProxy class
│   │   └── ...               # Supporting classes
│   └── /nftables-proxy       # NfTablesProxy implementation
├── /tls                      # TLS-specific functionality
│   ├── /sni                  # SNI handling components
│   └── /alerts               # TLS alerts system
└── /http                     # HTTP-specific functionality
    ├── /port80               # Port80Handler components
    ├── /router               # HTTP routing system
    └── /redirects            # Redirect handlers
```

## Implementation Plan

### Phase 1: Project Setup & Core Structure (Week 1)

- [ ] Create new directory structure
  - [ ] Create core subdirectories within `ts` directory
  - [ ] Set up barrel files (`index.ts`) in each directory

- [ ] Migrate core utilities
  - [ ] Move `ts/plugins.ts` → `ts/core/utils/plugins.ts`
  - [ ] Move `ts/common/types.ts` → `ts/core/models/common-types.ts`
  - [ ] Move `ts/common/eventUtils.ts` → `ts/core/utils/event-utils.ts`
  - [ ] Extract `ValidationUtils` → `ts/core/utils/validation-utils.ts`
  - [ ] Extract `IpUtils` → `ts/core/utils/ip-utils.ts`

- [ ] Update build and test scripts
  - [ ] Modify `package.json` build script for new structure
  - [ ] Create parallel test structure

### Phase 2: Forwarding System Migration (Weeks 1-2)

This component has the cleanest design, so we'll start migration here:

- [ ] Migrate forwarding types and interfaces
  - [ ] Move `ts/smartproxy/types/forwarding.types.ts` → `ts/forwarding/config/forwarding-types.ts`
  - [ ] Normalize interface names (remove 'I' prefix where appropriate)

- [ ] Migrate domain configuration
  - [ ] Move `ts/smartproxy/forwarding/domain-config.ts` → `ts/forwarding/config/domain-config.ts`
  - [ ] Move `ts/smartproxy/forwarding/domain-manager.ts` → `ts/forwarding/config/domain-manager.ts`

- [ ] Migrate handler implementations
  - [ ] Move base handler: `forwarding.handler.ts` → `ts/forwarding/handlers/base-handler.ts`
  - [ ] Move HTTP handler: `http.handler.ts` → `ts/forwarding/handlers/http-handler.ts`
  - [ ] Move passthrough handler: `https-passthrough.handler.ts` → `ts/forwarding/handlers/https-passthrough-handler.ts`
  - [ ] Move TLS termination handlers to respective files in `ts/forwarding/handlers/`
  - [ ] Move factory: `forwarding.factory.ts` → `ts/forwarding/factory/forwarding-factory.ts`

- [ ] Create proper forwarding system exports
  - [ ] Update all imports in forwarding components using relative paths
  - [ ] Create comprehensive barrel file in `ts/forwarding/index.ts`
  - [ ] Test forwarding system in isolation

### Phase 3: Certificate Management Migration (Week 2)

- [ ] Create certificate management structure
  - [ ] Create `ts/certificate/models/certificate-types.ts` for interfaces
  - [ ] Extract certificate events to `ts/certificate/events/`

- [ ] Migrate certificate providers
  - [ ] Move `ts/classes.pp.certprovisioner.ts` → `ts/certificate/providers/cert-provisioner.ts`
  - [ ] Move `ts/common/acmeFactory.ts` → `ts/certificate/acme/acme-factory.ts`
  - [ ] Extract ACME challenge handling to `ts/certificate/acme/challenge-handler.ts`

- [ ] Update certificate utilities
  - [ ] Move `ts/helpers.certificates.ts` → `ts/certificate/utils/certificate-helpers.ts`
  - [ ] Create proper exports in `ts/certificate/index.ts`

### Phase 4: TLS & SNI Handling Migration (Week 2-3)

- [ ] Migrate TLS alert system
  - [ ] Move `ts/smartproxy/classes.pp.tlsalert.ts` → `ts/tls/alerts/tls-alert.ts`
  - [ ] Extract common TLS utilities to `ts/tls/utils/tls-utils.ts`

- [ ] Migrate SNI handling
  - [ ] Move `ts/smartproxy/classes.pp.snihandler.ts` → `ts/tls/sni/sni-handler.ts`
  - [ ] Extract SNI extraction to `ts/tls/sni/sni-extraction.ts`
  - [ ] Extract ClientHello parsing to `ts/tls/sni/client-hello-parser.ts`

### Phase 5: HTTP Component Migration (Week 3)

- [ ] Migrate Port80Handler
  - [ ] Move `ts/port80handler/classes.port80handler.ts` → `ts/http/port80/port80-handler.ts`
  - [ ] Extract ACME challenge handling to `ts/http/port80/challenge-responder.ts`

- [ ] Migrate redirect handlers
  - [ ] Move `ts/redirect/classes.redirect.ts` → `ts/http/redirects/redirect-handler.ts`
  - [ ] Create `ts/http/redirects/ssl-redirect.ts` for specialized redirects

- [ ] Migrate router components
  - [ ] Move `ts/classes.router.ts` → `ts/http/router/proxy-router.ts`
  - [ ] Extract route matching to `ts/http/router/route-matcher.ts`

### Phase 6: Proxy Implementation Migration (Weeks 3-4)

- [ ] Migrate SmartProxy components
  - [ ] First, migrate interfaces to `ts/proxies/smart-proxy/models/`
  - [ ] Move core class: `ts/smartproxy/classes.smartproxy.ts` → `ts/proxies/smart-proxy/smart-proxy.ts`
  - [ ] Move supporting classes using consistent naming
  - [ ] Normalize interface names (SmartProxyOptions instead of IPortProxySettings)

- [ ] Migrate NetworkProxy components
  - [ ] First, migrate interfaces to `ts/proxies/network-proxy/models/`
  - [ ] Move core class: `ts/networkproxy/classes.np.networkproxy.ts` → `ts/proxies/network-proxy/network-proxy.ts`
  - [ ] Move supporting classes using consistent naming

- [ ] Migrate NfTablesProxy
  - [ ] Move `ts/nfttablesproxy/classes.nftablesproxy.ts` → `ts/proxies/nftables-proxy/nftables-proxy.ts`

### Phase 7: Integration & Main Module (Week 4-5)

- [ ] Create main entry points
  - [ ] Update `ts/index.ts` with all public exports
  - [ ] Ensure backward compatibility with type aliases
  - [ ] Implement proper namespace exports

- [ ] Update module dependencies
  - [ ] Update relative import paths in all modules
  - [ ] Resolve circular dependencies if found
  - [ ] Test cross-module integration

### Phase 8: Interface Normalization (Week 5)

- [ ] Standardize interface naming
  - [ ] Rename `IPortProxySettings` → `SmartProxyOptions`
  - [ ] Rename `IDomainConfig` → `DomainConfig`
  - [ ] Rename `IConnectionRecord` → `ConnectionRecord`
  - [ ] Rename `INetworkProxyOptions` → `NetworkProxyOptions`
  - [ ] Rename other interfaces for consistency

- [ ] Provide backward compatibility
  - [ ] Add type aliases for renamed interfaces
  - [ ] Ensure all exports are compatible with existing code

### Phase 9: Testing & Validation (Weeks 5-6)

- [ ] Reorganize test structure
  - [ ] Create test directories matching source structure
  - [ ] Move tests to appropriate directories
  - [ ] Update test imports and references

- [ ] Add test coverage for new components
  - [ ] Create unit tests for extracted utilities
  - [ ] Ensure integration tests cover all scenarios
  - [ ] Validate backward compatibility

### Phase 10: Documentation (Weeks 6-7)

- [ ] Update core documentation
  - [ ] Update README.md with new structure and examples
  - [ ] Create architecture diagram showing component relationships
  - [ ] Document import patterns and best practices

- [ ] Create specialized documentation
  - [ ] `ARCHITECTURE.md` for system overview
  - [ ] `FORWARDING.md` for forwarding system specifics
  - [ ] `CERTIFICATE.md` for certificate management details
  - [ ] `DEVELOPMENT.md` for contributor guidelines

- [ ] Update example files
  - [ ] Update existing examples to use new structure
  - [ ] Add new examples demonstrating key scenarios

### Phase 11: Release & Migration Guide (Week 8)

- [ ] Prepare for release
  - [ ] Final testing and validation
  - [ ] Performance comparison with previous version
  - [ ] Create detailed changelog

- [ ] Create migration guide
  - [ ] Document breaking changes
  - [ ] Provide upgrade instructions
  - [ ] Include code examples for common scenarios

## Detailed File Migration Table

| Current File | New File | Status |
|--------------|----------|--------|
| **Core/Common Files** | | |
| ts/common/types.ts | ts/core/models/common-types.ts | ❌ |
| ts/common/eventUtils.ts | ts/core/utils/event-utils.ts | ❌ |
| ts/common/acmeFactory.ts | ts/certificate/acme/acme-factory.ts | ❌ |
| ts/plugins.ts | ts/core/utils/plugins.ts | ❌ |
| ts/00_commitinfo_data.ts | ts/core/utils/commit-info.ts | ❌ |
| **Certificate Management** | | |
| ts/helpers.certificates.ts | ts/certificate/utils/certificate-helpers.ts | ❌ |
| ts/classes.pp.certprovisioner.ts | ts/certificate/providers/cert-provisioner.ts | ❌ |
| **TLS and SNI Handling** | | |
| ts/smartproxy/classes.pp.tlsalert.ts | ts/tls/alerts/tls-alert.ts | ❌ |
| ts/smartproxy/classes.pp.snihandler.ts | ts/tls/sni/sni-handler.ts | ❌ |
| **HTTP Components** | | |
| ts/port80handler/classes.port80handler.ts | ts/http/port80/port80-handler.ts | ❌ |
| ts/redirect/classes.redirect.ts | ts/http/redirects/redirect-handler.ts | ❌ |
| ts/classes.router.ts | ts/http/router/proxy-router.ts | ❌ |
| **SmartProxy Components** | | |
| ts/smartproxy/classes.smartproxy.ts | ts/proxies/smart-proxy/smart-proxy.ts | ❌ |
| ts/smartproxy/classes.pp.interfaces.ts | ts/proxies/smart-proxy/models/interfaces.ts | ❌ |
| ts/smartproxy/classes.pp.connectionhandler.ts | ts/proxies/smart-proxy/connection-handler.ts | ❌ |
| ts/smartproxy/classes.pp.connectionmanager.ts | ts/proxies/smart-proxy/connection-manager.ts | ❌ |
| ts/smartproxy/classes.pp.domainconfigmanager.ts | ts/proxies/smart-proxy/domain-config-manager.ts | ❌ |
| ts/smartproxy/classes.pp.portrangemanager.ts | ts/proxies/smart-proxy/port-range-manager.ts | ❌ |
| ts/smartproxy/classes.pp.securitymanager.ts | ts/proxies/smart-proxy/security-manager.ts | ❌ |
| ts/smartproxy/classes.pp.timeoutmanager.ts | ts/proxies/smart-proxy/timeout-manager.ts | ❌ |
| ts/smartproxy/classes.pp.networkproxybridge.ts | ts/proxies/smart-proxy/network-proxy-bridge.ts | ❌ |
| **NetworkProxy Components** | | |
| ts/networkproxy/classes.np.networkproxy.ts | ts/proxies/network-proxy/network-proxy.ts | ❌ |
| ts/networkproxy/classes.np.certificatemanager.ts | ts/proxies/network-proxy/certificate-manager.ts | ❌ |
| ts/networkproxy/classes.np.connectionpool.ts | ts/proxies/network-proxy/connection-pool.ts | ❌ |
| ts/networkproxy/classes.np.requesthandler.ts | ts/proxies/network-proxy/request-handler.ts | ❌ |
| ts/networkproxy/classes.np.websockethandler.ts | ts/proxies/network-proxy/websocket-handler.ts | ❌ |
| ts/networkproxy/classes.np.types.ts | ts/proxies/network-proxy/models/types.ts | ❌ |
| **NFTablesProxy Components** | | |
| ts/nfttablesproxy/classes.nftablesproxy.ts | ts/proxies/nftables-proxy/nftables-proxy.ts | ❌ |
| **Forwarding System** | | |
| ts/smartproxy/types/forwarding.types.ts | ts/forwarding/config/forwarding-types.ts | ❌ |
| ts/smartproxy/forwarding/domain-config.ts | ts/forwarding/config/domain-config.ts | ❌ |
| ts/smartproxy/forwarding/domain-manager.ts | ts/forwarding/config/domain-manager.ts | ❌ |
| ts/smartproxy/forwarding/forwarding.handler.ts | ts/forwarding/handlers/base-handler.ts | ❌ |
| ts/smartproxy/forwarding/http.handler.ts | ts/forwarding/handlers/http-handler.ts | ❌ |
| ts/smartproxy/forwarding/https-passthrough.handler.ts | ts/forwarding/handlers/https-passthrough-handler.ts | ❌ |
| ts/smartproxy/forwarding/https-terminate-to-http.handler.ts | ts/forwarding/handlers/https-terminate-to-http-handler.ts | ❌ |
| ts/smartproxy/forwarding/https-terminate-to-https.handler.ts | ts/forwarding/handlers/https-terminate-to-https-handler.ts | ❌ |
| ts/smartproxy/forwarding/forwarding.factory.ts | ts/forwarding/factory/forwarding-factory.ts | ❌ |
| ts/smartproxy/forwarding/index.ts | ts/forwarding/index.ts | ❌ |
| **Examples and Entry Points** | | |
| ts/examples/forwarding-example.ts | ts/examples/forwarding-example.ts | ❌ |
| ts/index.ts | ts/index.ts (updated) | ❌ |

## Import Strategy

Since path aliases will not be used, we'll maintain standard relative imports throughout the codebase:

1. **Import Strategy for Deeply Nested Files**
   ```typescript
   // Example: Importing from another component in a nested directory
   // From ts/forwarding/handlers/http-handler.ts to ts/core/utils/validation-utils.ts
   import { validateConfig } from '../../../core/utils/validation-utils.js';
   ```

2. **Barrel Files for Convenience**
   ```typescript
   // ts/forwarding/index.ts
   export * from './config/forwarding-types.js';
   export * from './handlers/base-handler.js';
   // ... other exports

   // Then in consuming code:
   import { ForwardingHandler, httpOnly } from '../../forwarding/index.js';
   ```

3. **Flattened Imports Where Sensible**
   ```typescript
   // Avoid excessive nesting with targeted exports
   // ts/index.ts will export key components for external use
   import { SmartProxy, NetworkProxy } from '../index.js';
   ```

## Expected Outcomes

### Improved Code Organization
- Related code will be grouped together in domain-specific directories
- Consistent naming conventions will make code navigation intuitive
- Clear module boundaries will prevent unintended dependencies

### Enhanced Developer Experience
- Standardized interface naming will improve type clarity
- Better documentation will help new contributors get started
- Clear and predictable file locations

### Maintainability Benefits
- Smaller, focused files with clear responsibilities
- Unified patterns for common operations
- Improved separation of concerns between components
- Better test organization matching source structure

### Performance and Compatibility
- No performance regression from structural changes
- Backward compatibility through type aliases and consistent exports
- Clear migration path for dependent projects

## Migration Strategy

To ensure a smooth transition, we'll follow this approach for each component:

1. Create the new file structure first
2. Migrate code while updating relative imports
3. Test each component as it's migrated
4. Only remove old files once all dependencies are updated
5. Use a phased approach to allow parallel work

This approach ensures the codebase remains functional throughout the restructuring process while progressively adopting the new organization.

## Measuring Success

We'll measure the success of this restructuring by:

1. Reduced complexity in the directory structure
2. Improved code coverage through better test organization
3. Faster onboarding time for new developers
4. Less time spent navigating the codebase
5. Cleaner git blame output showing cohesive component changes

## Special Considerations

- We'll maintain backward compatibility for all public APIs
- We'll provide detailed upgrade guides for any breaking changes
- We'll ensure the build process produces compatible output
- We'll preserve commit history using git move operations where possible