NFTables-SmartProxy Integration Plan

Overview

This document outlines a comprehensive plan to integrate the existing NFTables functionality with the SmartProxy core to provide advanced network-level routing capabilities. The NFTables proxy already exists in the codebase but is not fully integrated with the SmartProxy routing system. This integration will allow SmartProxy to leverage the power of Linux's NFTables firewall system for high-performance port forwarding, load balancing, and security filtering.

Current State

NFTablesProxy: A standalone implementation exists in ts/proxies/nftables-proxy/ with its own configuration and API.
SmartProxy: The main routing system with route-based configuration.
No Integration: Currently, these systems operate independently with no shared configuration or coordination.

Goals

Create a unified configuration system where SmartProxy routes can specify NFTables-based forwarding.
Allow SmartProxy to dynamically provision and manage NFTables rules based on route configuration.
Support advanced filtering and security rules through NFTables for better performance.
Ensure backward compatibility with existing setups.
Provide metrics integration between the systems.

Implementation Plan

Phase 1: Route Configuration Schema Extension

Extend Route Configuration Schema:
- Add new forwardingEngine option to IRouteAction to specify the forwarding implementation.
- Support values: 'node' (current NodeJS implementation) and 'nftables' (Linux NFTables).
- Add NFTables-specific configuration options to IRouteAction.

Update Type Definitions:

// In route-types.ts
export interface IRouteAction {
  type: 'forward' | 'redirect' | 'block';
  target?: IRouteTarget;
  security?: IRouteSecurity;
  options?: IRouteOptions;
  tls?: IRouteTlsOptions;
  forwardingEngine?: 'node' | 'nftables';  // New field
  nftables?: INfTablesOptions;             // New field
}

export interface INfTablesOptions {
  preserveSourceIP?: boolean;
  protocol?: 'tcp' | 'udp' | 'all';
  maxRate?: string;                // QoS rate limiting
  priority?: number;               // QoS priority
  tableName?: string;              // Optional custom table name
  useIPSets?: boolean;             // Use IP sets for performance
  useAdvancedNAT?: boolean;        // Use connection tracking
}

Phase 2: NFTablesManager Implementation

Create NFTablesManager Class:
- Create a new class to manage NFTables rules based on SmartProxy routes.
- Add methods to create, update, and remove NFTables rules.
- Design a rule naming scheme to track which rules correspond to which routes.

Implementation:

// In ts/proxies/smart-proxy/nftables-manager.ts
export class NFTablesManager {
  private rulesMap: Map<string, NfTablesProxy> = new Map();

  constructor(private options: ISmartProxyOptions) {}

  /**
   * Provision NFTables rules for a route
   */
  public async provisionRoute(route: IRouteConfig): Promise<boolean> {
    // Generate a unique ID for this route
    const routeId = this.generateRouteId(route);

    // Skip if route doesn't use NFTables
    if (route.action.forwardingEngine !== 'nftables') {
      return true;
    }

    // Create NFTables options from route configuration
    const nftOptions = this.createNfTablesOptions(route);

    // Create and start an NFTablesProxy instance
    const proxy = new NfTablesProxy(nftOptions);

    try {
      await proxy.start();
      this.rulesMap.set(routeId, proxy);
      return true;
    } catch (err) {
      console.error(`Failed to provision NFTables rules for route ${route.name}: ${err.message}`);
      return false;
    }
  }

  /**
   * Remove NFTables rules for a route
   */
  public async deprovisionRoute(route: IRouteConfig): Promise<boolean> {
    const routeId = this.generateRouteId(route);

    const proxy = this.rulesMap.get(routeId);
    if (!proxy) {
      return true; // Nothing to remove
    }

    try {
      await proxy.stop();
      this.rulesMap.delete(routeId);
      return true;
    } catch (err) {
      console.error(`Failed to deprovision NFTables rules for route ${route.name}: ${err.message}`);
      return false;
    }
  }

  /**
   * Update NFTables rules when route changes
   */
  public async updateRoute(oldRoute: IRouteConfig, newRoute: IRouteConfig): Promise<boolean> {
    // Remove old rules and add new ones
    await this.deprovisionRoute(oldRoute);
    return this.provisionRoute(newRoute);
  }

  /**
   * Generate a unique ID for a route
   */
  private generateRouteId(route: IRouteConfig): string {
    // Generate a unique ID based on route properties
    return `${route.name || 'unnamed'}-${JSON.stringify(route.match)}-${Date.now()}`;
  }

  /**
   * Create NFTablesProxy options from a route configuration
   */
  private createNfTablesOptions(route: IRouteConfig): NfTableProxyOptions {
    const { action } = route;

    // Ensure we have a target
    if (!action.target) {
      throw new Error('Route must have a target to use NFTables forwarding');
    }

    // Convert port specifications
    const fromPorts = this.expandPortRange(route.match.ports);

    // Determine target port
    let toPorts;
    if (action.target.port === 'preserve') {
      // 'preserve' means use the same ports as the source
      toPorts = fromPorts;
    } else if (typeof action.target.port === 'function') {
      // For function-based ports, we can't determine at setup time
      // Use the "preserve" approach and let NFTables handle it
      toPorts = fromPorts;
    } else {
      toPorts = action.target.port;
    }

    // Create options
    const options: NfTableProxyOptions = {
      fromPort: fromPorts,
      toPort: toPorts,
      toHost: typeof action.target.host === 'function' 
        ? 'localhost' // Can't determine at setup time, use localhost
        : (Array.isArray(action.target.host) 
           ? action.target.host[0] // Use first host for now
           : action.target.host),
      protocol: action.nftables?.protocol || 'tcp',
      preserveSourceIP: action.nftables?.preserveSourceIP,
      useIPSets: action.nftables?.useIPSets !== false,
      useAdvancedNAT: action.nftables?.useAdvancedNAT,
      enableLogging: this.options.enableDetailedLogging,
      deleteOnExit: true,
      tableName: action.nftables?.tableName || 'smartproxy'
    };

    // Add security-related options
    if (action.security?.ipAllowList?.length) {
      options.allowedSourceIPs = action.security.ipAllowList;
    }

    if (action.security?.ipBlockList?.length) {
      options.bannedSourceIPs = action.security.ipBlockList;
    }

    // Add QoS options
    if (action.nftables?.maxRate || action.nftables?.priority) {
      options.qos = {
        enabled: true,
        maxRate: action.nftables.maxRate,
        priority: action.nftables.priority
      };
    }

    return options;
  }

  /**
   * Expand port range specifications
   */
  private expandPortRange(ports: TPortRange): number | PortRange | Array<number | PortRange> {
    // Use RouteManager's expandPortRange to convert to actual port numbers
    const routeManager = new RouteManager(this.options);

    // Process different port specifications
    if (typeof ports === 'number') {
      return ports;
    } else if (Array.isArray(ports)) {
      const result: Array<number | PortRange> = [];

      for (const item of ports) {
        if (typeof item === 'number') {
          result.push(item);
        } else if ('from' in item && 'to' in item) {
          result.push({ from: item.from, to: item.to });
        }
      }

      return result;
    } else if ('from' in ports && 'to' in ports) {
      return { from: ports.from, to: ports.to };
    }

    // Fallback
    return 80;
  }

  /**
   * Get status of all managed rules
   */
  public async getStatus(): Promise<Record<string, NfTablesStatus>> {
    const result: Record<string, NfTablesStatus> = {};

    for (const [routeId, proxy] of this.rulesMap.entries()) {
      result[routeId] = await proxy.getStatus();
    }

    return result;
  }

  /**
   * Stop all NFTables rules
   */
  public async stop(): Promise<void> {
    // Stop all NFTables proxies
    const stopPromises = Array.from(this.rulesMap.values()).map(proxy => proxy.stop());
    await Promise.all(stopPromises);

    this.rulesMap.clear();
  }
}

Phase 3: SmartProxy Integration

Extend SmartProxy Class:
- Add NFTablesManager as a property of SmartProxy.
- Hook into route configuration to provision NFTables rules.
- Add methods to manage NFTables functionality.

Implementation:

// In ts/proxies/smart-proxy/smart-proxy.ts
import { NFTablesManager } from './nftables-manager.js';

export class SmartProxy {
  // Existing properties
  private nftablesManager: NFTablesManager;

  constructor(options: ISmartProxyOptions) {
    // Existing initialization

    // Initialize NFTablesManager
    this.nftablesManager = new NFTablesManager(options);
  }

  /**
   * Start the SmartProxy server
   */
  public async start(): Promise<void> {
    // Existing initialization

    // If we have routes, provision NFTables rules for them
    for (const route of this.settings.routes) {
      if (route.action.forwardingEngine === 'nftables') {
        await this.nftablesManager.provisionRoute(route);
      }
    }

    // Rest of existing start method
  }

  /**
   * Stop the SmartProxy server
   */
  public async stop(): Promise<void> {
    // Stop NFTablesManager first
    await this.nftablesManager.stop();

    // Rest of existing stop method
  }

  /**
   * Update routes
   */
  public async updateRoutes(routes: IRouteConfig[]): Promise<void> {
    // Get existing routes that use NFTables
    const oldNfTablesRoutes = this.settings.routes.filter(
      r => r.action.forwardingEngine === 'nftables'
    );

    // Get new routes that use NFTables
    const newNfTablesRoutes = routes.filter(
      r => r.action.forwardingEngine === 'nftables'
    );

    // Find routes to remove, update, or add
    for (const oldRoute of oldNfTablesRoutes) {
      const newRoute = newNfTablesRoutes.find(r => r.name === oldRoute.name);

      if (!newRoute) {
        // Route was removed
        await this.nftablesManager.deprovisionRoute(oldRoute);
      } else {
        // Route was updated
        await this.nftablesManager.updateRoute(oldRoute, newRoute);
      }
    }

    // Find new routes to add
    for (const newRoute of newNfTablesRoutes) {
      const oldRoute = oldNfTablesRoutes.find(r => r.name === newRoute.name);

      if (!oldRoute) {
        // New route
        await this.nftablesManager.provisionRoute(newRoute);
      }
    }

    // Update settings with the new routes
    this.settings.routes = routes;

    // Update route manager with new routes
    this.routeManager.updateRoutes(routes);
  }

  /**
   * Get NFTables status
   */
  public async getNfTablesStatus(): Promise<Record<string, NfTablesStatus>> {
    return this.nftablesManager.getStatus();
  }
}

Phase 4: Routing System Integration

Extend the Route-Connection-Handler:
- Modify to check if a route uses NFTables.
- Skip Node.js-based connection handling for NFTables routes.

Implementation:

// In ts/proxies/smart-proxy/route-connection-handler.ts
export class RouteConnectionHandler {
  // Existing methods

  /**
   * Route the connection based on match criteria
   */
  private routeConnection(
    socket: plugins.net.Socket, 
    record: IConnectionRecord,
    serverName: string,
    initialChunk?: Buffer
  ): void {
    // Find matching route
    const routeMatch = this.routeManager.findMatchingRoute({
      port: record.localPort,
      domain: serverName,
      clientIp: record.remoteIP,
      path: undefined,
      tlsVersion: undefined
    });

    if (!routeMatch) {
      // Existing code for no matching route
      return;
    }

    const route = routeMatch.route;

    // Check if this route uses NFTables for forwarding
    if (route.action.forwardingEngine === 'nftables') {
      // For NFTables routes, we don't need to do anything at the application level
      // The packet is forwarded at the kernel level

      // Log the connection
      console.log(
        `[${record.id}] Connection forwarded by NFTables: ${record.remoteIP} -> port ${record.localPort}`
      );

      // Just close the socket in our application since it's handled at kernel level
      socket.end();
      this.connectionManager.initiateCleanupOnce(record, 'nftables_handled');
      return;
    }

    // Existing code for handling the route
  }
}

Phase 5: CLI and Configuration Helpers

Add Helper Functions:
- Create helper functions for easy route creation with NFTables.
- Update the route-helpers.ts utility file.

Implementation:

// In ts/proxies/smart-proxy/utils/route-helpers.ts

/**
 * Create an NFTables-based route
 */
export function createNfTablesRoute(
  nameOrDomains: string | string[],
  target: { host: string; port: number | 'preserve' },
  options: {
    ports?: TPortRange;
    protocol?: 'tcp' | 'udp' | 'all';
    preserveSourceIP?: boolean;
    allowedIps?: string[];
    maxRate?: string;
    priority?: number;
    useTls?: boolean;
  } = {}
): IRouteConfig {
  // Determine if this is a name or domain
  let name: string;
  let domains: string | string[];

  if (Array.isArray(nameOrDomains) || nameOrDomains.includes('.')) {
    domains = nameOrDomains;
    name = Array.isArray(nameOrDomains) ? nameOrDomains[0] : nameOrDomains;
  } else {
    name = nameOrDomains;
    domains = []; // No domains
  }

  const route: IRouteConfig = {
    name,
    match: {
      domains,
      ports: options.ports || 80
    },
    action: {
      type: 'forward',
      target: {
        host: target.host,
        port: target.port
      },
      forwardingEngine: 'nftables',
      nftables: {
        protocol: options.protocol || 'tcp',
        preserveSourceIP: options.preserveSourceIP,
        maxRate: options.maxRate,
        priority: options.priority
      }
    }
  };

  // Add security if allowed IPs are specified
  if (options.allowedIps?.length) {
    route.action.security = {
      ipAllowList: options.allowedIps
    };
  }

  // Add TLS options if needed
  if (options.useTls) {
    route.action.tls = {
      mode: 'passthrough'
    };
  }

  return route;
}

/**
 * Create an NFTables-based TLS termination route
 */
export function createNfTablesTerminateRoute(
  nameOrDomains: string | string[],
  target: { host: string; port: number | 'preserve' },
  options: {
    ports?: TPortRange;
    protocol?: 'tcp' | 'udp' | 'all';
    preserveSourceIP?: boolean;
    allowedIps?: string[];
    maxRate?: string;
    priority?: number;
    certificate?: string | { cert: string; key: string };
  } = {}
): IRouteConfig {
  const route = createNfTablesRoute(
    nameOrDomains,
    target,
    {
      ...options,
      ports: options.ports || 443,
      useTls: false
    }
  );

  // Set TLS termination
  route.action.tls = {
    mode: 'terminate',
    certificate: options.certificate || 'auto'
  };

  return route;
}

Phase 6: Documentation and Testing

Update Documentation:
- Add NFTables integration documentation to README and API docs.
- Document the implementation and use cases.
Test Cases:
- Create test cases for NFTables-based routing.
- Test performance comparison with Node.js-based forwarding.
- Test security features with IP allowlists/blocklists.

// In test/test.nftables-integration.ts
import { SmartProxy } from '../ts/proxies/smart-proxy/index.js';
import { createNfTablesRoute } from '../ts/proxies/smart-proxy/utils/route-helpers.js';
import { expect, tap } from '@push.rocks/tapbundle';
import * as net from 'net';

// Test server and client utilities
let testServer: net.Server;
let smartProxy: SmartProxy;

const TEST_PORT = 4000;
const PROXY_PORT = 5000;
const TEST_DATA = 'Hello through NFTables!';

tap.test('setup NFTables integration test environment', async () => {
  // Create a test TCP server
  testServer = net.createServer((socket) => {
    socket.on('data', (data) => {
      socket.write(`Server says: ${data.toString()}`);
    });
  });
  
  await new Promise<void>((resolve) => {
    testServer.listen(TEST_PORT, () => {
      console.log(`Test server listening on port ${TEST_PORT}`);
      resolve();
    });
  });
  
  // Create SmartProxy with NFTables route
  smartProxy = new SmartProxy({
    routes: [
      createNfTablesRoute('test-nftables', {
        host: 'localhost',
        port: TEST_PORT
      }, {
        ports: PROXY_PORT,
        protocol: 'tcp'
      })
    ]
  });
  
  // Start the proxy
  await smartProxy.start();
});

tap.test('should forward TCP connections through NFTables', async () => {
  // Connect to the proxy port
  const client = new net.Socket();
  
  const response = await new Promise<string>((resolve, reject) => {
    let responseData = '';
    
    client.connect(PROXY_PORT, 'localhost', () => {
      client.write(TEST_DATA);
    });
    
    client.on('data', (data) => {
      responseData += data.toString();
      client.end();
    });
    
    client.on('end', () => {
      resolve(responseData);
    });
    
    client.on('error', (err) => {
      reject(err);
    });
  });
  
  expect(response).toEqual(`Server says: ${TEST_DATA}`);
});

tap.test('cleanup NFTables integration test environment', async () => {
  // Stop the proxy and test server
  await smartProxy.stop();
  
  await new Promise<void>((resolve) => {
    testServer.close(() => {
      resolve();
    });
  });
});

export default tap.start();

Expected Benefits

Performance: NFTables operates at the kernel level, offering much higher performance than Node.js-based routing.
Scalability: Handle more connections with less CPU and memory usage.
Security: Leverage kernel-level security features for better protection.
Integration: Unified configuration model between application and network layers.
Advanced Features: Support for QoS, rate limiting, and other advanced networking features.

Implementation Notes

This integration requires root/sudo access to configure NFTables rules.
Consider adding a capability check to gracefully fall back to Node.js routing if NFTables is not available.
The NFTables integration should be optional and SmartProxy should continue to work without it.
The integration provides a path for future extensions to other kernel-level networking features.

Timeline

Phase 1 (Route Configuration Schema): 1-2 days
Phase 2 (NFTablesManager): 2-3 days
Phase 3 (SmartProxy Integration): 1-2 days
Phase 4 (Routing System Integration): 1 day
Phase 5 (CLI and Helpers): 1 day
Phase 6 (Documentation and Testing): 2 days

Total Estimated Time: 8-11 days

20 KiB Raw Blame History

NFTables-SmartProxy Integration Plan

Overview

Current State

Goals

Implementation Plan

Phase 1: Route Configuration Schema Extension

Phase 2: NFTablesManager Implementation

Phase 3: SmartProxy Integration

Phase 4: Routing System Integration

Phase 5: CLI and Configuration Helpers

Phase 6: Documentation and Testing

Expected Benefits

Implementation Notes

Timeline

20 KiB

Raw Blame History