19.6.13

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@push.rocks/smartproxy",
-  "version": "19.6.12",
+  "version": "19.6.13",
   "private": false,
   "description": "A powerful proxy package with unified route-based configuration for high traffic management. Features include SSL/TLS support, flexible routing patterns, WebSocket handling, advanced security options, and automatic ACME certificate management.",
   "main": "dist_ts/index.js",

readme.hints.md

@@ -142,4 +142,45 @@ Keep-alive connections receive special treatment based on `keepAliveTreatment` s
 The system supports both receiving and sending PROXY protocol:
 - **Receiving**: Automatically detected from trusted proxy IPs (configured in `proxyIPs`)
 - **Sending**: Enabled per-route or globally via `sendProxyProtocol` setting
-- Real client IP is preserved and used for all connection tracking and security checks
+- Real client IP is preserved and used for all connection tracking and security checks
+
+## Metrics and Throughput Calculation
+
+The metrics system tracks throughput using per-second sampling:
+
+1. **Byte Recording**: Bytes are recorded as data flows through connections
+2. **Sampling**: Every second, accumulated bytes are stored as a sample
+3. **Rate Calculation**: Throughput is calculated by summing bytes over a time window
+4. **Per-Route/IP Tracking**: Separate ThroughputTracker instances for each route and IP
+
+Key implementation details:
+- Bytes are recorded in the bidirectional forwarding callbacks
+- The instant() method returns throughput over the last 1 second
+- The recent() method returns throughput over the last 10 seconds
+- Custom windows can be specified for different averaging periods
+
+### Throughput Spikes Issue
+
+There's a fundamental difference between application-layer and network-layer throughput:
+
+**Application Layer (what we measure)**:
+- Bytes are recorded when delivered to/from the application
+- Large chunks can arrive "instantly" due to kernel/Node.js buffering
+- Shows spikes when buffers are flushed (e.g., 20MB in 1 second = 160 Mbit/s)
+
+**Network Layer (what Unifi shows)**:
+- Actual packet flow through the network interface
+- Limited by physical network speed (e.g., 20 Mbit/s)
+- Data transfers over time, not in bursts
+
+The spikes occur because:
+1. Data flows over network at 20 Mbit/s (takes 8 seconds for 20MB)
+2. Kernel/Node.js buffers this incoming data
+3. When buffer is flushed, application receives large chunk at once
+4. We record entire chunk in current second, creating artificial spike
+
+**Potential Solutions**:
+1. Use longer window for "instant" measurements (e.g., 5 seconds instead of 1)
+2. Track socket write backpressure to estimate actual network flow
+3. Implement bandwidth estimation based on connection duration
+4. Accept that application-layer != network-layer throughput

ts/proxies/smart-proxy/throughput-tracker.ts

@@ -65,24 +65,18 @@ export class ThroughputTracker {
       return { in: 0, out: 0 };
     }
     
-    // Sum bytes in the window
+    // Calculate total bytes in window
     const totalBytesIn = relevantSamples.reduce((sum, s) => sum + s.bytesIn, 0);
     const totalBytesOut = relevantSamples.reduce((sum, s) => sum + s.bytesOut, 0);
     
-    // Calculate actual window duration (might be less than requested if not enough data)
-    const actualWindowSeconds = Math.min(
-      windowSeconds,
-      (now - relevantSamples[0].timestamp) / 1000
-    );
-    
-    // Avoid division by zero
-    if (actualWindowSeconds === 0) {
-      return { in: 0, out: 0 };
-    }
+    // Use actual number of seconds covered by samples for accurate rate
+    const oldestSampleTime = relevantSamples[0].timestamp;
+    const newestSampleTime = relevantSamples[relevantSamples.length - 1].timestamp;
+    const actualSeconds = Math.max(1, (newestSampleTime - oldestSampleTime) / 1000 + 1);
     
     return {
-      in: Math.round(totalBytesIn / actualWindowSeconds),
-      out: Math.round(totalBytesOut / actualWindowSeconds)
+      in: Math.round(totalBytesIn / actualSeconds),
+      out: Math.round(totalBytesOut / actualSeconds)
     };
   }