tspm/readme.plan.md

TSPM Real-Time Log Streaming Implementation Plan

Overview

Implementing real-time log streaming (tailing) functionality for TSPM using SmartIPC's pub/sub capabilities.

Approach: Hybrid Request + Subscribe

1. Initial getLogs request to fetch historical logs up to current point
2. Subscribe to pub/sub channel for real-time updates
3. Use sequence numbers to detect and handle gaps/duplicates
4. Per-process topics for granular subscriptions

Implementation Tasks

Core Changes

• Update IProcessLog interface with seq and runId fields
• Add nextSeq and runId fields to ProcessWrapper class
• Update addLog() methods to include sequencing
• Implement pub/sub publishing in daemon

IPC Client Updates

• Add subscribe/unsubscribe methods to TspmIpcClient
• Implement log streaming handler
• Add connection state management for subscriptions

CLI Enhancement

• Add --follow flag to logs command
• Implement streaming output with proper formatting
• Handle Ctrl+C gracefully to unsubscribe

Reliability Features

• Add backpressure handling (drop oldest when buffer full)
• Implement gap detection and recovery
• Add process restart detection via runId

Testing

• Test basic log streaming
• Test gap recovery
• Test high-volume logging scenarios
• Test process restart handling

Technical Details

Sequence Numbering

• Each log entry gets incrementing seq number per process
• runId changes on process restart
• Client tracks lastSeq to detect gaps

Topic Structure

• Format: logs.<processId>
• Daemon publishes to topic on new log entries
• Clients subscribe to specific process topics

Backpressure Strategy

• Circular buffer of 10,000 entries per process
• Drop oldest entries when buffer full
• Client can detect gaps via sequence numbers