56 lines
1.8 KiB
Markdown
56 lines
1.8 KiB
Markdown
# TSPM Real-Time Log Streaming Implementation Plan
|
|
|
|
## Overview
|
|
Implementing real-time log streaming (tailing) functionality for TSPM using SmartIPC's pub/sub capabilities.
|
|
|
|
## Approach: Hybrid Request + Subscribe
|
|
1. Initial getLogs request to fetch historical logs up to current point
|
|
2. Subscribe to pub/sub channel for real-time updates
|
|
3. Use sequence numbers to detect and handle gaps/duplicates
|
|
4. Per-process topics for granular subscriptions
|
|
|
|
## Implementation Tasks
|
|
|
|
### Core Changes
|
|
- [x] Update IProcessLog interface with seq and runId fields
|
|
- [x] Add nextSeq and runId fields to ProcessWrapper class
|
|
- [x] Update addLog() methods to include sequencing
|
|
- [x] Implement pub/sub publishing in daemon
|
|
|
|
### IPC Client Updates
|
|
- [x] Add subscribe/unsubscribe methods to TspmIpcClient
|
|
- [ ] Implement log streaming handler
|
|
- [ ] Add connection state management for subscriptions
|
|
|
|
### CLI Enhancement
|
|
- [x] Add --follow flag to logs command
|
|
- [x] Implement streaming output with proper formatting
|
|
- [x] Handle Ctrl+C gracefully to unsubscribe
|
|
|
|
### Reliability Features
|
|
- [x] Add backpressure handling (drop oldest when buffer full)
|
|
- [x] Implement gap detection and recovery
|
|
- [x] Add process restart detection via runId
|
|
|
|
### Testing
|
|
- [x] Test basic log streaming
|
|
- [x] Test gap recovery
|
|
- [x] Test high-volume logging scenarios
|
|
- [x] Test process restart handling
|
|
|
|
## Technical Details
|
|
|
|
### Sequence Numbering
|
|
- Each log entry gets incrementing seq number per process
|
|
- runId changes on process restart
|
|
- Client tracks lastSeq to detect gaps
|
|
|
|
### Topic Structure
|
|
- Format: `logs.<processId>`
|
|
- Daemon publishes to topic on new log entries
|
|
- Clients subscribe to specific process topics
|
|
|
|
### Backpressure Strategy
|
|
- Circular buffer of 10,000 entries per process
|
|
- Drop oldest entries when buffer full
|
|
- Client can detect gaps via sequence numbers |