feat(streaming): Add streaming support: chunked stream transfers, file send/receive, stream events and helpers

This commit is contained in:
2025-08-30 23:02:49 +00:00
parent 7ba064584b
commit 994b1d20fb
9 changed files with 678 additions and 11 deletions

View File

@@ -16,6 +16,7 @@ SmartIPC delivers bulletproof Inter-Process Communication for Node.js applicatio
- **CI/Test Ready** - Built-in helpers and race condition prevention for testing
- **Observable** - Real-time metrics, connection tracking, and health monitoring
- **Multiple Patterns** - Request/Response, Pub/Sub, and Fire-and-Forget messaging
- **Streaming Support** - Efficient, backpressureaware streaming for large data and files
## 📦 Installation
@@ -184,6 +185,90 @@ await publisher.publish('user.login', {
## 💪 Advanced Features
### 📦 Streaming Large Data & Files
SmartIPC supports efficient, backpressure-aware streaming of large payloads using chunked messages. Streams work both directions and emit a high-level `stream` event for consumption.
Client → Server streaming:
```typescript
// Server side: receive stream
server.on('stream', async (info, readable) => {
if (info.meta?.type === 'file') {
console.log('Receiving file', info.meta.basename, 'from', info.clientId);
}
// Pipe to disk or process chunks
await SmartIpc.pipeStreamToFile(readable, '/tmp/incoming.bin');
});
// Client side: send a stream
const readable = fs.createReadStream('/path/to/local.bin');
await client.sendStream(readable, {
meta: { type: 'file', basename: 'local.bin' },
chunkSize: 64 * 1024 // optional, defaults to 64k
});
```
Server → Client streaming:
```typescript
client.on('stream', async (info, readable) => {
console.log('Got stream from server', info.meta);
await SmartIpc.pipeStreamToFile(readable, '/tmp/from-server.bin');
});
await server.sendStreamToClient(client.getClientId(), fs.createReadStream('/path/server.bin'), {
meta: { type: 'file', basename: 'server.bin' }
});
```
High-level helpers for files:
```typescript
// Client → Server
await client.sendFile('/path/to/bigfile.iso');
// Server → Client
await server.sendFileToClient(clientId, '/path/to/backup.tar');
// Save an incoming stream to a file (both sides)
server.on('stream', async (info, readable) => {
await SmartIpc.pipeStreamToFile(readable, '/data/uploaded/' + info.meta?.basename);
});
```
Events & metadata:
- `channel/server/client` emit `stream` with `(info, readable)`
- `info` contains: `streamId`, `meta` (your metadata, e.g., filename/size), `headers`, and `clientId` (if available)
API summary:
- Client: `sendStream(readable, opts)`, `sendFile(filePath, opts)`, `cancelOutgoingStream(id)`, `cancelIncomingStream(id)`
- Server: `sendStreamToClient(clientId, readable, opts)`, `sendFileToClient(clientId, filePath, opts)`, `cancelIncomingStreamFromClient(clientId, id)`, `cancelOutgoingStreamToClient(clientId, id)`
- Utility: `SmartIpc.pipeStreamToFile(readable, filePath)`
Concurrency and cancelation:
```typescript
// Limit concurrent streams per connection
const server = SmartIpc.createServer({
id: 'svc', socketPath: '/tmp/svc.sock', maxConcurrentStreams: 2
});
// Cancel a stream from the receiver side
server.on('stream', (info, readable) => {
if (info.meta?.shouldCancel) {
(server as any).primaryChannel.cancelIncomingStream(info.streamId, { clientId: info.clientId });
}
});
```
Notes:
- Streaming uses chunked messages under the hood and respects socket backpressure.
- Include `meta` to share context like filename/size; its delivered with the `stream` event.
- Configure `maxConcurrentStreams` (default: 32) to guard resources.
### 🏁 Server Readiness Detection
Eliminate race conditions in tests and production: