588 lines
12 KiB
Markdown
588 lines
12 KiB
Markdown
# @git.zone/tstest/tapbundle_protocol
|
|
|
|
> 📡 Enhanced TAP Protocol V2 implementation for structured test communication
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
# tapbundle_protocol is included as part of @git.zone/tstest
|
|
pnpm install --save-dev @git.zone/tstest
|
|
```
|
|
|
|
## Overview
|
|
|
|
`@git.zone/tstest/tapbundle_protocol` implements Protocol V2, an enhanced version of the Test Anything Protocol (TAP) with support for structured metadata, real-time events, error diffs, and isomorphic operation. This protocol enables rich communication between test runners and test consumers while maintaining backward compatibility with standard TAP parsers.
|
|
|
|
## Key Features
|
|
|
|
- 📋 **TAP v13 Compliant** - Fully compatible with standard TAP consumers
|
|
- 🎯 **Enhanced Metadata** - Timing, tags, errors, diffs, and custom data
|
|
- 🔄 **Real-time Events** - Live test execution updates
|
|
- 🔍 **Structured Errors** - JSON error blocks with stack traces and diffs
|
|
- 📸 **Snapshot Support** - Built-in snapshot testing protocol
|
|
- 🌐 **Isomorphic** - Works in Node.js, browsers, Deno, and Bun
|
|
- 🏷️ **Protocol Markers** - Structured data using Unicode delimiters
|
|
|
|
## Protocol V2 Format
|
|
|
|
### Protocol Markers
|
|
|
|
Protocol V2 uses special Unicode markers to embed structured data within TAP output:
|
|
|
|
- `⟦TSTEST:` - Start marker
|
|
- `⟧` - End marker
|
|
|
|
These markers allow structured data to coexist with standard TAP without breaking compatibility.
|
|
|
|
### Example Output
|
|
|
|
```tap
|
|
⟦TSTEST:PROTOCOL:2.0.0⟧
|
|
TAP version 13
|
|
1..3
|
|
ok 1 - should add numbers ⟦TSTEST:time:42⟧
|
|
not ok 2 - should validate input
|
|
⟦TSTEST:META:{"time":156,"file":"test.ts","line":42}⟧
|
|
⟦TSTEST:ERROR⟧
|
|
{
|
|
"error": {
|
|
"message": "Expected 5 to equal 6",
|
|
"diff": {...}
|
|
}
|
|
}
|
|
⟦TSTEST:/ERROR⟧
|
|
ok 3 - should handle edge cases # SKIP not implemented ⟦TSTEST:SKIP:not implemented⟧
|
|
```
|
|
|
|
## API Reference
|
|
|
|
### ProtocolEmitter
|
|
|
|
Generates Protocol V2 messages. Used by tapbundle to emit test results.
|
|
|
|
#### `emitProtocolHeader()`
|
|
|
|
Emit the protocol version header.
|
|
|
|
```typescript
|
|
import { ProtocolEmitter } from '@git.zone/tstest/tapbundle_protocol';
|
|
|
|
const emitter = new ProtocolEmitter();
|
|
console.log(emitter.emitProtocolHeader());
|
|
// Output: ⟦TSTEST:PROTOCOL:2.0.0⟧
|
|
```
|
|
|
|
#### `emitTapVersion(version?)`
|
|
|
|
Emit TAP version line.
|
|
|
|
```typescript
|
|
console.log(emitter.emitTapVersion(13));
|
|
// Output: TAP version 13
|
|
```
|
|
|
|
#### `emitPlan(plan)`
|
|
|
|
Emit test plan.
|
|
|
|
```typescript
|
|
console.log(emitter.emitPlan({ start: 1, end: 5 }));
|
|
// Output: 1..5
|
|
|
|
console.log(emitter.emitPlan({ start: 1, end: 0, skipAll: 'Not ready' }));
|
|
// Output: 1..0 # Skipped: Not ready
|
|
```
|
|
|
|
#### `emitTest(result)`
|
|
|
|
Emit a test result with optional metadata.
|
|
|
|
```typescript
|
|
const lines = emitter.emitTest({
|
|
ok: true,
|
|
testNumber: 1,
|
|
description: 'should work correctly',
|
|
metadata: {
|
|
time: 45,
|
|
tags: ['unit', 'fast']
|
|
}
|
|
});
|
|
|
|
lines.forEach(line => console.log(line));
|
|
// Output:
|
|
// ok 1 - should work correctly ⟦TSTEST:time:45⟧
|
|
// ⟦TSTEST:META:{"tags":["unit","fast"]}⟧
|
|
```
|
|
|
|
#### `emitComment(comment)`
|
|
|
|
Emit a comment line.
|
|
|
|
```typescript
|
|
console.log(emitter.emitComment('Setup complete'));
|
|
// Output: # Setup complete
|
|
```
|
|
|
|
#### `emitBailout(reason)`
|
|
|
|
Emit a bailout (abort all tests).
|
|
|
|
```typescript
|
|
console.log(emitter.emitBailout('Database connection failed'));
|
|
// Output: Bail out! Database connection failed
|
|
```
|
|
|
|
#### `emitError(error)`
|
|
|
|
Emit a structured error block.
|
|
|
|
```typescript
|
|
const lines = emitter.emitError({
|
|
testNumber: 2,
|
|
error: {
|
|
message: 'Expected 5 to equal 6',
|
|
stack: 'Error: ...',
|
|
actual: 5,
|
|
expected: 6,
|
|
diff: '...'
|
|
}
|
|
});
|
|
|
|
lines.forEach(line => console.log(line));
|
|
// Output:
|
|
// ⟦TSTEST:ERROR⟧
|
|
// {
|
|
// "testNumber": 2,
|
|
// "error": { ... }
|
|
// }
|
|
// ⟦TSTEST:/ERROR⟧
|
|
```
|
|
|
|
#### `emitEvent(event)`
|
|
|
|
Emit a real-time test event.
|
|
|
|
```typescript
|
|
console.log(emitter.emitEvent({
|
|
eventType: 'test:started',
|
|
timestamp: Date.now(),
|
|
data: {
|
|
testNumber: 1,
|
|
description: 'should work'
|
|
}
|
|
}));
|
|
// Output: ⟦TSTEST:EVENT:{"eventType":"test:started",...}⟧
|
|
```
|
|
|
|
#### `emitSnapshot(snapshot)`
|
|
|
|
Emit snapshot data.
|
|
|
|
```typescript
|
|
const lines = emitter.emitSnapshot({
|
|
name: 'user-data',
|
|
content: { name: 'Alice', age: 30 },
|
|
format: 'json'
|
|
});
|
|
|
|
lines.forEach(line => console.log(line));
|
|
// Output:
|
|
// ⟦TSTEST:SNAPSHOT:user-data⟧
|
|
// {
|
|
// "name": "Alice",
|
|
// "age": 30
|
|
// }
|
|
// ⟦TSTEST:/SNAPSHOT⟧
|
|
```
|
|
|
|
### ProtocolParser
|
|
|
|
Parses Protocol V2 messages. Used by tstest to consume test results.
|
|
|
|
#### `parseLine(line)`
|
|
|
|
Parse a single line and return protocol messages.
|
|
|
|
```typescript
|
|
import { ProtocolParser } from '@git.zone/tstest/tapbundle_protocol';
|
|
|
|
const parser = new ProtocolParser();
|
|
|
|
const messages = parser.parseLine('ok 1 - test passed ⟦TSTEST:time:42⟧');
|
|
console.log(messages);
|
|
// Output:
|
|
// [{
|
|
// type: 'test',
|
|
// content: {
|
|
// ok: true,
|
|
// testNumber: 1,
|
|
// description: 'test passed',
|
|
// metadata: { time: 42 }
|
|
// }
|
|
// }]
|
|
```
|
|
|
|
#### Message Types
|
|
|
|
The parser returns different message types:
|
|
|
|
```typescript
|
|
interface IProtocolMessage {
|
|
type: 'test' | 'plan' | 'comment' | 'version' | 'bailout' | 'protocol' | 'snapshot' | 'error' | 'event';
|
|
content: any;
|
|
}
|
|
```
|
|
|
|
**Examples:**
|
|
|
|
```typescript
|
|
// Test result
|
|
{
|
|
type: 'test',
|
|
content: {
|
|
ok: true,
|
|
testNumber: 1,
|
|
description: 'test name',
|
|
metadata: { ... }
|
|
}
|
|
}
|
|
|
|
// Plan
|
|
{
|
|
type: 'plan',
|
|
content: {
|
|
start: 1,
|
|
end: 5
|
|
}
|
|
}
|
|
|
|
// Event
|
|
{
|
|
type: 'event',
|
|
content: {
|
|
eventType: 'test:started',
|
|
timestamp: 1234567890,
|
|
data: { ... }
|
|
}
|
|
}
|
|
|
|
// Error
|
|
{
|
|
type: 'error',
|
|
content: {
|
|
testNumber: 2,
|
|
error: {
|
|
message: '...',
|
|
stack: '...',
|
|
diff: '...'
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### `getProtocolVersion()`
|
|
|
|
Get the detected protocol version.
|
|
|
|
```typescript
|
|
const version = parser.getProtocolVersion();
|
|
console.log(version); // "2.0.0" or null
|
|
```
|
|
|
|
## TypeScript Types
|
|
|
|
### ITestResult
|
|
|
|
```typescript
|
|
interface ITestResult {
|
|
ok: boolean;
|
|
testNumber: number;
|
|
description: string;
|
|
directive?: {
|
|
type: 'skip' | 'todo';
|
|
reason?: string;
|
|
};
|
|
metadata?: ITestMetadata;
|
|
}
|
|
```
|
|
|
|
### ITestMetadata
|
|
|
|
```typescript
|
|
interface ITestMetadata {
|
|
// Timing
|
|
time?: number; // Test duration in milliseconds
|
|
startTime?: number; // Unix timestamp
|
|
endTime?: number; // Unix timestamp
|
|
|
|
// Status
|
|
skip?: string; // Skip reason
|
|
todo?: string; // Todo reason
|
|
retry?: number; // Current retry attempt
|
|
maxRetries?: number; // Max retries allowed
|
|
|
|
// Error details
|
|
error?: {
|
|
message: string;
|
|
stack?: string;
|
|
diff?: string;
|
|
actual?: any;
|
|
expected?: any;
|
|
code?: string;
|
|
};
|
|
|
|
// Test context
|
|
file?: string; // Source file path
|
|
line?: number; // Line number
|
|
column?: number; // Column number
|
|
|
|
// Custom data
|
|
tags?: string[]; // Test tags
|
|
custom?: Record<string, any>;
|
|
}
|
|
```
|
|
|
|
### ITestEvent
|
|
|
|
```typescript
|
|
interface ITestEvent {
|
|
eventType: EventType;
|
|
timestamp: number;
|
|
data: {
|
|
testNumber?: number;
|
|
description?: string;
|
|
suiteName?: string;
|
|
hookName?: string;
|
|
progress?: number; // 0-100
|
|
duration?: number;
|
|
error?: IEnhancedError;
|
|
[key: string]: any;
|
|
};
|
|
}
|
|
|
|
type EventType =
|
|
| 'test:queued'
|
|
| 'test:started'
|
|
| 'test:progress'
|
|
| 'test:completed'
|
|
| 'suite:started'
|
|
| 'suite:completed'
|
|
| 'hook:started'
|
|
| 'hook:completed'
|
|
| 'assertion:failed';
|
|
```
|
|
|
|
### IEnhancedError
|
|
|
|
```typescript
|
|
interface IEnhancedError {
|
|
message: string;
|
|
stack?: string;
|
|
diff?: IDiffResult;
|
|
actual?: any;
|
|
expected?: any;
|
|
code?: string;
|
|
type?: 'assertion' | 'timeout' | 'uncaught' | 'syntax' | 'runtime';
|
|
}
|
|
```
|
|
|
|
### IDiffResult
|
|
|
|
```typescript
|
|
interface IDiffResult {
|
|
type: 'string' | 'object' | 'array' | 'primitive';
|
|
changes: IDiffChange[];
|
|
context?: number; // Lines of context
|
|
}
|
|
|
|
interface IDiffChange {
|
|
type: 'add' | 'remove' | 'modify';
|
|
path?: string[]; // For object/array diffs
|
|
oldValue?: any;
|
|
newValue?: any;
|
|
line?: number; // For string diffs
|
|
content?: string;
|
|
}
|
|
```
|
|
|
|
## Protocol Constants
|
|
|
|
```typescript
|
|
import { PROTOCOL_MARKERS, PROTOCOL_VERSION } from '@git.zone/tstest/tapbundle_protocol';
|
|
|
|
console.log(PROTOCOL_VERSION); // "2.0.0"
|
|
console.log(PROTOCOL_MARKERS.START); // "⟦TSTEST:"
|
|
console.log(PROTOCOL_MARKERS.END); // "⟧"
|
|
```
|
|
|
|
### Available Markers
|
|
|
|
```typescript
|
|
const PROTOCOL_MARKERS = {
|
|
START: '⟦TSTEST:',
|
|
END: '⟧',
|
|
META_PREFIX: 'META:',
|
|
ERROR_PREFIX: 'ERROR',
|
|
ERROR_END: '/ERROR',
|
|
SNAPSHOT_PREFIX: 'SNAPSHOT:',
|
|
SNAPSHOT_END: '/SNAPSHOT',
|
|
PROTOCOL_PREFIX: 'PROTOCOL:',
|
|
SKIP_PREFIX: 'SKIP:',
|
|
TODO_PREFIX: 'TODO:',
|
|
EVENT_PREFIX: 'EVENT:',
|
|
};
|
|
```
|
|
|
|
## Usage Patterns
|
|
|
|
### Creating a Custom Test Runner
|
|
|
|
```typescript
|
|
import { ProtocolEmitter } from '@git.zone/tstest/tapbundle_protocol';
|
|
|
|
const emitter = new ProtocolEmitter();
|
|
|
|
// Emit protocol header
|
|
console.log(emitter.emitProtocolHeader());
|
|
console.log(emitter.emitTapVersion(13));
|
|
|
|
// Emit plan
|
|
console.log(emitter.emitPlan({ start: 1, end: 2 }));
|
|
|
|
// Run test 1
|
|
emitter.emitEvent({
|
|
eventType: 'test:started',
|
|
timestamp: Date.now(),
|
|
data: { testNumber: 1 }
|
|
}).split('\n').forEach(line => console.log(line));
|
|
|
|
const result1 = emitter.emitTest({
|
|
ok: true,
|
|
testNumber: 1,
|
|
description: 'first test',
|
|
metadata: { time: 45 }
|
|
});
|
|
result1.forEach(line => console.log(line));
|
|
|
|
// Run test 2
|
|
const result2 = emitter.emitTest({
|
|
ok: false,
|
|
testNumber: 2,
|
|
description: 'second test',
|
|
metadata: {
|
|
time: 120,
|
|
error: {
|
|
message: 'Assertion failed',
|
|
actual: 5,
|
|
expected: 6
|
|
}
|
|
}
|
|
});
|
|
result2.forEach(line => console.log(line));
|
|
```
|
|
|
|
### Parsing Test Output
|
|
|
|
```typescript
|
|
import { ProtocolParser } from '@git.zone/tstest/tapbundle_protocol';
|
|
import * as readline from 'readline';
|
|
|
|
const parser = new ProtocolParser();
|
|
|
|
const rl = readline.createInterface({
|
|
input: process.stdin,
|
|
output: process.stdout
|
|
});
|
|
|
|
rl.on('line', (line) => {
|
|
const messages = parser.parseLine(line);
|
|
|
|
messages.forEach(message => {
|
|
switch (message.type) {
|
|
case 'test':
|
|
console.log(`Test ${message.content.testNumber}: ${message.content.ok ? 'PASS' : 'FAIL'}`);
|
|
break;
|
|
case 'event':
|
|
console.log(`Event: ${message.content.eventType}`);
|
|
break;
|
|
case 'error':
|
|
console.error(`Error: ${message.content.error.message}`);
|
|
break;
|
|
}
|
|
});
|
|
});
|
|
```
|
|
|
|
### Building Test Dashboards
|
|
|
|
Real-time events enable building live test dashboards:
|
|
|
|
```typescript
|
|
const parser = new ProtocolParser();
|
|
|
|
parser.parseLine(line).forEach(message => {
|
|
if (message.type === 'event') {
|
|
const event = message.content;
|
|
|
|
switch (event.eventType) {
|
|
case 'test:started':
|
|
updateUI({ status: 'running', test: event.data.description });
|
|
break;
|
|
case 'test:completed':
|
|
updateUI({ status: 'done', duration: event.data.duration });
|
|
break;
|
|
case 'suite:started':
|
|
createSuiteCard(event.data.suiteName);
|
|
break;
|
|
}
|
|
}
|
|
});
|
|
```
|
|
|
|
## Backward Compatibility
|
|
|
|
Protocol V2 is fully backward compatible with standard TAP parsers:
|
|
|
|
- Protocol markers use Unicode characters that TAP parsers ignore
|
|
- Standard TAP output (ok/not ok, plan, comments) works everywhere
|
|
- Enhanced features gracefully degrade in standard TAP consumers
|
|
|
|
**Standard TAP View:**
|
|
```tap
|
|
TAP version 13
|
|
1..3
|
|
ok 1 - should add numbers
|
|
not ok 2 - should validate input
|
|
ok 3 - should handle edge cases # SKIP not implemented
|
|
```
|
|
|
|
**Protocol V2 View (same output):**
|
|
```tap
|
|
⟦TSTEST:PROTOCOL:2.0.0⟧
|
|
TAP version 13
|
|
1..3
|
|
ok 1 - should add numbers ⟦TSTEST:time:42⟧
|
|
not ok 2 - should validate input
|
|
⟦TSTEST:META:{"time":156}⟧
|
|
ok 3 - should handle edge cases # SKIP not implemented ⟦TSTEST:SKIP:not implemented⟧
|
|
```
|
|
|
|
## Isomorphic Design
|
|
|
|
This module works in all JavaScript environments:
|
|
|
|
- ✅ Node.js
|
|
- ✅ Browsers (via tapbundle)
|
|
- ✅ Deno
|
|
- ✅ Bun
|
|
- ✅ Web Workers
|
|
- ✅ Service Workers
|
|
|
|
No runtime-specific APIs are used, making it truly portable.
|
|
|
|
## Legal
|
|
|
|
This project is licensed under MIT.
|
|
|
|
© 2025 Task Venture Capital GmbH. All rights reserved.
|