git.zone/tstest
2
0
Fork 0
Code Issues 2 Pull Requests Packages Projects Releases Wiki Activity

feat(core): Implement Protocol V2 with enhanced settings and lifecycle hooks

Browse Source
This commit is contained in:
Philipp Kunz 2025-05-26 04:02:32 +00:00
parent 91880f8d42
commit 33d2ff1d4f
24 changed files with 2356 additions and 441 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
changelog.md
View File

@ -1,5 +1,15 @@
# Changelog
## 2025-05-26 - 2.1.0 - feat(core)
Implement Protocol V2 with enhanced settings and lifecycle hooks
- Migrated to Protocol V2 using Unicode markers and structured metadata with new ts_tapbundle_protocol module
- Refactored TAP parser/emitter to support improved protocol parsing and error reporting
- Integrated global settings via tap.settings() and lifecycle hooks (beforeAll/afterAll, beforeEach/afterEach)
- Enhanced expect wrapper with diff generation for clearer assertion failures
- Updated test loader to automatically run 00init.ts for proper test configuration
- Revised documentation (readme.hints.md, readme.plan.md) to reflect current implementation status and remaining work
## 2025-05-25 - 2.0.0 - BREAKING CHANGE(protocol)
Introduce protocol v2 implementation and update build configuration with revised build order, new tspublish files, and enhanced documentation

92
readme.hints.md
View File

@ -125,4 +125,94 @@ The protocol v2 implementation is contained in a separate `ts_tapbundle_protocol
This architectural decision ensures the protocol can be used in any JavaScript environment without modification and maintains proper build dependencies.
See `readme.protocol.md` for the full specification and `ts_tapbundle_protocol/` for the implementation.
See `readme.protocol.md` for the full specification and `ts_tapbundle_protocol/` for the implementation.
## Protocol V2 Implementation Status
The Protocol V2 has been implemented to fix issues with TAP protocol parsing when test descriptions contain special characters like `#`, `###SNAPSHOT###`, or protocol markers like `⟦TSTEST:ERROR⟧`.
### Implementation Details:
1. **Protocol Components**:
- `ProtocolEmitter` - Generates protocol v2 messages (used by tapbundle)
- `ProtocolParser` - Parses protocol v2 messages (used by tstest)
- Uses Unicode markers `⟦TSTEST:` and `⟧` to avoid conflicts with test content
2. **Current Status**:
- ✅ Basic protocol emission and parsing works
- ✅ Handles test descriptions with special characters correctly
- ✅ Supports metadata for timing, tags, errors
- ⚠️ Protocol messages sometimes appear in console output (parsing not catching all cases)
3. **Key Findings**:
- `tap.skip.test()` doesn't create actual test objects, just logs and increments counter
- `tap.todo()` method is not implemented (no `addTodo` method in Tap class)
- Protocol parser's `isBlockStart` was fixed to only match exact block markers, not partial matches in test descriptions
4. **Import Paths**:
- tstest imports from: `import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';`
- tapbundle imports from: `import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';`
## Test Configuration System (Phase 2)
The Test Configuration System has been implemented to provide global settings and lifecycle hooks for tests.
### Key Features:
1. **00init.ts Discovery**:
- Automatically detects `00init.ts` files in the same directory as test files
- Creates a temporary loader file that imports both `00init.ts` and the test file
- Loader files are cleaned up automatically after test execution
2. **Settings Inheritance**:
- Global settings from `00init.ts` → File-level settings → Test-level settings
- Settings include: timeout, retries, retryDelay, bail, concurrency
- Lifecycle hooks: beforeAll, afterAll, beforeEach, afterEach
3. **Implementation Details**:
- `SettingsManager` class handles settings inheritance and merging
- `tap.settings()` API allows configuration at any level
- Lifecycle hooks are integrated into test execution flow
### Important Development Notes:
1. **Local Development**: When developing tstest itself, use `node cli.js` instead of globally installed `tstest` to test changes
2. **Console Output Buffering**: Console output from tests is buffered and only displayed for failing tests. TAP-compliant comments (lines starting with `#`) are always shown.
3. **TypeScript Warnings**: Fixed async/await warnings in `movePreviousLogFiles()` by using sync versions of file operations
## Enhanced Communication Features (Phase 3)
The Enhanced Communication system has been implemented to provide rich, real-time feedback during test execution.
### Key Features:
1. **Event-Based Test Lifecycle Reporting**:
- `test:queued` - Test is ready to run
- `test:started` - Test execution begins
- `test:completed` - Test finishes (with pass/fail status)
- `suite:started` - Test suite/describe block begins
- `suite:completed` - Test suite/describe block ends
- `hook:started` - Lifecycle hook (beforeEach/afterEach) begins
- `hook:completed` - Lifecycle hook finishes
- `assertion:failed` - Assertion failure with detailed information
2. **Visual Diff Output for Assertion Failures**:
- **String Diffs**: Character-by-character comparison with colored output
- **Object/Array Diffs**: Deep property comparison showing added/removed/changed properties
- **Primitive Diffs**: Clear display of expected vs actual values
- **Colorized Output**: Green for expected, red for actual, yellow for differences
- **Smart Formatting**: Multi-line strings and complex objects are formatted for readability
3. **Real-Time Test Progress API**:
- Tests emit progress events as they execute
- tstest parser processes events and updates display in real-time
- Structured event format carries rich metadata (timing, errors, diffs)
- Seamless integration with existing TAP protocol via Protocol V2
### Implementation Details:
- Events are transmitted via Protocol V2's `EVENT` block type
- Event data is JSON-encoded within protocol markers
- Parser handles events asynchronously for real-time updates
- Visual diffs are generated using custom diff algorithms for each data type

172
readme.plan.md
View File

@ -2,33 +2,33 @@
!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
## Improved Internal Protocol (NEW - Critical)
## Improved Internal Protocol (NEW - Critical) ✅ COMPLETED
### Current Issues
- TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
- Fragile regex parsing that breaks with special characters
- Limited extensibility for new metadata types
### Current Issues ✅ RESOLVED
- ✅ TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
- ✅ Fragile regex parsing that breaks with special characters
- ✅ Limited extensibility for new metadata types
### Proposed Solution: Protocol V2
- Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
- Structured JSON metadata format
- Separate protocol blocks for complex data (errors, snapshots)
- Complete replacement of v1 (no backwards compatibility needed)
### Proposed Solution: Protocol V2 ✅ IMPLEMENTED
- ✅ Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
- ✅ Structured JSON metadata format
- ✅ Separate protocol blocks for complex data (errors, snapshots)
- ✅ Complete replacement of v1 (no backwards compatibility needed)
### Implementation
- Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
- Phase 2: Replace all v1 code in both tstest and tapbundle with v2
- Phase 3: Delete all v1 parsing and generation code
### Implementation ✅ COMPLETED
- ✅ Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
- ✅ Phase 2: Replace all v1 code in both tstest and tapbundle with v2
- ✅ Phase 3: Delete all v1 parsing and generation code
#### ts_tapbundle_protocol Directory
The protocol v2 implementation will be contained in the `ts_tapbundle_protocol` directory as isomorphic TypeScript code:
- **Isomorphic Design**: All code must work in both browser and Node.js environments
- **No Node.js Imports**: No Node.js-specific modules allowed (no fs, path, child_process, etc.)
- **Protocol Classes**: Contains classes implementing all sides of the protocol:
- `ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
- `ProtocolParser`: For parsing protocol v2 messages (used by tstest)
- `ProtocolMessage`: Base classes for different message types
- `ProtocolTypes`: TypeScript interfaces and types for protocol structures
- ✅ `ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
- ✅ `ProtocolParser`: For parsing protocol v2 messages (used by tstest)
- ✅ `ProtocolMessage`: Base classes for different message types
- ✅ `ProtocolTypes`: TypeScript interfaces and types for protocol structures
- **Pure TypeScript**: Only browser-compatible APIs and pure TypeScript/JavaScript code
- **Build Integration**:
- Compiled by `pnpm build` (via tsbuild) to `dist_ts_tapbundle_protocol/`
@ -92,19 +92,19 @@ interface TapSettings {
3. **Application**: Apply settings to test execution
4. **Advanced**: Parallel execution and snapshot configuration
## 1. Enhanced Communication Between tapbundle and tstest
## 1. Enhanced Communication Between tapbundle and tstest ✅ COMPLETED
### 1.1 Real-time Test Progress API
- Create a bidirectional communication channel between tapbundle and tstest
- Emit events for test lifecycle stages (start, progress, completion)
- Allow tstest to subscribe to tapbundle events for better progress reporting
- Implement a standardized message format for test metadata
### 1.1 Real-time Test Progress API ✅ COMPLETED
- ✅ Create a bidirectional communication channel between tapbundle and tstest
- ✅ Emit events for test lifecycle stages (start, progress, completion)
- ✅ Allow tstest to subscribe to tapbundle events for better progress reporting
- ✅ Implement a standardized message format for test metadata
### 1.2 Rich Error Reporting
- Pass structured error objects from tapbundle to tstest
- Include stack traces, code snippets, and contextual information
- Support for error categorization (assertion failures, timeouts, uncaught exceptions)
- Visual diff output for failed assertions
### 1.2 Rich Error Reporting ✅ COMPLETED
- ✅ Pass structured error objects from tapbundle to tstest
- ✅ Include stack traces, code snippets, and contextual information
- ✅ Support for error categorization (assertion failures, timeouts, uncaught exceptions)
- ✅ Visual diff output for failed assertions
## 2. Enhanced toolsArg Functionality
@ -155,7 +155,7 @@ tap.test('performance test', async (toolsArg) => {
- Fast feedback loop for development
- Integration with IDE/editor plugins
### 5.3 Advanced Test Filtering (Partial)
### 5.3 Advanced Test Filtering (Partial) ⚠️
```typescript
// Exclude tests by pattern (not yet implemented)
tstest --exclude "**/slow/**"
@ -197,38 +197,38 @@ tstest --changed
## Implementation Phases
### Phase 1: Improved Internal Protocol (Priority: Critical) (NEW)
1. Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
- Implement ProtocolEmitter class for message generation
- Implement ProtocolParser class for message parsing
- Define ProtocolMessage types and interfaces
- Ensure all code is browser and Node.js compatible
- Add tspublish.json to configure build order
2. Update build configuration to compile ts_tapbundle_protocol first
3. Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
4. Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
5. Delete all v1 TAP parsing code from tstest
6. Delete all v1 TAP generation code from tapbundle
7. Test with real-world test suites containing special characters
### Phase 1: Improved Internal Protocol (Priority: Critical) ✅ COMPLETED
1. ✅ Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
- ✅ Implement ProtocolEmitter class for message generation
- ✅ Implement ProtocolParser class for message parsing
- ✅ Define ProtocolMessage types and interfaces
- ✅ Ensure all code is browser and Node.js compatible
- ✅ Add tspublish.json to configure build order
2. ✅ Update build configuration to compile ts_tapbundle_protocol first
3. ✅ Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
4. ✅ Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
5. ✅ Delete all v1 TAP parsing code from tstest
6. ✅ Delete all v1 TAP generation code from tapbundle
7. ✅ Test with real-world test suites containing special characters
### Phase 2: Test Configuration System (Priority: High)
1. Implement tap.settings() API with TypeScript interfaces
2. Add 00init.ts discovery and loading mechanism
3. Implement settings inheritance and merge logic
4. Apply settings to test execution (timeouts, retries, etc.)
### Phase 2: Test Configuration System (Priority: High) ✅ COMPLETED
1. ✅ Implement tap.settings() API with TypeScript interfaces
2. ✅ Add 00init.ts discovery and loading mechanism
3. ✅ Implement settings inheritance and merge logic
4. ✅ Apply settings to test execution (timeouts, retries, etc.)
### Phase 3: Enhanced Communication (Priority: High)
1. Build on Protocol V2 for richer communication
2. Implement real-time test progress API
3. Add structured error reporting with diffs and traces
### Phase 3: Enhanced Communication (Priority: High) ✅ COMPLETED
1. ✅ Build on Protocol V2 for richer communication
2. ✅ Implement real-time test progress API
3. ✅ Add structured error reporting with diffs and traces
### Phase 4: Developer Experience (Priority: Medium)
### Phase 4: Developer Experience (Priority: Medium) ❌ NOT STARTED
1. Add watch mode
2. Implement custom reporters
3. Complete advanced test filtering options
4. Add performance benchmarking API
### Phase 5: Analytics and Performance (Priority: Low)
### Phase 5: Analytics and Performance (Priority: Low) ❌ NOT STARTED
1. Build test analytics dashboard
2. Implement coverage integration
3. Create trend analysis tools
@ -252,4 +252,66 @@ tstest --changed
- Clean interfaces between tstest and tapbundle
- Extensible plugin architecture
- Standard test result format
- Compatible with existing CI/CD tools
- Compatible with existing CI/CD tools
## Summary of Remaining Work
### ✅ Completed
- **Protocol V2**: Full implementation with Unicode delimiters, structured metadata, and special character handling
- **Test Configuration System**: tap.settings() API, 00init.ts discovery, settings inheritance, lifecycle hooks
- **Enhanced Communication**: Event-based test lifecycle reporting, visual diff output for assertion failures, real-time test progress API
- **Rich Error Reporting**: Stack traces, error metadata, and visual diffs through protocol
- **Tags Filtering**: `--tags` option for running specific tagged tests
### ✅ Existing Features (Not in Plan)
- **Timeout Support**: `--timeout` option and per-test timeouts
- **Test Retries**: `tap.retry()` for flaky test handling
- **Parallel Tests**: `.testParallel()` for concurrent execution
- **Snapshot Testing**: Basic implementation with `toMatchSnapshot()`
- **Test Lifecycle**: `describe()` blocks with `beforeEach`/`afterEach`
- **Skip Tests**: `tap.skip.test()` (though it doesn't create test objects)
- **Log Files**: `--logfile` option saves output to `.nogit/testlogs/`
- **Test Range**: `--startFrom` and `--stopAt` for partial runs
### ⚠️ Partially Completed
- **Advanced Test Filtering**: Have `--tags` but missing `--exclude`, `--failed`, `--changed`
### ❌ Not Started
#### High Priority
#### Medium Priority
2. **Developer Experience**
- Watch mode for file changes
- Custom reporters (JSON, JUnit, HTML, Markdown)
- Performance benchmarking API
- Better error messages with suggestions
3. **Enhanced toolsArg**
- Test data injection
- Context sharing between tests
- Parameterized tests
4. **Test Organization**
- Hierarchical test suites
- Nested describe blocks
- Suite-level lifecycle hooks
#### Low Priority
5. **Analytics and Performance**
- Test analytics dashboard
- Code coverage integration
- Trend analysis
- Flaky test detection
### Known Issues to Fix
- **tap.todo()**: Method exists but has no implementation
- **tap.skip.test()**: Doesn't create test objects, just logs (breaks test count)
- **Protocol Output**: Some protocol messages still appear in console output
- **Only Tests**: `tap.only.test()` exists but `--only` mode not fully implemented
### Next Recommended Steps
1. Add Watch Mode - high developer value
2. Implement Custom Reporters - important for CI/CD integration
3. Fix known issues: tap.todo() and tap.skip.test() implementations
4. Implement performance benchmarking API

41
test/config-test/00init.ts Normal file
View File

@ -0,0 +1,41 @@
import { tap } from '../../ts_tapbundle/index.js';
// TAP-compliant comment output
console.log('# 🚀 00init.ts: LOADED AND EXECUTING');
console.log('# 🚀 00init.ts: Setting up global test configuration');
// Add a global variable to verify 00init.ts was loaded
(global as any).__00INIT_LOADED = true;
// Configure global test settings
tap.settings({
// Set a default timeout of 5 seconds for all tests
timeout: 5000,
// Enable retries for flaky tests
retries: 2,
retryDelay: 1000,
// Show test duration
showTestDuration: true,
// Global lifecycle hooks
beforeAll: async () => {
console.log('Global beforeAll: Initializing test environment');
},
afterAll: async () => {
console.log('Global afterAll: Cleaning up test environment');
},
beforeEach: async (testName: string) => {
console.log(`Global beforeEach: Starting test "${testName}"`);
},
afterEach: async (testName: string, passed: boolean) => {
console.log(`Global afterEach: Test "${testName}" ${passed ? 'passed' : 'failed'}`);
}
});
console.log('# 🚀 00init.ts: Configuration COMPLETE');
console.log('# 🚀 00init.ts: tap.settings() called successfully');

44
test/config-test/test.config.ts Normal file
View File

@ -0,0 +1,44 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
// TAP-compliant comment output
console.log('# 🔍 TEST FILE LOADED - test.config.ts');
// Check if 00init.ts was loaded
const initLoaded = (global as any).__00INIT_LOADED;
console.log(`# 🔍 00init.ts loaded: ${initLoaded === true}`);
// Test that uses the global timeout setting
tap.test('Test with global timeout', async (toolsArg) => {
// This test should complete within the 5 second timeout set in 00init.ts
await toolsArg.delayFor(2000); // 2 seconds
expect(true).toBeTrue();
});
// Test that demonstrates retries
tap.test('Test with retries', async () => {
// This test will use the global retry setting (2 retries)
console.log('Running test that might be flaky');
// Simulate a flaky test that passes on second try
const randomValue = Math.random();
console.log(`Random value: ${randomValue}`);
// Always pass for demonstration
expect(true).toBeTrue();
});
// Test with custom timeout that overrides global
tap.timeout(1000).test('Test with custom timeout', async (toolsArg) => {
// This test has a 1 second timeout, overriding the global 5 seconds
await toolsArg.delayFor(500); // 500ms - should pass
expect(true).toBeTrue();
});
// Test to verify lifecycle hooks are working
tap.test('Test lifecycle hooks', async () => {
console.log('Inside test: lifecycle hooks should have run');
expect(true).toBeTrue();
});
// Start the test suite
tap.start();

22
test/config-test/test.file-settings.ts Normal file
View File

@ -0,0 +1,22 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
// Override global settings for this file
tap.settings({
timeout: 2000, // Override global timeout to 2 seconds
retries: 0, // Disable retries for this file
});
tap.test('Test with file-level timeout', async (toolsArg) => {
// This should use the file-level timeout of 2 seconds
console.log('Running with file-level timeout of 2 seconds');
await toolsArg.delayFor(1000); // 1 second - should pass
expect(true).toBeTrue();
});
tap.test('Test without retries', async () => {
// This test should not retry even if it fails
console.log('This test has no retries (file-level setting)');
expect(true).toBeTrue();
});
tap.start();

56
test/tapbundle/test.diff-demo.ts Normal file
View File

@ -0,0 +1,56 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
tap.test('should show string diff', async () => {
const expected = `Hello World
This is a test
of multiline strings`;
const actual = `Hello World
This is a demo
of multiline strings`;
// This will fail and show a diff
expect(actual).toEqual(expected);
});
tap.test('should show object diff', async () => {
const expected = {
name: 'John',
age: 30,
city: 'New York',
hobbies: ['reading', 'coding']
};
const actual = {
name: 'John',
age: 31,
city: 'Boston',
hobbies: ['reading', 'gaming']
};
// This will fail and show a diff
expect(actual).toEqual(expected);
});
tap.test('should show array diff', async () => {
const expected = [1, 2, 3, 4, 5];
const actual = [1, 2, 3, 5, 6];
// This will fail and show a diff
expect(actual).toEqual(expected);
});
tap.test('should show primitive diff', async () => {
const expected = 42;
const actual = 43;
// This will fail and show a diff
expect(actual).toBe(expected);
});
tap.test('should pass without diff', async () => {
expect(true).toBe(true);
expect('hello').toEqual('hello');
});
tap.start({ throwOnError: false });

23
test/tapbundle/test.simple-diff.ts Normal file
View File

@ -0,0 +1,23 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
tap.test('should show string diff', async () => {
const expected = `line 1
line 2
line 3`;
const actual = `line 1
line changed
line 3`;
try {
expect(actual).toEqual(expected);
} catch (e) {
// Expected to fail
}
});
tap.test('should pass', async () => {
expect('hello').toEqual('hello');
});
tap.start({ throwOnError: false });

2
ts/00_commitinfo_data.ts
View File

@ -3,6 +3,6 @@
*/
export const commitinfo = {
name: '@git.zone/tstest',
version: '2.0.0',
version: '2.1.0',
description: 'a test utility to run tests that match test/**/*.ts'
}

385
ts/tstest.classes.tap.parser.ts
View File

@ -8,28 +8,27 @@ import * as plugins from './tstest.plugins.js';
import { TapTestResult } from './tstest.classes.tap.testresult.js';
import * as logPrefixes from './tstest.logprefixes.js';
import { TsTestLogger } from './tstest.logging.js';
import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';
import type { IProtocolMessage, ITestResult, IPlanLine, IErrorBlock, ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
export class TapParser {
testStore: TapTestResult[] = [];
expectedTestsRegex = /([0-9]*)\.\.([0-9]*)$/;
expectedTests: number;
receivedTests: number;
expectedTests: number = 0;
receivedTests: number = 0;
testStatusRegex = /(ok|not\sok)\s([0-9]+)\s-\s(.*?)(\s#\s(.*))?$/;
activeTapTestResult: TapTestResult;
collectingErrorDetails: boolean = false;
currentTestError: string[] = [];
pretaskRegex = /^::__PRETASK:(.*)$/;
private logger: TsTestLogger;
private protocolParser: ProtocolParser;
private protocolVersion: string | null = null;
/**
* the constructor for TapParser
*/
constructor(public fileName: string, logger?: TsTestLogger) {
this.logger = logger;
this.protocolParser = new ProtocolParser();
}
/**
@ -75,137 +74,299 @@ export class TapParser {
logLineArray.pop();
}
// lets parse the log information
// Process each line through the protocol parser
for (const logLine of logLineArray) {
let logLineIsTapProtocol = false;
if (!this.expectedTests && this.expectedTestsRegex.test(logLine)) {
logLineIsTapProtocol = true;
const regexResult = this.expectedTestsRegex.exec(logLine);
this.expectedTests = parseInt(regexResult[2]);
if (this.logger) {
this.logger.tapOutput(`Expecting ${this.expectedTests} tests!`);
const messages = this.protocolParser.parseLine(logLine);
if (messages.length > 0) {
// Handle protocol messages
for (const message of messages) {
this._handleProtocolMessage(message, logLine);
}
// initiating first TapResult
this._getNewTapTestResult();
} else if (this.pretaskRegex.test(logLine)) {
logLineIsTapProtocol = true;
const pretaskContentMatch = this.pretaskRegex.exec(logLine);
if (pretaskContentMatch && pretaskContentMatch[1]) {
if (this.logger) {
this.logger.tapOutput(`Pretask -> ${pretaskContentMatch[1]}: Success.`);
}
}
} else if (this.testStatusRegex.test(logLine)) {
logLineIsTapProtocol = true;
const regexResult = this.testStatusRegex.exec(logLine);
// const testId = parseInt(regexResult[2]); // Currently unused
const testOk = (() => {
if (regexResult[1] === 'ok') {
return true;
}
return false;
})();
const testSubject = regexResult[3].trim();
const testMetadata = regexResult[5]; // This will be either "time=XXXms" or "SKIP reason" or "TODO reason"
let testDuration = 0;
if (testMetadata) {
const timeMatch = testMetadata.match(/time=(\d+)ms/);
// const skipMatch = testMetadata.match(/SKIP\s*(.*)/); // Currently unused
// const todoMatch = testMetadata.match(/TODO\s*(.*)/); // Currently unused
if (timeMatch) {
testDuration = parseInt(timeMatch[1]);
}
// Skip/todo handling could be added here in the future
}
// test for protocol error - disabled as it's not critical
// The test ID mismatch can occur when tests are filtered, skipped, or use todo
// if (testId !== this.activeTapTestResult.id) {
// if (this.logger) {
// this.logger.error('Something is strange! Test Ids are not equal!');
// }
// }
this.activeTapTestResult.setTestResult(testOk);
if (testOk) {
if (this.logger) {
this.logger.testResult(testSubject, true, testDuration);
}
} else {
// Start collecting error details for failed test
this.collectingErrorDetails = true;
this.currentTestError = [];
if (this.logger) {
this.logger.testResult(testSubject, false, testDuration);
}
}
}
if (!logLineIsTapProtocol) {
} else {
// Not a protocol message, handle as console output
if (this.activeTapTestResult) {
this.activeTapTestResult.addLogLine(logLine);
}
// Check for snapshot communication
// Check for snapshot communication (legacy)
const snapshotMatch = logLine.match(/###SNAPSHOT###(.+)###SNAPSHOT###/);
if (snapshotMatch) {
const base64Data = snapshotMatch[1];
try {
const snapshotData = JSON.parse(Buffer.from(base64Data, 'base64').toString());
this.handleSnapshot(snapshotData);
} catch (error) {
} catch (error: any) {
if (this.logger) {
this.logger.testConsoleOutput(`Error parsing snapshot data: ${error.message}`);
}
}
} else {
// Check if we're collecting error details
if (this.collectingErrorDetails) {
// Check if this line is an error detail (starts with Error: or has stack trace characteristics)
if (logLine.trim().startsWith('Error:') || logLine.trim().match(/^\s*at\s/)) {
this.currentTestError.push(logLine);
} else if (this.currentTestError.length > 0) {
// End of error details, show the error
const errorMessage = this.currentTestError.join('\n');
if (this.logger) {
this.logger.testErrorDetails(errorMessage);
}
this.collectingErrorDetails = false;
this.currentTestError = [];
}
}
// Don't output TAP error details as console output when we're collecting them
if (!this.collectingErrorDetails || (!logLine.trim().startsWith('Error:') && !logLine.trim().match(/^\s*at\s/))) {
if (this.logger) {
// This is console output from the test file, not TAP protocol
this.logger.testConsoleOutput(logLine);
}
}
} else if (this.logger) {
// This is console output from the test file
this.logger.testConsoleOutput(logLine);
}
}
}
}
if (this.activeTapTestResult && this.activeTapTestResult.testSettled) {
// Ensure any pending error is shown before settling the test
if (this.collectingErrorDetails && this.currentTestError.length > 0) {
const errorMessage = this.currentTestError.join('\n');
private _handleProtocolMessage(message: IProtocolMessage, originalLine: string) {
switch (message.type) {
case 'protocol':
this.protocolVersion = message.content.version;
if (this.logger) {
this.logger.tapOutput(`Protocol version: ${this.protocolVersion}`);
}
break;
case 'version':
// TAP version, we can ignore this
break;
case 'plan':
const plan = message.content as IPlanLine;
this.expectedTests = plan.end - plan.start + 1;
if (plan.skipAll) {
if (this.logger) {
this.logger.testErrorDetails(errorMessage);
this.logger.tapOutput(`Skipping all tests: ${plan.skipAll}`);
}
this.collectingErrorDetails = false;
this.currentTestError = [];
} else {
if (this.logger) {
this.logger.tapOutput(`Expecting ${this.expectedTests} tests!`);
}
}
// Initialize first TapResult
this._getNewTapTestResult();
break;
case 'test':
const testResult = message.content as ITestResult;
// Update active test result
this.activeTapTestResult.setTestResult(testResult.ok);
// Extract test duration from metadata
let testDuration = 0;
if (testResult.metadata?.time) {
testDuration = testResult.metadata.time;
}
// Log test result
if (this.logger) {
if (testResult.ok) {
this.logger.testResult(testResult.description, true, testDuration);
} else {
this.logger.testResult(testResult.description, false, testDuration);
// If there's error metadata, show it
if (testResult.metadata?.error) {
const error = testResult.metadata.error;
let errorDetails = error.message;
if (error.stack) {
errorDetails = error.stack;
}
this.logger.testErrorDetails(errorDetails);
}
}
}
// Handle directives (skip/todo)
if (testResult.directive) {
if (this.logger) {
if (testResult.directive.type === 'skip') {
this.logger.testConsoleOutput(`Test skipped: ${testResult.directive.reason || 'No reason given'}`);
} else if (testResult.directive.type === 'todo') {
this.logger.testConsoleOutput(`Test todo: ${testResult.directive.reason || 'No reason given'}`);
}
}
}
// Mark test as settled and move to next
this.activeTapTestResult.testSettled = true;
this.testStore.push(this.activeTapTestResult);
this._getNewTapTestResult();
break;
case 'comment':
if (this.logger) {
// Check if it's a pretask comment
const pretaskMatch = message.content.match(/^Pretask -> (.+): Success\.$/);
if (pretaskMatch) {
this.logger.tapOutput(message.content);
} else {
this.logger.testConsoleOutput(message.content);
}
}
break;
case 'bailout':
if (this.logger) {
this.logger.error(`Bail out! ${message.content}`);
}
break;
case 'error':
const errorBlock = message.content as IErrorBlock;
if (this.logger && errorBlock.error) {
let errorDetails = errorBlock.error.message;
if (errorBlock.error.stack) {
errorDetails = errorBlock.error.stack;
}
this.logger.testErrorDetails(errorDetails);
}
break;
case 'snapshot':
// Handle new protocol snapshot format
const snapshot = message.content;
this.handleSnapshot({
path: snapshot.name,
content: typeof snapshot.content === 'string' ? snapshot.content : JSON.stringify(snapshot.content),
action: 'compare' // Default action
});
break;
case 'event':
const event = message.content as ITestEvent;
this._handleTestEvent(event);
break;
}
}
private _handleTestEvent(event: ITestEvent) {
if (!this.logger) return;
switch (event.eventType) {
case 'test:queued':
// We can track queued tests if needed
break;
case 'test:started':
this.logger.testConsoleOutput(cs(`Test starting: ${event.data.description}`, 'cyan'));
if (event.data.retry) {
this.logger.testConsoleOutput(cs(` Retry attempt ${event.data.retry}`, 'orange'));
}
break;
case 'test:progress':
if (event.data.progress !== undefined) {
this.logger.testConsoleOutput(cs(` Progress: ${event.data.progress}%`, 'cyan'));
}
break;
case 'test:completed':
// Test completion is already handled by the test result
// This event provides additional timing info if needed
break;
case 'suite:started':
this.logger.testConsoleOutput(cs(`\nSuite: ${event.data.suiteName}`, 'blue'));
break;
case 'suite:completed':
this.logger.testConsoleOutput(cs(`Suite completed: ${event.data.suiteName}\n`, 'blue'));
break;
case 'hook:started':
this.logger.testConsoleOutput(cs(` Hook: ${event.data.hookName}`, 'cyan'));
break;
case 'hook:completed':
// Silent unless there's an error
if (event.data.error) {
this.logger.testConsoleOutput(cs(` Hook failed: ${event.data.hookName}`, 'red'));
}
break;
case 'assertion:failed':
// Enhanced assertion failure with diff
if (event.data.error) {
this._displayAssertionError(event.data.error);
}
break;
}
}
private _displayAssertionError(error: any) {
if (!this.logger) return;
// Display error message
if (error.message) {
this.logger.testErrorDetails(error.message);
}
// Display visual diff if available
if (error.diff) {
this._displayDiff(error.diff, error.expected, error.actual);
}
}
private _displayDiff(diff: any, expected: any, actual: any) {
if (!this.logger) return;
this.logger.testConsoleOutput(cs('\n Diff:', 'cyan'));
switch (diff.type) {
case 'string':
this._displayStringDiff(diff.changes);
break;
case 'object':
this._displayObjectDiff(diff.changes, expected, actual);
break;
case 'array':
this._displayArrayDiff(diff.changes, expected, actual);
break;
case 'primitive':
this._displayPrimitiveDiff(diff.changes);
break;
}
}
private _displayStringDiff(changes: any[]) {
for (const change of changes) {
const linePrefix = ` Line ${change.line + 1}: `;
if (change.type === 'add') {
this.logger.testConsoleOutput(cs(`${linePrefix}+ ${change.content}`, 'green'));
} else if (change.type === 'remove') {
this.logger.testConsoleOutput(cs(`${linePrefix}- ${change.content}`, 'red'));
}
}
}
private _displayObjectDiff(changes: any[], expected: any, actual: any) {
this.logger.testConsoleOutput(cs(' Expected:', 'red'));
this.logger.testConsoleOutput(` ${JSON.stringify(expected, null, 2)}`);
this.logger.testConsoleOutput(cs(' Actual:', 'green'));
this.logger.testConsoleOutput(` ${JSON.stringify(actual, null, 2)}`);
this.logger.testConsoleOutput(cs('\n Changes:', 'cyan'));
for (const change of changes) {
const path = change.path.join('.');
if (change.type === 'add') {
this.logger.testConsoleOutput(cs(` + ${path}: ${JSON.stringify(change.newValue)}`, 'green'));
} else if (change.type === 'remove') {
this.logger.testConsoleOutput(cs(` - ${path}: ${JSON.stringify(change.oldValue)}`, 'red'));
} else if (change.type === 'modify') {
this.logger.testConsoleOutput(cs(` ~ ${path}:`, 'cyan'));
this.logger.testConsoleOutput(cs(` - ${JSON.stringify(change.oldValue)}`, 'red'));
this.logger.testConsoleOutput(cs(` + ${JSON.stringify(change.newValue)}`, 'green'));
}
}
}
private _displayArrayDiff(changes: any[], expected: any[], actual: any[]) {
this._displayObjectDiff(changes, expected, actual);
}
private _displayPrimitiveDiff(changes: any[]) {
const change = changes[0];
if (change) {
this.logger.testConsoleOutput(cs(` Expected: ${JSON.stringify(change.oldValue)}`, 'red'));
this.logger.testConsoleOutput(cs(` Actual: ${JSON.stringify(change.newValue)}`, 'green'));
}
}
/**
* returns all tests that are not completed
@ -353,4 +514,4 @@ export class TapParser {
}
}
}
}
}

46
ts/tstest.classes.tstest.ts
View File

@ -161,9 +161,45 @@ export class TsTest {
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
}
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(
`tsrun ${fileNameArg}${tsrunOptions}`
);
// Check for 00init.ts file in test directory
const testDir = plugins.path.dirname(fileNameArg);
const initFile = plugins.path.join(testDir, '00init.ts');
let runCommand = `tsrun ${fileNameArg}${tsrunOptions}`;
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
// If 00init.ts exists, run it first
if (initFileExists) {
// Create a temporary loader file that imports both 00init.ts and the test file
const absoluteInitFile = plugins.path.resolve(initFile);
const absoluteTestFile = plugins.path.resolve(fileNameArg);
const loaderContent = `
import '${absoluteInitFile.replace(/\\/g, '/')}';
import '${absoluteTestFile.replace(/\\/g, '/')}';
`;
const loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(fileNameArg)}`);
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
runCommand = `tsrun ${loaderPath}${tsrunOptions}`;
}
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
// If we created a loader file, clean it up after test execution
if (initFileExists) {
const loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(fileNameArg)}`);
const cleanup = () => {
try {
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
plugins.smartfile.fs.removeSync(loaderPath);
}
} catch (e) {
// Ignore cleanup errors
}
};
execResultStreaming.childProcess.on('exit', cleanup);
execResultStreaming.childProcess.on('error', cleanup);
}
// Handle timeout if specified
if (this.timeoutSeconds !== null) {
@ -382,10 +418,10 @@ export class TsTest {
try {
// Delete 00err and 00diff directories if they exist
if (await plugins.smartfile.fs.isDirectory(errDir)) {
await plugins.smartfile.fs.remove(errDir);
plugins.smartfile.fs.removeSync(errDir);
}
if (await plugins.smartfile.fs.isDirectory(diffDir)) {
await plugins.smartfile.fs.remove(diffDir);
plugins.smartfile.fs.removeSync(diffDir);
}
// Get all .log files in log directory (not in subdirectories)

8
ts_tapbundle/index.ts
View File

@ -1,11 +1,7 @@
export { tap } from './tapbundle.classes.tap.js';
export { TapWrap } from './tapbundle.classes.tapwrap.js';
export { webhelpers } from './webhelpers.js';
// Protocol utilities (for future protocol v2)
export * from './tapbundle.protocols.js';
export { TapTools } from './tapbundle.classes.taptools.js';
import { expect } from '@push.rocks/smartexpect';
export { expect };
// Export enhanced expect with diff generation
export { expect, setProtocolEmitter } from './tapbundle.expect.wrapper.js';

117
ts_tapbundle/tapbundle.classes.settingsmanager.ts Normal file
View File

@ -0,0 +1,117 @@
import type { ITapSettings, ISettingsManager } from './tapbundle.interfaces.js';
export class SettingsManager implements ISettingsManager {
private globalSettings: ITapSettings = {};
private fileSettings: ITapSettings = {};
private testSettings: Map<string, ITapSettings> = new Map();
// Default settings
private defaultSettings: ITapSettings = {
timeout: undefined, // No timeout by default
slowThreshold: 1000, // 1 second
bail: false,
retries: 0,
retryDelay: 0,
suppressConsole: false,
verboseErrors: true,
showTestDuration: true,
maxConcurrency: 5,
isolateTests: false,
enableSnapshots: true,
snapshotDirectory: '.snapshots',
updateSnapshots: false,
};
/**
* Get merged settings for current context
*/
public getSettings(): ITapSettings {
return this.mergeSettings(
this.defaultSettings,
this.globalSettings,
this.fileSettings
);
}
/**
* Set global settings (from 00init.ts or tap.settings())
*/
public setGlobalSettings(settings: ITapSettings): void {
this.globalSettings = { ...this.globalSettings, ...settings };
}
/**
* Set file-level settings
*/
public setFileSettings(settings: ITapSettings): void {
this.fileSettings = { ...this.fileSettings, ...settings };
}
/**
* Set test-specific settings
*/
public setTestSettings(testId: string, settings: ITapSettings): void {
const existingSettings = this.testSettings.get(testId) || {};
this.testSettings.set(testId, { ...existingSettings, ...settings });
}
/**
* Get settings for specific test
*/
public getTestSettings(testId: string): ITapSettings {
const testSpecificSettings = this.testSettings.get(testId) || {};
return this.mergeSettings(
this.defaultSettings,
this.globalSettings,
this.fileSettings,
testSpecificSettings
);
}
/**
* Merge settings with proper inheritance
* Later settings override earlier ones
*/
private mergeSettings(...settingsArray: ITapSettings[]): ITapSettings {
const result: ITapSettings = {};
for (const settings of settingsArray) {
// Simple properties - later values override
if (settings.timeout !== undefined) result.timeout = settings.timeout;
if (settings.slowThreshold !== undefined) result.slowThreshold = settings.slowThreshold;
if (settings.bail !== undefined) result.bail = settings.bail;
if (settings.retries !== undefined) result.retries = settings.retries;
if (settings.retryDelay !== undefined) result.retryDelay = settings.retryDelay;
if (settings.suppressConsole !== undefined) result.suppressConsole = settings.suppressConsole;
if (settings.verboseErrors !== undefined) result.verboseErrors = settings.verboseErrors;
if (settings.showTestDuration !== undefined) result.showTestDuration = settings.showTestDuration;
if (settings.maxConcurrency !== undefined) result.maxConcurrency = settings.maxConcurrency;
if (settings.isolateTests !== undefined) result.isolateTests = settings.isolateTests;
if (settings.enableSnapshots !== undefined) result.enableSnapshots = settings.enableSnapshots;
if (settings.snapshotDirectory !== undefined) result.snapshotDirectory = settings.snapshotDirectory;
if (settings.updateSnapshots !== undefined) result.updateSnapshots = settings.updateSnapshots;
// Lifecycle hooks - later ones override
if (settings.beforeAll !== undefined) result.beforeAll = settings.beforeAll;
if (settings.afterAll !== undefined) result.afterAll = settings.afterAll;
if (settings.beforeEach !== undefined) result.beforeEach = settings.beforeEach;
if (settings.afterEach !== undefined) result.afterEach = settings.afterEach;
// Environment variables - merge
if (settings.env) {
result.env = { ...result.env, ...settings.env };
}
}
return result;
}
/**
* Clear all settings (useful for testing)
*/
public clearSettings(): void {
this.globalSettings = {};
this.fileSettings = {};
this.testSettings.clear();
}
}

259
ts_tapbundle/tapbundle.classes.tap.ts
View File

@ -2,6 +2,9 @@ import * as plugins from './tapbundle.plugins.js';
import { type IPreTaskFunction, PreTask } from './tapbundle.classes.pretask.js';
import { TapTest, type ITestFunction } from './tapbundle.classes.taptest.js';
import { ProtocolEmitter, type ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
import type { ITapSettings } from './tapbundle.interfaces.js';
import { SettingsManager } from './tapbundle.classes.settingsmanager.js';
export interface ITestSuite {
description: string;
@ -102,6 +105,8 @@ class TestBuilder<T> {
}
export class Tap<T> {
private protocolEmitter = new ProtocolEmitter();
private settingsManager = new SettingsManager();
private _skipCount = 0;
private _filterTags: string[] = [];
@ -139,12 +144,27 @@ export class Tap<T> {
*/
public skip = {
test: (descriptionArg: string, functionArg: ITestFunction<T>) => {
console.log(`skipped test: ${descriptionArg}`);
this._skipCount++;
const skippedTest = this.test(descriptionArg, functionArg, 'skip');
return skippedTest;
},
testParallel: (descriptionArg: string, functionArg: ITestFunction<T>) => {
console.log(`skipped test: ${descriptionArg}`);
this._skipCount++;
const skippedTest = new TapTest<T>({
description: descriptionArg,
testFunction: functionArg,
parallel: true,
});
// Mark as skip mode
skippedTest.tapTools.markAsSkipped('Marked as skip');
// Add to appropriate test list
if (this._currentSuite) {
this._currentSuite.tests.push(skippedTest);
} else {
this._tapTests.push(skippedTest);
}
return skippedTest;
},
};
@ -153,7 +173,65 @@ export class Tap<T> {
*/
public only = {
test: (descriptionArg: string, testFunctionArg: ITestFunction<T>) => {
this.test(descriptionArg, testFunctionArg, 'only');
return this.test(descriptionArg, testFunctionArg, 'only');
},
testParallel: (descriptionArg: string, testFunctionArg: ITestFunction<T>) => {
const onlyTest = new TapTest<T>({
description: descriptionArg,
testFunction: testFunctionArg,
parallel: true,
});
// Add to only tests list
this._tapTestsOnly.push(onlyTest);
return onlyTest;
},
};
/**
* mark a test as todo (not yet implemented)
*/
public todo = {
test: (descriptionArg: string, functionArg?: ITestFunction<T>) => {
const defaultFunc = (async () => {}) as ITestFunction<T>;
const todoTest = new TapTest<T>({
description: descriptionArg,
testFunction: functionArg || defaultFunc,
parallel: false,
});
// Mark as todo
todoTest.tapTools.todo('Marked as todo');
// Add to appropriate test list
if (this._currentSuite) {
this._currentSuite.tests.push(todoTest);
} else {
this._tapTests.push(todoTest);
}
return todoTest;
},
testParallel: (descriptionArg: string, functionArg?: ITestFunction<T>) => {
const defaultFunc = (async () => {}) as ITestFunction<T>;
const todoTest = new TapTest<T>({
description: descriptionArg,
testFunction: functionArg || defaultFunc,
parallel: true,
});
// Mark as todo
todoTest.tapTools.todo('Marked as todo');
// Add to appropriate test list
if (this._currentSuite) {
this._currentSuite.tests.push(todoTest);
} else {
this._tapTests.push(todoTest);
}
return todoTest;
},
};
@ -163,6 +241,21 @@ export class Tap<T> {
private _currentSuite: ITestSuite | null = null;
private _rootSuites: ITestSuite[] = [];
/**
* Configure global test settings
*/
public settings(settings: ITapSettings): this {
this.settingsManager.setGlobalSettings(settings);
return this;
}
/**
* Get current test settings
*/
public getSettings(): ITapSettings {
return this.settingsManager.getSettings();
}
/**
* Normal test function, will run one by one
* @param testDescription - A description of what the test does
@ -179,14 +272,26 @@ export class Tap<T> {
parallel: false,
});
// No options applied here - use the fluent builder syntax instead
// Apply default settings from settings manager
const settings = this.settingsManager.getSettings();
if (settings.timeout !== undefined) {
localTest.timeoutMs = settings.timeout;
}
if (settings.retries !== undefined) {
localTest.tapTools.retry(settings.retries);
}
// Handle skip mode
if (modeArg === 'skip') {
localTest.tapTools.markAsSkipped('Marked as skip');
}
// If we're in a suite, add test to the suite
if (this._currentSuite) {
this._currentSuite.tests.push(localTest);
} else {
// Otherwise add to global test list
if (modeArg === 'normal') {
if (modeArg === 'normal' || modeArg === 'skip') {
this._tapTests.push(localTest);
} else if (modeArg === 'only') {
this._tapTestsOnly.push(localTest);
@ -211,6 +316,15 @@ export class Tap<T> {
parallel: true,
});
// Apply default settings from settings manager
const settings = this.settingsManager.getSettings();
if (settings.timeout !== undefined) {
localTest.timeoutMs = settings.timeout;
}
if (settings.retries !== undefined) {
localTest.tapTools.retry(settings.retries);
}
if (this._currentSuite) {
this._currentSuite.tests.push(localTest);
} else {
@ -336,8 +450,27 @@ export class Tap<T> {
await preTask.run();
}
// Count actual tests that will be run
console.log(`1..${concerningTests.length}`);
// Emit protocol header and TAP version
console.log(this.protocolEmitter.emitProtocolHeader());
console.log(this.protocolEmitter.emitTapVersion(13));
// Emit test plan
const plan = {
start: 1,
end: concerningTests.length
};
console.log(this.protocolEmitter.emitPlan(plan));
// Run global beforeAll hook if configured
const settings = this.settingsManager.getSettings();
if (settings.beforeAll) {
try {
await settings.beforeAll();
} catch (error) {
console.error('Error in beforeAll hook:', error);
throw error;
}
}
// Run tests from suites with lifecycle hooks
let testKey = 0;
@ -365,6 +498,33 @@ export class Tap<T> {
});
for (const currentTest of nonSuiteTests) {
// Wrap test function with global lifecycle hooks
const originalFunction = currentTest.testFunction;
const testName = currentTest.description;
currentTest.testFunction = async (tapTools) => {
// Run global beforeEach if configured
if (settings.beforeEach) {
await settings.beforeEach(testName);
}
// Run the actual test
let testPassed = true;
let result: any;
try {
result = await originalFunction(tapTools);
} catch (error) {
testPassed = false;
throw error;
} finally {
// Run global afterEach if configured
if (settings.afterEach) {
await settings.afterEach(testName, testPassed);
}
}
return result;
};
const testPromise = currentTest.run(testKey++);
if (currentTest.parallel) {
promiseArray.push(testPromise);
@ -394,6 +554,16 @@ export class Tap<T> {
console.log(failReason);
}
// Run global afterAll hook if configured
if (settings.afterAll) {
try {
await settings.afterAll();
} catch (error) {
console.error('Error in afterAll hook:', error);
// Don't throw here, we want to complete the test run
}
}
if (optionsArg && optionsArg.throwOnError && failReasons.length > 0) {
if (!smartenvInstance.isBrowser && typeof process !== 'undefined') process.exit(1);
}
@ -402,6 +572,13 @@ export class Tap<T> {
}
}
/**
* Emit an event
*/
private emitEvent(event: ITestEvent) {
console.log(this.protocolEmitter.emitEvent(event));
}
/**
* Run tests in a suite with lifecycle hooks
*/
@ -412,6 +589,14 @@ export class Tap<T> {
context: { testKey: number }
) {
for (const suite of suites) {
// Emit suite:started event
this.emitEvent({
eventType: 'suite:started',
timestamp: Date.now(),
data: {
suiteName: suite.description
}
});
// Run beforeEach from parent suites
const beforeEachFunctions: ITestFunction<any>[] = [];
let currentSuite: ITestSuite | null = suite;
@ -426,27 +611,46 @@ export class Tap<T> {
for (const test of suite.tests) {
// Create wrapper test function that includes lifecycle hooks
const originalFunction = test.testFunction;
const testName = test.description;
test.testFunction = async (tapTools) => {
// Run all beforeEach hooks
// Run global beforeEach if configured
const settings = this.settingsManager.getSettings();
if (settings.beforeEach) {
await settings.beforeEach(testName);
}
// Run all suite beforeEach hooks
for (const beforeEach of beforeEachFunctions) {
await beforeEach(tapTools);
}
// Run the actual test
const result = await originalFunction(tapTools);
// Run afterEach hooks in reverse order
const afterEachFunctions: ITestFunction<any>[] = [];
currentSuite = suite;
while (currentSuite) {
if (currentSuite.afterEach) {
afterEachFunctions.push(currentSuite.afterEach);
let testPassed = true;
let result: any;
try {
result = await originalFunction(tapTools);
} catch (error) {
testPassed = false;
throw error;
} finally {
// Run afterEach hooks in reverse order
const afterEachFunctions: ITestFunction<any>[] = [];
currentSuite = suite;
while (currentSuite) {
if (currentSuite.afterEach) {
afterEachFunctions.push(currentSuite.afterEach);
}
currentSuite = currentSuite.parent || null;
}
for (const afterEach of afterEachFunctions) {
await afterEach(tapTools);
}
// Run global afterEach if configured
if (settings.afterEach) {
await settings.afterEach(testName, testPassed);
}
currentSuite = currentSuite.parent || null;
}
for (const afterEach of afterEachFunctions) {
await afterEach(tapTools);
}
return result;
@ -462,6 +666,15 @@ export class Tap<T> {
// Recursively run child suites
await this._runSuite(suite, suite.children, promiseArray, context);
// Emit suite:completed event
this.emitEvent({
eventType: 'suite:completed',
timestamp: Date.now(),
data: {
suiteName: suite.description
}
});
}
}

187
ts_tapbundle/tapbundle.classes.taptest.ts
View File

@ -1,6 +1,8 @@
import * as plugins from './tapbundle.plugins.js';
import { tapCreator } from './tapbundle.tapcreator.js';
import { TapTools, SkipError } from './tapbundle.classes.taptools.js';
import { ProtocolEmitter, type ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
import { setProtocolEmitter } from './tapbundle.expect.wrapper.js';
// imported interfaces
import { Deferred } from '@push.rocks/smartpromise';
@ -32,6 +34,7 @@ export class TapTest<T = unknown> {
public testPromise: Promise<TapTest<T>> = this.testDeferred.promise;
private testResultDeferred: Deferred<T> = plugins.smartpromise.defer();
public testResultPromise: Promise<T> = this.testResultDeferred.promise;
private protocolEmitter = new ProtocolEmitter();
/**
* constructor
*/
@ -48,6 +51,13 @@ export class TapTest<T = unknown> {
this.testFunction = optionsArg.testFunction;
}
/**
* Emit an event
*/
private emitEvent(event: ITestEvent) {
console.log(this.protocolEmitter.emitEvent(event));
}
/**
* run the test
*/
@ -55,11 +65,74 @@ export class TapTest<T = unknown> {
this.testKey = testKeyArg;
const testNumber = testKeyArg + 1;
// Emit test:queued event
this.emitEvent({
eventType: 'test:queued',
timestamp: Date.now(),
data: {
testNumber,
description: this.description
}
});
// Handle todo tests
if (this.isTodo) {
const todoText = this.todoReason ? `# TODO ${this.todoReason}` : '# TODO';
console.log(`ok ${testNumber} - ${this.description} ${todoText}`);
const testResult = {
ok: true,
testNumber,
description: this.description,
directive: {
type: 'todo' as const,
reason: this.todoReason
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'success';
// Emit test:completed event for todo test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: 0,
error: undefined
}
});
this.testDeferred.resolve(this);
return;
}
// Handle pre-marked skip tests
if (this.tapTools.isSkipped) {
const testResult = {
ok: true,
testNumber,
description: this.description,
directive: {
type: 'skip' as const,
reason: this.tapTools.skipReason || 'Marked as skip'
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'skipped';
// Emit test:completed event for skipped test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: 0,
error: undefined
}
});
this.testDeferred.resolve(this);
return;
}
@ -71,6 +144,20 @@ export class TapTest<T = unknown> {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
this.hrtMeasurement.start();
// Emit test:started event
this.emitEvent({
eventType: 'test:started',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
retry: attempt > 0 ? attempt : undefined
}
});
// Set protocol emitter for enhanced expect
setProtocolEmitter(this.protocolEmitter);
try {
// Set up timeout if specified
let timeoutHandle: any;
@ -97,10 +184,32 @@ export class TapTest<T = unknown> {
}
this.hrtMeasurement.stop();
console.log(
`ok ${testNumber} - ${this.description} # time=${this.hrtMeasurement.milliSeconds}ms`,
);
const testResult = {
ok: true,
testNumber,
description: this.description,
metadata: {
time: this.hrtMeasurement.milliSeconds,
tags: this.tags.length > 0 ? this.tags : undefined,
file: this.fileName
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'success';
// Emit test:completed event
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: this.hrtMeasurement.milliSeconds,
error: undefined
}
});
this.testDeferred.resolve(this);
this.testResultDeferred.resolve(testReturnValue);
return; // Success, exit retry loop
@ -110,8 +219,31 @@ export class TapTest<T = unknown> {
// Handle skip
if (err instanceof SkipError || err.name === 'SkipError') {
console.log(`ok ${testNumber} - ${this.description} # SKIP ${err.message.replace('Skipped: ', '')}`);
const testResult = {
ok: true,
testNumber,
description: this.description,
directive: {
type: 'skip' as const,
reason: err.message.replace('Skipped: ', '')
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'skipped';
// Emit test:completed event for skipped test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: this.hrtMeasurement.milliSeconds,
error: undefined
}
});
this.testDeferred.resolve(this);
return;
}
@ -120,17 +252,48 @@ export class TapTest<T = unknown> {
// If we have retries left, try again
if (attempt < maxRetries) {
console.log(
`# Retry ${attempt + 1}/${maxRetries} for test: ${this.description}`,
);
console.log(this.protocolEmitter.emitComment(`Retry ${attempt + 1}/${maxRetries} for test: ${this.description}`));
this.tapTools._incrementRetryCount();
continue;
}
// Final failure
console.log(
`not ok ${testNumber} - ${this.description} # time=${this.hrtMeasurement.milliSeconds}ms`,
);
const testResult = {
ok: false,
testNumber,
description: this.description,
metadata: {
time: this.hrtMeasurement.milliSeconds,
retry: this.tapTools.retryCount,
maxRetries: maxRetries > 0 ? maxRetries : undefined,
error: {
message: lastError.message || String(lastError),
stack: lastError.stack,
code: lastError.code
},
tags: this.tags.length > 0 ? this.tags : undefined,
file: this.fileName
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
// Emit test:completed event for failed test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: this.hrtMeasurement.milliSeconds,
error: {
message: lastError.message || String(lastError),
stack: lastError.stack,
type: 'runtime' as const
}
}
});
this.testDeferred.resolve(this);
this.testResultDeferred.resolve(err);

28
ts_tapbundle/tapbundle.classes.taptools.ts
View File

@ -22,6 +22,10 @@ export class TapTools {
public testData: any = {};
private static _sharedContext = new Map<string, any>();
private _snapshotPath: string = '';
// Flags for skip/todo
private _isSkipped = false;
private _skipReason?: string;
constructor(TapTestArg: TapTest<any>) {
this._tapTest = TapTestArg;
@ -45,9 +49,33 @@ export class TapTools {
* skip the rest of the test
*/
public skip(reason?: string): never {
this._isSkipped = true;
this._skipReason = reason;
const skipMessage = reason ? `Skipped: ${reason}` : 'Skipped';
throw new SkipError(skipMessage);
}
/**
* Mark test as skipped without throwing (for pre-marking)
*/
public markAsSkipped(reason?: string): void {
this._isSkipped = true;
this._skipReason = reason;
}
/**
* Check if test is marked as skipped
*/
public get isSkipped(): boolean {
return this._isSkipped;
}
/**
* Get skip reason
*/
public get skipReason(): string | undefined {
return this._skipReason;
}
/**
* conditionally skip the rest of the test

81
ts_tapbundle/tapbundle.expect.wrapper.ts Normal file
View File

@ -0,0 +1,81 @@
import { expect as smartExpect } from '@push.rocks/smartexpect';
import { generateDiff } from './tapbundle.utilities.diff.js';
import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';
import type { IEnhancedError } from '../dist_ts_tapbundle_protocol/index.js';
// Store the protocol emitter for event emission
let protocolEmitter: ProtocolEmitter | null = null;
/**
* Set the protocol emitter for enhanced error reporting
*/
export function setProtocolEmitter(emitter: ProtocolEmitter) {
protocolEmitter = emitter;
}
/**
* Enhanced expect wrapper that captures assertion failures and generates diffs
*/
export function createEnhancedExpect() {
return new Proxy(smartExpect, {
apply(target, thisArg, argumentsList: any[]) {
const expectation = target.apply(thisArg, argumentsList);
// Wrap common assertion methods
const wrappedExpectation = new Proxy(expectation, {
get(target, prop, receiver) {
const originalValue = Reflect.get(target, prop, receiver);
// Wrap assertion methods that compare values
if (typeof prop === 'string' && typeof originalValue === 'function' && ['toEqual', 'toBe', 'toMatch', 'toContain'].includes(prop)) {
return function(expected: any) {
try {
return originalValue.apply(target, arguments);
} catch (error: any) {
// Enhance the error with diff information
const actual = argumentsList[0];
const enhancedError: IEnhancedError = {
message: error.message,
stack: error.stack,
actual,
expected,
type: 'assertion'
};
// Generate diff if applicable
if (prop === 'toEqual' || prop === 'toBe') {
const diff = generateDiff(expected, actual);
if (diff) {
enhancedError.diff = diff;
}
}
// Emit assertion:failed event if protocol emitter is available
if (protocolEmitter) {
const event = {
eventType: 'assertion:failed' as const,
timestamp: Date.now(),
data: {
error: enhancedError
}
};
console.log(protocolEmitter.emitEvent(event));
}
// Re-throw the enhanced error
throw error;
}
};
}
return originalValue;
}
});
return wrappedExpectation;
}
});
}
// Create the enhanced expect function
export const expect = createEnhancedExpect();

46
ts_tapbundle/tapbundle.interfaces.ts Normal file
View File

@ -0,0 +1,46 @@
export interface ITapSettings {
// Timing
timeout?: number; // Default timeout for all tests (ms)
slowThreshold?: number; // Mark tests as slow if they exceed this (ms)
// Execution Control
bail?: boolean; // Stop on first test failure
retries?: number; // Number of retries for failed tests
retryDelay?: number; // Delay between retries (ms)
// Output Control
suppressConsole?: boolean; // Suppress console output in passing tests
verboseErrors?: boolean; // Show full stack traces
showTestDuration?: boolean; // Show duration for each test
// Parallel Execution
maxConcurrency?: number; // Max parallel tests (for .para files)
isolateTests?: boolean; // Run each test in fresh context
// Lifecycle Hooks
beforeAll?: () => Promise<void> | void;
afterAll?: () => Promise<void> | void;
beforeEach?: (testName: string) => Promise<void> | void;
afterEach?: (testName: string, passed: boolean) => Promise<void> | void;
// Environment
env?: Record<string, string>; // Additional environment variables
// Features
enableSnapshots?: boolean; // Enable snapshot testing
snapshotDirectory?: string; // Custom snapshot directory
updateSnapshots?: boolean; // Update snapshots instead of comparing
}
export interface ISettingsManager {
// Get merged settings for current context
getSettings(): ITapSettings;
// Apply settings at different levels
setGlobalSettings(settings: ITapSettings): void;
setFileSettings(settings: ITapSettings): void;
setTestSettings(testId: string, settings: ITapSettings): void;
// Get settings for specific test
getTestSettings(testId: string): ITapSettings;
}

226
ts_tapbundle/tapbundle.protocols.ts
View File

@ -1,226 +0,0 @@
/**
* Internal protocol constants and utilities for improved TAP communication
* between tapbundle and tstest
*/
export const PROTOCOL = {
VERSION: '2.0',
MARKERS: {
START: '⟦TSTEST:',
END: '⟧',
BLOCK_END: '⟦/TSTEST:',
},
TYPES: {
META: 'META',
ERROR: 'ERROR',
SKIP: 'SKIP',
TODO: 'TODO',
SNAPSHOT: 'SNAPSHOT',
PROTOCOL: 'PROTOCOL',
}
} as const;
export interface TestMetadata {
// Timing
time?: number; // milliseconds
startTime?: number; // Unix timestamp
endTime?: number; // Unix timestamp
// Status
skip?: string; // skip reason
todo?: string; // todo reason
retry?: number; // retry attempt
maxRetries?: number; // max retries allowed
// Error details
error?: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
};
// Test context
file?: string; // source file
line?: number; // line number
column?: number; // column number
// Custom data
tags?: string[]; // test tags
custom?: Record<string, any>;
}
export class ProtocolEncoder {
/**
* Encode metadata for inline inclusion
*/
static encodeInline(type: string, data: any): string {
if (typeof data === 'string') {
return `${PROTOCOL.MARKERS.START}${type}:${data}${PROTOCOL.MARKERS.END}`;
}
return `${PROTOCOL.MARKERS.START}${type}:${JSON.stringify(data)}${PROTOCOL.MARKERS.END}`;
}
/**
* Encode block data for multi-line content
*/
static encodeBlock(type: string, data: any): string[] {
const lines: string[] = [];
lines.push(`${PROTOCOL.MARKERS.START}${type}${PROTOCOL.MARKERS.END}`);
if (typeof data === 'string') {
lines.push(data);
} else {
lines.push(JSON.stringify(data, null, 2));
}
lines.push(`${PROTOCOL.MARKERS.BLOCK_END}${type}${PROTOCOL.MARKERS.END}`);
return lines;
}
/**
* Create a TAP line with metadata
*/
static createTestLine(
status: 'ok' | 'not ok',
number: number,
description: string,
metadata?: TestMetadata
): string {
let line = `${status} ${number} - ${description}`;
if (metadata) {
// For skip/todo, use inline format for compatibility
if (metadata.skip) {
line += ` ${this.encodeInline(PROTOCOL.TYPES.SKIP, metadata.skip)}`;
} else if (metadata.todo) {
line += ` ${this.encodeInline(PROTOCOL.TYPES.TODO, metadata.todo)}`;
} else {
// For other metadata, append inline
const metaCopy = { ...metadata };
delete metaCopy.error; // Error details go in separate block
if (Object.keys(metaCopy).length > 0) {
line += ` ${this.encodeInline(PROTOCOL.TYPES.META, metaCopy)}`;
}
}
}
return line;
}
}
export class ProtocolDecoder {
/**
* Extract all protocol markers from a line
*/
static extractMarkers(line: string): Array<{type: string, data: any, start: number, end: number}> {
const markers: Array<{type: string, data: any, start: number, end: number}> = [];
let searchFrom = 0;
while (true) {
const start = line.indexOf(PROTOCOL.MARKERS.START, searchFrom);
if (start === -1) break;
const end = line.indexOf(PROTOCOL.MARKERS.END, start);
if (end === -1) break;
const content = line.substring(start + PROTOCOL.MARKERS.START.length, end);
const colonIndex = content.indexOf(':');
if (colonIndex !== -1) {
const type = content.substring(0, colonIndex);
const dataStr = content.substring(colonIndex + 1);
let data: any;
try {
// Try to parse as JSON first
data = JSON.parse(dataStr);
} catch {
// If not JSON, treat as string
data = dataStr;
}
markers.push({
type,
data,
start,
end: end + PROTOCOL.MARKERS.END.length
});
}
searchFrom = end + 1;
}
return markers;
}
/**
* Remove protocol markers from a line
*/
static cleanLine(line: string): string {
const markers = this.extractMarkers(line);
// Remove markers from end to start to preserve indices
let cleanedLine = line;
for (let i = markers.length - 1; i >= 0; i--) {
const marker = markers[i];
cleanedLine = cleanedLine.substring(0, marker.start) +
cleanedLine.substring(marker.end);
}
return cleanedLine.trim();
}
/**
* Parse a test line and extract metadata
*/
static parseTestLine(line: string): {
cleaned: string;
metadata: TestMetadata;
} {
const markers = this.extractMarkers(line);
const metadata: TestMetadata = {};
for (const marker of markers) {
switch (marker.type) {
case PROTOCOL.TYPES.META:
Object.assign(metadata, marker.data);
break;
case PROTOCOL.TYPES.SKIP:
metadata.skip = marker.data;
break;
case PROTOCOL.TYPES.TODO:
metadata.todo = marker.data;
break;
}
}
return {
cleaned: this.cleanLine(line),
metadata
};
}
/**
* Check if a line starts a protocol block
*/
static isBlockStart(line: string): {isBlock: boolean, type?: string} {
const trimmed = line.trim();
if (trimmed.startsWith(PROTOCOL.MARKERS.START) && trimmed.endsWith(PROTOCOL.MARKERS.END)) {
const content = trimmed.slice(PROTOCOL.MARKERS.START.length, -PROTOCOL.MARKERS.END.length);
if (!content.includes(':')) {
return { isBlock: true, type: content };
}
}
return { isBlock: false };
}
/**
* Check if a line ends a protocol block
*/
static isBlockEnd(line: string, type: string): boolean {
return line.trim() === `${PROTOCOL.MARKERS.BLOCK_END}${type}${PROTOCOL.MARKERS.END}`;
}
}

188
ts_tapbundle/tapbundle.utilities.diff.ts Normal file
View File

@ -0,0 +1,188 @@
import type { IDiffResult, IDiffChange } from '../dist_ts_tapbundle_protocol/index.js';
/**
* Generate a diff between two values
*/
export function generateDiff(expected: any, actual: any, context: number = 3): IDiffResult | null {
// Handle same values
if (expected === actual) {
return null;
}
// Determine diff type based on values
if (typeof expected === 'string' && typeof actual === 'string') {
return generateStringDiff(expected, actual, context);
} else if (Array.isArray(expected) && Array.isArray(actual)) {
return generateArrayDiff(expected, actual);
} else if (expected && actual && typeof expected === 'object' && typeof actual === 'object') {
return generateObjectDiff(expected, actual);
} else {
return generatePrimitiveDiff(expected, actual);
}
}
/**
* Generate diff for primitive values
*/
function generatePrimitiveDiff(expected: any, actual: any): IDiffResult {
return {
type: 'primitive',
changes: [{
type: 'modify',
oldValue: expected,
newValue: actual
}]
};
}
/**
* Generate diff for strings (line-by-line)
*/
function generateStringDiff(expected: string, actual: string, context: number): IDiffResult {
const expectedLines = expected.split('\n');
const actualLines = actual.split('\n');
const changes: IDiffChange[] = [];
// Simple line-by-line diff
const maxLines = Math.max(expectedLines.length, actualLines.length);
for (let i = 0; i < maxLines; i++) {
const expectedLine = expectedLines[i];
const actualLine = actualLines[i];
if (expectedLine === undefined) {
changes.push({
type: 'add',
line: i,
content: actualLine
});
} else if (actualLine === undefined) {
changes.push({
type: 'remove',
line: i,
content: expectedLine
});
} else if (expectedLine !== actualLine) {
changes.push({
type: 'remove',
line: i,
content: expectedLine
});
changes.push({
type: 'add',
line: i,
content: actualLine
});
}
}
return {
type: 'string',
changes,
context
};
}
/**
* Generate diff for arrays
*/
function generateArrayDiff(expected: any[], actual: any[]): IDiffResult {
const changes: IDiffChange[] = [];
const maxLength = Math.max(expected.length, actual.length);
for (let i = 0; i < maxLength; i++) {
const expectedItem = expected[i];
const actualItem = actual[i];
if (i >= expected.length) {
changes.push({
type: 'add',
path: [String(i)],
newValue: actualItem
});
} else if (i >= actual.length) {
changes.push({
type: 'remove',
path: [String(i)],
oldValue: expectedItem
});
} else if (!deepEqual(expectedItem, actualItem)) {
changes.push({
type: 'modify',
path: [String(i)],
oldValue: expectedItem,
newValue: actualItem
});
}
}
return {
type: 'array',
changes
};
}
/**
* Generate diff for objects
*/
function generateObjectDiff(expected: any, actual: any): IDiffResult {
const changes: IDiffChange[] = [];
const allKeys = new Set([...Object.keys(expected), ...Object.keys(actual)]);
for (const key of allKeys) {
const expectedValue = expected[key];
const actualValue = actual[key];
if (!(key in expected)) {
changes.push({
type: 'add',
path: [key],
newValue: actualValue
});
} else if (!(key in actual)) {
changes.push({
type: 'remove',
path: [key],
oldValue: expectedValue
});
} else if (!deepEqual(expectedValue, actualValue)) {
changes.push({
type: 'modify',
path: [key],
oldValue: expectedValue,
newValue: actualValue
});
}
}
return {
type: 'object',
changes
};
}
/**
* Deep equality check
*/
function deepEqual(a: any, b: any): boolean {
if (a === b) return true;
if (a === null || b === null) return false;
if (typeof a !== typeof b) return false;
if (typeof a === 'object') {
if (Array.isArray(a) && Array.isArray(b)) {
if (a.length !== b.length) return false;
return a.every((item, index) => deepEqual(item, b[index]));
}
const keysA = Object.keys(a);
const keysB = Object.keys(b);
if (keysA.length !== keysB.length) return false;
return keysA.every(key => deepEqual(a[key], b[key]));
}
return false;
}

13
ts_tapbundle_protocol/index.ts Normal file
View File

@ -0,0 +1,13 @@
// Protocol V2 - Isomorphic implementation for improved TAP protocol
// This module is designed to work in both browser and Node.js environments
export * from './protocol.types.js';
export * from './protocol.emitter.js';
export * from './protocol.parser.js';
// Re-export main classes for convenience
export { ProtocolEmitter } from './protocol.emitter.js';
export { ProtocolParser } from './protocol.parser.js';
// Re-export constants
export { PROTOCOL_MARKERS, PROTOCOL_VERSION } from './protocol.types.js';

196
ts_tapbundle_protocol/protocol.emitter.ts Normal file
View File

@ -0,0 +1,196 @@
import type {
ITestResult,
ITestMetadata,
IPlanLine,
ISnapshotData,
IErrorBlock,
ITestEvent
} from './protocol.types.js';
import {
PROTOCOL_MARKERS,
PROTOCOL_VERSION
} from './protocol.types.js';
/**
* ProtocolEmitter generates Protocol V2 messages
* This class is used by tapbundle to emit test results in the new protocol format
*/
export class ProtocolEmitter {
/**
* Emit protocol version header
*/
public emitProtocolHeader(): string {
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.PROTOCOL_PREFIX}${PROTOCOL_VERSION}${PROTOCOL_MARKERS.END}`;
}
/**
* Emit TAP version line
*/
public emitTapVersion(version: number = 13): string {
return `TAP version ${version}`;
}
/**
* Emit test plan
*/
public emitPlan(plan: IPlanLine): string {
if (plan.skipAll) {
return `1..0 # Skipped: ${plan.skipAll}`;
}
return `${plan.start}..${plan.end}`;
}
/**
* Emit a test result
*/
public emitTest(result: ITestResult): string[] {
const lines: string[] = [];
// Build the basic TAP line
let tapLine = result.ok ? 'ok' : 'not ok';
tapLine += ` ${result.testNumber}`;
tapLine += ` - ${result.description}`;
// Add directive if present
if (result.directive) {
tapLine += ` # ${result.directive.type.toUpperCase()}`;
if (result.directive.reason) {
tapLine += ` ${result.directive.reason}`;
}
}
// Add inline metadata for simple cases
if (result.metadata && this.shouldUseInlineMetadata(result.metadata)) {
const metaStr = this.createInlineMetadata(result.metadata);
if (metaStr) {
tapLine += ` ${metaStr}`;
}
}
lines.push(tapLine);
// Add block metadata for complex cases
if (result.metadata && !this.shouldUseInlineMetadata(result.metadata)) {
lines.push(...this.createBlockMetadata(result.metadata, result.testNumber));
}
return lines;
}
/**
* Emit a comment line
*/
public emitComment(comment: string): string {
return `# ${comment}`;
}
/**
* Emit bailout
*/
public emitBailout(reason: string): string {
return `Bail out! ${reason}`;
}
/**
* Emit snapshot data
*/
public emitSnapshot(snapshot: ISnapshotData): string[] {
const lines: string[] = [];
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_PREFIX}${snapshot.name}${PROTOCOL_MARKERS.END}`);
if (snapshot.format === 'json') {
lines.push(JSON.stringify(snapshot.content, null, 2));
} else {
lines.push(String(snapshot.content));
}
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_END}${PROTOCOL_MARKERS.END}`);
return lines;
}
/**
* Emit error block
*/
public emitError(error: IErrorBlock): string[] {
const lines: string[] = [];
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_PREFIX}${PROTOCOL_MARKERS.END}`);
lines.push(JSON.stringify(error, null, 2));
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_END}${PROTOCOL_MARKERS.END}`);
return lines;
}
/**
* Emit test event
*/
public emitEvent(event: ITestEvent): string {
const eventJson = JSON.stringify(event);
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.EVENT_PREFIX}${eventJson}${PROTOCOL_MARKERS.END}`;
}
/**
* Check if metadata should be inline
*/
private shouldUseInlineMetadata(metadata: ITestMetadata): boolean {
// Use inline for simple metadata (time, retry, simple skip/todo)
const hasComplexData = metadata.error ||
metadata.custom ||
(metadata.tags && metadata.tags.length > 0) ||
metadata.file ||
metadata.line;
return !hasComplexData;
}
/**
* Create inline metadata string
*/
private createInlineMetadata(metadata: ITestMetadata): string {
const parts: string[] = [];
if (metadata.time !== undefined) {
parts.push(`time:${metadata.time}`);
}
if (metadata.retry !== undefined) {
parts.push(`retry:${metadata.retry}`);
}
if (metadata.skip) {
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SKIP_PREFIX}${metadata.skip}${PROTOCOL_MARKERS.END}`;
}
if (metadata.todo) {
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.TODO_PREFIX}${metadata.todo}${PROTOCOL_MARKERS.END}`;
}
if (parts.length > 0) {
return `${PROTOCOL_MARKERS.START}${parts.join(',')}${PROTOCOL_MARKERS.END}`;
}
return '';
}
/**
* Create block metadata lines
*/
private createBlockMetadata(metadata: ITestMetadata, testNumber?: number): string[] {
const lines: string[] = [];
// Create a clean metadata object without skip/todo (handled inline)
const blockMeta = { ...metadata };
delete blockMeta.skip;
delete blockMeta.todo;
// Emit metadata block
const metaJson = JSON.stringify(blockMeta);
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.META_PREFIX}${metaJson}${PROTOCOL_MARKERS.END}`);
// Emit separate error block if present
if (metadata.error) {
lines.push(...this.emitError({ testNumber, error: metadata.error }));
}
return lines;
}
}

407
ts_tapbundle_protocol/protocol.parser.ts Normal file
View File

@ -0,0 +1,407 @@
import type {
ITestResult,
ITestMetadata,
IPlanLine,
IProtocolMessage,
ISnapshotData,
IErrorBlock,
ITestEvent
} from './protocol.types.js';
import {
PROTOCOL_MARKERS
} from './protocol.types.js';
/**
* ProtocolParser parses Protocol V2 messages
* This class is used by tstest to parse test results from the new protocol format
*/
export class ProtocolParser {
private protocolVersion: string | null = null;
private inBlock = false;
private blockType: string | null = null;
private blockContent: string[] = [];
/**
* Parse a single line and return protocol messages
*/
public parseLine(line: string): IProtocolMessage[] {
const messages: IProtocolMessage[] = [];
// Handle block content
if (this.inBlock) {
if (this.isBlockEnd(line)) {
messages.push(this.finalizeBlock());
this.inBlock = false;
this.blockType = null;
this.blockContent = [];
} else {
this.blockContent.push(line);
}
return messages;
}
// Check for block start
if (this.isBlockStart(line)) {
this.inBlock = true;
this.blockType = this.extractBlockType(line);
return messages;
}
// Check for protocol version
const protocolVersion = this.parseProtocolVersion(line);
if (protocolVersion) {
this.protocolVersion = protocolVersion;
messages.push({
type: 'protocol',
content: { version: protocolVersion }
});
return messages;
}
// Parse TAP version
const tapVersion = this.parseTapVersion(line);
if (tapVersion !== null) {
messages.push({
type: 'version',
content: tapVersion
});
return messages;
}
// Parse plan
const plan = this.parsePlan(line);
if (plan) {
messages.push({
type: 'plan',
content: plan
});
return messages;
}
// Parse bailout
const bailout = this.parseBailout(line);
if (bailout) {
messages.push({
type: 'bailout',
content: bailout
});
return messages;
}
// Parse comment
if (this.isComment(line)) {
messages.push({
type: 'comment',
content: line.substring(2) // Remove "# "
});
return messages;
}
// Parse test result
const testResult = this.parseTestResult(line);
if (testResult) {
messages.push({
type: 'test',
content: testResult
});
return messages;
}
// Parse event
const event = this.parseEvent(line);
if (event) {
messages.push({
type: 'event',
content: event
});
return messages;
}
return messages;
}
/**
* Parse protocol version header
*/
private parseProtocolVersion(line: string): string | null {
const match = this.extractProtocolData(line, PROTOCOL_MARKERS.PROTOCOL_PREFIX);
return match;
}
/**
* Parse TAP version line
*/
private parseTapVersion(line: string): number | null {
const match = line.match(/^TAP version (\d+)$/);
if (match) {
return parseInt(match[1], 10);
}
return null;
}
/**
* Parse plan line
*/
private parsePlan(line: string): IPlanLine | null {
// Skip all plan
const skipMatch = line.match(/^1\.\.0\s*#\s*Skipped:\s*(.*)$/);
if (skipMatch) {
return {
start: 1,
end: 0,
skipAll: skipMatch[1]
};
}
// Normal plan
const match = line.match(/^(\d+)\.\.(\d+)$/);
if (match) {
return {
start: parseInt(match[1], 10),
end: parseInt(match[2], 10)
};
}
return null;
}
/**
* Parse bailout
*/
private parseBailout(line: string): string | null {
const match = line.match(/^Bail out!\s*(.*)$/);
return match ? match[1] : null;
}
/**
* Parse event
*/
private parseEvent(line: string): ITestEvent | null {
const eventData = this.extractProtocolData(line, PROTOCOL_MARKERS.EVENT_PREFIX);
if (eventData) {
try {
return JSON.parse(eventData) as ITestEvent;
} catch (e) {
// Invalid JSON, ignore
return null;
}
}
return null;
}
/**
* Check if line is a comment
*/
private isComment(line: string): boolean {
return line.startsWith('# ');
}
/**
* Parse test result line
*/
private parseTestResult(line: string): ITestResult | null {
// First extract any inline metadata
const metadata = this.extractInlineMetadata(line);
const cleanLine = this.removeInlineMetadata(line);
// Parse the TAP part
const tapMatch = cleanLine.match(/^(ok|not ok)\s+(\d+)\s*-?\s*(.*)$/);
if (!tapMatch) {
return null;
}
const result: ITestResult = {
ok: tapMatch[1] === 'ok',
testNumber: parseInt(tapMatch[2], 10),
description: tapMatch[3].trim()
};
// Parse directive
const directiveMatch = result.description.match(/^(.*?)\s*#\s*(SKIP|TODO)\s*(.*)$/i);
if (directiveMatch) {
result.description = directiveMatch[1].trim();
result.directive = {
type: directiveMatch[2].toLowerCase() as 'skip' | 'todo',
reason: directiveMatch[3] || undefined
};
}
// Add metadata if found
if (metadata) {
result.metadata = metadata;
}
return result;
}
/**
* Extract inline metadata from line
*/
private extractInlineMetadata(line: string): ITestMetadata | null {
const metadata: ITestMetadata = {};
let hasData = false;
// Extract skip reason
const skipData = this.extractProtocolData(line, PROTOCOL_MARKERS.SKIP_PREFIX);
if (skipData) {
metadata.skip = skipData;
hasData = true;
}
// Extract todo reason
const todoData = this.extractProtocolData(line, PROTOCOL_MARKERS.TODO_PREFIX);
if (todoData) {
metadata.todo = todoData;
hasData = true;
}
// Extract META JSON
const metaData = this.extractProtocolData(line, PROTOCOL_MARKERS.META_PREFIX);
if (metaData) {
try {
Object.assign(metadata, JSON.parse(metaData));
hasData = true;
} catch (e) {
// Invalid JSON, ignore
}
}
// Extract simple key:value pairs
const simpleMatch = line.match(new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]+)${this.escapeRegex(PROTOCOL_MARKERS.END)}`));
if (simpleMatch && !simpleMatch[1].includes(':')) {
// Not a prefixed format, might be key:value pairs
const pairs = simpleMatch[1].split(',');
for (const pair of pairs) {
const [key, value] = pair.split(':');
if (key && value) {
if (key === 'time') {
metadata.time = parseInt(value, 10);
hasData = true;
} else if (key === 'retry') {
metadata.retry = parseInt(value, 10);
hasData = true;
}
}
}
}
return hasData ? metadata : null;
}
/**
* Remove inline metadata from line
*/
private removeInlineMetadata(line: string): string {
// Remove all protocol markers
const regex = new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}[^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*${this.escapeRegex(PROTOCOL_MARKERS.END)}`, 'g');
return line.replace(regex, '').trim();
}
/**
* Extract protocol data with specific prefix
*/
private extractProtocolData(line: string, prefix: string): string | null {
const regex = new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}${this.escapeRegex(prefix)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*)${this.escapeRegex(PROTOCOL_MARKERS.END)}`);
const match = line.match(regex);
return match ? match[1] : null;
}
/**
* Check if line starts a block
*/
private isBlockStart(line: string): boolean {
// Only match if the line is exactly the block marker (after trimming)
const trimmed = line.trim();
return trimmed === `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_PREFIX}${PROTOCOL_MARKERS.END}` ||
(trimmed.startsWith(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_PREFIX}`) &&
trimmed.endsWith(PROTOCOL_MARKERS.END) &&
!trimmed.includes(' '));
}
/**
* Check if line ends a block
*/
private isBlockEnd(line: string): boolean {
return line.includes(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_END}${PROTOCOL_MARKERS.END}`) ||
line.includes(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_END}${PROTOCOL_MARKERS.END}`);
}
/**
* Extract block type from start line
*/
private extractBlockType(line: string): string | null {
if (line.includes(PROTOCOL_MARKERS.ERROR_PREFIX)) {
return 'error';
}
if (line.includes(PROTOCOL_MARKERS.SNAPSHOT_PREFIX)) {
const match = line.match(new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}${this.escapeRegex(PROTOCOL_MARKERS.SNAPSHOT_PREFIX)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*)${this.escapeRegex(PROTOCOL_MARKERS.END)}`));
return match ? `snapshot:${match[1]}` : 'snapshot';
}
return null;
}
/**
* Finalize current block
*/
private finalizeBlock(): IProtocolMessage {
const content = this.blockContent.join('\n');
if (this.blockType === 'error') {
try {
const errorData = JSON.parse(content) as IErrorBlock;
return {
type: 'error',
content: errorData
};
} catch (e) {
return {
type: 'error',
content: { error: { message: content } }
};
}
}
if (this.blockType?.startsWith('snapshot:')) {
const name = this.blockType.substring(9);
let parsedContent = content;
let format: 'json' | 'text' = 'text';
try {
parsedContent = JSON.parse(content);
format = 'json';
} catch (e) {
// Not JSON, keep as text
}
return {
type: 'snapshot',
content: {
name,
content: parsedContent,
format
} as ISnapshotData
};
}
// Fallback
return {
type: 'comment',
content: content
};
}
/**
* Escape regex special characters
*/
private escapeRegex(str: string): string {
return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
/**
* Get protocol version
*/
public getProtocolVersion(): string | null {
return this.protocolVersion;
}
}

148
ts_tapbundle_protocol/protocol.types.ts Normal file
View File

@ -0,0 +1,148 @@
// Protocol V2 Types and Interfaces
// This file contains all type definitions for the improved TAP protocol
export interface ITestMetadata {
// Timing
time?: number; // milliseconds
startTime?: number; // Unix timestamp
endTime?: number; // Unix timestamp
// Status
skip?: string; // skip reason
todo?: string; // todo reason
retry?: number; // retry attempt
maxRetries?: number; // max retries allowed
// Error details
error?: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
code?: string;
};
// Test context
file?: string; // source file
line?: number; // line number
column?: number; // column number
// Custom data
tags?: string[]; // test tags
custom?: Record<string, any>;
}
export interface ITestResult {
ok: boolean;
testNumber: number;
description: string;
directive?: {
type: 'skip' | 'todo';
reason?: string;
};
metadata?: ITestMetadata;
}
export interface IPlanLine {
start: number;
end: number;
skipAll?: string;
}
export interface IProtocolMessage {
type: 'test' | 'plan' | 'comment' | 'version' | 'bailout' | 'protocol' | 'snapshot' | 'error' | 'event';
content: any;
}
export interface IProtocolVersion {
version: string;
features?: string[];
}
export interface ISnapshotData {
name: string;
content: any;
format?: 'json' | 'text' | 'binary';
}
export interface IErrorBlock {
testNumber?: number;
error: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
};
}
// Enhanced Communication Types
export type EventType =
| 'test:queued'
| 'test:started'
| 'test:progress'
| 'test:completed'
| 'suite:started'
| 'suite:completed'
| 'hook:started'
| 'hook:completed'
| 'assertion:failed';
export interface ITestEvent {
eventType: EventType;
timestamp: number;
data: {
testNumber?: number;
description?: string;
suiteName?: string;
hookName?: string;
progress?: number; // 0-100
duration?: number;
error?: IEnhancedError;
[key: string]: any;
};
}
export interface IEnhancedError {
message: string;
stack?: string;
diff?: IDiffResult;
actual?: any;
expected?: any;
code?: string;
type?: 'assertion' | 'timeout' | 'uncaught' | 'syntax' | 'runtime';
}
export interface IDiffResult {
type: 'string' | 'object' | 'array' | 'primitive';
changes: IDiffChange[];
context?: number; // lines of context
}
export interface IDiffChange {
type: 'add' | 'remove' | 'modify';
path?: string[]; // for object/array diffs
oldValue?: any;
newValue?: any;
line?: number; // for string diffs
content?: string;
}
// Protocol markers
export const PROTOCOL_MARKERS = {
START: '⟦TSTEST:',
END: '⟧',
META_PREFIX: 'META:',
ERROR_PREFIX: 'ERROR',
ERROR_END: '/ERROR',
SNAPSHOT_PREFIX: 'SNAPSHOT:',
SNAPSHOT_END: '/SNAPSHOT',
PROTOCOL_PREFIX: 'PROTOCOL:',
SKIP_PREFIX: 'SKIP:',
TODO_PREFIX: 'TODO:',
EVENT_PREFIX: 'EVENT:',
} as const;
// Protocol version
export const PROTOCOL_VERSION = '2.0.0';