fix(protocol): Fix inline timing metadata parsing and enhance test coverage for performance metrics and timing edge cases

This commit is contained in:
2025-05-26 08:22:26 +00:00
parent 845f146e91
commit 899045e6aa
7 changed files with 651 additions and 4 deletions

View File

@@ -265,4 +265,59 @@ Previously reported issues with these methods have been resolved:
- Other tests are not executed but still counted
- Both regular and parallel only tests supported
These fixes ensure accurate test counts and proper TAP-compliant output for all test states.
These fixes ensure accurate test counts and proper TAP-compliant output for all test states.
## Test Timing Implementation
### Timing Architecture
Test timing is captured using `@push.rocks/smarttime`'s `HrtMeasurement` class, which provides high-resolution timing:
1. **Timing Capture**:
- Each `TapTest` instance has its own `HrtMeasurement`
- Timer starts immediately before test function execution
- Timer stops after test completes (or fails/times out)
- Millisecond precision is used for reporting
2. **Protocol Integration**:
- Timing is embedded in TAP output using Protocol V2 markers
- Inline format for simple timing: `ok 1 - test name ⟦TSTEST:time:123⟧`
- Block format for complex metadata: `⟦TSTEST:META:{"time":456,"file":"test.ts"}⟧`
3. **Performance Metrics Calculation**:
- Average is calculated from sum of individual test times, not total runtime
- Slowest test detection prefers tests with >0ms duration
- Failed tests still contribute their execution time to metrics
### Edge Cases and Considerations
1. **Sub-millisecond Tests**:
- Very fast tests may report 0ms due to millisecond rounding
- Performance metrics handle this by showing "All tests completed in <1ms" when appropriate
2. **Special Test States**:
- **Skipped tests**: Report 0ms (not executed)
- **Todo tests**: Report 0ms (not executed)
- **Failed tests**: Report actual execution time before failure
- **Timeout tests**: Report time until timeout occurred
3. **Parallel Test Timing**:
- Each parallel test tracks its own execution time independently
- Parallel tests may have overlapping execution periods
- Total suite time reflects wall-clock time, not sum of test times
4. **Hook Timing**:
- `beforeEach`/`afterEach` hooks are not included in individual test times
- Only the actual test function execution is measured
5. **Retry Timing**:
- When tests retry, only the final attempt's duration is reported
- Each retry attempt emits separate `test:started` events
### Parser Fix for Timing Metadata
The protocol parser was fixed to correctly handle inline timing metadata:
- Changed condition from `!simpleMatch[1].includes(':')` to check for simple key:value pairs
- Excludes prefixed formats (META:, SKIP:, TODO:, EVENT:) while parsing simple formats like `time:250`
This ensures timing metadata is correctly extracted and displayed in test results.