fix(protocol): Fix inline timing metadata parsing and enhance test coverage for performance metrics and timing edge cases
This commit is contained in:
@@ -265,4 +265,59 @@ Previously reported issues with these methods have been resolved:
|
||||
- Other tests are not executed but still counted
|
||||
- Both regular and parallel only tests supported
|
||||
|
||||
These fixes ensure accurate test counts and proper TAP-compliant output for all test states.
|
||||
These fixes ensure accurate test counts and proper TAP-compliant output for all test states.
|
||||
|
||||
## Test Timing Implementation
|
||||
|
||||
### Timing Architecture
|
||||
|
||||
Test timing is captured using `@push.rocks/smarttime`'s `HrtMeasurement` class, which provides high-resolution timing:
|
||||
|
||||
1. **Timing Capture**:
|
||||
- Each `TapTest` instance has its own `HrtMeasurement`
|
||||
- Timer starts immediately before test function execution
|
||||
- Timer stops after test completes (or fails/times out)
|
||||
- Millisecond precision is used for reporting
|
||||
|
||||
2. **Protocol Integration**:
|
||||
- Timing is embedded in TAP output using Protocol V2 markers
|
||||
- Inline format for simple timing: `ok 1 - test name ⟦TSTEST:time:123⟧`
|
||||
- Block format for complex metadata: `⟦TSTEST:META:{"time":456,"file":"test.ts"}⟧`
|
||||
|
||||
3. **Performance Metrics Calculation**:
|
||||
- Average is calculated from sum of individual test times, not total runtime
|
||||
- Slowest test detection prefers tests with >0ms duration
|
||||
- Failed tests still contribute their execution time to metrics
|
||||
|
||||
### Edge Cases and Considerations
|
||||
|
||||
1. **Sub-millisecond Tests**:
|
||||
- Very fast tests may report 0ms due to millisecond rounding
|
||||
- Performance metrics handle this by showing "All tests completed in <1ms" when appropriate
|
||||
|
||||
2. **Special Test States**:
|
||||
- **Skipped tests**: Report 0ms (not executed)
|
||||
- **Todo tests**: Report 0ms (not executed)
|
||||
- **Failed tests**: Report actual execution time before failure
|
||||
- **Timeout tests**: Report time until timeout occurred
|
||||
|
||||
3. **Parallel Test Timing**:
|
||||
- Each parallel test tracks its own execution time independently
|
||||
- Parallel tests may have overlapping execution periods
|
||||
- Total suite time reflects wall-clock time, not sum of test times
|
||||
|
||||
4. **Hook Timing**:
|
||||
- `beforeEach`/`afterEach` hooks are not included in individual test times
|
||||
- Only the actual test function execution is measured
|
||||
|
||||
5. **Retry Timing**:
|
||||
- When tests retry, only the final attempt's duration is reported
|
||||
- Each retry attempt emits separate `test:started` events
|
||||
|
||||
### Parser Fix for Timing Metadata
|
||||
|
||||
The protocol parser was fixed to correctly handle inline timing metadata:
|
||||
- Changed condition from `!simpleMatch[1].includes(':')` to check for simple key:value pairs
|
||||
- Excludes prefixed formats (META:, SKIP:, TODO:, EVENT:) while parsing simple formats like `time:250`
|
||||
|
||||
This ensures timing metadata is correctly extracted and displayed in test results.
|
Reference in New Issue
Block a user