git.zone/tstest
2
0
Fork 0
Code Issues 2 Pull Requests Packages Projects Releases Wiki Activity

feat(core): Implement Protocol V2 with enhanced settings and lifecycle hooks

Browse Source
This commit is contained in:
Philipp Kunz
2025-05-26 04:02:32 +00:00
parent 91880f8d42
commit 33d2ff1d4f
24 changed files with 2356 additions and 441 deletions
Download Patch File Download Diff File Expand all files Collapse all files

172
readme.plan.md
View File

@@ -2,33 +2,33 @@
!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
## Improved Internal Protocol (NEW - Critical)
## Improved Internal Protocol (NEW - Critical) ✅ COMPLETED
### Current Issues
- TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
- Fragile regex parsing that breaks with special characters
- Limited extensibility for new metadata types
### Current Issues ✅ RESOLVED
- ✅ TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
- ✅ Fragile regex parsing that breaks with special characters
- ✅ Limited extensibility for new metadata types
### Proposed Solution: Protocol V2
- Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
- Structured JSON metadata format
- Separate protocol blocks for complex data (errors, snapshots)
- Complete replacement of v1 (no backwards compatibility needed)
### Proposed Solution: Protocol V2 ✅ IMPLEMENTED
- ✅ Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
- ✅ Structured JSON metadata format
- ✅ Separate protocol blocks for complex data (errors, snapshots)
- ✅ Complete replacement of v1 (no backwards compatibility needed)
### Implementation
- Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
- Phase 2: Replace all v1 code in both tstest and tapbundle with v2
- Phase 3: Delete all v1 parsing and generation code
### Implementation ✅ COMPLETED
- ✅ Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
- ✅ Phase 2: Replace all v1 code in both tstest and tapbundle with v2
- ✅ Phase 3: Delete all v1 parsing and generation code
#### ts_tapbundle_protocol Directory
The protocol v2 implementation will be contained in the `ts_tapbundle_protocol` directory as isomorphic TypeScript code:
- **Isomorphic Design**: All code must work in both browser and Node.js environments
- **No Node.js Imports**: No Node.js-specific modules allowed (no fs, path, child_process, etc.)
- **Protocol Classes**: Contains classes implementing all sides of the protocol:
- `ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
- `ProtocolParser`: For parsing protocol v2 messages (used by tstest)
- `ProtocolMessage`: Base classes for different message types
- `ProtocolTypes`: TypeScript interfaces and types for protocol structures
- ✅ `ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
- ✅ `ProtocolParser`: For parsing protocol v2 messages (used by tstest)
- ✅ `ProtocolMessage`: Base classes for different message types
- ✅ `ProtocolTypes`: TypeScript interfaces and types for protocol structures
- **Pure TypeScript**: Only browser-compatible APIs and pure TypeScript/JavaScript code
- **Build Integration**:
- Compiled by `pnpm build` (via tsbuild) to `dist_ts_tapbundle_protocol/`
@@ -92,19 +92,19 @@ interface TapSettings {
3. **Application**: Apply settings to test execution
4. **Advanced**: Parallel execution and snapshot configuration
## 1. Enhanced Communication Between tapbundle and tstest
## 1. Enhanced Communication Between tapbundle and tstest ✅ COMPLETED
### 1.1 Real-time Test Progress API
- Create a bidirectional communication channel between tapbundle and tstest
- Emit events for test lifecycle stages (start, progress, completion)
- Allow tstest to subscribe to tapbundle events for better progress reporting
- Implement a standardized message format for test metadata
### 1.1 Real-time Test Progress API ✅ COMPLETED
- ✅ Create a bidirectional communication channel between tapbundle and tstest
- ✅ Emit events for test lifecycle stages (start, progress, completion)
- ✅ Allow tstest to subscribe to tapbundle events for better progress reporting
- ✅ Implement a standardized message format for test metadata
### 1.2 Rich Error Reporting
- Pass structured error objects from tapbundle to tstest
- Include stack traces, code snippets, and contextual information
- Support for error categorization (assertion failures, timeouts, uncaught exceptions)
- Visual diff output for failed assertions
### 1.2 Rich Error Reporting ✅ COMPLETED
- ✅ Pass structured error objects from tapbundle to tstest
- ✅ Include stack traces, code snippets, and contextual information
- ✅ Support for error categorization (assertion failures, timeouts, uncaught exceptions)
- ✅ Visual diff output for failed assertions
## 2. Enhanced toolsArg Functionality
@@ -155,7 +155,7 @@ tap.test('performance test', async (toolsArg) => {
- Fast feedback loop for development
- Integration with IDE/editor plugins
### 5.3 Advanced Test Filtering (Partial)
### 5.3 Advanced Test Filtering (Partial) ⚠️
```typescript
// Exclude tests by pattern (not yet implemented)
tstest --exclude "**/slow/**"
@@ -197,38 +197,38 @@ tstest --changed
## Implementation Phases
### Phase 1: Improved Internal Protocol (Priority: Critical) (NEW)
1. Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
- Implement ProtocolEmitter class for message generation
- Implement ProtocolParser class for message parsing
- Define ProtocolMessage types and interfaces
- Ensure all code is browser and Node.js compatible
- Add tspublish.json to configure build order
2. Update build configuration to compile ts_tapbundle_protocol first
3. Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
4. Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
5. Delete all v1 TAP parsing code from tstest
6. Delete all v1 TAP generation code from tapbundle
7. Test with real-world test suites containing special characters
### Phase 1: Improved Internal Protocol (Priority: Critical) ✅ COMPLETED
1. ✅ Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
- ✅ Implement ProtocolEmitter class for message generation
- ✅ Implement ProtocolParser class for message parsing
- ✅ Define ProtocolMessage types and interfaces
- ✅ Ensure all code is browser and Node.js compatible
- ✅ Add tspublish.json to configure build order
2. ✅ Update build configuration to compile ts_tapbundle_protocol first
3. ✅ Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
4. ✅ Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
5. ✅ Delete all v1 TAP parsing code from tstest
6. ✅ Delete all v1 TAP generation code from tapbundle
7. ✅ Test with real-world test suites containing special characters
### Phase 2: Test Configuration System (Priority: High)
1. Implement tap.settings() API with TypeScript interfaces
2. Add 00init.ts discovery and loading mechanism
3. Implement settings inheritance and merge logic
4. Apply settings to test execution (timeouts, retries, etc.)
### Phase 2: Test Configuration System (Priority: High) ✅ COMPLETED
1. ✅ Implement tap.settings() API with TypeScript interfaces
2. ✅ Add 00init.ts discovery and loading mechanism
3. ✅ Implement settings inheritance and merge logic
4. ✅ Apply settings to test execution (timeouts, retries, etc.)
### Phase 3: Enhanced Communication (Priority: High)
1. Build on Protocol V2 for richer communication
2. Implement real-time test progress API
3. Add structured error reporting with diffs and traces
### Phase 3: Enhanced Communication (Priority: High) ✅ COMPLETED
1. ✅ Build on Protocol V2 for richer communication
2. ✅ Implement real-time test progress API
3. ✅ Add structured error reporting with diffs and traces
### Phase 4: Developer Experience (Priority: Medium)
### Phase 4: Developer Experience (Priority: Medium) ❌ NOT STARTED
1. Add watch mode
2. Implement custom reporters
3. Complete advanced test filtering options
4. Add performance benchmarking API
### Phase 5: Analytics and Performance (Priority: Low)
### Phase 5: Analytics and Performance (Priority: Low) ❌ NOT STARTED
1. Build test analytics dashboard
2. Implement coverage integration
3. Create trend analysis tools
@@ -252,4 +252,66 @@ tstest --changed
- Clean interfaces between tstest and tapbundle
- Extensible plugin architecture
- Standard test result format
- Compatible with existing CI/CD tools
- Compatible with existing CI/CD tools
## Summary of Remaining Work
### ✅ Completed
- **Protocol V2**: Full implementation with Unicode delimiters, structured metadata, and special character handling
- **Test Configuration System**: tap.settings() API, 00init.ts discovery, settings inheritance, lifecycle hooks
- **Enhanced Communication**: Event-based test lifecycle reporting, visual diff output for assertion failures, real-time test progress API
- **Rich Error Reporting**: Stack traces, error metadata, and visual diffs through protocol
- **Tags Filtering**: `--tags` option for running specific tagged tests
### ✅ Existing Features (Not in Plan)
- **Timeout Support**: `--timeout` option and per-test timeouts
- **Test Retries**: `tap.retry()` for flaky test handling
- **Parallel Tests**: `.testParallel()` for concurrent execution
- **Snapshot Testing**: Basic implementation with `toMatchSnapshot()`
- **Test Lifecycle**: `describe()` blocks with `beforeEach`/`afterEach`
- **Skip Tests**: `tap.skip.test()` (though it doesn't create test objects)
- **Log Files**: `--logfile` option saves output to `.nogit/testlogs/`
- **Test Range**: `--startFrom` and `--stopAt` for partial runs
### ⚠️ Partially Completed
- **Advanced Test Filtering**: Have `--tags` but missing `--exclude`, `--failed`, `--changed`
### ❌ Not Started
#### High Priority
#### Medium Priority
2. **Developer Experience**
- Watch mode for file changes
- Custom reporters (JSON, JUnit, HTML, Markdown)
- Performance benchmarking API
- Better error messages with suggestions
3. **Enhanced toolsArg**
- Test data injection
- Context sharing between tests
- Parameterized tests
4. **Test Organization**
- Hierarchical test suites
- Nested describe blocks
- Suite-level lifecycle hooks
#### Low Priority
5. **Analytics and Performance**
- Test analytics dashboard
- Code coverage integration
- Trend analysis
- Flaky test detection
### Known Issues to Fix
- **tap.todo()**: Method exists but has no implementation
- **tap.skip.test()**: Doesn't create test objects, just logs (breaks test count)
- **Protocol Output**: Some protocol messages still appear in console output
- **Only Tests**: `tap.only.test()` exists but `--only` mode not fully implemented
### Next Recommended Steps
1. Add Watch Mode - high developer value
2. Implement Custom Reporters - important for CI/CD integration
3. Fix known issues: tap.todo() and tap.skip.test() implementations
4. Implement performance benchmarking API