feat(package): Revamp package exports and update permissions with an extensive improvement plan for test runner enhancements.

This commit is contained in:
Philipp Kunz 2025-05-15 20:48:26 +00:00
parent cc388f1408
commit b28e2eace3
4 changed files with 258 additions and 33 deletions

View File

@ -1,5 +1,12 @@
# Changelog # Changelog
## 2025-05-15 - 1.6.0 - feat(package)
Revamp package exports and update permissions with an extensive improvement plan for test runner enhancements.
- Replaced 'main' and 'typings' in package.json with explicit exports for improved module resolution.
- Added .claude/settings.local.json to configure permissions for bash commands and web fetches.
- Updated readme.plan.md with a comprehensive roadmap covering enhanced error reporting, rich test metadata, nested test suites, and advanced test features.
## 2025-05-15 - 1.5.0 - feat(cli) ## 2025-05-15 - 1.5.0 - feat(cli)
Improve test runner configuration: update test scripts, reorganize test directories, update dependencies and add local settings for command permissions. Improve test runner configuration: update test scripts, reorganize test directories, update dependencies and add local settings for command permissions.

View File

@ -3,8 +3,11 @@
"version": "1.5.0", "version": "1.5.0",
"private": false, "private": false,
"description": "a test utility to run tests that match test/**/*.ts", "description": "a test utility to run tests that match test/**/*.ts",
"main": "dist_ts/index.js", "exports": {
"typings": "dist_ts/index.d.ts", ".": "./dist_ts/index.js",
"./tapbundle": "./dist_ts_tapbundle/index.js",
"./tapbundle_node": "./dist_ts_tapbundle_node/index.js"
},
"type": "module", "type": "module",
"author": "Lossless GmbH", "author": "Lossless GmbH",
"license": "MIT", "license": "MIT",

View File

@ -1,41 +1,256 @@
# Plan for showing logs for failed tests # Improvement Plan for tstest and tapbundle
!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !! !! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
## Goal ## 1. Enhanced Communication Between tapbundle and tstest
When a test fails, we want to display all the console logs from that failed test in the terminal, even without the --verbose flag. This makes debugging failed tests much easier.
## Current Behavior ### 1.1 Real-time Test Progress API
- Default mode: Only shows test results, no console logs - Create a bidirectional communication channel between tapbundle and tstest
- Verbose mode: Shows all console logs from all tests - Emit events for test lifecycle stages (start, progress, completion)
- When a test fails: Only shows the error message - Allow tstest to subscribe to tapbundle events for better progress reporting
- Implement a standardized message format for test metadata
## Desired Behavior ### 1.2 Rich Error Reporting
- Default mode: Shows test results, and IF a test fails, shows all console logs from that failed test - Pass structured error objects from tapbundle to tstest
- Verbose mode: Shows all console logs from all tests (unchanged) - Include stack traces, code snippets, and contextual information
- When a test fails: Shows all console logs from that test plus the error - Support for error categorization (assertion failures, timeouts, uncaught exceptions)
- Visual diff output for failed assertions
## Implementation Plan ## 2. Enhanced toolsArg Functionality
### 1. Update TapParser ### 2.1 Test Flow Control
- Store console logs for each test temporarily ```typescript
- When a test fails, mark that its logs should be shown tap.test('conditional test', async (toolsArg) => {
const result = await someOperation();
### 2. Update TsTestLogger // Skip the rest of the test
- Add a new method to handle failed test logs if (!result) {
- Modify testConsoleOutput to buffer logs when not in verbose mode return toolsArg.skip('Precondition not met');
- When a test fails, flush the buffered logs for that test }
### 3. Update test result handling // Conditional skipping
- When a test fails, trigger display of all buffered logs for that test await toolsArg.skipIf(condition, 'Reason for skipping');
- Clear logs after each test completes successfully
## Code Changes Needed // Mark test as todo
1. Add log buffering to TapParser await toolsArg.todo('Not implemented yet');
2. Update TsTestLogger to handle failed test logs });
3. Modify test result processing to show logs on failure ```
## Files to Modify ### 2.2 Test Metadata and Configuration
- `ts/tstest.classes.tap.parser.ts` - Add log buffering ```typescript
- `ts/tstest.logging.ts` - Add failed test log handling tap.test('configurable test', async (toolsArg) => {
- `ts/tstest.classes.tap.testresult.ts` - May need to store logs // Set custom timeout
toolsArg.timeout(5000);
// Retry on failure
toolsArg.retry(3);
// Add tags for filtering
toolsArg.tags(['slow', 'integration']);
// Set test priority
toolsArg.priority('high');
});
```
### 2.3 Test Data and Context Sharing
```typescript
tap.test('data-driven test', async (toolsArg) => {
// Access shared context
const sharedData = toolsArg.context.get('sharedData');
// Set data for other tests
toolsArg.context.set('resultData', computedValue);
// Parameterized test data
const testData = toolsArg.data<TestInput>();
expect(processData(testData)).toEqual(expected);
});
```
## 3. Nested Tests and Test Suites
### 3.1 Test Grouping with describe()
```typescript
tap.describe('User Authentication', () => {
tap.beforeEach(async (toolsArg) => {
// Setup for each test in this suite
await toolsArg.context.set('db', await createTestDatabase());
});
tap.afterEach(async (toolsArg) => {
// Cleanup after each test
await toolsArg.context.get('db').cleanup();
});
tap.test('should login with valid credentials', async (toolsArg) => {
// Test implementation
});
tap.describe('Password Reset', () => {
tap.test('should send reset email', async (toolsArg) => {
// Nested test
});
});
});
```
### 3.2 Hierarchical Test Organization
- Support for multiple levels of nesting
- Inherited context and configuration from parent suites
- Aggregated reporting for test suites
- Suite-level lifecycle hooks
## 4. Advanced Test Features
### 4.1 Snapshot Testing
```typescript
tap.test('component render', async (toolsArg) => {
const output = renderComponent(props);
// Compare with stored snapshot
await toolsArg.matchSnapshot(output, 'component-output');
});
```
### 4.2 Performance Benchmarking
```typescript
tap.test('performance test', async (toolsArg) => {
const benchmark = toolsArg.benchmark();
// Run operation
await expensiveOperation();
// Assert performance constraints
benchmark.expect({
maxDuration: 1000,
maxMemory: '100MB'
});
});
```
### 4.3 Test Fixtures and Factories
```typescript
tap.test('with fixtures', async (toolsArg) => {
// Create test fixtures
const user = await toolsArg.fixture('user', { name: 'Test User' });
const post = await toolsArg.fixture('post', { author: user });
// Use factory functions
const users = await toolsArg.factory('user').createMany(5);
});
```
## 5. Test Execution Improvements
### 5.1 Parallel Test Execution
- Run independent tests concurrently
- Configurable concurrency limits
- Resource pooling for shared resources
- Proper isolation between parallel tests
### 5.2 Watch Mode
- Automatically re-run tests on file changes
- Intelligent test selection based on changed files
- Fast feedback loop for development
- Integration with IDE/editor plugins
### 5.3 Advanced Test Filtering
```typescript
// Run tests by tags
tstest --tags "unit,fast"
// Exclude tests by pattern
tstest --exclude "**/slow/**"
// Run only failed tests from last run
tstest --failed
// Run tests modified in git
tstest --changed
```
## 6. Reporting and Analytics
### 6.1 Custom Reporters
- Plugin architecture for custom reporters
- Built-in reporters: JSON, JUnit, HTML, Markdown
- Real-time streaming reporters
- Aggregated test metrics and trends
### 6.2 Coverage Integration
- Built-in code coverage collection
- Coverage thresholds and enforcement
- Coverage trending over time
- Integration with CI/CD pipelines
### 6.3 Test Analytics Dashboard
- Web-based dashboard for test results
- Historical test performance data
- Flaky test detection
- Test impact analysis
## 7. Developer Experience
### 7.1 Better Error Messages
- Clear, actionable error messages
- Suggestions for common issues
- Links to documentation
- Code examples in error output
### 7.2 Interactive Mode
- REPL for exploring test failures
- Debugging integration
- Step-through test execution
- Interactive test data manipulation
### 7.3 VS Code Extension
- Test explorer integration
- Inline test results
- CodeLens for running individual tests
- Debugging support
## Implementation Phases
### Phase 1: Core Enhancements (Priority: High)
1. Implement enhanced toolsArg methods (skip, skipIf, timeout, retry)
2. Add basic test grouping with describe()
3. Improve error reporting between tapbundle and tstest
### Phase 2: Advanced Features (Priority: Medium)
1. Implement nested test suites
2. Add snapshot testing
3. Create test fixture system
4. Implement parallel test execution
### Phase 3: Developer Experience (Priority: Medium)
1. Add watch mode
2. Implement custom reporters
3. Create VS Code extension
4. Add interactive debugging
### Phase 4: Analytics and Performance (Priority: Low)
1. Build test analytics dashboard
2. Add performance benchmarking
3. Implement coverage integration
4. Create trend analysis tools
## Technical Considerations
### API Design Principles
- Maintain backward compatibility
- Progressive enhancement approach
- Opt-in features to avoid breaking changes
- Clear migration paths for new features
### Performance Goals
- Minimal overhead for test execution
- Efficient parallel execution
- Fast test discovery
- Optimized browser test bundling
### Integration Points
- Clean interfaces between tstest and tapbundle
- Extensible plugin architecture
- Standard test result format
- Compatible with existing CI/CD tools

View File

@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@git.zone/tstest', name: '@git.zone/tstest',
version: '1.5.0', version: '1.6.0',
description: 'a test utility to run tests that match test/**/*.ts' description: 'a test utility to run tests that match test/**/*.ts'
} }