Compare commits

...

10 Commits

12 changed files with 175 additions and 34 deletions

View File

@ -1,5 +1,41 @@
# Changelog # Changelog
## 2025-05-25 - 2.0.0 - BREAKING CHANGE(protocol)
Introduce protocol v2 implementation and update build configuration with revised build order, new tspublish files, and enhanced documentation
- Added ts_tapbundle_protocol directory with isomorphic implementation for protocol v2
- Updated readme.hints.md and readme.plan.md to explain the complete replacement of the v1 protocol and new build process
- Revised build order in tspublish.json files across ts, ts_tapbundle, ts_tapbundle_node, and ts_tapbundle_protocol
- Introduced .claude/settings.local.json with updated permission settings for CLI and build tools
## 2025-05-24 - 1.11.5 - fix(tstest)
Fix timeout handling to correctly evaluate TAP results after killing the test process.
- Added call to evaluateFinalResult() after killing the process in runInNode to ensure final TAP output is processed.
## 2025-05-24 - 1.11.4 - fix(logging)
Improve warning logging and add permission settings file
- Replace multiple logger.error calls with logger.warning for tests running over 1 minute
- Add warning method in tstest logger to display warning messages consistently
- Introduce .claude/settings.local.json to configure allowed permissions
## 2025-05-24 - 1.11.3 - fix(tstest)
Add timeout warning for long-running tests and introduce local settings configuration
- Add .claude/settings.local.json with permission configuration for local development
- Implement a timeout warning timer that notifies when tests run longer than 1 minute without an explicit timeout
- Clear the timeout warning timer upon test completion
- Remove unused import of logPrefixes in tstest.classes.tstest.ts
## 2025-05-24 - 1.11.2 - fix(tstest)
Improve timeout and error handling in test execution along with TAP parser timeout logic improvements.
- In the TAP parser, ensure that expected tests are properly set when no tests are defined to avoid false negatives on timeout.
- Use smartshell's terminate method and fallback kill to properly stop the entire process tree on timeout.
- Clean up browser, server, and WebSocket instances reliably even when a timeout occurs.
- Minor improvements in log file filtering and error logging for better clarity.
## 2025-05-24 - 1.11.1 - fix(tstest) ## 2025-05-24 - 1.11.1 - fix(tstest)
Clear timeout identifiers after successful test execution and add local CLAUDE settings Clear timeout identifiers after successful test execution and add local CLAUDE settings

View File

@ -1,6 +1,6 @@
{ {
"name": "@git.zone/tstest", "name": "@git.zone/tstest",
"version": "1.11.1", "version": "2.0.0",
"private": false, "private": false,
"description": "a test utility to run tests that match test/**/*.ts", "description": "a test utility to run tests that match test/**/*.ts",
"exports": { "exports": {

View File

@ -40,9 +40,17 @@ This project integrates tstest with tapbundle through a modular architecture:
- Automatically detects browser environment and only enables in browser context - Automatically detects browser environment and only enables in browser context
3. **Build System** 3. **Build System**
- Uses `tsbuild tsfolders` to compile TypeScript - Uses `tsbuild tsfolders` to compile TypeScript (invoked by `pnpm build`)
- Maintains separate output directories: `/dist_ts/`, `/dist_ts_tapbundle/`, `/dist_ts_tapbundle_node/` - Maintains separate output directories: `/dist_ts/`, `/dist_ts_tapbundle/`, `/dist_ts_tapbundle_node/`, `/dist_ts_tapbundle_protocol/`
- Compilation order is resolved automatically based on dependencies - Compilation order is resolved automatically based on dependencies in tspublish.json files
- Protocol imports use compiled dist directories:
```typescript
// In ts/tstest.classes.tap.parser.ts
import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';
// In ts_tapbundle/tapbundle.classes.tap.ts
import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';
```
### Test Scripts ### Test Scripts
@ -102,6 +110,19 @@ A new internal protocol is being designed that will:
- Use Unicode delimiters `⟦TSTEST:⟧` that won't conflict with test content - Use Unicode delimiters `⟦TSTEST:⟧` that won't conflict with test content
- Support structured JSON metadata - Support structured JSON metadata
- Allow rich error reporting with stack traces and diffs - Allow rich error reporting with stack traces and diffs
- Maintain backwards compatibility during migration - Completely replace v1 protocol (no backwards compatibility)
See `readme.protocol.md` for the full specification and `tapbundle.protocols.ts` for the implementation utilities. ### ts_tapbundle_protocol Directory
The protocol v2 implementation is contained in a separate `ts_tapbundle_protocol` directory:
- **Isomorphic Code**: All protocol code works in both browser and Node.js environments
- **No Platform Dependencies**: No Node.js-specific imports, ensuring true cross-platform compatibility
- **Clean Separation**: Protocol logic is isolated from platform-specific code in tstest and tapbundle
- **Shared Implementation**: Both tstest (parser) and tapbundle (emitter) use the same protocol classes
- **Build Process**:
- Compiled by `pnpm build` via tsbuild to `dist_ts_tapbundle_protocol/`
- Build order managed through tspublish.json files
- Other modules import from the compiled dist directory, not source
This architectural decision ensures the protocol can be used in any JavaScript environment without modification and maintains proper build dependencies.
See `readme.protocol.md` for the full specification and `ts_tapbundle_protocol/` for the implementation.

View File

@ -13,12 +13,27 @@
- Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names - Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
- Structured JSON metadata format - Structured JSON metadata format
- Separate protocol blocks for complex data (errors, snapshots) - Separate protocol blocks for complex data (errors, snapshots)
- Backwards compatible with gradual migration - Complete replacement of v1 (no backwards compatibility needed)
### Implementation ### Implementation
- Phase 1: Add protocol v2 parser alongside v1 - Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
- Phase 2: Generate v2 by default with --legacy flag for v1 - Phase 2: Replace all v1 code in both tstest and tapbundle with v2
- Phase 3: Full migration to v2 in next major version - Phase 3: Delete all v1 parsing and generation code
#### ts_tapbundle_protocol Directory
The protocol v2 implementation will be contained in the `ts_tapbundle_protocol` directory as isomorphic TypeScript code:
- **Isomorphic Design**: All code must work in both browser and Node.js environments
- **No Node.js Imports**: No Node.js-specific modules allowed (no fs, path, child_process, etc.)
- **Protocol Classes**: Contains classes implementing all sides of the protocol:
- `ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
- `ProtocolParser`: For parsing protocol v2 messages (used by tstest)
- `ProtocolMessage`: Base classes for different message types
- `ProtocolTypes`: TypeScript interfaces and types for protocol structures
- **Pure TypeScript**: Only browser-compatible APIs and pure TypeScript/JavaScript code
- **Build Integration**:
- Compiled by `pnpm build` (via tsbuild) to `dist_ts_tapbundle_protocol/`
- Build order defined in tspublish.json files
- Imported by ts and ts_tapbundle modules from the compiled dist directory
See `readme.protocol.md` for detailed specification. See `readme.protocol.md` for detailed specification.
@ -183,10 +198,18 @@ tstest --changed
## Implementation Phases ## Implementation Phases
### Phase 1: Improved Internal Protocol (Priority: Critical) (NEW) ### Phase 1: Improved Internal Protocol (Priority: Critical) (NEW)
1. Implement Protocol V2 parser in tstest 1. Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
2. Add protocol version negotiation - Implement ProtocolEmitter class for message generation
3. Update tapbundle to generate V2 format with feature flag - Implement ProtocolParser class for message parsing
4. Test with real-world test suites containing special characters - Define ProtocolMessage types and interfaces
- Ensure all code is browser and Node.js compatible
- Add tspublish.json to configure build order
2. Update build configuration to compile ts_tapbundle_protocol first
3. Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
4. Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
5. Delete all v1 TAP parsing code from tstest
6. Delete all v1 TAP generation code from tapbundle
7. Test with real-world test suites containing special characters
### Phase 2: Test Configuration System (Priority: High) ### Phase 2: Test Configuration System (Priority: High)
1. Implement tap.settings() API with TypeScript interfaces 1. Implement tap.settings() API with TypeScript interfaces
@ -214,10 +237,10 @@ tstest --changed
## Technical Considerations ## Technical Considerations
### API Design Principles ### API Design Principles
- Maintain backward compatibility - Clean, modern API design without legacy constraints
- Progressive enhancement approach - Progressive enhancement approach
- Opt-in features to avoid breaking changes - Well-documented features and APIs
- Clear migration paths for new features - Clear, simple interfaces
### Performance Goals ### Performance Goals
- Minimal overhead for test execution - Minimal overhead for test execution

View File

@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@git.zone/tstest', name: '@git.zone/tstest',
version: '1.11.1', version: '2.0.0',
description: 'a test utility to run tests that match test/**/*.ts' description: 'a test utility to run tests that match test/**/*.ts'
} }

View File

@ -1,3 +1,3 @@
{ {
"order": 2 "order": 4
} }

View File

@ -36,18 +36,20 @@ export class TapParser {
* Handle test file timeout * Handle test file timeout
*/ */
public handleTimeout(timeoutSeconds: number) { public handleTimeout(timeoutSeconds: number) {
// If no tests have been defined yet, set expected to 1
if (this.expectedTests === 0) {
this.expectedTests = 1;
}
// Create a fake failing test result for timeout // Create a fake failing test result for timeout
this._getNewTapTestResult(); this._getNewTapTestResult();
this.activeTapTestResult.testOk = false; this.activeTapTestResult.testOk = false;
this.activeTapTestResult.testSettled = true; this.activeTapTestResult.testSettled = true;
this.testStore.push(this.activeTapTestResult); this.testStore.push(this.activeTapTestResult);
// Set expected vs received to force failure
this.expectedTests = 1;
this.receivedTests = 0;
// Log the timeout error // Log the timeout error
if (this.logger) { if (this.logger) {
// First log the test result
this.logger.testResult( this.logger.testResult(
`Test file timeout`, `Test file timeout`,
false, false,
@ -55,9 +57,9 @@ export class TapParser {
`Error: Test file exceeded timeout of ${timeoutSeconds} seconds` `Error: Test file exceeded timeout of ${timeoutSeconds} seconds`
); );
this.logger.testErrorDetails(`Test execution was terminated after ${timeoutSeconds} seconds`); this.logger.testErrorDetails(`Test execution was terminated after ${timeoutSeconds} seconds`);
// Force file end with failure
this.logger.testFileEnd(0, 1, timeoutSeconds * 1000);
} }
// Don't call evaluateFinalResult here, let the caller handle it
} }
private _getNewTapTestResult() { private _getNewTapTestResult() {

View File

@ -1,6 +1,5 @@
import * as plugins from './tstest.plugins.js'; import * as plugins from './tstest.plugins.js';
import * as paths from './tstest.paths.js'; import * as paths from './tstest.paths.js';
import * as logPrefixes from './tstest.logprefixes.js';
import { coloredString as cs } from '@push.rocks/consolecolor'; import { coloredString as cs } from '@push.rocks/consolecolor';
@ -19,6 +18,7 @@ export class TsTest {
public startFromFile: number | null; public startFromFile: number | null;
public stopAtFile: number | null; public stopAtFile: number | null;
public timeoutSeconds: number | null; public timeoutSeconds: number | null;
private timeoutWarningTimer: NodeJS.Timeout | null = null;
public smartshellInstance = new plugins.smartshell.Smartshell({ public smartshellInstance = new plugins.smartshell.Smartshell({
executor: 'bash', executor: 'bash',
@ -45,6 +45,15 @@ export class TsTest {
await this.movePreviousLogFiles(); await this.movePreviousLogFiles();
} }
// Start timeout warning timer if no timeout was specified
if (this.timeoutSeconds === null) {
this.timeoutWarningTimer = setTimeout(() => {
this.logger.warning('Test is running for more than 1 minute.');
this.logger.warning('Consider using --timeout option to set a timeout for test files.');
this.logger.warning('Example: tstest test --timeout=300 (for 5 minutes)');
}, 60000); // 1 minute
}
const testGroups = await this.testDir.getTestFileGroups(); const testGroups = await this.testDir.getTestFileGroups();
const allFiles = [...testGroups.serial, ...Object.values(testGroups.parallelGroups).flat()]; const allFiles = [...testGroups.serial, ...Object.values(testGroups.parallelGroups).flat()];
@ -83,6 +92,12 @@ export class TsTest {
} }
} }
// Clear the timeout warning timer if it was set
if (this.timeoutWarningTimer) {
clearTimeout(this.timeoutWarningTimer);
this.timeoutWarningTimer = null;
}
tapCombinator.evaluate(); tapCombinator.evaluate();
} }
@ -156,8 +171,9 @@ export class TsTest {
let timeoutId: NodeJS.Timeout; let timeoutId: NodeJS.Timeout;
const timeoutPromise = new Promise<void>((_resolve, reject) => { const timeoutPromise = new Promise<void>((_resolve, reject) => {
timeoutId = setTimeout(() => { timeoutId = setTimeout(async () => {
execResultStreaming.childProcess.kill('SIGTERM'); // Use smartshell's terminate() to kill entire process tree
await execResultStreaming.terminate();
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`)); reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
}, timeoutMs); }, timeoutMs);
}); });
@ -172,6 +188,13 @@ export class TsTest {
} catch (error) { } catch (error) {
// Handle timeout error // Handle timeout error
tapParser.handleTimeout(this.timeoutSeconds); tapParser.handleTimeout(this.timeoutSeconds);
// Ensure entire process tree is killed if still running
try {
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
} catch (killError) {
// Process tree might already be dead
}
await tapParser.evaluateFinalResult();
} }
} else { } else {
await tapParser.handleTapProcess(execResultStreaming.childProcess); await tapParser.handleTapProcess(execResultStreaming.childProcess);
@ -321,13 +344,29 @@ export class TsTest {
await evaluatePromise; await evaluatePromise;
} }
await this.smartbrowserInstance.stop(); // Always clean up resources, even on timeout
await server.stop(); try {
wss.close(); await this.smartbrowserInstance.stop();
} catch (error) {
// Browser might already be stopped
}
try {
await server.stop();
} catch (error) {
// Server might already be stopped
}
try {
wss.close();
} catch (error) {
// WebSocket server might already be closed
}
console.log( console.log(
`${cs('=> ', 'blue')} Stopped ${cs(fileNameArg, 'orange')} chromium instance and server.` `${cs('=> ', 'blue')} Stopped ${cs(fileNameArg, 'orange')} chromium instance and server.`
); );
// lets create the tap parser // Always evaluate final result (handleTimeout just sets up the test state)
await tapParser.evaluateFinalResult(); await tapParser.evaluateFinalResult();
return tapParser; return tapParser;
} }
@ -351,7 +390,7 @@ export class TsTest {
// Get all .log files in log directory (not in subdirectories) // Get all .log files in log directory (not in subdirectories)
const files = await plugins.smartfile.fs.listFileTree(logDir, '*.log'); const files = await plugins.smartfile.fs.listFileTree(logDir, '*.log');
const logFiles = files.filter(file => !file.includes('/')); const logFiles = files.filter((file: string) => !file.includes('/'));
if (logFiles.length === 0) { if (logFiles.length === 0) {
return; return;

View File

@ -443,6 +443,20 @@ export class TsTestLogger {
this.log(this.format(`\n${status}`, statusColor)); this.log(this.format(`\n${status}`, statusColor));
} }
// Warning display
warning(message: string) {
if (this.options.json) {
this.logJson({ event: 'warning', message });
return;
}
if (this.options.quiet) {
console.log(`WARNING: ${message}`);
} else {
this.log(this.format(` ⚠️ ${message}`, 'orange'));
}
}
// Error display // Error display
error(message: string, file?: string, stack?: string) { error(message: string, file?: string, stack?: string) {
if (this.options.json) { if (this.options.json) {

View File

@ -1,3 +1,3 @@
{ {
"order": 1 "order": 2
} }

View File

@ -0,0 +1,3 @@
{
"order": 3
}

View File

@ -0,0 +1,3 @@
{
"order": 1
}