Compare commits

...

75 Commits

Author SHA1 Message Date
d05ec21b73 2.4.3 2025-10-12 17:49:38 +00:00
956a880a4a fix(docs): Update documentation: expand README with multi-runtime architecture, add module READMEs, and add local dev settings 2025-10-12 17:49:38 +00:00
ee11b1ac17 2.4.2 2025-10-10 16:55:48 +00:00
054cbb6b3c fix(deno): Enable additional Deno permissions for runtime adapters and add local dev settings 2025-10-10 16:55:48 +00:00
ecf11efb4c 2.4.1 2025-10-10 16:45:06 +00:00
1de674e91d fix(runtime/deno): Enable Deno runtime tests by adding required permissions and local settings 2025-10-10 16:45:06 +00:00
9fa2c23ab2 2.4.0 2025-10-10 16:35:22 +00:00
36715c9139 feat(runtime): Add runtime adapters, filename runtime parser and migration tool; integrate runtime selection into TsTest and add tests 2025-10-10 16:35:22 +00:00
ee0aca9ff7 2.3.8 2025-09-12 18:51:28 +00:00
aaebe75326 fix(tstest): Improve free port selection for Chrome runner and bump smartnetwork dependency 2025-09-12 18:51:28 +00:00
265ed702ee 2.3.7 2025-09-12 14:09:28 +00:00
efbaded1f3 fix(tests): Remove flaky dynamic-ports browser test and add local dev tool settings 2025-09-12 14:09:28 +00:00
799a60188f feat(tstest): Implement dynamic port allocation for HTTP and WebSocket connections, add tests for port validation 2025-09-12 14:06:03 +00:00
3c38a53d9d 2.3.6 2025-09-03 12:37:57 +00:00
cca01b51ec fix(tstest): Update deps, fix chrome server route for static bundles, add local tool settings and CI ignore 2025-09-03 12:37:57 +00:00
84843ad359 2.3.5 2025-08-18 02:40:44 +00:00
7a8ae95be2 fix(core): Use SmartRequest with Buffer for binary downloads, tighten static route handling, bump dependencies and add workspace/config files 2025-08-18 02:40:44 +00:00
133e0eda8b 2.3.4 2025-08-16 18:07:57 +00:00
14e32b06de fix(ci): Add local Claude settings to allow required WebFetch and Bash permissions for local tooling and tests 2025-08-16 18:07:57 +00:00
48aebb1eac 2.3.3 2025-08-16 18:01:44 +00:00
733b2249d0 fix(dependencies): Bump dependency versions and add local Claude settings 2025-08-16 18:01:44 +00:00
008844a9e2 fix(tapbundle): Fix TypeScript IDE warning about tapTools parameter possibly being undefined 2025-07-24 22:24:52 +00:00
e4fc6623ea 2.3.1 2025-05-26 16:07:17 +00:00
70435cce45 fix(tapParser/logger): Fix test duration reporting and summary formatting in TAP parser and logger 2025-05-26 16:07:17 +00:00
c26145205f 2.3.0 2025-05-26 14:20:56 +00:00
82fc22653b feat(cli): Add --version option and warn against global tstest usage in the tstest project 2025-05-26 14:20:55 +00:00
3d85f54be0 2.2.6 2025-05-26 14:04:41 +00:00
9464c17c15 fix(tstest): Improve timeout warning timer management and summary output formatting in the test runner. 2025-05-26 14:04:40 +00:00
91b99ce304 2.2.5 2025-05-26 08:22:26 +00:00
899045e6aa fix(protocol): Fix inline timing metadata parsing and enhance test coverage for performance metrics and timing edge cases 2025-05-26 08:22:26 +00:00
845f146e91 2.2.4 2025-05-26 05:01:06 +00:00
d1f8652fc7 fix(logging): Improve performance metrics reporting and add local permissions configuration 2025-05-26 05:01:06 +00:00
f717078558 2.2.3 2025-05-26 04:55:42 +00:00
d2c0e533b5 fix(readme/ts/tstest.plugins): Update npm package scope and documentation to use @git.zone instead of @gitzone, and add local settings configuration. 2025-05-26 04:55:42 +00:00
d3c7fce595 2.2.2 2025-05-26 04:46:25 +00:00
570e2d6b3b fix(config): Cleanup project configuration by adding local CLAUDE settings and removing redundant license files 2025-05-26 04:46:25 +00:00
b7f4b7b3b8 2.2.1 2025-05-26 04:40:10 +00:00
424046b0de fix(repo configuration): Update repository metadata to use git.zone naming and add local permission settings 2025-05-26 04:40:10 +00:00
0f762f2063 2.2.0 2025-05-26 04:37:38 +00:00
82757c4abc feat(watch mode): Add watch mode support with CLI options and enhanced documentation 2025-05-26 04:37:38 +00:00
7aaeed0dc6 fix: Implement tap.todo(), fix tap.skip.test() to create test objects, and ensure tap.only.test() works correctly
- tap.todo.test() now creates proper test objects marked as todo
- tap.skip.test() creates test objects instead of just logging
- tap.only.test() properly filters to run only marked tests
- Added markAsSkipped() method for pre-test skip marking
- All test types now contribute to accurate test counts
- Updated documentation to reflect these fixes

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-26 04:07:05 +00:00
c98bd85829 2.1.0 2025-05-26 04:02:32 +00:00
33d2ff1d4f feat(core): Implement Protocol V2 with enhanced settings and lifecycle hooks 2025-05-26 04:02:32 +00:00
91880f8d42 2.0.0 2025-05-25 19:37:14 +00:00
7b1732abcc BREAKING CHANGE(protocol): Introduce protocol v2 implementation and update build configuration with revised build order, new tspublish files, and enhanced documentation 2025-05-25 19:37:13 +00:00
7d09b39f2b 1.11.5 2025-05-24 14:33:24 +00:00
96efba5903 fix(tstest): Fix timeout handling to correctly evaluate TAP results after killing the test process. 2025-05-24 14:33:24 +00:00
3c535a8a77 1.11.4 2025-05-24 11:10:49 +00:00
0954265095 fix(logging): Improve warning logging and add permission settings file 2025-05-24 11:10:49 +00:00
e1d90589bc 1.11.3 2025-05-24 10:52:15 +00:00
33f705d961 fix(tstest): Add timeout warning for long-running tests and introduce local settings configuration 2025-05-24 10:52:15 +00:00
13b11ab1bf 1.11.2 2025-05-24 02:22:34 +00:00
63280e4a9a fix(tstest): Improve timeout and error handling in test execution along with TAP parser timeout logic improvements. 2025-05-24 02:22:34 +00:00
23addc2d2f 1.11.1 2025-05-24 01:42:24 +00:00
3649114c8d fix(tstest): Clear timeout identifiers after successful test execution and add local CLAUDE settings 2025-05-24 01:42:24 +00:00
2841aba8a4 1.11.0 2025-05-24 01:32:41 +00:00
31bf090410 feat(cli): Add new timeout and file range options with enhanced logfile diff logging 2025-05-24 01:32:41 +00:00
b525754035 1.10.2 2025-05-24 00:59:30 +00:00
aa10fc4ab3 fix(tstest-logging): Improve log file handling with log rotation and diff reporting 2025-05-24 00:59:30 +00:00
3eb8ef22e5 1.10.1 2025-05-23 23:18:35 +00:00
763dc89f59 fix(tstest): Improve file range filtering and summary logging by skipping test files outside the specified range and reporting them in the final summary. 2025-05-23 23:18:35 +00:00
e0d8ede450 1.10.0 2025-05-23 23:05:38 +00:00
27c950c1a1 feat(cli): Add --startFrom and --stopAt options to filter test files by range 2025-05-23 23:05:38 +00:00
83b324b09f 1.9.4 2025-05-23 22:33:34 +00:00
63a2879cb4 fix(docs): Update documentation and configuration for legal notices and CI permissions. This commit adds a new local settings file for tool permissions, refines the legal and trademark sections in the readme, and improves glob test files with clearer log messages. 2025-05-23 22:33:34 +00:00
1a375fa689 1.9.3 2025-05-23 22:27:12 +00:00
c48887a820 fix(tstest): Fix test timing display issue and update TAP protocol documentation 2025-05-23 22:27:12 +00:00
02aeb8195e 1.9.2 2025-05-23 21:31:39 +00:00
53d3dc55e6 fix(logging): Fix log file naming to prevent collisions and update logging system documentation. 2025-05-23 21:31:39 +00:00
a82fdc0f26 1.9.1 2025-05-23 18:53:08 +00:00
cfcb99de76 fix(dependencies): Update dependency versions and add local configuration files 2025-05-23 18:53:08 +00:00
a3a4ded41e 1.9.0 2025-05-16 01:00:06 +00:00
03d478d6ff feat(docs): Update documentation to embed tapbundle and clarify module exports for browser compatibility; also add CI permission settings. 2025-05-16 01:00:06 +00:00
77e53bd68a 1.8.0 2025-05-16 00:54:10 +00:00
946e467c26 feat(documentation): Enhance README with detailed test features and update local settings for build permissions. 2025-05-16 00:54:10 +00:00
58 changed files with 12129 additions and 3173 deletions

1
.serena/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
/cache

68
.serena/project.yml Normal file
View File

@@ -0,0 +1,68 @@
# language of the project (csharp, python, rust, java, typescript, go, cpp, or ruby)
# * For C, use cpp
# * For JavaScript, use typescript
# Special requirements:
# * csharp: Requires the presence of a .sln file in the project folder.
language: typescript
# whether to use the project's gitignore file to ignore files
# Added on 2025-04-07
ignore_all_files_in_gitignore: true
# list of additional paths to ignore
# same syntax as gitignore, so you can use * and **
# Was previously called `ignored_dirs`, please update your config if you are using that.
# Added (renamed) on 2025-04-07
ignored_paths: []
# whether the project is in read-only mode
# If set to true, all editing tools will be disabled and attempts to use them will result in an error
# Added on 2025-04-18
read_only: false
# list of tool names to exclude. We recommend not excluding any tools, see the readme for more details.
# Below is the complete list of tools for convenience.
# To make sure you have the latest list of tools, and to view their descriptions,
# execute `uv run scripts/print_tool_overview.py`.
#
# * `activate_project`: Activates a project by name.
# * `check_onboarding_performed`: Checks whether project onboarding was already performed.
# * `create_text_file`: Creates/overwrites a file in the project directory.
# * `delete_lines`: Deletes a range of lines within a file.
# * `delete_memory`: Deletes a memory from Serena's project-specific memory store.
# * `execute_shell_command`: Executes a shell command.
# * `find_referencing_code_snippets`: Finds code snippets in which the symbol at the given location is referenced.
# * `find_referencing_symbols`: Finds symbols that reference the symbol at the given location (optionally filtered by type).
# * `find_symbol`: Performs a global (or local) search for symbols with/containing a given name/substring (optionally filtered by type).
# * `get_current_config`: Prints the current configuration of the agent, including the active and available projects, tools, contexts, and modes.
# * `get_symbols_overview`: Gets an overview of the top-level symbols defined in a given file.
# * `initial_instructions`: Gets the initial instructions for the current project.
# Should only be used in settings where the system prompt cannot be set,
# e.g. in clients you have no control over, like Claude Desktop.
# * `insert_after_symbol`: Inserts content after the end of the definition of a given symbol.
# * `insert_at_line`: Inserts content at a given line in a file.
# * `insert_before_symbol`: Inserts content before the beginning of the definition of a given symbol.
# * `list_dir`: Lists files and directories in the given directory (optionally with recursion).
# * `list_memories`: Lists memories in Serena's project-specific memory store.
# * `onboarding`: Performs onboarding (identifying the project structure and essential tasks, e.g. for testing or building).
# * `prepare_for_new_conversation`: Provides instructions for preparing for a new conversation (in order to continue with the necessary context).
# * `read_file`: Reads a file within the project directory.
# * `read_memory`: Reads the memory with the given name from Serena's project-specific memory store.
# * `remove_project`: Removes a project from the Serena configuration.
# * `replace_lines`: Replaces a range of lines within a file with new content.
# * `replace_symbol_body`: Replaces the full definition of a symbol.
# * `restart_language_server`: Restarts the language server, may be necessary when edits not through Serena happen.
# * `search_for_pattern`: Performs a search for a pattern in the project.
# * `summarize_changes`: Provides instructions for summarizing the changes made to the codebase.
# * `switch_modes`: Activates modes by providing a list of their names
# * `think_about_collected_information`: Thinking tool for pondering the completeness of collected information.
# * `think_about_task_adherence`: Thinking tool for determining whether the agent is still on track with the current task.
# * `think_about_whether_you_are_done`: Thinking tool for determining whether the task is truly completed.
# * `write_memory`: Writes a named memory (for future reference) to Serena's project-specific memory store.
excluded_tools: []
# initial prompt for the project. It will always be given to the LLM upon activating the project
# (contrary to the memories, which are loaded on demand).
initial_prompt: ""
project_name: "tstest"

View File

@@ -1,5 +1,299 @@
# Changelog # Changelog
## 2025-10-11 - 2.4.3 - fix(docs)
Update documentation: expand README with multi-runtime architecture, add module READMEs, and add local dev settings
- Expanded project README: fixed typos, clarified availability header, and added a detailed Multi-Runtime Architecture section (runtimes, naming conventions, migration tool, examples, and runtime-specific notes).
- Inserted additional example output and adjusted JSON/example sections to reflect multi-runtime flows and updated totals/durations in examples.
- Added dedicated README files for ts_tapbundle, ts_tapbundle_node, and ts_tapbundle_protocol modules with API overviews and usage guides.
- Added .claude/settings.local.json to provide local development permissions/settings used by the project tooling.
- Minor formatting and documentation cleanup (whitespace, headings, and changelog entries).
## 2025-10-10 - 2.4.2 - fix(deno)
Enable additional Deno permissions for runtime adapters and add local dev settings
- Add --allow-sys, --allow-import and --node-modules-dir to the default Deno permission set used by the Deno runtime adapter
- Include the new permission flags in the fallback permissions array when constructing Deno command args
- Add .claude/settings.local.json to capture local development permissions and helper commands
## 2025-10-10 - 2.4.1 - fix(runtime/deno)
Enable Deno runtime tests by adding required permissions and local settings
- ts/tstest.classes.runtime.deno.ts: expanded default Deno permissions to include --allow-net, --allow-write and --sloppy-imports to allow network access, file writes and permissive JS/TS imports
- ts/tstest.classes.runtime.deno.ts: updated fallback permissions used when building the Deno command to match the new default set
- Added .claude/settings.local.json with a set of allowed local commands/permissions used for local development/CI tooling
## 2025-10-10 - 2.4.0 - feat(runtime)
Add runtime adapters, filename runtime parser and migration tool; integrate runtime selection into TsTest and add tests
- Introduce RuntimeAdapter abstraction and RuntimeAdapterRegistry to manage multiple runtimes
- Add runtime adapters: NodeRuntimeAdapter, ChromiumRuntimeAdapter, DenoRuntimeAdapter and BunRuntimeAdapter
- Add filename runtime parser utilities: parseTestFilename, isLegacyFilename and getLegacyMigrationTarget
- Add Migration class to detect and (dry-run) migrate legacy test filenames to the new naming convention
- Integrate runtime registry into TsTest and choose execution adapters based on parsed runtimes; show deprecation warnings for legacy naming
- Add tests covering runtime parsing and migration: test/test.runtime.parser.node.ts and test/test.migration.node.ts
## 2025-09-12 - 2.3.8 - fix(tstest)
Improve free port selection for Chrome runner and bump smartnetwork dependency
- Use randomized port selection when finding free HTTP and WebSocket ports to reduce collision probability in concurrent runs
- Ensure WebSocket port search excludes the chosen HTTP port so the two ports will not conflict
- Simplify failure handling: throw early if a free WebSocket port cannot be found instead of retrying with a less robust fallback
- Bump @push.rocks/smartnetwork dependency from ^4.2.0 to ^4.4.0 to pick up new findFreePort options
## 2025-09-12 - 2.3.7 - fix(tests)
Remove flaky dynamic-ports browser test and add local dev tool settings
- Removed test/tapbundle/test.dynamicports.ts — deletes a browser test that relied on injected dynamic WebSocket ports (reduces flaky CI/browser runs).
- Added .claude/settings.local.json — local development settings for the CLAUDE helper (grants allowed dev/automation commands and webfetch permissions).
## 2025-09-03 - 2.3.6 - fix(tstest)
Update deps, fix chrome server route for static bundles, add local tool settings and CI ignore
- Bump devDependency @git.zone/tsbuild to ^2.6.8
- Bump dependencies: @api.global/typedserver to ^3.0.78, @push.rocks/smartlog to ^3.1.9, @push.rocks/smartrequest to ^4.3.1
- Fix test server static route in ts/tstest.classes.tstest.ts: replace '(.*)' with '/*splat' so bundled test files are served correctly in Chromium runs
- Add .claude/settings.local.json with local permissions for development tasks
- Add .serena/.gitignore to ignore /cache
## 2025-08-18 - 2.3.5 - fix(core)
Use SmartRequest with Buffer for binary downloads, tighten static route handling, bump dependencies and add workspace/config files
- ts_tapbundle_node/classes.testfileprovider.ts: switch to SmartRequest.create().url(...).get() and convert response to a Buffer before writing to disk to fix binary download handling for the Docker Alpine image.
- ts/tstest.classes.tstest.ts: change server.addRoute from '*' to '(.*)' so the typedserver static handler uses a proper regex route.
- package.json: bump several dependencies (e.g. @api.global/typedserver, @git.zone/tsbuild, @push.rocks/smartfile, @push.rocks/smartpath, @push.rocks/smartrequest, @push.rocks/smartshell) to newer patch/minor versions.
- pnpm-workspace.yaml: add onlyBuiltDependencies list (esbuild, mongodb-memory-server, puppeteer).
- Remove registry setting from .npmrc (cleanup).
- Add project/agent config files: .serena/project.yml and .claude/settings.local.json for local tooling/agent configuration.
## 2025-08-16 - 2.3.4 - fix(ci)
Add local Claude settings to allow required WebFetch and Bash permissions for local tooling and tests
- Add .claude/settings.local.json to configure allowed permissions for local assistant/automation
- Grants WebFetch access for code.foss.global and www.npmjs.com
- Allows various Bash commands used by local tasks and test runs (mkdir, tsbuild, pnpm, node, tsx, tstest, ls, rm, grep, cat)
- No runtime/library code changes — configuration only
## 2025-08-16 - 2.3.3 - fix(dependencies)
Bump dependency versions and add local Claude settings
- Bumped devDependency @git.zone/tsbuild ^2.6.3 → ^2.6.4
- Updated @git.zone/tsbundle ^2.2.5 → ^2.5.1
- Updated @push.rocks/consolecolor ^2.0.2 → ^2.0.3
- Updated @push.rocks/qenv ^6.1.0 → ^6.1.3
- Updated @push.rocks/smartchok ^1.0.34 → ^1.1.1
- Updated @push.rocks/smartenv ^5.0.12 → ^5.0.13
- Updated @push.rocks/smartfile ^11.2.3 → ^11.2.5
- Updated @push.rocks/smarts3 ^2.2.5 → ^2.2.6
- Updated @push.rocks/smartshell ^3.2.3 → ^3.2.4
- Updated ws ^8.18.2 → ^8.18.3
- Added .claude/settings.local.json for local Claude permissions and tooling (local-only configuration)
## 2025-07-24 - 2.3.2 - fix(tapbundle)
Fix TypeScript IDE warning about tapTools parameter possibly being undefined
- Changed ITestFunction from interface with optional parameter to union type
- Updated test runner to handle both function signatures (with and without tapTools)
- Resolves IDE warnings while maintaining backward compatibility
## 2025-05-26 - 2.3.1 - fix(tapParser/logger)
Fix test duration reporting and summary formatting in TAP parser and logger
- Introduce startTime in TapParser to capture the overall test duration
- Pass computed duration to logger methods in evaluateFinalResult for accurate timing
- Update summary output to format duration in a human-readable way (ms vs. s)
- Add local permission settings configuration to .claude/settings.local.json
## 2025-05-26 - 2.3.0 - feat(cli)
Add '--version' option and warn against global tstest usage in the tstest project
- Introduced a new '--version' CLI flag that prints the version from package.json
- Added logic in ts/index.ts to detect if tstest is run globally within its own project and issue a warning
- Added .claude/settings.local.json to configure allowed permissions for various commands
## 2025-05-26 - 2.2.6 - fix(tstest)
Improve timeout warning timer management and summary output formatting in the test runner.
- Removed the global timeoutWarningTimer and replaced it with local warning timers in runInNode and runInChrome methods.
- Added warnings when test files run for over one minute if no timeout is specified.
- Ensured proper clearing of warning timers on successful completion or timeout.
- Enhanced quiet mode summary output to clearly display passed and failed test counts.
## 2025-05-26 - 2.2.5 - fix(protocol)
Fix inline timing metadata parsing and enhance test coverage for performance metrics and timing edge cases
- Updated the protocol parser to correctly parse inline key:value pairs while excluding prefixed formats (META:, SKIP:, TODO:, EVENT:)
- Added new tests for performance metrics, timing edge cases, and protocol timing to verify accurate timing capture and retry handling
- Expanded documentation in readme.hints.md to detail the updated timing implementation and parser fixes
## 2025-05-26 - 2.2.4 - fix(logging)
Improve performance metrics reporting and add local permissions configuration
- Add .claude/settings.local.json to configure allowed permissions for various commands
- Update tstest logging: compute average test duration from actual durations and adjust slowest test display formatting
## 2025-05-26 - 2.2.3 - fix(readme/ts/tstest.plugins)
Update npm package scope and documentation to use '@git.zone' instead of '@gitzone', and add local settings configuration.
- Changed npm package links and source repository URLs in readme from '@gitzone/tstest' to '@git.zone/tstest'.
- Updated comments in ts/tstest.plugins.ts to reflect the correct '@git.zone' scope.
- Added .claude/settings.local.json file with local permission settings.
## 2025-05-26 - 2.2.2 - fix(config)
Cleanup project configuration by adding local CLAUDE settings and removing redundant license files
- Added .claude/settings.local.json with updated permissions for CLI and build tasks
- Removed license and license.md files to streamline repository content
## 2025-05-26 - 2.2.1 - fix(repo configuration)
Update repository metadata to use 'git.zone' naming and add local permission settings
- Changed githost from 'gitlab.com' to 'code.foss.global' and gitscope from 'gitzone' to 'git.zone' in npmextra.json
- Updated npm package name from '@gitzone/tstest' to '@git.zone/tstest' in npmextra.json and readme.md
- Added .claude/settings.local.json with new permission configuration
## 2025-05-26 - 2.2.0 - feat(watch mode)
Add watch mode support with CLI options and enhanced documentation
- Introduce '--watch' (or '-w') and '--watch-ignore' CLI flags for automatic test re-runs
- Integrate @push.rocks/smartchok for file watching with 300ms debouncing
- Update readme.md and readme.hints.md with detailed instructions and examples for watch mode
- Add a demo test file (test/watch-demo/test.demo.ts) to illustrate the new feature
- Add smartchok dependency in package.json
## 2025-05-26 - 2.1.0 - feat(core)
Implement Protocol V2 with enhanced settings and lifecycle hooks
- Migrated to Protocol V2 using Unicode markers and structured metadata with new ts_tapbundle_protocol module
- Refactored TAP parser/emitter to support improved protocol parsing and error reporting
- Integrated global settings via tap.settings() and lifecycle hooks (beforeAll/afterAll, beforeEach/afterEach)
- Enhanced expect wrapper with diff generation for clearer assertion failures
- Updated test loader to automatically run 00init.ts for proper test configuration
- Revised documentation (readme.hints.md, readme.plan.md) to reflect current implementation status and remaining work
## 2025-05-25 - 2.0.0 - BREAKING CHANGE(protocol)
Introduce protocol v2 implementation and update build configuration with revised build order, new tspublish files, and enhanced documentation
- Added ts_tapbundle_protocol directory with isomorphic implementation for protocol v2
- Updated readme.hints.md and readme.plan.md to explain the complete replacement of the v1 protocol and new build process
- Revised build order in tspublish.json files across ts, ts_tapbundle, ts_tapbundle_node, and ts_tapbundle_protocol
- Introduced .claude/settings.local.json with updated permission settings for CLI and build tools
## 2025-05-24 - 1.11.5 - fix(tstest)
Fix timeout handling to correctly evaluate TAP results after killing the test process.
- Added call to evaluateFinalResult() after killing the process in runInNode to ensure final TAP output is processed.
## 2025-05-24 - 1.11.4 - fix(logging)
Improve warning logging and add permission settings file
- Replace multiple logger.error calls with logger.warning for tests running over 1 minute
- Add warning method in tstest logger to display warning messages consistently
- Introduce .claude/settings.local.json to configure allowed permissions
## 2025-05-24 - 1.11.3 - fix(tstest)
Add timeout warning for long-running tests and introduce local settings configuration
- Add .claude/settings.local.json with permission configuration for local development
- Implement a timeout warning timer that notifies when tests run longer than 1 minute without an explicit timeout
- Clear the timeout warning timer upon test completion
- Remove unused import of logPrefixes in tstest.classes.tstest.ts
## 2025-05-24 - 1.11.2 - fix(tstest)
Improve timeout and error handling in test execution along with TAP parser timeout logic improvements.
- In the TAP parser, ensure that expected tests are properly set when no tests are defined to avoid false negatives on timeout.
- Use smartshell's terminate method and fallback kill to properly stop the entire process tree on timeout.
- Clean up browser, server, and WebSocket instances reliably even when a timeout occurs.
- Minor improvements in log file filtering and error logging for better clarity.
## 2025-05-24 - 1.11.1 - fix(tstest)
Clear timeout identifiers after successful test execution and add local CLAUDE settings
- Ensure timeout IDs are cleared when tests complete to prevent lingering timeouts
- Add .claude/settings.local.json with updated permission settings for CLI commands
## 2025-05-24 - 1.11.0 - feat(cli)
Add new timeout and file range options with enhanced logfile diff logging
- Introduce --timeout <seconds> option to safeguard tests from running too long
- Add --startFrom and --stopAt options to control the range of test files executed
- Enhance logfile organization by automatically moving previous logs and generating diff reports for failed or changed test outputs
- Update CLI argument parsing and internal timeout handling for both Node.js and browser tests
## 2025-05-24 - 1.10.2 - fix(tstest-logging)
Improve log file handling with log rotation and diff reporting
- Add .claude/settings.local.json to configure allowed shell and web operations
- Introduce movePreviousLogFiles function to archive previous log files when --logfile is used
- Enhance logging to generate error copies and diff reports between current and previous logs
- Add type annotations for console overrides in browser evaluations for improved stability
## 2025-05-23 - 1.10.1 - fix(tstest)
Improve file range filtering and summary logging by skipping test files outside the specified range and reporting them in the final summary.
- Introduce runSingleTestOrSkip to check file index against startFrom/stopAt values.
- Log skipped files with appropriate messages and add them to the summary.
- Update the logger to include total skipped files in the test summary.
- Add permission settings in .claude/settings.local.json to support new operations.
## 2025-05-23 - 1.10.0 - feat(cli)
Add --startFrom and --stopAt options to filter test files by range
- Introduced CLI options --startFrom and --stopAt in ts/index.ts for selective test execution
- Added validation to ensure provided range values are positive and startFrom is not greater than stopAt
- Propagated file range filtering into test grouping in tstest.classes.tstest.ts, applying the range filter across serial and parallel groups
- Updated usage messages to include the new options
## 2025-05-23 - 1.9.4 - fix(docs)
Update documentation and configuration for legal notices and CI permissions. This commit adds a new local settings file for tool permissions, refines the legal and trademark sections in the readme, and improves glob test files with clearer log messages.
- Added .claude/settings.local.json to configure permissions for various CLI commands
- Revised legal and trademark documentation in the readme to clarify company ownership and usage guidelines
- Updated glob test files with improved console log messages for better clarity during test discovery
## 2025-05-23 - 1.9.3 - fix(tstest)
Fix test timing display issue and update TAP protocol documentation
- Changed TAP parser regex to non-greedy pattern to correctly separate test timing metadata
- Enhanced readme.hints.md with detailed explanation of test timing fix and planned protocol upgrades
- Updated readme.md with improved usage examples for tapbundle and comprehensive test framework documentation
- Added new protocol design document (readme.protocol.md) and improvement plan (readme.plan.md) outlining future changes
- Introduced .claude/settings.local.json update for npm and CLI permissions
- Exported protocol utilities and added tapbundle protocol implementation for future enhancements
## 2025-05-23 - 1.9.2 - fix(logging)
Fix log file naming to prevent collisions and update logging system documentation.
- Enhance safe filename generation in tstest logging to preserve directory structure using double underscores.
- Update readme.hints.md to include detailed logging system documentation and behavior.
- Add .claude/settings.local.json with updated permissions for build tools.
## 2025-05-23 - 1.9.1 - fix(dependencies)
Update dependency versions and add local configuration files
- Bump @git.zone/tsbuild from ^2.5.1 to ^2.6.3
- Bump @types/node from ^22.15.18 to ^22.15.21
- Bump @push.rocks/smartexpect from ^2.4.2 to ^2.5.0
- Bump @push.rocks/smartfile from ^11.2.0 to ^11.2.3
- Bump @push.rocks/smartlog from ^3.1.1 to ^3.1.8
- Add .npmrc with npm registry configuration
- Add .claude/settings.local.json for local permissions
## 2025-05-16 - 1.9.0 - feat(docs)
Update documentation to embed tapbundle and clarify module exports for browser compatibility; also add CI permission settings.
- Embed tapbundle directly into tstest to simplify usage and ensure browser support.
- Update import paths in examples from '@push.rocks/tapbundle' to '@git.zone/tstest/tapbundle'.
- Revise the changelog to reflect version 1.8.0 improvements including enhanced test lifecycle hooks and parallel execution fixes.
- Add .claude/settings.local.json to configure CI-related permissions and tool operations.
## 2025-05-16 - 1.8.0 - feat(documentation)
Enhance README with detailed test features and update local settings for build permissions.
- Expanded the documentation to include tag filtering, parallel test execution groups, lifecycle hooks, snapshot testing, timeout control, retry logic, and test fixtures
- Updated .claude/settings.local.json to allow additional permissions for various build and test commands
## 2025-05-16 - 1.7.0 - feat(tstest) ## 2025-05-16 - 1.7.0 - feat(tstest)
Enhance tstest with fluent API, suite grouping, tag filtering, fixture & snapshot testing, and parallel execution improvements Enhance tstest with fluent API, suite grouping, tag filtering, fixture & snapshot testing, and parallel execution improvements

19
license.md Normal file
View File

@@ -0,0 +1,19 @@
Copyright (c) 2014 Task Venture Capital GmbH (hello@task.vc)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -6,11 +6,11 @@
"gitzone": { "gitzone": {
"projectType": "npm", "projectType": "npm",
"module": { "module": {
"githost": "gitlab.com", "githost": "code.foss.global",
"gitscope": "gitzone", "gitscope": "git.zone",
"gitrepo": "tstest", "gitrepo": "tstest",
"description": "a test utility to run tests that match test/**/*.ts", "description": "a test utility to run tests that match test/**/*.ts",
"npmPackagename": "@gitzone/tstest", "npmPackagename": "@git.zone/tstest",
"license": "MIT" "license": "MIT"
} }
} }

View File

@@ -1,6 +1,6 @@
{ {
"name": "@git.zone/tstest", "name": "@git.zone/tstest",
"version": "1.7.0", "version": "2.4.3",
"private": false, "private": false,
"description": "a test utility to run tests that match test/**/*.ts", "description": "a test utility to run tests that match test/**/*.ts",
"exports": { "exports": {
@@ -24,33 +24,35 @@
"buildDocs": "tsdoc" "buildDocs": "tsdoc"
}, },
"devDependencies": { "devDependencies": {
"@git.zone/tsbuild": "^2.5.1", "@git.zone/tsbuild": "^2.6.8",
"@types/node": "^22.15.18" "@types/node": "^22.15.21"
}, },
"dependencies": { "dependencies": {
"@api.global/typedserver": "^3.0.74", "@api.global/typedserver": "^3.0.78",
"@git.zone/tsbundle": "^2.2.5", "@git.zone/tsbundle": "^2.5.1",
"@git.zone/tsrun": "^1.3.3", "@git.zone/tsrun": "^1.3.3",
"@push.rocks/consolecolor": "^2.0.2", "@push.rocks/consolecolor": "^2.0.3",
"@push.rocks/qenv": "^6.1.0", "@push.rocks/qenv": "^6.1.3",
"@push.rocks/smartbrowser": "^2.0.8", "@push.rocks/smartbrowser": "^2.0.8",
"@push.rocks/smartchok": "^1.1.1",
"@push.rocks/smartcrypto": "^2.0.4", "@push.rocks/smartcrypto": "^2.0.4",
"@push.rocks/smartdelay": "^3.0.5", "@push.rocks/smartdelay": "^3.0.5",
"@push.rocks/smartenv": "^5.0.12", "@push.rocks/smartenv": "^5.0.13",
"@push.rocks/smartexpect": "^2.4.2", "@push.rocks/smartexpect": "^2.5.0",
"@push.rocks/smartfile": "^11.2.0", "@push.rocks/smartfile": "^11.2.7",
"@push.rocks/smartjson": "^5.0.20", "@push.rocks/smartjson": "^5.0.20",
"@push.rocks/smartlog": "^3.1.1", "@push.rocks/smartlog": "^3.1.9",
"@push.rocks/smartmongo": "^2.0.12", "@push.rocks/smartmongo": "^2.0.12",
"@push.rocks/smartpath": "^5.0.18", "@push.rocks/smartnetwork": "^4.4.0",
"@push.rocks/smartpath": "^6.0.0",
"@push.rocks/smartpromise": "^4.2.3", "@push.rocks/smartpromise": "^4.2.3",
"@push.rocks/smartrequest": "^2.1.0", "@push.rocks/smartrequest": "^4.3.1",
"@push.rocks/smarts3": "^2.2.5", "@push.rocks/smarts3": "^2.2.6",
"@push.rocks/smartshell": "^3.2.3", "@push.rocks/smartshell": "^3.3.0",
"@push.rocks/smarttime": "^4.1.1", "@push.rocks/smarttime": "^4.1.1",
"@types/ws": "^8.18.1", "@types/ws": "^8.18.1",
"figures": "^6.1.0", "figures": "^6.1.0",
"ws": "^8.18.2" "ws": "^8.18.3"
}, },
"files": [ "files": [
"ts/**/*", "ts/**/*",

5965
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

4
pnpm-workspace.yaml Normal file
View File

@@ -0,0 +1,4 @@
onlyBuiltDependencies:
- esbuild
- mongodb-memory-server
- puppeteer

View File

@@ -12,7 +12,7 @@ This project integrates tstest with tapbundle through a modular architecture:
### Test Execution Flow ### Test Execution Flow
1. **CLI Entry Point** (`cli.js` <20> `cli.ts.js` <20> `cli.child.ts`) 1. **CLI Entry Point** (`cli.js` <20> `cli.ts.js` <20> `cli.child.ts`)
- The CLI uses tsx to run TypeScript files directly - The CLI uses tsx to run TypeScript files directly
- Accepts glob patterns to find test files - Accepts glob patterns to find test files
- Supports options like `--verbose`, `--quiet`, `--web` - Supports options like `--verbose`, `--quiet`, `--web`
@@ -40,9 +40,17 @@ This project integrates tstest with tapbundle through a modular architecture:
- Automatically detects browser environment and only enables in browser context - Automatically detects browser environment and only enables in browser context
3. **Build System** 3. **Build System**
- Uses `tsbuild tsfolders` to compile TypeScript - Uses `tsbuild tsfolders` to compile TypeScript (invoked by `pnpm build`)
- Maintains separate output directories: `/dist_ts/`, `/dist_ts_tapbundle/`, `/dist_ts_tapbundle_node/` - Maintains separate output directories: `/dist_ts/`, `/dist_ts_tapbundle/`, `/dist_ts_tapbundle_node/`, `/dist_ts_tapbundle_protocol/`
- Compilation order is resolved automatically based on dependencies - Compilation order is resolved automatically based on dependencies in tspublish.json files
- Protocol imports use compiled dist directories:
```typescript
// In ts/tstest.classes.tap.parser.ts
import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';
// In ts_tapbundle/tapbundle.classes.tap.ts
import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';
```
### Test Scripts ### Test Scripts
@@ -60,3 +68,256 @@ The framework automatically detects the runtime environment:
- WebHelpers are only enabled in browser environment - WebHelpers are only enabled in browser environment
This architecture allows for seamless testing across both Node.js and browser environments while maintaining a clean separation of concerns. This architecture allows for seamless testing across both Node.js and browser environments while maintaining a clean separation of concerns.
## Logging System
### Log File Naming (Fixed in v1.9.1)
When using the `--logfile` flag, tstest creates log files in `.nogit/testlogs/`. The log file naming was updated to preserve directory structure and prevent collisions:
- **Old behavior**: `test/tapbundle/test.ts` → `.nogit/testlogs/test.log`
- **New behavior**: `test/tapbundle/test.ts` → `.nogit/testlogs/test__tapbundle__test.log`
This fix ensures that test files with the same basename in different directories don't overwrite each other's logs. The implementation:
1. Takes the relative path from the current working directory
2. Replaces path separators (`/`) with double underscores (`__`)
3. Removes the `.ts` extension
4. Creates a flat filename that preserves the directory structure
### Test Timing Display (Fixed in v1.9.2)
Fixed an issue where test timing was displayed incorrectly with duplicate values like:
- Before: `✅ test name # time=133ms (0ms)`
- After: `✅ test name (133ms)`
The issue was in the TAP parser regex which was greedily capturing the entire line including the TAP timing comment. Changed the regex from `(.*)` to `(.*?)` to make it non-greedy, properly separating the test name from the timing metadata.
## Protocol Limitations and Improvements
### Current TAP Protocol Issues
The current implementation uses standard TAP format with metadata in comments:
```
ok 1 - test name # time=123ms
```
This has several limitations:
1. **Delimiter Conflict**: Test descriptions containing `#` can break parsing
2. **Regex Fragility**: Complex regex patterns that are hard to maintain
3. **Limited Metadata**: Difficult to add rich error information or custom data
### Planned Protocol V2
A new internal protocol is being designed that will:
- Use Unicode delimiters `⟦TSTEST:⟧` that won't conflict with test content
- Support structured JSON metadata
- Allow rich error reporting with stack traces and diffs
- Completely replace v1 protocol (no backwards compatibility)
### ts_tapbundle_protocol Directory
The protocol v2 implementation is contained in a separate `ts_tapbundle_protocol` directory:
- **Isomorphic Code**: All protocol code works in both browser and Node.js environments
- **No Platform Dependencies**: No Node.js-specific imports, ensuring true cross-platform compatibility
- **Clean Separation**: Protocol logic is isolated from platform-specific code in tstest and tapbundle
- **Shared Implementation**: Both tstest (parser) and tapbundle (emitter) use the same protocol classes
- **Build Process**:
- Compiled by `pnpm build` via tsbuild to `dist_ts_tapbundle_protocol/`
- Build order managed through tspublish.json files
- Other modules import from the compiled dist directory, not source
This architectural decision ensures the protocol can be used in any JavaScript environment without modification and maintains proper build dependencies.
See `readme.protocol.md` for the full specification and `ts_tapbundle_protocol/` for the implementation.
## Protocol V2 Implementation Status
The Protocol V2 has been implemented to fix issues with TAP protocol parsing when test descriptions contain special characters like `#`, `###SNAPSHOT###`, or protocol markers like `⟦TSTEST:ERROR⟧`.
### Implementation Details:
1. **Protocol Components**:
- `ProtocolEmitter` - Generates protocol v2 messages (used by tapbundle)
- `ProtocolParser` - Parses protocol v2 messages (used by tstest)
- Uses Unicode markers `⟦TSTEST:` and `` to avoid conflicts with test content
2. **Current Status**:
- ✅ Basic protocol emission and parsing works
- ✅ Handles test descriptions with special characters correctly
- ✅ Supports metadata for timing, tags, errors
- ⚠️ Protocol messages sometimes appear in console output (parsing not catching all cases)
3. **Key Findings**:
- `tap.skip.test()` doesn't create actual test objects, just logs and increments counter
- `tap.todo()` method is not implemented (no `addTodo` method in Tap class)
- Protocol parser's `isBlockStart` was fixed to only match exact block markers, not partial matches in test descriptions
4. **Import Paths**:
- tstest imports from: `import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';`
- tapbundle imports from: `import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';`
## Test Configuration System (Phase 2)
The Test Configuration System has been implemented to provide global settings and lifecycle hooks for tests.
### Key Features:
1. **00init.ts Discovery**:
- Automatically detects `00init.ts` files in the same directory as test files
- Creates a temporary loader file that imports both `00init.ts` and the test file
- Loader files are cleaned up automatically after test execution
2. **Settings Inheritance**:
- Global settings from `00init.ts` → File-level settings → Test-level settings
- Settings include: timeout, retries, retryDelay, bail, concurrency
- Lifecycle hooks: beforeAll, afterAll, beforeEach, afterEach
3. **Implementation Details**:
- `SettingsManager` class handles settings inheritance and merging
- `tap.settings()` API allows configuration at any level
- Lifecycle hooks are integrated into test execution flow
### Important Development Notes:
1. **Local Development**: When developing tstest itself, use `node cli.js` instead of globally installed `tstest` to test changes
2. **Console Output Buffering**: Console output from tests is buffered and only displayed for failing tests. TAP-compliant comments (lines starting with `#`) are always shown.
3. **TypeScript Warnings**: Fixed async/await warnings in `movePreviousLogFiles()` by using sync versions of file operations
## Enhanced Communication Features (Phase 3)
The Enhanced Communication system has been implemented to provide rich, real-time feedback during test execution.
### Key Features:
1. **Event-Based Test Lifecycle Reporting**:
- `test:queued` - Test is ready to run
- `test:started` - Test execution begins
- `test:completed` - Test finishes (with pass/fail status)
- `suite:started` - Test suite/describe block begins
- `suite:completed` - Test suite/describe block ends
- `hook:started` - Lifecycle hook (beforeEach/afterEach) begins
- `hook:completed` - Lifecycle hook finishes
- `assertion:failed` - Assertion failure with detailed information
2. **Visual Diff Output for Assertion Failures**:
- **String Diffs**: Character-by-character comparison with colored output
- **Object/Array Diffs**: Deep property comparison showing added/removed/changed properties
- **Primitive Diffs**: Clear display of expected vs actual values
- **Colorized Output**: Green for expected, red for actual, yellow for differences
- **Smart Formatting**: Multi-line strings and complex objects are formatted for readability
3. **Real-Time Test Progress API**:
- Tests emit progress events as they execute
- tstest parser processes events and updates display in real-time
- Structured event format carries rich metadata (timing, errors, diffs)
- Seamless integration with existing TAP protocol via Protocol V2
### Implementation Details:
- Events are transmitted via Protocol V2's `EVENT` block type
- Event data is JSON-encoded within protocol markers
- Parser handles events asynchronously for real-time updates
- Visual diffs are generated using custom diff algorithms for each data type
## Watch Mode (Phase 4)
tstest now supports watch mode for automatic test re-runs on file changes.
### Usage
```bash
tstest test/**/*.ts --watch
tstest test/specific.ts -w
```
### Features
- **Automatic Re-runs**: Tests re-run when any watched file changes
- **Debouncing**: Multiple rapid changes are batched (300ms delay)
- **Clear Output**: Console is cleared before each run for clean results
- **Status Updates**: Shows which files triggered the re-run
- **Graceful Exit**: Press Ctrl+C to stop watching
### Options
- `--watch` or `-w`: Enable watch mode
- `--watch-ignore`: Comma-separated patterns to ignore (e.g., `--watch-ignore node_modules,dist`)
### Implementation Details
- Uses `@push.rocks/smartchok` for cross-platform file watching
- Watches the entire project directory from where tests are run
- Ignores changes matching the ignore patterns
- Shows "Waiting for file changes..." between runs
## Fixed Issues
### tap.skip.test(), tap.todo(), and tap.only.test() (Fixed)
Previously reported issues with these methods have been resolved:
1. **tap.skip.test()** - Now properly creates test objects that are counted in test results
- Tests marked with `skip.test()` appear in the test count
- Shows as passed with skip directive in TAP output
- `markAsSkipped()` method added to handle pre-test skip marking
2. **tap.todo.test()** - Fully implemented with test object creation
- Supports both `tap.todo.test('description')` and `tap.todo.test('description', testFunc)`
- Todo tests are counted and marked with todo directive
- Both regular and parallel todo tests supported
3. **tap.only.test()** - Works correctly for focused testing
- When `.only` tests exist, only those tests run
- Other tests are not executed but still counted
- Both regular and parallel only tests supported
These fixes ensure accurate test counts and proper TAP-compliant output for all test states.
## Test Timing Implementation
### Timing Architecture
Test timing is captured using `@push.rocks/smarttime`'s `HrtMeasurement` class, which provides high-resolution timing:
1. **Timing Capture**:
- Each `TapTest` instance has its own `HrtMeasurement`
- Timer starts immediately before test function execution
- Timer stops after test completes (or fails/times out)
- Millisecond precision is used for reporting
2. **Protocol Integration**:
- Timing is embedded in TAP output using Protocol V2 markers
- Inline format for simple timing: `ok 1 - test name ⟦TSTEST:time:123⟧`
- Block format for complex metadata: `⟦TSTEST:META:{"time":456,"file":"test.ts"}⟧`
3. **Performance Metrics Calculation**:
- Average is calculated from sum of individual test times, not total runtime
- Slowest test detection prefers tests with >0ms duration
- Failed tests still contribute their execution time to metrics
### Edge Cases and Considerations
1. **Sub-millisecond Tests**:
- Very fast tests may report 0ms due to millisecond rounding
- Performance metrics handle this by showing "All tests completed in <1ms" when appropriate
2. **Special Test States**:
- **Skipped tests**: Report 0ms (not executed)
- **Todo tests**: Report 0ms (not executed)
- **Failed tests**: Report actual execution time before failure
- **Timeout tests**: Report time until timeout occurred
3. **Parallel Test Timing**:
- Each parallel test tracks its own execution time independently
- Parallel tests may have overlapping execution periods
- Total suite time reflects wall-clock time, not sum of test times
4. **Hook Timing**:
- `beforeEach`/`afterEach` hooks are not included in individual test times
- Only the actual test function execution is measured
5. **Retry Timing**:
- When tests retry, only the final attempt's duration is reported
- Each retry attempt emits separate `test:started` events
### Parser Fix for Timing Metadata
The protocol parser was fixed to correctly handle inline timing metadata:
- Changed condition from `!simpleMatch[1].includes(':')` to check for simple key:value pairs
- Excludes prefixed formats (META:, SKIP:, TODO:, EVENT:) while parsing simple formats like `time:250`
This ensures timing metadata is correctly extracted and displayed in test results.

887
readme.md
View File

@@ -1,9 +1,9 @@
# @gitzone/tstest # @git.zone/tstest
🧪 **A powerful, modern test runner for TypeScript** - making your test runs beautiful and informative! 🧪 **A powerful, modern test runner for TypeScript** - making your test runs beautiful and informative across multiple runtimes!
## Availabililty and Links ## Availability and Links
* [npmjs.org (npm package)](https://www.npmjs.com/package/@gitzone/tstest) * [npmjs.org (npm package)](https://www.npmjs.com/package/@git.zone/tstest)
* [code.foss.global (source)](https://code.foss.global/gitzone/tstest) * [code.foss.global (source)](https://code.foss.global/git.zone/tstest)
## Why tstest? ## Why tstest?
@@ -12,22 +12,167 @@
### ✨ Key Features ### ✨ Key Features
- 🎯 **Smart Test Execution** - Run all tests, single files, or use glob patterns - 🎯 **Smart Test Execution** - Run all tests, single files, or use glob patterns
- 🚀 **Multi-Runtime Support** - Run tests in Node.js, Deno, Bun, and Chromium
- 🎨 **Beautiful Output** - Color-coded results with emojis and clean formatting - 🎨 **Beautiful Output** - Color-coded results with emojis and clean formatting
- 📊 **Multiple Output Modes** - Choose from normal, quiet, verbose, or JSON output - 📊 **Multiple Output Modes** - Choose from normal, quiet, verbose, or JSON output
- 🔍 **Automatic Discovery** - Finds all your test files automatically - 🔍 **Automatic Discovery** - Finds all your test files automatically
- 🌐 **Cross-Environment** - Supports Node.js and browser testing
- 📝 **Detailed Logging** - Optional file logging for debugging - 📝 **Detailed Logging** - Optional file logging for debugging
-**Performance Metrics** - See which tests are slow -**Performance Metrics** - See which tests are slow
- 🤖 **CI/CD Ready** - JSON output mode for automation - 🤖 **CI/CD Ready** - JSON output mode for automation
- 🏷️ **Tag-based Filtering** - Run only tests with specific tags
- 🎯 **Parallel Test Execution** - Run tests in parallel groups
- 🔧 **Test Lifecycle Hooks** - beforeEach/afterEach support
- 📸 **Snapshot Testing** - Compare test outputs with saved snapshots
-**Timeout Control** - Set custom timeouts for tests
- 🔁 **Retry Logic** - Automatically retry failing tests
- 🛠️ **Test Fixtures** - Create reusable test data
- 👀 **Watch Mode** - Automatically re-run tests on file changes
- 📊 **Real-time Progress** - Live test execution progress updates
- 🎨 **Visual Diffs** - Beautiful side-by-side diffs for failed assertions
- 🔄 **Event-based Reporting** - Real-time test lifecycle events
## Installation ## Installation
```bash ```bash
npm install --save-dev @gitzone/tstest npm install --save-dev @git.zone/tstest
# or with pnpm # or with pnpm
pnpm add -D @gitzone/tstest pnpm add -D @git.zone/tstest
``` ```
## Multi-Runtime Architecture
tstest supports running your tests across multiple JavaScript runtimes, allowing you to verify cross-platform compatibility easily.
### Supported Runtimes
- **Node.js** - Default runtime, uses tsrun for TypeScript execution
- **Chromium** - Browser environment testing with full DOM support
- **Deno** - Secure TypeScript/JavaScript runtime with modern features
- **Bun** - Ultra-fast all-in-one JavaScript runtime
### Test File Naming Convention
Name your test files with runtime specifiers to control where they run:
| Pattern | Runtimes | Example |
|---------|----------|---------|
| `*.ts` | Node.js only (default) | `test.api.ts` |
| `*.node.ts` | Node.js only | `test.server.node.ts` |
| `*.chromium.ts` | Chromium browser | `test.dom.chromium.ts` |
| `*.deno.ts` | Deno runtime | `test.http.deno.ts` |
| `*.bun.ts` | Bun runtime | `test.fast.bun.ts` |
| `*.node+chromium.ts` | Both Node.js and Chromium | `test.isomorphic.node+chromium.ts` |
| `*.node+deno.ts` | Both Node.js and Deno | `test.cross.node+deno.ts` |
| `*.deno+bun.ts` | Both Deno and Bun | `test.modern.deno+bun.ts` |
| `*.chromium.nonci.ts` | Chromium, skip in CI | `test.visual.chromium.nonci.ts` |
**Multi-Runtime Examples:**
```typescript
// test.api.node+deno+bun.ts - runs in Node.js, Deno, and Bun
import { expect, tap } from '@git.zone/tstest/tapbundle';
tap.test('cross-runtime HTTP test', async () => {
const response = await fetch('https://api.example.com/test');
expect(response.status).toEqual(200);
});
export default tap.start();
```
### Runtime Execution Order
When multiple runtimes are specified, tests execute in this order:
1. Node.js
2. Bun
3. Deno
4. Chromium
### Legacy Naming (Deprecated)
The following patterns are still supported but deprecated. Use the migration tool to update:
| Legacy Pattern | Modern Equivalent | Migration Command |
|----------------|-------------------|-------------------|
| `*.browser.ts` | `*.chromium.ts` | `tstest migrate` |
| `*.both.ts` | `*.node+chromium.ts` | `tstest migrate` |
When running legacy files, tstest shows a deprecation warning with the suggested new name.
### Migration Tool
Migrate your test files from legacy naming to the new convention:
```bash
# Dry run - see what would change
tstest migrate --dry-run
# Apply migrations (uses git mv to preserve history)
tstest migrate --write
```
**Migration Features:**
- ✅ Uses `git mv` to preserve file history
- ✅ Idempotent - safe to run multiple times
- ✅ Dry-run by default for safety
- ✅ Colored output showing all changes
- ✅ Handles modifiers like `.nonci` correctly
Example output:
```
============================================================
Test File Migration Tool
============================================================
🔍 DRY RUN MODE - No files will be modified
Found 3 legacy test file(s)
Would migrate:
test.browser.ts
→ test.chromium.ts
Would migrate:
test.both.ts
→ test.node+chromium.ts
Would migrate:
test.auth.browser.nonci.ts
→ test.auth.chromium.nonci.ts
============================================================
Summary:
Total legacy files: 3
Successfully migrated: 3
Errors: 0
============================================================
To apply these changes, run:
tstest migrate --write
```
### Runtime-Specific Permissions
#### Deno Runtime
Tests run with these permissions by default:
```bash
--allow-read
--allow-env
--allow-net
--allow-write
--allow-sys
--allow-import # Enables npm packages and Node.js built-ins
--node-modules-dir # Node.js compatibility mode
--sloppy-imports # Allows .js imports to resolve to .ts files
```
Configure custom permissions in your test file or via environment variables.
#### Bun Runtime
Bun runs with its native TypeScript support and full access to Node.js APIs.
## Usage ## Usage
### Basic Test Execution ### Basic Test Execution
@@ -60,7 +205,14 @@ tstest "test/unit/*.ts"
| `--verbose`, `-v` | Show all console output from tests | | `--verbose`, `-v` | Show all console output from tests |
| `--no-color` | Disable colored output | | `--no-color` | Disable colored output |
| `--json` | Output results as JSON | | `--json` | Output results as JSON |
| `--logfile` | Save detailed logs to `.nogit/testlogs/[testname].log` | | `--logfile` | Save detailed logs with automatic error and diff tracking |
| `--tags <tags>` | Run only tests with specific tags (comma-separated) |
| `--timeout <seconds>` | Timeout test files after specified seconds |
| `--startFrom <n>` | Start running from test file number n |
| `--stopAt <n>` | Stop running at test file number n |
| `--watch`, `-w` | Watch for file changes and re-run tests |
| `--watch-ignore <patterns>` | Ignore patterns in watch mode (comma-separated) |
| `--only` | Run only tests marked with .only |
### Example Outputs ### Example Outputs
@@ -71,18 +223,29 @@ tstest "test/unit/*.ts"
Pattern: test Pattern: test
Found: 4 test file(s) Found: 4 test file(s)
▶️ test/test.ts (1/4) ━━━ Part 1: Node.js ━━━
Runtime: node.js
✅ prepare test (1ms) ▶️ test/test.node+deno.ts (1/4)
Summary: 1/1 PASSED Runtime: Node.js
✅ HTTP request works (12ms)
✅ JSON parsing works (3ms)
Summary: 2/2 PASSED in 1.2s
━━━ Part 2: Deno ━━━
▶️ test/test.node+deno.ts (1/4)
Runtime: Deno
✅ HTTP request works (15ms)
✅ JSON parsing works (2ms)
Summary: 2/2 PASSED in 1.1s
📊 Test Summary 📊 Test Summary
┌────────────────────────────────┐ ┌────────────────────────────────┐
│ Total Files: 4 │ │ Total Files: 4 │
│ Total Tests: 4 │ Total Tests: 8
│ Passed: 4 │ Passed: 8
│ Failed: 0 │ │ Failed: 0 │
│ Duration: 542ms │ │ Duration: 2.4s │
└────────────────────────────────┘ └────────────────────────────────┘
ALL TESTS PASSED! 🎉 ALL TESTS PASSED! 🎉
@@ -120,57 +283,560 @@ Perfect for CI/CD pipelines:
{"event":"summary","summary":{"totalFiles":4,"totalTests":4,"totalPassed":4,"totalFailed":0,"totalDuration":542}} {"event":"summary","summary":{"totalFiles":4,"totalTests":4,"totalPassed":4,"totalFailed":0,"totalDuration":542}}
``` ```
## Test File Naming Conventions ## Writing Tests with tapbundle
tstest supports different test environments through file naming: tstest includes tapbundle, a powerful TAP-based test framework. Import it from the embedded tapbundle:
| Pattern | Environment | Example |
|---------|-------------|---------|
| `*.ts` | Node.js (default) | `test.basic.ts` |
| `*.node.ts` | Node.js only | `test.api.node.ts` |
| `*.chrome.ts` | Chrome browser | `test.dom.chrome.ts` |
| `*.browser.ts` | Browser environment | `test.ui.browser.ts` |
| `*.both.ts` | Both Node.js and browser | `test.isomorphic.both.ts` |
### Writing Tests
tstest uses TAP (Test Anything Protocol) for test output. Use `@pushrocks/tapbundle` for the best experience:
```typescript ```typescript
import { expect, tap } from '@push.rocks/tapbundle'; import { expect, tap } from '@git.zone/tstest/tapbundle';
tap.test('my awesome test', async () => { tap.test('my awesome test', async () => {
const result = await myFunction(); const result = await myFunction();
expect(result).toEqual('expected value'); expect(result).toEqual('expected value');
}); });
tap.start(); export default tap.start();
```
**Module Exports**
tstest provides multiple exports for different use cases:
- `@git.zone/tstest` - Main CLI and test runner functionality
- `@git.zone/tstest/tapbundle` - Browser-compatible test framework
- `@git.zone/tstest/tapbundle_node` - Node.js-specific test utilities
## tapbundle Test Framework
### Basic Test Syntax
```typescript
import { tap, expect } from '@git.zone/tstest/tapbundle';
// Basic test
tap.test('should perform basic arithmetic', async () => {
expect(2 + 2).toEqual(4);
});
// Async test with tools
tap.test('async operations', async (tools) => {
await tools.delayFor(100); // delay for 100ms
const result = await fetchData();
expect(result).toBeDefined();
});
// Start test execution
export default tap.start();
```
### Test Modifiers and Chaining
```typescript
// Skip a test
tap.skip.test('not ready yet', async () => {
// This test will be skipped
});
// Run only this test (exclusive)
tap.only.test('focus on this', async () => {
// Only this test will run
});
// Todo test - creates actual test object marked as todo
tap.todo.test('implement later', async () => {
// This test will be counted but marked as todo
});
// Chaining modifiers
tap.timeout(5000)
.retry(3)
.tags('api', 'integration')
.test('complex test', async (tools) => {
// Test with 5s timeout, 3 retries, and tags
});
```
### Test Organization with describe()
```typescript
tap.describe('User Management', () => {
let testDatabase;
tap.beforeEach(async () => {
testDatabase = await createTestDB();
});
tap.afterEach(async () => {
await testDatabase.cleanup();
});
tap.test('should create user', async () => {
const user = await testDatabase.createUser({ name: 'John' });
expect(user.id).toBeDefined();
});
tap.describe('User Permissions', () => {
tap.test('should set admin role', async () => {
// Nested describe blocks
});
});
});
```
### Test Tools (Available in Test Function)
Every test function receives a `tools` parameter with utilities:
```typescript
tap.test('using test tools', async (tools) => {
// Delay utilities
await tools.delayFor(1000); // delay for 1000ms
await tools.delayForRandom(100, 500); // random delay between 100-500ms
// Skip test conditionally
tools.skipIf(process.env.CI === 'true', 'Skipping in CI');
// Skip test unconditionally
if (!apiKeyAvailable) {
tools.skip('API key not available');
}
// Mark as todo
tools.todo('Needs implementation');
// Retry configuration
tools.retry(3); // Set retry count
// Timeout configuration
tools.timeout(10000); // Set timeout to 10s
// Context sharing between tests
tools.context.set('userId', 12345);
const userId = tools.context.get('userId');
// Deferred promises
const deferred = tools.defer();
setTimeout(() => deferred.resolve('done'), 100);
await deferred.promise;
// Colored console output
const coloredString = await tools.coloredString('Success!', 'green');
console.log(coloredString);
// Error handling helper
const error = await tools.returnError(async () => {
throw new Error('Expected error');
});
expect(error).toBeInstanceOf(Error);
});
```
### Snapshot Testing
```typescript
tap.test('snapshot test', async (tools) => {
const output = generateComplexOutput();
// Compare with saved snapshot
await tools.matchSnapshot(output);
// Named snapshots for multiple checks in one test
await tools.matchSnapshot(output.header, 'header');
await tools.matchSnapshot(output.body, 'body');
});
// Update snapshots with: UPDATE_SNAPSHOTS=true tstest test/
```
### Test Fixtures
```typescript
// Define reusable fixtures
tap.defineFixture('testUser', async (data) => ({
id: Date.now(),
name: data?.name || 'Test User',
email: data?.email || 'test@example.com',
created: new Date()
}));
tap.defineFixture('testPost', async (data) => ({
id: Date.now(),
title: data?.title || 'Test Post',
authorId: data?.authorId || 1
}));
// Use fixtures in tests
tap.test('fixture test', async (tools) => {
const user = await tools.fixture('testUser', { name: 'John' });
const post = await tools.fixture('testPost', { authorId: user.id });
expect(post.authorId).toEqual(user.id);
// Factory pattern for multiple instances
const users = await tools.factory('testUser').createMany(5);
expect(users).toHaveLength(5);
});
```
### Parallel Test Execution
```typescript
// Parallel tests within a file
tap.testParallel('parallel test 1', async () => {
await heavyOperation();
});
tap.testParallel('parallel test 2', async () => {
await anotherHeavyOperation();
});
// File naming for parallel groups
// test.api.para__1.ts - runs in parallel with other para__1 files
// test.db.para__1.ts - runs in parallel with other para__1 files
// test.auth.para__2.ts - runs after para__1 group completes
```
### Assertions with expect()
tapbundle uses @push.rocks/smartexpect for assertions:
```typescript
// Basic assertions
expect(value).toEqual(5);
expect(value).not.toEqual(10);
expect(obj).toDeepEqual({ a: 1, b: 2 });
// Type assertions
expect('hello').toBeTypeofString();
expect(42).toBeTypeofNumber();
expect(true).toBeTypeofBoolean();
expect([]).toBeArray();
expect({}).toBeTypeOf('object');
// Comparison assertions
expect(5).toBeGreaterThan(3);
expect(3).toBeLessThan(5);
expect(5).toBeGreaterThanOrEqual(5);
expect(5).toBeLessThanOrEqual(5);
expect(0.1 + 0.2).toBeCloseTo(0.3, 10);
// Truthiness
expect(true).toBeTrue();
expect(false).toBeFalse();
expect('text').toBeTruthy();
expect(0).toBeFalsy();
expect(null).toBeNull();
expect(undefined).toBeUndefined();
expect(null).toBeNullOrUndefined();
// String assertions
expect('hello world').toStartWith('hello');
expect('hello world').toEndWith('world');
expect('hello world').toInclude('lo wo');
expect('hello world').toMatch(/^hello/);
expect('option').toBeOneOf(['choice', 'option', 'alternative']);
// Array assertions
expect([1, 2, 3]).toContain(2);
expect([1, 2, 3]).toContainAll([1, 3]);
expect([1, 2, 3]).toExclude(4);
expect([1, 2, 3]).toHaveLength(3);
expect([]).toBeEmptyArray();
expect([{ id: 1 }]).toContainEqual({ id: 1 });
// Object assertions
expect(obj).toHaveProperty('name');
expect(obj).toHaveProperty('user.email', 'test@example.com');
expect(obj).toHaveDeepProperty(['level1', 'level2']);
expect(obj).toMatchObject({ name: 'John' });
// Function assertions
expect(() => { throw new Error('test'); }).toThrow();
expect(() => { throw new Error('test'); }).toThrow(Error);
expect(() => { throw new Error('test error'); }).toThrowErrorMatching(/test/);
expect(myFunction).not.toThrow();
// Promise assertions
await expect(Promise.resolve('value')).resolves.toEqual('value');
await expect(Promise.reject(new Error('fail'))).rejects.toThrow();
// Custom assertions
expect(7).customAssertion(
value => value % 2 === 1,
'Value is not odd'
);
```
### Pre-tasks
Run setup tasks before tests start:
```typescript
tap.preTask('setup database', async () => {
await initializeTestDatabase();
console.log('Database initialized');
});
tap.preTask('load environment', async () => {
await loadTestEnvironment();
});
// Pre-tasks run in order before any tests
```
### Tag-based Test Filtering
```typescript
// Tag individual tests
tap.tags('unit', 'api')
.test('api unit test', async () => {
// Test code
});
tap.tags('integration', 'slow')
.test('database integration', async () => {
// Test code
});
// Run only tests with specific tags
// tstest test/ --tags unit,api
```
### Context Sharing
Share data between tests:
```typescript
tap.test('first test', async (tools) => {
const sessionId = await createSession();
tools.context.set('sessionId', sessionId);
});
tap.test('second test', async (tools) => {
const sessionId = tools.context.get('sessionId');
expect(sessionId).toBeDefined();
// Cleanup
tools.context.delete('sessionId');
});
```
### Browser Testing with webhelpers
For browser-specific tests:
```typescript
import { tap, webhelpers } from '@git.zone/tstest/tapbundle';
tap.test('DOM manipulation', async () => {
// Create DOM elements from HTML strings
const element = await webhelpers.fixture(webhelpers.html`
<div class="test-container">
<h1>Test Title</h1>
<button id="test-btn">Click Me</button>
</div>
`);
expect(element.querySelector('h1').textContent).toEqual('Test Title');
// Simulate interactions
const button = element.querySelector('#test-btn');
button.click();
});
tap.test('CSS testing', async () => {
const styles = webhelpers.css`
.test-class {
color: red;
font-size: 16px;
}
`;
// styles is a string that can be injected into the page
expect(styles).toInclude('color: red');
});
```
### Advanced Error Handling
```typescript
tap.test('error handling', async (tools) => {
// Capture errors without failing the test
const error = await tools.returnError(async () => {
await functionThatThrows();
});
expect(error).toBeInstanceOf(Error);
expect(error.message).toEqual('Expected error message');
});
```
### Test Wrap
Create wrapped test environments:
```typescript
import { TapWrap } from '@git.zone/tstest/tapbundle';
const tapWrap = new TapWrap({
before: async () => {
console.log('Before all tests');
await globalSetup();
},
after: async () => {
console.log('After all tests');
await globalCleanup();
}
});
// Tests registered here will have the wrap lifecycle
tapWrap.tap.test('wrapped test', async () => {
// This test runs with the wrap setup/teardown
});
``` ```
## Advanced Features ## Advanced Features
### Glob Pattern Support ### Watch Mode
Automatically re-run tests when files change:
Run specific test patterns:
```bash ```bash
# Run all unit tests # Watch all files in the project
tstest "test/unit/**/*.ts" tstest test/ --watch
# Run all integration tests # Watch with custom ignore patterns
tstest "test/integration/*.test.ts" tstest test/ --watch --watch-ignore "dist/**,coverage/**"
# Run multiple patterns # Short form
tstest "test/**/*.spec.ts" "test/**/*.test.ts" tstest test/ -w
``` ```
### Automatic Logging **Features:**
- 👀 Shows which files triggered the re-run
- ⏱️ 300ms debouncing to batch rapid changes
- 🔄 Clears console between runs for clean output
- 📁 Intelligently ignores common non-source files
### Real-time Test Progress
tstest provides real-time updates during test execution:
```
▶️ test/api.test.ts (1/4)
Runtime: node.js
⏳ Running: api endpoint validation...
✅ api endpoint validation (145ms)
⏳ Running: error handling...
✅ error handling (23ms)
Summary: 2/2 PASSED
```
### Visual Diffs for Failed Assertions
When assertions fail, tstest shows beautiful side-by-side diffs:
```
❌ should return correct user data
String Diff:
- Expected
+ Received
- Hello World
+ Hello Universe
Object Diff:
{
name: "John",
- age: 30,
+ age: 31,
email: "john@example.com"
}
```
### Enhanced Test Logging
The `--logfile` option provides intelligent test logging with automatic organization:
Use `--logfile` to automatically save test output:
```bash ```bash
tstest test/ --logfile tstest test/ --logfile
``` ```
This creates detailed logs in `.nogit/testlogs/[testname].log` for each test file. **Log Organization:**
- **Current Run**: `.nogit/testlogs/[testname].log`
- **Previous Run**: `.nogit/testlogs/previous/[testname].log`
- **Failed Tests**: `.nogit/testlogs/00err/[testname].log`
- **Changed Output**: `.nogit/testlogs/00diff/[testname].log`
**Features:**
- Previous logs are automatically moved to the `previous/` folder
- Failed tests create copies in `00err/` for quick identification
- Tests with changed output create diff reports in `00diff/`
- The `00err/` and `00diff/` folders are cleared on each run
**Example Diff Report:**
```
DIFF REPORT: test__api__integration.log
Generated: 2025-05-24T01:29:13.847Z
================================================================================
- [Line 8] ✅ api test passes (150ms)
+ [Line 8] ✅ api test passes (165ms)
================================================================================
Previous version had 40 lines
Current version has 40 lines
```
### Test Timeout Protection
Prevent runaway tests with the `--timeout` option:
```bash
# Timeout any test file that runs longer than 60 seconds
tstest test/ --timeout 60
# Shorter timeout for unit tests
tstest test/unit/ --timeout 10
```
When a test exceeds the timeout:
- The test process is terminated (SIGTERM)
- The test is marked as failed
- An error log is created in `.nogit/testlogs/00err/`
- Clear error message shows the timeout duration
### Test File Range Control
Run specific ranges of test files using `--startFrom` and `--stopAt`:
```bash
# Run tests starting from the 5th file
tstest test/ --startFrom 5
# Run only files 5 through 10
tstest test/ --startFrom 5 --stopAt 10
# Run only the first 3 test files
tstest test/ --stopAt 3
```
This is particularly useful for:
- Debugging specific test failures in large test suites
- Running tests in chunks on different CI runners
- Quickly testing changes to specific test files
The output shows which files are skipped:
```
⏭️ test/auth.test.ts (1/10)
Skipped: before start range (5)
⏭️ test/user.test.ts (2/10)
Skipped: before start range (5)
▶️ test/api.test.ts (5/10)
Runtime: node.js
✅ api endpoints work (145ms)
```
### Performance Analysis ### Performance Analysis
@@ -181,6 +847,26 @@ In verbose mode, see performance metrics:
Slowest test: api integration test (486ms) Slowest test: api integration test (486ms)
``` ```
### Parallel Test Groups
Tests can be organized into parallel groups for concurrent execution:
```
━━━ Parallel Group: para__1 ━━━
▶️ test/auth.para__1.ts
▶️ test/user.para__1.ts
... tests run concurrently ...
──────────────────────────────────
━━━ Parallel Group: para__2 ━━━
▶️ test/db.para__2.ts
▶️ test/api.para__2.ts
... tests run concurrently ...
──────────────────────────────────
```
Files with the same parallel group suffix (e.g., `para__1`) run simultaneously, while different groups run sequentially.
### CI/CD Integration ### CI/CD Integration
For continuous integration, combine quiet and JSON modes: For continuous integration, combine quiet and JSON modes:
@@ -192,13 +878,116 @@ tstest test/ --json > test-results.json
tstest test/ --quiet tstest test/ --quiet
``` ```
## Contribution **Advanced CI Example:**
```bash
# Run tests with comprehensive logging and safety features
tstest test/ \
--timeout 300 \
--logfile \
--json > test-results.json
We are always happy for code contributions. If you are not the code contributing type that is ok. Still, maintaining Open Source repositories takes considerable time and thought. If you like the quality of what we do and our modules are useful to you we would appreciate a little monthly contribution: You can [contribute one time](https://lossless.link/contribute-onetime) or [contribute monthly](https://lossless.link/contribute). :) # Run specific test chunks in parallel CI jobs
tstest test/ --startFrom 1 --stopAt 10 # Job 1
tstest test/ --startFrom 11 --stopAt 20 # Job 2
tstest test/ --startFrom 21 # Job 3
```
## License ### Debugging Failed Tests
> MIT licensed | **&copy;** [Lossless GmbH](https://lossless.gmbh) When tests fail, use the enhanced logging features:
| By using this npm module you agree to our [privacy policy](https://lossless.gmbH/privacy)
[![repo-footer](https://lossless.gitlab.io/publicrelations/repofooter.svg)](https://maintainedby.lossless.com) ```bash
# Run with logging to capture detailed output
tstest test/ --logfile --verbose
# Check error logs
ls .nogit/testlogs/00err/
# Review diffs for flaky tests
cat .nogit/testlogs/00diff/test__api__endpoints.log
# Re-run specific failed tests
tstest test/api/endpoints.test.ts --verbose --timeout 60
```
## Changelog
### Version 2.4.0
- 🚀 **Multi-Runtime Architecture** - Support for Deno, Bun, Node.js, and Chromium
- 🔀 **New Naming Convention** - Flexible `.runtime1+runtime2.ts` pattern
- 🔄 **Migration Tool** - Easy migration from legacy naming (`.browser.ts`, `.both.ts`)
- 🦕 **Deno Support** - Full Deno runtime with Node.js compatibility
- 🐰 **Bun Support** - Ultra-fast Bun runtime integration
-**Dynamic Port Selection** - Random port allocation (30000-40000) prevents conflicts
- 🏗️ **Runtime Adapter Pattern** - Extensible architecture for adding new runtimes
- 📝 **Deprecation Warnings** - Helpful migration suggestions for legacy naming
-**Comprehensive Tests** - Full test coverage for parser and migration tool
### Version 1.11.0
- 👀 Added Watch Mode with `--watch`/`-w` flag for automatic test re-runs
- 📊 Implemented real-time test progress updates with event streaming
- 🎨 Added visual diffs for failed assertions with side-by-side comparison
- 🔄 Enhanced event-based test lifecycle reporting
- ⚙️ Added test configuration system with `.tstest.json` files
- 🚀 Implemented Protocol V2 with Unicode delimiters for better TAP parsing
- 🐛 Fixed `tap.todo()` to create proper test objects
- 🐛 Fixed `tap.skip.test()` to correctly create and count test objects
- 🐛 Fixed `tap.only.test()` implementation with `--only` flag support
- 📁 Added settings inheritance for cascading test configuration
- ⏱️ Added debouncing for file change events in watch mode
### Version 1.10.0
- ⏱️ Added `--timeout <seconds>` option for test file timeout protection
- 🎯 Added `--startFrom <n>` and `--stopAt <n>` options for test file range control
- 📁 Enhanced `--logfile` with intelligent log organization:
- Previous logs moved to `previous/` folder
- Failed tests copied to `00err/` folder
- Changed tests create diff reports in `00diff/` folder
- 🔍 Improved test discovery to show skipped files with clear reasons
- 🐛 Fixed TypeScript compilation warnings and unused variables
- 📊 Test summaries now include skipped file counts
### Version 1.9.2
- 🐛 Fixed test timing display issue (removed duplicate timing in output)
- 📝 Improved internal protocol design documentation
- 🔧 Added protocol v2 utilities for future improvements
### Version 1.9.1
- 🐛 Fixed log file naming to preserve directory structure
- 📁 Log files now prevent collisions: `test__dir__file.log`
### Version 1.9.0
- 📚 Comprehensive documentation update
- 🏗️ Embedded tapbundle for better integration
- 🌐 Full browser compatibility
### Version 1.8.0
- 📦 Embedded tapbundle directly into tstest project
- 🌐 Made tapbundle fully browser-compatible
- 📸 Added snapshot testing with base64-encoded communication protocol
- 🏷️ Introduced tag-based test filtering
- 🔧 Enhanced test lifecycle hooks (beforeEach/afterEach)
- 🎯 Fixed parallel test execution and grouping
- ⏳ Improved timeout and retry mechanisms
- 🛠️ Added test fixtures for reusable test data
- 📊 Enhanced TAP parser for better test reporting
- 🐛 Fixed glob pattern handling in shell scripts
## License and Legal Information
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
### Company Information
Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

View File

@@ -2,61 +2,115 @@
!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !! !! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
## 1. Enhanced Communication Between tapbundle and tstest ## Improved Internal Protocol (NEW - Critical) ✅ COMPLETED
### 1.1 Real-time Test Progress API ### Current Issues ✅ RESOLVED
- Create a bidirectional communication channel between tapbundle and tstest - ✅ TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
- Emit events for test lifecycle stages (start, progress, completion) - ✅ Fragile regex parsing that breaks with special characters
- Allow tstest to subscribe to tapbundle events for better progress reporting - ✅ Limited extensibility for new metadata types
- Implement a standardized message format for test metadata
### 1.2 Rich Error Reporting ### Proposed Solution: Protocol V2 ✅ IMPLEMENTED
- Pass structured error objects from tapbundle to tstest - ✅ Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
- Include stack traces, code snippets, and contextual information - ✅ Structured JSON metadata format
- Support for error categorization (assertion failures, timeouts, uncaught exceptions) - ✅ Separate protocol blocks for complex data (errors, snapshots)
- Visual diff output for failed assertions - ✅ Complete replacement of v1 (no backwards compatibility needed)
### Implementation ✅ COMPLETED
- ✅ Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
- ✅ Phase 2: Replace all v1 code in both tstest and tapbundle with v2
- ✅ Phase 3: Delete all v1 parsing and generation code
#### ts_tapbundle_protocol Directory
The protocol v2 implementation will be contained in the `ts_tapbundle_protocol` directory as isomorphic TypeScript code:
- **Isomorphic Design**: All code must work in both browser and Node.js environments
- **No Node.js Imports**: No Node.js-specific modules allowed (no fs, path, child_process, etc.)
- **Protocol Classes**: Contains classes implementing all sides of the protocol:
-`ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
-`ProtocolParser`: For parsing protocol v2 messages (used by tstest)
-`ProtocolMessage`: Base classes for different message types
-`ProtocolTypes`: TypeScript interfaces and types for protocol structures
- **Pure TypeScript**: Only browser-compatible APIs and pure TypeScript/JavaScript code
- **Build Integration**:
- Compiled by `pnpm build` (via tsbuild) to `dist_ts_tapbundle_protocol/`
- Build order defined in tspublish.json files
- Imported by ts and ts_tapbundle modules from the compiled dist directory
See `readme.protocol.md` for detailed specification.
## Test Configuration System (NEW)
### Global Test Configuration via 00init.ts
- **Discovery**: Check for `test/00init.ts` before running tests
- **Execution**: Import and execute before any test files if found
- **Purpose**: Define project-wide default test settings
### tap.settings() API
```typescript
interface TapSettings {
// Timing
timeout?: number; // Default timeout for all tests (ms)
slowThreshold?: number; // Mark tests as slow if they exceed this (ms)
// Execution Control
bail?: boolean; // Stop on first test failure
retries?: number; // Number of retries for failed tests
retryDelay?: number; // Delay between retries (ms)
// Output Control
suppressConsole?: boolean; // Suppress console output in passing tests
verboseErrors?: boolean; // Show full stack traces
showTestDuration?: boolean; // Show duration for each test
// Parallel Execution
maxConcurrency?: number; // Max parallel tests (for .para files)
isolateTests?: boolean; // Run each test in fresh context
// Lifecycle Hooks
beforeAll?: () => Promise<void> | void;
afterAll?: () => Promise<void> | void;
beforeEach?: (testName: string) => Promise<void> | void;
afterEach?: (testName: string, passed: boolean) => Promise<void> | void;
// Environment
env?: Record<string, string>; // Additional environment variables
// Features
enableSnapshots?: boolean; // Enable snapshot testing
snapshotDirectory?: string; // Custom snapshot directory
updateSnapshots?: boolean; // Update snapshots instead of comparing
}
```
### Settings Inheritance
- Global (00init.ts) → File level → Test level
- More specific settings override less specific ones
- Arrays/objects are merged, primitives are replaced
### Implementation Phases
1. **Core Infrastructure**: Settings storage and merge logic
2. **Discovery**: 00init.ts loading mechanism
3. **Application**: Apply settings to test execution
4. **Advanced**: Parallel execution and snapshot configuration
## 1. Enhanced Communication Between tapbundle and tstest ✅ COMPLETED
### 1.1 Real-time Test Progress API ✅ COMPLETED
- ✅ Create a bidirectional communication channel between tapbundle and tstest
- ✅ Emit events for test lifecycle stages (start, progress, completion)
- ✅ Allow tstest to subscribe to tapbundle events for better progress reporting
- ✅ Implement a standardized message format for test metadata
### 1.2 Rich Error Reporting ✅ COMPLETED
- ✅ Pass structured error objects from tapbundle to tstest
- ✅ Include stack traces, code snippets, and contextual information
- ✅ Support for error categorization (assertion failures, timeouts, uncaught exceptions)
- ✅ Visual diff output for failed assertions
## 2. Enhanced toolsArg Functionality ## 2. Enhanced toolsArg Functionality
### 2.1 Test Flow Control ✅ ### 2.3 Test Data and Context Sharing (Partial)
```typescript
tap.test('conditional test', async (toolsArg) => {
const result = await someOperation();
// Skip the rest of the test
if (!result) {
return toolsArg.skip('Precondition not met');
}
// Conditional skipping
await toolsArg.skipIf(condition, 'Reason for skipping');
// Mark test as todo
await toolsArg.todo('Not implemented yet');
});
```
### 2.2 Test Metadata and Configuration ✅
```typescript
// Fluent syntax ✅
tap.tags('slow', 'integration')
.priority('high')
.timeout(5000)
.retry(3)
.test('configurable test', async (toolsArg) => {
// Test implementation
});
```
### 2.3 Test Data and Context Sharing ✅
```typescript ```typescript
tap.test('data-driven test', async (toolsArg) => { tap.test('data-driven test', async (toolsArg) => {
// Access shared context ✅
const sharedData = toolsArg.context.get('sharedData');
// Set data for other tests ✅
toolsArg.context.set('resultData', computedValue);
// Parameterized test data (not yet implemented) // Parameterized test data (not yet implemented)
const testData = toolsArg.data<TestInput>(); const testData = toolsArg.data<TestInput>();
expect(processData(testData)).toEqual(expected); expect(processData(testData)).toEqual(expected);
@@ -65,32 +119,7 @@ tap.test('data-driven test', async (toolsArg) => {
## 3. Nested Tests and Test Suites ## 3. Nested Tests and Test Suites
### 3.1 Test Grouping with describe() ✅ ### 3.2 Hierarchical Test Organization (Not yet implemented)
```typescript
tap.describe('User Authentication', () => {
tap.beforeEach(async (toolsArg) => {
// Setup for each test in this suite
await toolsArg.context.set('db', await createTestDatabase());
});
tap.afterEach(async (toolsArg) => {
// Cleanup after each test
await toolsArg.context.get('db').cleanup();
});
tap.test('should login with valid credentials', async (toolsArg) => {
// Test implementation
});
tap.describe('Password Reset', () => {
tap.test('should send reset email', async (toolsArg) => {
// Nested test
});
});
});
```
### 3.2 Hierarchical Test Organization
- Support for multiple levels of nesting - Support for multiple levels of nesting
- Inherited context and configuration from parent suites - Inherited context and configuration from parent suites
- Aggregated reporting for test suites - Aggregated reporting for test suites
@@ -98,15 +127,7 @@ tap.describe('User Authentication', () => {
## 4. Advanced Test Features ## 4. Advanced Test Features
### 4.1 Snapshot Testing ### 4.1 Snapshot Testing ✅ (Basic implementation complete)
```typescript
tap.test('component render', async (toolsArg) => {
const output = renderComponent(props);
// Compare with stored snapshot
await toolsArg.matchSnapshot(output, 'component-output');
});
```
### 4.2 Performance Benchmarking ### 4.2 Performance Benchmarking
```typescript ```typescript
@@ -124,42 +145,20 @@ tap.test('performance test', async (toolsArg) => {
}); });
``` ```
### 4.3 Test Fixtures and Factories ✅
```typescript
tap.test('with fixtures', async (toolsArg) => {
// Create test fixtures
const user = await toolsArg.fixture('user', { name: 'Test User' });
const post = await toolsArg.fixture('post', { author: user });
// Use factory functions
const users = await toolsArg.factory('user').createMany(5);
});
```
## 5. Test Execution Improvements ## 5. Test Execution Improvements
### 5.1 Parallel Test Execution ✅
- Run independent tests concurrently ✅
- Configurable concurrency limits (via file naming convention)
- Resource pooling for shared resources
- Proper isolation between parallel tests ✅
Implementation: ### 5.2 Watch Mode ✅ COMPLETED
- Tests with `para__<groupNumber>` in filename run in parallel
- Different groups run sequentially
- Tests without `para__` run serially
### 5.2 Watch Mode
- Automatically re-run tests on file changes - Automatically re-run tests on file changes
- Intelligent test selection based on changed files - Debounced file change detection (300ms)
- Fast feedback loop for development - Clear console output between runs
- Integration with IDE/editor plugins - Shows which files triggered re-runs
- Graceful exit with Ctrl+C
- `--watch-ignore` option for excluding patterns
### 5.3 Advanced Test Filtering ✅ (partially) ### 5.3 Advanced Test Filtering (Partial) ⚠️
```typescript ```typescript
// Run tests by tags ✅
tstest --tags "unit,fast"
// Exclude tests by pattern (not yet implemented) // Exclude tests by pattern (not yet implemented)
tstest --exclude "**/slow/**" tstest --exclude "**/slow/**"
@@ -198,58 +197,52 @@ tstest --changed
- Links to documentation - Links to documentation
- Code examples in error output - Code examples in error output
### 7.2 Interactive Mode (Needs Detailed Specification)
- REPL for exploring test failures
- Need to define: How to enter interactive mode? When tests fail?
- What commands/features should be available in the REPL?
- Debugging integration
- Node.js inspector protocol integration?
- Breakpoint support?
- Step-through test execution
- Pause between tests?
- Step into/over/out functionality?
- Interactive test data manipulation
- Modify test inputs on the fly?
- Inspect intermediate values?
### 7.3 ~~VS Code Extension~~ (Scratched)
- ~~Test explorer integration~~
- ~~Inline test results~~
- ~~CodeLens for running individual tests~~
- ~~Debugging support~~
## Implementation Phases ## Implementation Phases
### Phase 1: Core Enhancements (Priority: High) ✅ ### Phase 1: Improved Internal Protocol (Priority: Critical) ✅ COMPLETED
1. Implement enhanced toolsArg methods (skip, skipIf, timeout, retry) ✅ 1. ✅ Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
2. Add basic test grouping with describe() ✅ - ✅ Implement ProtocolEmitter class for message generation
3. Improve error reporting between tapbundle and tstest ✅ - ✅ Implement ProtocolParser class for message parsing
- ✅ Define ProtocolMessage types and interfaces
- ✅ Ensure all code is browser and Node.js compatible
- ✅ Add tspublish.json to configure build order
2. ✅ Update build configuration to compile ts_tapbundle_protocol first
3. ✅ Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
4. ✅ Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
5. ✅ Delete all v1 TAP parsing code from tstest
6. ✅ Delete all v1 TAP generation code from tapbundle
7. ✅ Test with real-world test suites containing special characters
### Phase 2: Advanced Features (Priority: Medium) ### Phase 2: Test Configuration System (Priority: High) ✅ COMPLETED
1. Implement nested test suites ✅ (basic describe support) 1. Implement tap.settings() API with TypeScript interfaces
2. Add snapshot testing ✅ 2. Add 00init.ts discovery and loading mechanism
3. Create test fixture system ✅ 3. ✅ Implement settings inheritance and merge logic
4. Implement parallel test execution ✅ 4. ✅ Apply settings to test execution (timeouts, retries, etc.)
### Phase 3: Developer Experience (Priority: Medium) ### Phase 3: Enhanced Communication (Priority: High) ✅ COMPLETED
1. ✅ Build on Protocol V2 for richer communication
2. ✅ Implement real-time test progress API
3. ✅ Add structured error reporting with diffs and traces
### Phase 4: Developer Experience (Priority: Medium) ❌ NOT STARTED
1. Add watch mode 1. Add watch mode
2. Implement custom reporters 2. Implement custom reporters
3. ~~Create VS Code extension~~ (Scratched) 3. Complete advanced test filtering options
4. Add interactive debugging (Needs detailed spec first) 4. Add performance benchmarking API
### Phase 4: Analytics and Performance (Priority: Low) ### Phase 5: Analytics and Performance (Priority: Low) ❌ NOT STARTED
1. Build test analytics dashboard 1. Build test analytics dashboard
2. Add performance benchmarking 2. Implement coverage integration
3. Implement coverage integration 3. Create trend analysis tools
4. Create trend analysis tools 4. Add test impact analysis
## Technical Considerations ## Technical Considerations
### API Design Principles ### API Design Principles
- Maintain backward compatibility - Clean, modern API design without legacy constraints
- Progressive enhancement approach - Progressive enhancement approach
- Opt-in features to avoid breaking changes - Well-documented features and APIs
- Clear migration paths for new features - Clear, simple interfaces
### Performance Goals ### Performance Goals
- Minimal overhead for test execution - Minimal overhead for test execution
@@ -262,3 +255,67 @@ tstest --changed
- Extensible plugin architecture - Extensible plugin architecture
- Standard test result format - Standard test result format
- Compatible with existing CI/CD tools - Compatible with existing CI/CD tools
## Summary of Remaining Work
### ✅ Completed
- **Protocol V2**: Full implementation with Unicode delimiters, structured metadata, and special character handling
- **Test Configuration System**: tap.settings() API, 00init.ts discovery, settings inheritance, lifecycle hooks
- **Enhanced Communication**: Event-based test lifecycle reporting, visual diff output for assertion failures, real-time test progress API
- **Rich Error Reporting**: Stack traces, error metadata, and visual diffs through protocol
- **Tags Filtering**: `--tags` option for running specific tagged tests
### ✅ Existing Features (Not in Plan)
- **Timeout Support**: `--timeout` option and per-test timeouts
- **Test Retries**: `tap.retry()` for flaky test handling
- **Parallel Tests**: `.testParallel()` for concurrent execution
- **Snapshot Testing**: Basic implementation with `toMatchSnapshot()`
- **Test Lifecycle**: `describe()` blocks with `beforeEach`/`afterEach`
- **Skip Tests**: `tap.skip.test()` (though it doesn't create test objects)
- **Log Files**: `--logfile` option saves output to `.nogit/testlogs/`
- **Test Range**: `--startFrom` and `--stopAt` for partial runs
### ⚠️ Partially Completed
- **Advanced Test Filtering**: Have `--tags` but missing `--exclude`, `--failed`, `--changed`
### ❌ Not Started
#### High Priority
#### Medium Priority
2. **Developer Experience**
- Watch mode for file changes
- Custom reporters (JSON, JUnit, HTML, Markdown)
- Performance benchmarking API
- Better error messages with suggestions
3. **Enhanced toolsArg**
- Test data injection
- Context sharing between tests
- Parameterized tests
4. **Test Organization**
- Hierarchical test suites
- Nested describe blocks
- Suite-level lifecycle hooks
#### Low Priority
5. **Analytics and Performance**
- Test analytics dashboard
- Code coverage integration
- Trend analysis
- Flaky test detection
### Recently Fixed Issues ✅
- **tap.todo()**: Now fully implemented with test object creation
- **tap.skip.test()**: Now creates test objects and maintains accurate test count
- **tap.only.test()**: Works correctly - when .only tests exist, only those run
### Remaining Minor Issues
- **Protocol Output**: Some protocol messages still appear in console output
### Next Recommended Steps
1. Add Watch Mode (Phase 4) - high developer value for fast feedback
2. Implement Custom Reporters - important for CI/CD integration
3. Implement performance benchmarking API
4. Add better error messages with suggestions

287
readme.protocol.md Normal file
View File

@@ -0,0 +1,287 @@
# Improved Internal Protocol Design
## Current Issues with TAP Protocol
1. **Delimiter Conflict**: Using `#` for metadata conflicts with test descriptions containing `#`
2. **Ambiguous Parsing**: No clear boundary between test name and metadata
3. **Limited Extensibility**: Adding new metadata requires regex changes
4. **Mixed Concerns**: Protocol data mixed with human-readable output
## Proposed Internal Protocol v2
### Design Principles
1. **Clear Separation**: Protocol data must be unambiguously separated from user content
2. **Extensibility**: Easy to add new metadata without breaking parsers
3. **Backwards Compatible**: Can coexist with standard TAP for gradual migration
4. **Machine Readable**: Structured format for reliable parsing
5. **Human Friendly**: Still readable in raw form
### Protocol Options
#### Option 1: Special Delimiters
```
ok 1 - test description ::TSTEST:: {"time":123,"retry":0}
not ok 2 - another test ::TSTEST:: {"time":45,"error":"timeout"}
ok 3 - skipped test ::TSTEST:: {"time":0,"skip":"not ready"}
```
**Pros**:
- Simple to implement
- Backwards compatible with TAP parsers (they ignore the suffix)
- Easy to parse with split()
**Cons**:
- Still could conflict if test name contains `::TSTEST::`
- Not standard TAP
#### Option 2: Separate Metadata Lines
```
ok 1 - test description
::METADATA:: {"test":1,"time":123,"retry":0}
not ok 2 - another test
::METADATA:: {"test":2,"time":45,"error":"timeout"}
```
**Pros**:
- Complete separation of concerns
- No chance of conflicts
- Can include arbitrary metadata
**Cons**:
- Requires correlation between lines
- More complex parsing
#### Option 3: YAML Blocks (TAP 13 Compatible)
```
ok 1 - test description
---
time: 123
retry: 0
...
not ok 2 - another test
---
time: 45
error: timeout
stack: |
Error: timeout
at Test.run (test.js:10:5)
...
```
**Pros**:
- Standard TAP 13 feature
- Structured data format
- Human readable
- Extensible
**Cons**:
- More verbose
- YAML parsing overhead
#### Option 4: Binary Protocol Markers (Recommended)
```
ok 1 - test description
␛[TSTEST:eyJ0aW1lIjoxMjMsInJldHJ5IjowfQ==]␛
not ok 2 - another test
␛[TSTEST:eyJ0aW1lIjo0NSwiZXJyb3IiOiJ0aW1lb3V0In0=]␛
```
Using ASCII escape character (␛ = \x1B) with base64 encoded JSON.
**Pros**:
- Zero chance of accidental conflicts
- Compact
- Fast to parse
- Invisible in most terminals
**Cons**:
- Not human readable in raw form
- Requires base64 encoding/decoding
### Recommended Implementation: Hybrid Approach
Use multiple strategies based on context:
1. **For timing and basic metadata**: Use structured delimiters
```
ok 1 - test name ⟦time:123,retry:0⟧
```
2. **For complex data (errors, snapshots)**: Use separate protocol lines
```
ok 1 - test failed
⟦TSTEST:ERROR⟧
{"message":"Assertion failed","stack":"...","diff":"..."}
⟦/TSTEST:ERROR⟧
```
3. **For human-readable output**: Keep standard TAP comments
```
# Test suite: User Authentication
ok 1 - should login
```
### Implementation Plan
#### Phase 1: Parser Enhancement
1. Add new protocol parser alongside existing TAP parser
2. Support both old and new formats during transition
3. Add protocol version negotiation
#### Phase 2: Metadata Structure
```typescript
interface TestMetadata {
// Timing
time: number; // milliseconds
startTime?: number; // Unix timestamp
endTime?: number; // Unix timestamp
// Status
skip?: string; // skip reason
todo?: string; // todo reason
retry?: number; // retry attempt
maxRetries?: number; // max retries allowed
// Error details
error?: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
};
// Test context
file?: string; // source file
line?: number; // line number
column?: number; // column number
// Custom data
tags?: string[]; // test tags
custom?: Record<string, any>;
}
```
#### Phase 3: Protocol Messages
##### Success Message
```
ok 1 - user authentication works
⟦TSTEST:META:{"time":123,"tags":["auth","unit"]}⟧
```
##### Failure Message
```
not ok 2 - login fails with invalid password
⟦TSTEST:META:{"time":45,"retry":1,"maxRetries":3}⟧
⟦TSTEST:ERROR⟧
{
"message": "Expected 401 but got 500",
"stack": "Error: Expected 401 but got 500\n at Test.run (auth.test.ts:25:10)",
"actual": 500,
"expected": 401
}
⟦/TSTEST:ERROR⟧
```
##### Skip Message
```
ok 3 - database integration test ⟦TSTEST:SKIP:No database connection⟧
```
##### Snapshot Communication
```
⟦TSTEST:SNAPSHOT:user-profile⟧
{
"name": "John Doe",
"email": "john@example.com",
"roles": ["user", "admin"]
}
⟦/TSTEST:SNAPSHOT⟧
```
### Migration Strategy
1. **Version Detection**: First line indicates protocol version
```
⟦TSTEST:PROTOCOL:2.0⟧
TAP version 13
```
2. **Gradual Rollout**:
- v1.10: Add protocol v2 parser, keep v1 generator
- v1.11: Generate v2 by default, v1 with --legacy flag
- v2.0: Remove v1 support
3. **Feature Flags**:
```typescript
tap.settings({
protocol: 'v2', // or 'v1', 'auto'
protocolFeatures: {
structuredErrors: true,
enhancedTiming: true,
binaryMarkers: false
}
});
```
### Benefits of New Protocol
1. **Reliability**: No more regex fragility or description conflicts
2. **Performance**: Faster parsing with clear boundaries
3. **Extensibility**: Easy to add new metadata fields
4. **Debugging**: Rich error information with stack traces and diffs
5. **Integration**: Better IDE and CI/CD tool integration
6. **Forward Compatible**: Room for future enhancements
### Example Parser Implementation
```typescript
class ProtocolV2Parser {
private readonly MARKER_START = '⟦TSTEST:';
private readonly MARKER_END = '⟧';
parseMetadata(line: string): TestMetadata | null {
const start = line.lastIndexOf(this.MARKER_START);
if (start === -1) return null;
const end = line.indexOf(this.MARKER_END, start);
if (end === -1) return null;
const content = line.substring(start + this.MARKER_START.length, end);
const [type, data] = content.split(':', 2);
switch (type) {
case 'META':
return JSON.parse(data);
case 'SKIP':
return { skip: data };
case 'TODO':
return { todo: data };
default:
return null;
}
}
parseTestLine(line: string): ParsedTest {
// First extract any metadata
const metadata = this.parseMetadata(line);
// Then parse the TAP part (without metadata)
const cleanLine = this.removeMetadata(line);
const tapResult = this.parseTAP(cleanLine);
return { ...tapResult, metadata };
}
}
```
### Next Steps
1. Implement proof of concept with basic metadata support
2. Test with real-world test suites for edge cases
3. Benchmark parsing performance
4. Get feedback from users
5. Finalize protocol specification
6. Implement in both tapbundle and tstest

View File

@@ -0,0 +1,41 @@
import { tap } from '../../ts_tapbundle/index.js';
// TAP-compliant comment output
console.log('# 🚀 00init.ts: LOADED AND EXECUTING');
console.log('# 🚀 00init.ts: Setting up global test configuration');
// Add a global variable to verify 00init.ts was loaded
(global as any).__00INIT_LOADED = true;
// Configure global test settings
tap.settings({
// Set a default timeout of 5 seconds for all tests
timeout: 5000,
// Enable retries for flaky tests
retries: 2,
retryDelay: 1000,
// Show test duration
showTestDuration: true,
// Global lifecycle hooks
beforeAll: async () => {
console.log('Global beforeAll: Initializing test environment');
},
afterAll: async () => {
console.log('Global afterAll: Cleaning up test environment');
},
beforeEach: async (testName: string) => {
console.log(`Global beforeEach: Starting test "${testName}"`);
},
afterEach: async (testName: string, passed: boolean) => {
console.log(`Global afterEach: Test "${testName}" ${passed ? 'passed' : 'failed'}`);
}
});
console.log('# 🚀 00init.ts: Configuration COMPLETE');
console.log('# 🚀 00init.ts: tap.settings() called successfully');

View File

@@ -0,0 +1,44 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
// TAP-compliant comment output
console.log('# 🔍 TEST FILE LOADED - test.config.ts');
// Check if 00init.ts was loaded
const initLoaded = (global as any).__00INIT_LOADED;
console.log(`# 🔍 00init.ts loaded: ${initLoaded === true}`);
// Test that uses the global timeout setting
tap.test('Test with global timeout', async (toolsArg) => {
// This test should complete within the 5 second timeout set in 00init.ts
await toolsArg.delayFor(2000); // 2 seconds
expect(true).toBeTrue();
});
// Test that demonstrates retries
tap.test('Test with retries', async () => {
// This test will use the global retry setting (2 retries)
console.log('Running test that might be flaky');
// Simulate a flaky test that passes on second try
const randomValue = Math.random();
console.log(`Random value: ${randomValue}`);
// Always pass for demonstration
expect(true).toBeTrue();
});
// Test with custom timeout that overrides global
tap.timeout(1000).test('Test with custom timeout', async (toolsArg) => {
// This test has a 1 second timeout, overriding the global 5 seconds
await toolsArg.delayFor(500); // 500ms - should pass
expect(true).toBeTrue();
});
// Test to verify lifecycle hooks are working
tap.test('Test lifecycle hooks', async () => {
console.log('Inside test: lifecycle hooks should have run');
expect(true).toBeTrue();
});
// Start the test suite
tap.start();

View File

@@ -0,0 +1,22 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
// Override global settings for this file
tap.settings({
timeout: 2000, // Override global timeout to 2 seconds
retries: 0, // Disable retries for this file
});
tap.test('Test with file-level timeout', async (toolsArg) => {
// This should use the file-level timeout of 2 seconds
console.log('Running with file-level timeout of 2 seconds');
await toolsArg.delayFor(1000); // 1 second - should pass
expect(true).toBeTrue();
});
tap.test('Test without retries', async () => {
// This test should not retry even if it fails
console.log('This test has no retries (file-level setting)');
expect(true).toBeTrue();
});
tap.start();

View File

@@ -0,0 +1,8 @@
import { tap } from '../../ts_tapbundle/index.js';
tap.test('spec file test', async () => {
console.log('This is a .spec.ts file that should be found by glob');
return true;
});
tap.start();

View File

@@ -0,0 +1,8 @@
import { tap } from '../../../ts_tapbundle/index.js';
tap.test('nested glob pattern test', async () => {
console.log('This test file is in a nested directory');
return true;
});
tap.start();

View File

@@ -0,0 +1,8 @@
import { tap } from '../../ts_tapbundle/index.js';
tap.test('glob pattern test', async () => {
console.log('This test file should be found by glob patterns');
return true;
});
tap.start();

View File

@@ -0,0 +1,167 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
// Create tests with known, distinct timing patterns to verify metrics calculation
tap.test('metric test 1 - 10ms baseline', async (tools) => {
await tools.delayFor(10);
expect(true).toBeTrue();
});
tap.test('metric test 2 - 20ms double baseline', async (tools) => {
await tools.delayFor(20);
expect(true).toBeTrue();
});
tap.test('metric test 3 - 30ms triple baseline', async (tools) => {
await tools.delayFor(30);
expect(true).toBeTrue();
});
tap.test('metric test 4 - 40ms quadruple baseline', async (tools) => {
await tools.delayFor(40);
expect(true).toBeTrue();
});
tap.test('metric test 5 - 50ms quintuple baseline', async (tools) => {
await tools.delayFor(50);
expect(true).toBeTrue();
});
// Test that should be the slowest
tap.test('metric test slowest - 200ms intentionally slow', async (tools) => {
await tools.delayFor(200);
expect(true).toBeTrue();
});
// Tests to verify edge cases in average calculation
tap.test('metric test fast 1 - minimal work', async () => {
expect(1).toEqual(1);
});
tap.test('metric test fast 2 - minimal work', async () => {
expect(2).toEqual(2);
});
tap.test('metric test fast 3 - minimal work', async () => {
expect(3).toEqual(3);
});
// Test to verify that failed tests still contribute to timing metrics
tap.test('metric test that fails - 60ms before failure', async (tools) => {
await tools.delayFor(60);
expect(true).toBeFalse(); // This will fail
});
// Describe block with timing to test aggregation
tap.describe('performance metrics in describe block', () => {
tap.test('described test 1 - 15ms', async (tools) => {
await tools.delayFor(15);
expect(true).toBeTrue();
});
tap.test('described test 2 - 25ms', async (tools) => {
await tools.delayFor(25);
expect(true).toBeTrue();
});
tap.test('described test 3 - 35ms', async (tools) => {
await tools.delayFor(35);
expect(true).toBeTrue();
});
});
// Test timing with hooks
tap.describe('performance with hooks', () => {
let hookTime = 0;
tap.beforeEach(async () => {
// Hooks shouldn't count toward test time
await new Promise(resolve => setTimeout(resolve, 10));
hookTime += 10;
});
tap.afterEach(async () => {
// Hooks shouldn't count toward test time
await new Promise(resolve => setTimeout(resolve, 10));
hookTime += 10;
});
tap.test('test with hooks 1 - should only count test time', async (tools) => {
await tools.delayFor(30);
expect(true).toBeTrue();
// Test time should be ~30ms, not 50ms (including hooks)
});
tap.test('test with hooks 2 - should only count test time', async (tools) => {
await tools.delayFor(40);
expect(true).toBeTrue();
// Test time should be ~40ms, not 60ms (including hooks)
});
});
// Parallel tests to verify timing is captured correctly
tap.describe('parallel timing verification', () => {
const startTimes: Map<string, number> = new Map();
const endTimes: Map<string, number> = new Map();
tap.testParallel('parallel metric 1 - 80ms', async (tools) => {
startTimes.set('p1', Date.now());
await tools.delayFor(80);
endTimes.set('p1', Date.now());
expect(true).toBeTrue();
});
tap.testParallel('parallel metric 2 - 90ms', async (tools) => {
startTimes.set('p2', Date.now());
await tools.delayFor(90);
endTimes.set('p2', Date.now());
expect(true).toBeTrue();
});
tap.testParallel('parallel metric 3 - 100ms', async (tools) => {
startTimes.set('p3', Date.now());
await tools.delayFor(100);
endTimes.set('p3', Date.now());
expect(true).toBeTrue();
});
tap.test('verify parallel execution', async () => {
// This test runs after parallel tests
// Verify they actually ran in parallel by checking overlapping times
if (startTimes.size === 3 && endTimes.size === 3) {
const p1Start = startTimes.get('p1')!;
const p2Start = startTimes.get('p2')!;
const p3Start = startTimes.get('p3')!;
const p1End = endTimes.get('p1')!;
const p2End = endTimes.get('p2')!;
const p3End = endTimes.get('p3')!;
// Start times should be very close (within 50ms)
expect(Math.abs(p1Start - p2Start)).toBeLessThan(50);
expect(Math.abs(p2Start - p3Start)).toBeLessThan(50);
// There should be overlap in execution
const p1Overlaps = p1Start < p2End && p1End > p2Start;
const p2Overlaps = p2Start < p3End && p2End > p3Start;
expect(p1Overlaps || p2Overlaps).toBeTrue();
} else {
// Skip verification if parallel tests didn't run yet
expect(true).toBeTrue();
}
});
});
// Test to ensure average calculation handles mixed timing correctly
tap.test('final metrics test - 5ms minimal', async (tools) => {
await tools.delayFor(5);
expect(true).toBeTrue();
console.log('\n📊 Expected Performance Metrics Summary:');
console.log('- Tests include a mix of durations from <1ms to 200ms');
console.log('- Slowest test should be "metric test slowest" at ~200ms');
console.log('- Average should be calculated from individual test times');
console.log('- Failed test should still contribute its 60ms to timing');
console.log('- Parallel tests should show their individual times (80ms, 90ms, 100ms)');
});
tap.start();

View File

@@ -0,0 +1,214 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
tap.test('ultra-fast test - should capture sub-millisecond timing', async () => {
// This test does almost nothing, should complete in < 1ms
const x = 1 + 1;
expect(x).toEqual(2);
});
tap.test('test with exact 1ms delay', async (tools) => {
const start = Date.now();
await tools.delayFor(1);
const elapsed = Date.now() - start;
// Should be at least 1ms but could be more due to event loop
expect(elapsed).toBeGreaterThanOrEqual(1);
});
tap.test('test with 10ms delay', async (tools) => {
await tools.delayFor(10);
expect(true).toBeTrue();
});
tap.test('test with 100ms delay', async (tools) => {
await tools.delayFor(100);
expect(true).toBeTrue();
});
tap.test('test with 250ms delay', async (tools) => {
await tools.delayFor(250);
expect(true).toBeTrue();
});
tap.test('test with 500ms delay', async (tools) => {
await tools.delayFor(500);
expect(true).toBeTrue();
});
tap.test('test with variable processing time', async (tools) => {
// Simulate variable processing
const iterations = 1000000;
let sum = 0;
for (let i = 0; i < iterations; i++) {
sum += Math.sqrt(i);
}
expect(sum).toBeGreaterThan(0);
// Add a small delay to ensure measurable time
await tools.delayFor(5);
});
tap.test('test with multiple async operations', async () => {
// Multiple promises in parallel
const results = await Promise.all([
new Promise(resolve => setTimeout(() => resolve(1), 10)),
new Promise(resolve => setTimeout(() => resolve(2), 20)),
new Promise(resolve => setTimeout(() => resolve(3), 30))
]);
expect(results).toEqual([1, 2, 3]);
// This should take at least 30ms (the longest delay)
});
tap.test('test with synchronous heavy computation', async () => {
// Heavy synchronous computation
const fibonacci = (n: number): number => {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
};
// Calculate fibonacci(30) - should take measurable time
const result = fibonacci(30);
expect(result).toEqual(832040);
});
// Test with retry to see if timing accumulates correctly
tap.retry(2).test('test with retry - fails first then passes', async (tools) => {
// Get or initialize retry count
const retryCount = tools.context.get('retryCount') || 0;
tools.context.set('retryCount', retryCount + 1);
await tools.delayFor(50);
if (retryCount === 0) {
throw new Error('First attempt fails');
}
expect(retryCount).toEqual(1);
});
// Test timeout handling
tap.timeout(100).test('test with timeout - should complete just in time', async (tools) => {
await tools.delayFor(80); // Just under the timeout
expect(true).toBeTrue();
});
// Skip test - should show 0ms
tap.skip.test('skipped test - should report 0ms', async (tools) => {
await tools.delayFor(1000); // This won't execute
expect(true).toBeTrue();
});
// Todo test - should show 0ms
tap.todo.test('todo test - should report 0ms', async (tools) => {
await tools.delayFor(1000); // This won't execute
expect(true).toBeTrue();
});
// Test with skip inside
tap.test('test that skips conditionally - should show time until skip', async (tools) => {
await tools.delayFor(25);
const shouldSkip = true;
if (shouldSkip) {
tools.skip('Skipping after 25ms');
}
// This won't execute
await tools.delayFor(1000);
expect(true).toBeTrue();
});
// Test with very precise timing
tap.test('test with precise timing measurements', async (tools) => {
const measurements: number[] = [];
for (let i = 0; i < 5; i++) {
const start = process.hrtime.bigint();
await tools.delayFor(10);
const end = process.hrtime.bigint();
const durationMs = Number(end - start) / 1_000_000;
measurements.push(durationMs);
}
// All measurements should be at least 10ms
measurements.forEach(m => {
expect(m).toBeGreaterThanOrEqual(10);
});
// But not too much more (accounting for timer precision)
measurements.forEach(m => {
expect(m).toBeLessThan(20);
});
});
// Test that intentionally has 0 actual work
tap.test('empty test - absolute minimum execution time', async () => {
// Literally nothing
});
// Test with promise that resolves immediately
tap.test('test with immediate promise resolution', async () => {
await Promise.resolve();
expect(true).toBeTrue();
});
// Test with microtask queue
tap.test('test with microtask queue processing', async () => {
let value = 0;
await Promise.resolve().then(() => {
value = 1;
return Promise.resolve();
}).then(() => {
value = 2;
return Promise.resolve();
}).then(() => {
value = 3;
});
expect(value).toEqual(3);
});
// Test to verify timing accumulation in describe blocks
tap.describe('timing in describe blocks', () => {
let startTime: number;
tap.beforeEach(async () => {
startTime = Date.now();
await new Promise(resolve => setTimeout(resolve, 5));
});
tap.afterEach(async () => {
await new Promise(resolve => setTimeout(resolve, 5));
});
tap.test('first test in describe', async (tools) => {
await tools.delayFor(10);
const elapsed = Date.now() - startTime;
expect(elapsed).toBeGreaterThanOrEqual(10);
});
tap.test('second test in describe', async (tools) => {
await tools.delayFor(20);
const elapsed = Date.now() - startTime;
expect(elapsed).toBeGreaterThanOrEqual(20);
});
});
// Parallel tests to see timing differences
tap.testParallel('parallel test 1 - 100ms', async (tools) => {
await tools.delayFor(100);
expect(true).toBeTrue();
});
tap.testParallel('parallel test 2 - 50ms', async (tools) => {
await tools.delayFor(50);
expect(true).toBeTrue();
});
tap.testParallel('parallel test 3 - 150ms', async (tools) => {
await tools.delayFor(150);
expect(true).toBeTrue();
});
tap.start();

View File

@@ -0,0 +1,204 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
import { ProtocolParser, ProtocolEmitter } from '../../ts_tapbundle_protocol/index.js';
// Test the protocol's ability to emit and parse timing metadata
tap.test('protocol should correctly emit timing metadata', async () => {
const emitter = new ProtocolEmitter();
const testResult = {
ok: true,
testNumber: 1,
description: 'test with timing',
metadata: {
time: 123
}
};
const lines = emitter.emitTest(testResult);
// Should have inline timing metadata
expect(lines.length).toEqual(1);
expect(lines[0]).toInclude('⟦TSTEST:time:123⟧');
});
tap.test('protocol should correctly parse timing metadata', async () => {
const parser = new ProtocolParser();
const line = 'ok 1 - test with timing ⟦TSTEST:time:456⟧';
const messages = parser.parseLine(line);
expect(messages.length).toEqual(1);
expect(messages[0].type).toEqual('test');
const content = messages[0].content as any;
expect(content.metadata).toBeDefined();
expect(content.metadata.time).toEqual(456);
});
tap.test('protocol should handle 0ms timing', async () => {
const parser = new ProtocolParser();
const line = 'ok 1 - ultra fast test ⟦TSTEST:time:0⟧';
const messages = parser.parseLine(line);
const content = messages[0].content as any;
expect(content.metadata.time).toEqual(0);
});
tap.test('protocol should handle large timing values', async () => {
const parser = new ProtocolParser();
const line = 'ok 1 - slow test ⟦TSTEST:time:999999⟧';
const messages = parser.parseLine(line);
const content = messages[0].content as any;
expect(content.metadata.time).toEqual(999999);
});
tap.test('protocol should handle timing with other metadata', async () => {
const emitter = new ProtocolEmitter();
const testResult = {
ok: true,
testNumber: 1,
description: 'complex test',
metadata: {
time: 789,
file: 'test.ts',
tags: ['slow', 'integration']
}
};
const lines = emitter.emitTest(testResult);
// Should use block metadata format for complex metadata
expect(lines.length).toBeGreaterThan(1);
expect(lines[1]).toInclude('META:');
expect(lines[1]).toInclude('"time":789');
});
tap.test('protocol should parse timing from block metadata', async () => {
const parser = new ProtocolParser();
const lines = [
'ok 1 - complex test',
'⟦TSTEST:META:{"time":321,"file":"test.ts"}⟧'
];
let testResult: any;
for (const line of lines) {
const messages = parser.parseLine(line);
if (messages.length > 0 && messages[0].type === 'test') {
testResult = messages[0].content;
}
}
expect(testResult).toBeDefined();
expect(testResult.metadata).toBeUndefined(); // Metadata comes separately in block format
});
tap.test('timing for skipped tests should be 0 or missing', async () => {
const emitter = new ProtocolEmitter();
const testResult = {
ok: true,
testNumber: 1,
description: 'skipped test',
directive: {
type: 'skip' as const,
reason: 'Not ready'
},
metadata: {
time: 0
}
};
const lines = emitter.emitTest(testResult);
expect(lines[0]).toInclude('# SKIP');
// If time is 0, it might be included or omitted
if (lines[0].includes('⟦TSTEST:')) {
expect(lines[0]).toInclude('time:0');
}
});
tap.test('protocol should handle fractional milliseconds', async () => {
const emitter = new ProtocolEmitter();
// Even though we use integers, test that protocol handles them correctly
const testResult = {
ok: true,
testNumber: 1,
description: 'precise test',
metadata: {
time: 123 // Protocol uses integers for milliseconds
}
};
const lines = emitter.emitTest(testResult);
expect(lines[0]).toInclude('time:123');
});
tap.test('protocol should handle timing in retry scenarios', async () => {
const emitter = new ProtocolEmitter();
const testResult = {
ok: true,
testNumber: 1,
description: 'retry test',
metadata: {
time: 200,
retry: 2
}
};
const lines = emitter.emitTest(testResult);
// Should include both time and retry
expect(lines[0]).toMatch(/time:200.*retry:2|retry:2.*time:200/);
});
// Test actual timing capture
tap.test('HrtMeasurement should capture accurate timing', async (tools) => {
// Import HrtMeasurement
const { HrtMeasurement } = await import('@push.rocks/smarttime');
const measurement = new HrtMeasurement();
measurement.start();
await tools.delayFor(50);
measurement.stop();
// Should be at least 50ms
expect(measurement.milliSeconds).toBeGreaterThanOrEqual(50);
// But not too much more (allow for some overhead)
expect(measurement.milliSeconds).toBeLessThan(100);
});
tap.test('multiple timing measurements should be independent', async (tools) => {
const { HrtMeasurement } = await import('@push.rocks/smarttime');
const measurement1 = new HrtMeasurement();
const measurement2 = new HrtMeasurement();
measurement1.start();
await tools.delayFor(25);
measurement2.start();
await tools.delayFor(25);
measurement1.stop();
await tools.delayFor(25);
measurement2.stop();
// measurement1 should be ~50ms (25ms + 25ms)
expect(measurement1.milliSeconds).toBeGreaterThanOrEqual(50);
expect(measurement1.milliSeconds).toBeLessThan(70);
// measurement2 should be ~50ms (25ms + 25ms)
expect(measurement2.milliSeconds).toBeGreaterThanOrEqual(50);
expect(measurement2.milliSeconds).toBeLessThan(70);
});
tap.start();

111
test/test.migration.node.ts Normal file
View File

@@ -0,0 +1,111 @@
import { expect, tap } from '../ts_tapbundle/index.js';
import { Migration } from '../ts/tstest.classes.migration.js';
import * as plugins from '../ts/tstest.plugins.js';
import * as paths from '../ts/tstest.paths.js';
tap.test('Migration - can initialize', async () => {
const migration = new Migration({
baseDir: process.cwd(),
dryRun: true,
});
expect(migration).toBeInstanceOf(Migration);
});
tap.test('Migration - findLegacyFiles returns empty for no legacy files', async () => {
const migration = new Migration({
baseDir: process.cwd(),
pattern: 'test/test.migration.node.ts', // This file itself, not legacy
dryRun: true,
});
const legacyFiles = await migration.findLegacyFiles();
expect(legacyFiles).toEqual([]);
});
tap.test('Migration - generateReport works', async () => {
const migration = new Migration({
baseDir: process.cwd(),
dryRun: true,
});
const report = await migration.generateReport();
expect(report).toBeTypeOf('string');
expect(report).toContain('Test File Migration Report');
});
tap.test('Migration - detects legacy files when they exist', async () => {
// Create a temporary legacy test file
const tempDir = plugins.path.join(process.cwd(), '.nogit', 'test_migration');
await plugins.smartfile.fs.ensureEmptyDir(tempDir);
const legacyFile = plugins.path.join(tempDir, 'test.browser.ts');
await plugins.smartfile.memory.toFs('// Legacy test file\nexport default Promise.resolve();', legacyFile);
const migration = new Migration({
baseDir: tempDir,
pattern: '**/*.ts',
dryRun: true,
});
const legacyFiles = await migration.findLegacyFiles();
expect(legacyFiles.length).toEqual(1);
expect(legacyFiles[0]).toContain('test.browser.ts');
// Clean up
await plugins.smartfile.fs.removeSync(tempDir);
});
tap.test('Migration - detects both legacy pattern', async () => {
// Create temporary legacy files
const tempDir = plugins.path.join(process.cwd(), '.nogit', 'test_migration_both');
await plugins.smartfile.fs.ensureEmptyDir(tempDir);
const browserFile = plugins.path.join(tempDir, 'test.browser.ts');
const bothFile = plugins.path.join(tempDir, 'test.both.ts');
await plugins.smartfile.memory.toFs('// Browser test\nexport default Promise.resolve();', browserFile);
await plugins.smartfile.memory.toFs('// Both test\nexport default Promise.resolve();', bothFile);
const migration = new Migration({
baseDir: tempDir,
pattern: '**/*.ts',
dryRun: true,
});
const legacyFiles = await migration.findLegacyFiles();
expect(legacyFiles.length).toEqual(2);
// Clean up
await plugins.smartfile.fs.removeSync(tempDir);
});
tap.test('Migration - dry run does not modify files', async () => {
// Create a temporary legacy test file
const tempDir = plugins.path.join(process.cwd(), '.nogit', 'test_migration_dryrun');
await plugins.smartfile.fs.ensureEmptyDir(tempDir);
const legacyFile = plugins.path.join(tempDir, 'test.browser.ts');
await plugins.smartfile.memory.toFs('// Legacy test file\nexport default Promise.resolve();', legacyFile);
const migration = new Migration({
baseDir: tempDir,
pattern: '**/*.ts',
dryRun: true,
verbose: false,
});
const summary = await migration.run();
expect(summary.dryRun).toEqual(true);
expect(summary.totalLegacyFiles).toEqual(1);
expect(summary.migratedCount).toEqual(1); // Dry run still counts as "would migrate"
// Verify original file still exists
const fileExists = await plugins.smartfile.fs.fileExists(legacyFile);
expect(fileExists).toEqual(true);
// Clean up
await plugins.smartfile.fs.removeSync(tempDir);
});
export default tap.start();

View File

@@ -0,0 +1,167 @@
import { expect, tap } from '../ts_tapbundle/index.js';
import { parseTestFilename, isLegacyFilename, getLegacyMigrationTarget } from '../ts/tstest.classes.runtime.parser.js';
tap.test('parseTestFilename - single runtime', async () => {
const parsed = parseTestFilename('test.node.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['node']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - chromium runtime', async () => {
const parsed = parseTestFilename('test.chromium.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['chromium']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - multiple runtimes', async () => {
const parsed = parseTestFilename('test.node+chromium.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['node', 'chromium']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - deno+bun runtime', async () => {
const parsed = parseTestFilename('test.deno+bun.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['deno', 'bun']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - with nonci modifier', async () => {
const parsed = parseTestFilename('test.chromium.nonci.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['chromium']);
expect(parsed.modifiers).toEqual(['nonci']);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - multi-runtime with nonci', async () => {
const parsed = parseTestFilename('test.node+chromium.nonci.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['node', 'chromium']);
expect(parsed.modifiers).toEqual(['nonci']);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - legacy browser', async () => {
const parsed = parseTestFilename('test.browser.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['chromium']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(true);
});
tap.test('parseTestFilename - legacy both', async () => {
const parsed = parseTestFilename('test.both.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['node', 'chromium']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(true);
});
tap.test('parseTestFilename - legacy browser with nonci', async () => {
const parsed = parseTestFilename('test.browser.nonci.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['chromium']);
expect(parsed.modifiers).toEqual(['nonci']);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(true);
});
tap.test('parseTestFilename - complex basename', async () => {
const parsed = parseTestFilename('test.some.feature.node.ts');
expect(parsed.baseName).toEqual('test.some.feature');
expect(parsed.runtimes).toEqual(['node']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - default to node when no runtime', async () => {
const parsed = parseTestFilename('test.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['node']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - tsx extension', async () => {
const parsed = parseTestFilename('test.chromium.tsx');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['chromium']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('tsx');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('parseTestFilename - deduplicates runtime tokens', async () => {
const parsed = parseTestFilename('test.node+node.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['node']);
expect(parsed.modifiers).toEqual([]);
expect(parsed.extension).toEqual('ts');
expect(parsed.isLegacy).toEqual(false);
});
tap.test('isLegacyFilename - detects browser', async () => {
expect(isLegacyFilename('test.browser.ts')).toEqual(true);
});
tap.test('isLegacyFilename - detects both', async () => {
expect(isLegacyFilename('test.both.ts')).toEqual(true);
});
tap.test('isLegacyFilename - rejects new naming', async () => {
expect(isLegacyFilename('test.node.ts')).toEqual(false);
expect(isLegacyFilename('test.chromium.ts')).toEqual(false);
expect(isLegacyFilename('test.node+chromium.ts')).toEqual(false);
});
tap.test('getLegacyMigrationTarget - browser to chromium', async () => {
const target = getLegacyMigrationTarget('test.browser.ts');
expect(target).toEqual('test.chromium.ts');
});
tap.test('getLegacyMigrationTarget - both to node+chromium', async () => {
const target = getLegacyMigrationTarget('test.both.ts');
expect(target).toEqual('test.node+chromium.ts');
});
tap.test('getLegacyMigrationTarget - browser with nonci', async () => {
const target = getLegacyMigrationTarget('test.browser.nonci.ts');
expect(target).toEqual('test.chromium.nonci.ts');
});
tap.test('getLegacyMigrationTarget - both with nonci', async () => {
const target = getLegacyMigrationTarget('test.both.nonci.ts');
expect(target).toEqual('test.node+chromium.nonci.ts');
});
tap.test('getLegacyMigrationTarget - returns null for non-legacy', async () => {
const target = getLegacyMigrationTarget('test.node.ts');
expect(target).toEqual(null);
});
tap.test('parseTestFilename - handles full paths', async () => {
const parsed = parseTestFilename('/path/to/test.node+chromium.ts');
expect(parsed.baseName).toEqual('test');
expect(parsed.runtimes).toEqual(['node', 'chromium']);
expect(parsed.original).toEqual('test.node+chromium.ts');
});
export default tap.start();

View File

@@ -0,0 +1,17 @@
import { tap, expect } from '../../ts_tapbundle/index.js';
// This test file demonstrates watch mode
// Try modifying this file while running: tstest test/watch-demo --watch
let counter = 1;
tap.test('demo test that changes', async () => {
expect(counter).toEqual(1);
console.log(`Test run at: ${new Date().toISOString()}`);
});
tap.test('another test', async () => {
expect('hello').toEqual('hello');
});
tap.start();

View File

@@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@git.zone/tstest', name: '@git.zone/tstest',
version: '1.7.0', version: '2.4.3',
description: 'a test utility to run tests that match test/**/*.ts' description: 'a test utility to run tests that match test/**/*.ts'
} }

View File

@@ -8,17 +8,68 @@ export enum TestExecutionMode {
} }
export const runCli = async () => { export const runCli = async () => {
// Check if we're using global tstest in the tstest project itself
try {
const packageJsonPath = `${process.cwd()}/package.json`;
const fs = await import('fs');
if (fs.existsSync(packageJsonPath)) {
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
if (packageJson.name === '@git.zone/tstest') {
// Check if we're running from a global installation
const execPath = process.argv[1];
// Debug: log the paths (uncomment for debugging)
// console.log('DEBUG: Checking global tstest usage...');
// console.log('execPath:', execPath);
// console.log('cwd:', process.cwd());
// console.log('process.argv:', process.argv);
// Check if this is running from global installation
const isLocalCli = execPath.includes(process.cwd());
const isGlobalPnpm = process.argv.some(arg => arg.includes('.pnpm') && !arg.includes(process.cwd()));
const isGlobalNpm = process.argv.some(arg => arg.includes('npm/node_modules') && !arg.includes(process.cwd()));
if (!isLocalCli && (isGlobalPnpm || isGlobalNpm || !execPath.includes('node_modules'))) {
console.error('\n⚠ WARNING: You are using a globally installed tstest in the tstest project itself!');
console.error(' This means you are NOT testing your local changes.');
console.error(' Please use one of these commands instead:');
console.error(' • node cli.js <test-path>');
console.error(' • pnpm test <test-path>');
console.error(' • ./cli.js <test-path> (if executable)\n');
}
}
}
} catch (error) {
// Silently ignore any errors in this check
}
// Parse command line arguments // Parse command line arguments
const args = process.argv.slice(2); const args = process.argv.slice(2);
const logOptions: LogOptions = {}; const logOptions: LogOptions = {};
let testPath: string | null = null; let testPath: string | null = null;
let tags: string[] = []; let tags: string[] = [];
let startFromFile: number | null = null;
let stopAtFile: number | null = null;
let timeoutSeconds: number | null = null;
let watchMode: boolean = false;
let watchIgnorePatterns: string[] = [];
// Parse options // Parse options
for (let i = 0; i < args.length; i++) { for (let i = 0; i < args.length; i++) {
const arg = args[i]; const arg = args[i];
switch (arg) { switch (arg) {
case '--version':
// Get version from package.json
try {
const fs = await import('fs');
const packagePath = new URL('../package.json', import.meta.url).pathname;
const packageData = JSON.parse(await fs.promises.readFile(packagePath, 'utf8'));
console.log(`tstest version ${packageData.version}`);
} catch (error) {
console.log('tstest version unknown');
}
process.exit(0);
break;
case '--quiet': case '--quiet':
case '-q': case '-q':
logOptions.quiet = true; logOptions.quiet = true;
@@ -42,6 +93,57 @@ export const runCli = async () => {
tags = args[++i].split(','); tags = args[++i].split(',');
} }
break; break;
case '--startFrom':
if (i + 1 < args.length) {
const value = parseInt(args[++i], 10);
if (isNaN(value) || value < 1) {
console.error('Error: --startFrom must be a positive integer');
process.exit(1);
}
startFromFile = value;
} else {
console.error('Error: --startFrom requires a number argument');
process.exit(1);
}
break;
case '--stopAt':
if (i + 1 < args.length) {
const value = parseInt(args[++i], 10);
if (isNaN(value) || value < 1) {
console.error('Error: --stopAt must be a positive integer');
process.exit(1);
}
stopAtFile = value;
} else {
console.error('Error: --stopAt requires a number argument');
process.exit(1);
}
break;
case '--timeout':
if (i + 1 < args.length) {
const value = parseInt(args[++i], 10);
if (isNaN(value) || value < 1) {
console.error('Error: --timeout must be a positive integer (seconds)');
process.exit(1);
}
timeoutSeconds = value;
} else {
console.error('Error: --timeout requires a number argument (seconds)');
process.exit(1);
}
break;
case '--watch':
case '-w':
watchMode = true;
break;
case '--watch-ignore':
if (i + 1 < args.length) {
watchIgnorePatterns = args[++i].split(',');
} else {
console.error('Error: --watch-ignore requires a comma-separated list of patterns');
process.exit(1);
}
break;
default: default:
if (!arg.startsWith('-')) { if (!arg.startsWith('-')) {
testPath = arg; testPath = arg;
@@ -49,16 +151,28 @@ export const runCli = async () => {
} }
} }
// Validate test file range options
if (startFromFile !== null && stopAtFile !== null && startFromFile > stopAtFile) {
console.error('Error: --startFrom cannot be greater than --stopAt');
process.exit(1);
}
if (!testPath) { if (!testPath) {
console.error('You must specify a test directory/file/pattern as argument. Please try again.'); console.error('You must specify a test directory/file/pattern as argument. Please try again.');
console.error('\nUsage: tstest <path> [options]'); console.error('\nUsage: tstest <path> [options]');
console.error('\nOptions:'); console.error('\nOptions:');
console.error(' --version Show version information');
console.error(' --quiet, -q Minimal output'); console.error(' --quiet, -q Minimal output');
console.error(' --verbose, -v Verbose output'); console.error(' --verbose, -v Verbose output');
console.error(' --no-color Disable colored output'); console.error(' --no-color Disable colored output');
console.error(' --json Output results as JSON'); console.error(' --json Output results as JSON');
console.error(' --logfile Write logs to .nogit/testlogs/[testfile].log'); console.error(' --logfile Write logs to .nogit/testlogs/[testfile].log');
console.error(' --tags Run only tests with specified tags (comma-separated)'); console.error(' --tags <tags> Run only tests with specified tags (comma-separated)');
console.error(' --startFrom <n> Start running from test file number n');
console.error(' --stopAt <n> Stop running at test file number n');
console.error(' --timeout <s> Timeout test files after s seconds');
console.error(' --watch, -w Watch for file changes and re-run tests');
console.error(' --watch-ignore Patterns to ignore in watch mode (comma-separated)');
process.exit(1); process.exit(1);
} }
@@ -73,6 +187,16 @@ export const runCli = async () => {
executionMode = TestExecutionMode.DIRECTORY; executionMode = TestExecutionMode.DIRECTORY;
} }
const tsTestInstance = new TsTest(process.cwd(), testPath, executionMode, logOptions, tags); const tsTestInstance = new TsTest(process.cwd(), testPath, executionMode, logOptions, tags, startFromFile, stopAtFile, timeoutSeconds);
if (watchMode) {
await tsTestInstance.runWatch(watchIgnorePatterns);
} else {
await tsTestInstance.run(); await tsTestInstance.run();
}
}; };
// Execute CLI when this file is run directly
if (import.meta.url === `file://${process.argv[1]}`) {
runCli();
}

View File

@@ -1,3 +1,3 @@
{ {
"order": 2 "order": 4
} }

View File

@@ -0,0 +1,316 @@
import * as plugins from './tstest.plugins.js';
import { coloredString as cs } from '@push.rocks/consolecolor';
import { parseTestFilename, getLegacyMigrationTarget, isLegacyFilename } from './tstest.classes.runtime.parser.js';
/**
* Migration result for a single file
*/
export interface MigrationResult {
/**
* Original file path
*/
oldPath: string;
/**
* New file path after migration
*/
newPath: string;
/**
* Whether the migration was performed
*/
migrated: boolean;
/**
* Error message if migration failed
*/
error?: string;
}
/**
* Migration summary
*/
export interface MigrationSummary {
/**
* Total number of legacy files found
*/
totalLegacyFiles: number;
/**
* Number of files successfully migrated
*/
migratedCount: number;
/**
* Number of files that failed to migrate
*/
errorCount: number;
/**
* Individual migration results
*/
results: MigrationResult[];
/**
* Whether this was a dry run
*/
dryRun: boolean;
}
/**
* Migration options
*/
export interface MigrationOptions {
/**
* Base directory to search for test files
* Default: process.cwd()
*/
baseDir?: string;
/**
* Glob pattern for finding test files
* Default: '** /*test*.ts' (without space)
*/
pattern?: string;
/**
* Dry run mode - don't actually rename files
* Default: true
*/
dryRun?: boolean;
/**
* Verbose output
* Default: false
*/
verbose?: boolean;
}
/**
* Migration class for renaming legacy test files to new naming convention
*
* Migrations:
* - .browser.ts → .chromium.ts
* - .both.ts → .node+chromium.ts
* - .both.nonci.ts → .node+chromium.nonci.ts
* - .browser.nonci.ts → .chromium.nonci.ts
*/
export class Migration {
private options: Required<MigrationOptions>;
constructor(options: MigrationOptions = {}) {
this.options = {
baseDir: options.baseDir || process.cwd(),
pattern: options.pattern || '**/test*.ts',
dryRun: options.dryRun !== undefined ? options.dryRun : true,
verbose: options.verbose || false,
};
}
/**
* Find all legacy test files in the base directory
*/
async findLegacyFiles(): Promise<string[]> {
const files = await plugins.smartfile.fs.listFileTree(
this.options.baseDir,
this.options.pattern
);
const legacyFiles: string[] = [];
for (const file of files) {
const fileName = plugins.path.basename(file);
if (isLegacyFilename(fileName)) {
const absolutePath = plugins.path.isAbsolute(file)
? file
: plugins.path.join(this.options.baseDir, file);
legacyFiles.push(absolutePath);
}
}
return legacyFiles;
}
/**
* Migrate a single file
*/
private async migrateFile(filePath: string): Promise<MigrationResult> {
const fileName = plugins.path.basename(filePath);
const dirName = plugins.path.dirname(filePath);
try {
// Get the new filename
const newFileName = getLegacyMigrationTarget(fileName);
if (!newFileName) {
return {
oldPath: filePath,
newPath: filePath,
migrated: false,
error: 'File is not a legacy file',
};
}
const newPath = plugins.path.join(dirName, newFileName);
// Check if target file already exists
if (await plugins.smartfile.fs.fileExists(newPath)) {
return {
oldPath: filePath,
newPath,
migrated: false,
error: `Target file already exists: ${newPath}`,
};
}
if (!this.options.dryRun) {
// Check if we're in a git repository
const isGitRepo = await this.isGitRepository(this.options.baseDir);
if (isGitRepo) {
// Use git mv to preserve history
const smartshell = new plugins.smartshell.Smartshell({
executor: 'bash',
pathDirectories: [],
});
const gitCommand = `cd "${this.options.baseDir}" && git mv "${filePath}" "${newPath}"`;
const result = await smartshell.exec(gitCommand);
if (result.exitCode !== 0) {
throw new Error(`git mv failed: ${result.stderr}`);
}
} else {
// Not a git repository - cannot migrate without git
throw new Error('Migration requires a git repository. We have git!');
}
}
return {
oldPath: filePath,
newPath,
migrated: true,
};
} catch (error) {
return {
oldPath: filePath,
newPath: filePath,
migrated: false,
error: error.message,
};
}
}
/**
* Check if a directory is a git repository
*/
private async isGitRepository(dir: string): Promise<boolean> {
try {
const gitDir = plugins.path.join(dir, '.git');
return await plugins.smartfile.fs.isDirectory(gitDir);
} catch {
return false;
}
}
/**
* Run the migration
*/
async run(): Promise<MigrationSummary> {
const legacyFiles = await this.findLegacyFiles();
console.log('');
console.log(cs('='.repeat(60), 'blue'));
console.log(cs('Test File Migration Tool', 'blue'));
console.log(cs('='.repeat(60), 'blue'));
console.log('');
if (this.options.dryRun) {
console.log(cs('🔍 DRY RUN MODE - No files will be modified', 'orange'));
console.log('');
}
console.log(`Found ${legacyFiles.length} legacy test file(s)`);
console.log('');
const results: MigrationResult[] = [];
let migratedCount = 0;
let errorCount = 0;
for (const file of legacyFiles) {
const result = await this.migrateFile(file);
results.push(result);
if (result.migrated) {
migratedCount++;
const oldName = plugins.path.basename(result.oldPath);
const newName = plugins.path.basename(result.newPath);
if (this.options.dryRun) {
console.log(cs(` Would migrate:`, 'cyan'));
} else {
console.log(cs(` ✓ Migrated:`, 'green'));
}
console.log(` ${oldName}`);
console.log(cs(`${newName}`, 'green'));
console.log('');
} else if (result.error) {
errorCount++;
console.log(cs(` ✗ Failed: ${plugins.path.basename(result.oldPath)}`, 'red'));
console.log(cs(` ${result.error}`, 'red'));
console.log('');
}
}
console.log(cs('='.repeat(60), 'blue'));
console.log(`Summary:`);
console.log(` Total legacy files: ${legacyFiles.length}`);
console.log(` Successfully migrated: ${migratedCount}`);
console.log(` Errors: ${errorCount}`);
console.log(cs('='.repeat(60), 'blue'));
if (this.options.dryRun && legacyFiles.length > 0) {
console.log('');
console.log(cs('To apply these changes, run:', 'orange'));
console.log(cs(' tstest migrate --write', 'orange'));
}
console.log('');
return {
totalLegacyFiles: legacyFiles.length,
migratedCount,
errorCount,
results,
dryRun: this.options.dryRun,
};
}
/**
* Create a migration report without performing the migration
*/
async generateReport(): Promise<string> {
const legacyFiles = await this.findLegacyFiles();
let report = '';
report += 'Test File Migration Report\n';
report += '='.repeat(60) + '\n';
report += '\n';
report += `Found ${legacyFiles.length} legacy test file(s)\n`;
report += '\n';
for (const file of legacyFiles) {
const fileName = plugins.path.basename(file);
const newFileName = getLegacyMigrationTarget(fileName);
if (newFileName) {
report += `${fileName}\n`;
report += `${newFileName}\n`;
report += '\n';
}
}
report += '='.repeat(60) + '\n';
return report;
}
}

View File

@@ -0,0 +1,245 @@
import * as plugins from './tstest.plugins.js';
import type { Runtime } from './tstest.classes.runtime.parser.js';
import { TapParser } from './tstest.classes.tap.parser.js';
/**
* Runtime-specific configuration options
*/
export interface RuntimeOptions {
/**
* Environment variables to pass to the runtime
*/
env?: Record<string, string>;
/**
* Additional command-line arguments
*/
extraArgs?: string[];
/**
* Working directory for test execution
*/
cwd?: string;
/**
* Timeout in milliseconds (0 = no timeout)
*/
timeout?: number;
}
/**
* Deno-specific configuration options
*/
export interface DenoOptions extends RuntimeOptions {
/**
* Permissions to grant to Deno
* Default: ['--allow-read', '--allow-env']
*/
permissions?: string[];
/**
* Path to deno.json config file
*/
configPath?: string;
/**
* Path to import map file
*/
importMap?: string;
}
/**
* Chromium-specific configuration options
*/
export interface ChromiumOptions extends RuntimeOptions {
/**
* Chromium launch arguments
*/
launchArgs?: string[];
/**
* Headless mode (default: true)
*/
headless?: boolean;
/**
* Port range for HTTP server
*/
portRange?: { min: number; max: number };
}
/**
* Command configuration returned by createCommand()
*/
export interface RuntimeCommand {
/**
* The main command executable (e.g., 'node', 'deno', 'bun')
*/
command: string;
/**
* Command-line arguments
*/
args: string[];
/**
* Environment variables
*/
env?: Record<string, string>;
/**
* Working directory
*/
cwd?: string;
}
/**
* Runtime availability check result
*/
export interface RuntimeAvailability {
/**
* Whether the runtime is available
*/
available: boolean;
/**
* Version string if available
*/
version?: string;
/**
* Error message if not available
*/
error?: string;
}
/**
* Abstract base class for runtime adapters
* Each runtime (Node, Chromium, Deno, Bun) implements this interface
*/
export abstract class RuntimeAdapter {
/**
* Runtime identifier
*/
abstract readonly id: Runtime;
/**
* Human-readable display name
*/
abstract readonly displayName: string;
/**
* Check if this runtime is available on the system
* @returns Availability information including version
*/
abstract checkAvailable(): Promise<RuntimeAvailability>;
/**
* Create the command configuration for executing a test
* @param testFile - Absolute path to the test file
* @param options - Runtime-specific options
* @returns Command configuration
*/
abstract createCommand(testFile: string, options?: RuntimeOptions): RuntimeCommand;
/**
* Execute a test file and return a TAP parser
* @param testFile - Absolute path to the test file
* @param index - Test index (for display)
* @param total - Total number of tests (for display)
* @param options - Runtime-specific options
* @returns TAP parser with test results
*/
abstract run(
testFile: string,
index: number,
total: number,
options?: RuntimeOptions
): Promise<TapParser>;
/**
* Get the default options for this runtime
* Can be overridden by subclasses
*/
protected getDefaultOptions(): RuntimeOptions {
return {
timeout: 0,
extraArgs: [],
env: {},
};
}
/**
* Merge user options with defaults
*/
protected mergeOptions<T extends RuntimeOptions>(userOptions?: T): T {
const defaults = this.getDefaultOptions();
return {
...defaults,
...userOptions,
env: { ...defaults.env, ...userOptions?.env },
extraArgs: [...(defaults.extraArgs || []), ...(userOptions?.extraArgs || [])],
} as T;
}
}
/**
* Registry for runtime adapters
* Manages all available runtime implementations
*/
export class RuntimeAdapterRegistry {
private adapters: Map<Runtime, RuntimeAdapter> = new Map();
/**
* Register a runtime adapter
*/
register(adapter: RuntimeAdapter): void {
this.adapters.set(adapter.id, adapter);
}
/**
* Get an adapter by runtime ID
*/
get(runtime: Runtime): RuntimeAdapter | undefined {
return this.adapters.get(runtime);
}
/**
* Get all registered adapters
*/
getAll(): RuntimeAdapter[] {
return Array.from(this.adapters.values());
}
/**
* Check which runtimes are available on the system
*/
async checkAvailability(): Promise<Map<Runtime, RuntimeAvailability>> {
const results = new Map<Runtime, RuntimeAvailability>();
for (const [runtime, adapter] of this.adapters) {
const availability = await adapter.checkAvailable();
results.set(runtime, availability);
}
return results;
}
/**
* Get adapters for a list of runtimes, in order
* @param runtimes - Ordered list of runtimes
* @returns Adapters in the same order, skipping any that aren't registered
*/
getAdaptersForRuntimes(runtimes: Runtime[]): RuntimeAdapter[] {
const adapters: RuntimeAdapter[] = [];
for (const runtime of runtimes) {
const adapter = this.get(runtime);
if (adapter) {
adapters.push(adapter);
}
}
return adapters;
}
}

View File

@@ -0,0 +1,219 @@
import * as plugins from './tstest.plugins.js';
import { coloredString as cs } from '@push.rocks/consolecolor';
import {
RuntimeAdapter,
type RuntimeOptions,
type RuntimeCommand,
type RuntimeAvailability,
} from './tstest.classes.runtime.adapter.js';
import { TapParser } from './tstest.classes.tap.parser.js';
import { TsTestLogger } from './tstest.logging.js';
import type { Runtime } from './tstest.classes.runtime.parser.js';
/**
* Bun runtime adapter
* Executes tests using the Bun runtime with native TypeScript support
*/
export class BunRuntimeAdapter extends RuntimeAdapter {
readonly id: Runtime = 'bun';
readonly displayName: string = 'Bun';
constructor(
private logger: TsTestLogger,
private smartshellInstance: any, // SmartShell instance from @push.rocks/smartshell
private timeoutSeconds: number | null,
private filterTags: string[]
) {
super();
}
/**
* Check if Bun is available
*/
async checkAvailable(): Promise<RuntimeAvailability> {
try {
const result = await this.smartshellInstance.exec('bun --version', {
cwd: process.cwd(),
onError: () => {
// Ignore error
}
});
if (result.exitCode !== 0) {
return {
available: false,
error: 'Bun not found. Install from: https://bun.sh/',
};
}
// Bun version is just the version number
const version = result.stdout.trim();
return {
available: true,
version: `Bun ${version}`,
};
} catch (error) {
return {
available: false,
error: error.message,
};
}
}
/**
* Create command configuration for Bun test execution
*/
createCommand(testFile: string, options?: RuntimeOptions): RuntimeCommand {
const mergedOptions = this.mergeOptions(options);
const args: string[] = ['run'];
// Add extra args
if (mergedOptions.extraArgs && mergedOptions.extraArgs.length > 0) {
args.push(...mergedOptions.extraArgs);
}
// Add test file
args.push(testFile);
// Set environment variables
const env = { ...mergedOptions.env };
if (this.filterTags.length > 0) {
env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
}
return {
command: 'bun',
args,
env,
cwd: mergedOptions.cwd,
};
}
/**
* Execute a test file in Bun
*/
async run(
testFile: string,
index: number,
total: number,
options?: RuntimeOptions
): Promise<TapParser> {
this.logger.testFileStart(testFile, this.displayName, index, total);
const tapParser = new TapParser(testFile + ':bun', this.logger);
const mergedOptions = this.mergeOptions(options);
// Build Bun command
const command = this.createCommand(testFile, mergedOptions);
const fullCommand = `${command.command} ${command.args.join(' ')}`;
// Set filter tags as environment variable
if (this.filterTags.length > 0) {
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
}
// Check for 00init.ts file in test directory
const testDir = plugins.path.dirname(testFile);
const initFile = plugins.path.join(testDir, '00init.ts');
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
let runCommand = fullCommand;
let loaderPath: string | null = null;
// If 00init.ts exists, create a loader file
if (initFileExists) {
const absoluteInitFile = plugins.path.resolve(initFile);
const absoluteTestFile = plugins.path.resolve(testFile);
const loaderContent = `
import '${absoluteInitFile.replace(/\\/g, '/')}';
import '${absoluteTestFile.replace(/\\/g, '/')}';
`;
loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(testFile)}`);
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
// Rebuild command with loader file
const loaderCommand = this.createCommand(loaderPath, mergedOptions);
runCommand = `${loaderCommand.command} ${loaderCommand.args.join(' ')}`;
}
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
// If we created a loader file, clean it up after test execution
if (loaderPath) {
const cleanup = () => {
try {
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
plugins.smartfile.fs.removeSync(loaderPath);
}
} catch (e) {
// Ignore cleanup errors
}
};
execResultStreaming.childProcess.on('exit', cleanup);
execResultStreaming.childProcess.on('error', cleanup);
}
// Start warning timer if no timeout was specified
let warningTimer: NodeJS.Timeout | null = null;
if (this.timeoutSeconds === null) {
warningTimer = setTimeout(() => {
console.error('');
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
console.error(cs(` File: ${testFile}`, 'orange'));
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
console.error('');
}, 60000); // 1 minute
}
// Handle timeout if specified
if (this.timeoutSeconds !== null) {
const timeoutMs = this.timeoutSeconds * 1000;
let timeoutId: NodeJS.Timeout;
const timeoutPromise = new Promise<void>((_resolve, reject) => {
timeoutId = setTimeout(async () => {
// Use smartshell's terminate() to kill entire process tree
await execResultStreaming.terminate();
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
}, timeoutMs);
});
try {
await Promise.race([
tapParser.handleTapProcess(execResultStreaming.childProcess),
timeoutPromise
]);
// Clear timeout if test completed successfully
clearTimeout(timeoutId);
} catch (error) {
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Handle timeout error
tapParser.handleTimeout(this.timeoutSeconds);
// Ensure entire process tree is killed if still running
try {
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
} catch (killError) {
// Process tree might already be dead
}
await tapParser.evaluateFinalResult();
}
} else {
await tapParser.handleTapProcess(execResultStreaming.childProcess);
}
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
return tapParser;
}
}

View File

@@ -0,0 +1,293 @@
import * as plugins from './tstest.plugins.js';
import * as paths from './tstest.paths.js';
import { coloredString as cs } from '@push.rocks/consolecolor';
import {
RuntimeAdapter,
type ChromiumOptions,
type RuntimeCommand,
type RuntimeAvailability,
} from './tstest.classes.runtime.adapter.js';
import { TapParser } from './tstest.classes.tap.parser.js';
import { TsTestLogger } from './tstest.logging.js';
import type { Runtime } from './tstest.classes.runtime.parser.js';
/**
* Chromium runtime adapter
* Executes tests in a headless Chromium browser
*/
export class ChromiumRuntimeAdapter extends RuntimeAdapter {
readonly id: Runtime = 'chromium';
readonly displayName: string = 'Chromium';
constructor(
private logger: TsTestLogger,
private tsbundleInstance: any, // TsBundle instance from @push.rocks/tsbundle
private smartbrowserInstance: any, // SmartBrowser instance from @push.rocks/smartbrowser
private timeoutSeconds: number | null
) {
super();
}
/**
* Check if Chromium is available
*/
async checkAvailable(): Promise<RuntimeAvailability> {
try {
// Check if smartbrowser is available and can start
// The browser binary is usually handled by @push.rocks/smartbrowser
return {
available: true,
version: 'Chromium (via smartbrowser)',
};
} catch (error) {
return {
available: false,
error: error.message || 'Chromium not available',
};
}
}
/**
* Create command configuration for Chromium test execution
* Note: Chromium tests don't use a traditional command, but this satisfies the interface
*/
createCommand(testFile: string, options?: ChromiumOptions): RuntimeCommand {
const mergedOptions = this.mergeOptions(options);
return {
command: 'chromium',
args: [],
env: mergedOptions.env,
cwd: mergedOptions.cwd,
};
}
/**
* Find free ports for HTTP server and WebSocket
*/
private async findFreePorts(): Promise<{ httpPort: number; wsPort: number }> {
const smartnetwork = new plugins.smartnetwork.SmartNetwork();
// Find random free HTTP port in range 30000-40000 to minimize collision chance
const httpPort = await smartnetwork.findFreePort(30000, 40000, { randomize: true });
if (!httpPort) {
throw new Error('Could not find a free HTTP port in range 30000-40000');
}
// Find random free WebSocket port, excluding the HTTP port to ensure they're different
const wsPort = await smartnetwork.findFreePort(30000, 40000, {
randomize: true,
exclude: [httpPort]
});
if (!wsPort) {
throw new Error('Could not find a free WebSocket port in range 30000-40000');
}
// Log selected ports for debugging
if (!this.logger.options.quiet) {
console.log(`Selected ports - HTTP: ${httpPort}, WebSocket: ${wsPort}`);
}
return { httpPort, wsPort };
}
/**
* Execute a test file in Chromium browser
*/
async run(
testFile: string,
index: number,
total: number,
options?: ChromiumOptions
): Promise<TapParser> {
this.logger.testFileStart(testFile, this.displayName, index, total);
// lets get all our paths sorted
const tsbundleCacheDirPath = plugins.path.join(paths.cwd, './.nogit/tstest_cache');
const bundleFileName = testFile.replace('/', '__') + '.js';
const bundleFilePath = plugins.path.join(tsbundleCacheDirPath, bundleFileName);
// lets bundle the test
await plugins.smartfile.fs.ensureEmptyDir(tsbundleCacheDirPath);
await this.tsbundleInstance.build(process.cwd(), testFile, bundleFilePath, {
bundler: 'esbuild',
});
// Find free ports for HTTP and WebSocket
const { httpPort, wsPort } = await this.findFreePorts();
// lets create a server
const server = new plugins.typedserver.servertools.Server({
cors: true,
port: httpPort,
});
server.addRoute(
'/test',
new plugins.typedserver.servertools.Handler('GET', async (_req, res) => {
res.type('.html');
res.write(`
<html>
<head>
<script>
globalThis.testdom = true;
globalThis.wsPort = ${wsPort};
</script>
</head>
<body></body>
</html>
`);
res.end();
})
);
server.addRoute('/*splat', new plugins.typedserver.servertools.HandlerStatic(tsbundleCacheDirPath));
await server.start();
// lets handle realtime comms
const tapParser = new TapParser(testFile + ':chrome', this.logger);
const wss = new plugins.ws.WebSocketServer({ port: wsPort });
wss.on('connection', (ws) => {
ws.on('message', (message) => {
const messageStr = message.toString();
if (messageStr.startsWith('console:')) {
const [, level, ...messageParts] = messageStr.split(':');
this.logger.browserConsole(messageParts.join(':'), level);
} else {
tapParser.handleTapLog(messageStr);
}
});
});
// lets do the browser bit with timeout handling
await this.smartbrowserInstance.start();
const evaluatePromise = this.smartbrowserInstance.evaluateOnPage(
`http://localhost:${httpPort}/test?bundleName=${bundleFileName}`,
async () => {
// lets enable real time comms
const ws = new WebSocket(`ws://localhost:${globalThis.wsPort}`);
await new Promise((resolve) => (ws.onopen = resolve));
// Ensure this function is declared with 'async'
const logStore = [];
const originalLog = console.log;
const originalError = console.error;
// Override console methods to capture the logs
console.log = (...args: any[]) => {
logStore.push(args.join(' '));
ws.send(args.join(' '));
originalLog(...args);
};
console.error = (...args: any[]) => {
logStore.push(args.join(' '));
ws.send(args.join(' '));
originalError(...args);
};
const bundleName = new URLSearchParams(window.location.search).get('bundleName');
originalLog(`::TSTEST IN CHROMIUM:: Relevant Script name is: ${bundleName}`);
try {
// Dynamically import the test module
const testModule = await import(`/${bundleName}`);
if (testModule && testModule.default && testModule.default instanceof Promise) {
// Execute the exported test function
await testModule.default;
} else if (testModule && testModule.default && typeof testModule.default.then === 'function') {
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
console.log('Test module default export is just promiselike: Something might be messing with your Promise implementation.');
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
await testModule.default;
} else if (globalThis.tapPromise && typeof globalThis.tapPromise.then === 'function') {
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
console.log('Using globalThis.tapPromise');
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
await testModule.default;
} else {
console.error('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
console.error('Test module does not export a default promise.');
console.error('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
console.log(`We got: ${JSON.stringify(testModule)}`);
}
} catch (err) {
console.error(err);
}
return logStore.join('\n');
}
);
// Start warning timer if no timeout was specified
let warningTimer: NodeJS.Timeout | null = null;
if (this.timeoutSeconds === null) {
warningTimer = setTimeout(() => {
console.error('');
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
console.error(cs(` File: ${testFile}`, 'orange'));
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
console.error('');
}, 60000); // 1 minute
}
// Handle timeout if specified
if (this.timeoutSeconds !== null) {
const timeoutMs = this.timeoutSeconds * 1000;
let timeoutId: NodeJS.Timeout;
const timeoutPromise = new Promise<void>((_resolve, reject) => {
timeoutId = setTimeout(() => {
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
}, timeoutMs);
});
try {
await Promise.race([
evaluatePromise,
timeoutPromise
]);
// Clear timeout if test completed successfully
clearTimeout(timeoutId);
} catch (error) {
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Handle timeout error
tapParser.handleTimeout(this.timeoutSeconds);
}
} else {
await evaluatePromise;
}
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Always clean up resources, even on timeout
try {
await this.smartbrowserInstance.stop();
} catch (error) {
// Browser might already be stopped
}
try {
await server.stop();
} catch (error) {
// Server might already be stopped
}
try {
wss.close();
} catch (error) {
// WebSocket server might already be closed
}
console.log(
`${cs('=> ', 'blue')} Stopped ${cs(testFile, 'orange')} chromium instance and server.`
);
// Always evaluate final result (handleTimeout just sets up the test state)
await tapParser.evaluateFinalResult();
return tapParser;
}
}

View File

@@ -0,0 +1,262 @@
import * as plugins from './tstest.plugins.js';
import { coloredString as cs } from '@push.rocks/consolecolor';
import {
RuntimeAdapter,
type DenoOptions,
type RuntimeCommand,
type RuntimeAvailability,
} from './tstest.classes.runtime.adapter.js';
import { TapParser } from './tstest.classes.tap.parser.js';
import { TsTestLogger } from './tstest.logging.js';
import type { Runtime } from './tstest.classes.runtime.parser.js';
/**
* Deno runtime adapter
* Executes tests using the Deno runtime
*/
export class DenoRuntimeAdapter extends RuntimeAdapter {
readonly id: Runtime = 'deno';
readonly displayName: string = 'Deno';
constructor(
private logger: TsTestLogger,
private smartshellInstance: any, // SmartShell instance from @push.rocks/smartshell
private timeoutSeconds: number | null,
private filterTags: string[]
) {
super();
}
/**
* Get default Deno options
*/
protected getDefaultOptions(): DenoOptions {
return {
...super.getDefaultOptions(),
permissions: [
'--allow-read',
'--allow-env',
'--allow-net',
'--allow-write',
'--allow-sys', // Allow system info access
'--allow-import', // Allow npm/node imports
'--node-modules-dir', // Enable Node.js compatibility mode
'--sloppy-imports', // Allow .js imports to resolve to .ts files
],
};
}
/**
* Check if Deno is available
*/
async checkAvailable(): Promise<RuntimeAvailability> {
try {
const result = await this.smartshellInstance.exec('deno --version', {
cwd: process.cwd(),
onError: () => {
// Ignore error
}
});
if (result.exitCode !== 0) {
return {
available: false,
error: 'Deno not found. Install from: https://deno.land/',
};
}
// Parse Deno version from output (first line is "deno X.Y.Z")
const versionMatch = result.stdout.match(/deno (\d+\.\d+\.\d+)/);
const version = versionMatch ? versionMatch[1] : 'unknown';
return {
available: true,
version: `Deno ${version}`,
};
} catch (error) {
return {
available: false,
error: error.message,
};
}
}
/**
* Create command configuration for Deno test execution
*/
createCommand(testFile: string, options?: DenoOptions): RuntimeCommand {
const mergedOptions = this.mergeOptions(options) as DenoOptions;
const args: string[] = ['run'];
// Add permissions
const permissions = mergedOptions.permissions || [
'--allow-read',
'--allow-env',
'--allow-net',
'--allow-write',
'--allow-sys',
'--allow-import',
'--node-modules-dir',
'--sloppy-imports',
];
args.push(...permissions);
// Add config file if specified
if (mergedOptions.configPath) {
args.push('--config', mergedOptions.configPath);
}
// Add import map if specified
if (mergedOptions.importMap) {
args.push('--import-map', mergedOptions.importMap);
}
// Add extra args
if (mergedOptions.extraArgs && mergedOptions.extraArgs.length > 0) {
args.push(...mergedOptions.extraArgs);
}
// Add test file
args.push(testFile);
// Set environment variables
const env = { ...mergedOptions.env };
if (this.filterTags.length > 0) {
env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
}
return {
command: 'deno',
args,
env,
cwd: mergedOptions.cwd,
};
}
/**
* Execute a test file in Deno
*/
async run(
testFile: string,
index: number,
total: number,
options?: DenoOptions
): Promise<TapParser> {
this.logger.testFileStart(testFile, this.displayName, index, total);
const tapParser = new TapParser(testFile + ':deno', this.logger);
const mergedOptions = this.mergeOptions(options) as DenoOptions;
// Build Deno command
const command = this.createCommand(testFile, mergedOptions);
const fullCommand = `${command.command} ${command.args.join(' ')}`;
// Set filter tags as environment variable
if (this.filterTags.length > 0) {
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
}
// Check for 00init.ts file in test directory
const testDir = plugins.path.dirname(testFile);
const initFile = plugins.path.join(testDir, '00init.ts');
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
let runCommand = fullCommand;
let loaderPath: string | null = null;
// If 00init.ts exists, create a loader file
if (initFileExists) {
const absoluteInitFile = plugins.path.resolve(initFile);
const absoluteTestFile = plugins.path.resolve(testFile);
const loaderContent = `
import '${absoluteInitFile.replace(/\\/g, '/')}';
import '${absoluteTestFile.replace(/\\/g, '/')}';
`;
loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(testFile)}`);
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
// Rebuild command with loader file
const loaderCommand = this.createCommand(loaderPath, mergedOptions);
runCommand = `${loaderCommand.command} ${loaderCommand.args.join(' ')}`;
}
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
// If we created a loader file, clean it up after test execution
if (loaderPath) {
const cleanup = () => {
try {
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
plugins.smartfile.fs.removeSync(loaderPath);
}
} catch (e) {
// Ignore cleanup errors
}
};
execResultStreaming.childProcess.on('exit', cleanup);
execResultStreaming.childProcess.on('error', cleanup);
}
// Start warning timer if no timeout was specified
let warningTimer: NodeJS.Timeout | null = null;
if (this.timeoutSeconds === null) {
warningTimer = setTimeout(() => {
console.error('');
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
console.error(cs(` File: ${testFile}`, 'orange'));
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
console.error('');
}, 60000); // 1 minute
}
// Handle timeout if specified
if (this.timeoutSeconds !== null) {
const timeoutMs = this.timeoutSeconds * 1000;
let timeoutId: NodeJS.Timeout;
const timeoutPromise = new Promise<void>((_resolve, reject) => {
timeoutId = setTimeout(async () => {
// Use smartshell's terminate() to kill entire process tree
await execResultStreaming.terminate();
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
}, timeoutMs);
});
try {
await Promise.race([
tapParser.handleTapProcess(execResultStreaming.childProcess),
timeoutPromise
]);
// Clear timeout if test completed successfully
clearTimeout(timeoutId);
} catch (error) {
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Handle timeout error
tapParser.handleTimeout(this.timeoutSeconds);
// Ensure entire process tree is killed if still running
try {
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
} catch (killError) {
// Process tree might already be dead
}
await tapParser.evaluateFinalResult();
}
} else {
await tapParser.handleTapProcess(execResultStreaming.childProcess);
}
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
return tapParser;
}
}

View File

@@ -0,0 +1,222 @@
import * as plugins from './tstest.plugins.js';
import { coloredString as cs } from '@push.rocks/consolecolor';
import {
RuntimeAdapter,
type RuntimeOptions,
type RuntimeCommand,
type RuntimeAvailability,
} from './tstest.classes.runtime.adapter.js';
import { TapParser } from './tstest.classes.tap.parser.js';
import { TsTestLogger } from './tstest.logging.js';
import type { Runtime } from './tstest.classes.runtime.parser.js';
/**
* Node.js runtime adapter
* Executes tests using tsrun (TypeScript runner for Node.js)
*/
export class NodeRuntimeAdapter extends RuntimeAdapter {
readonly id: Runtime = 'node';
readonly displayName: string = 'Node.js';
constructor(
private logger: TsTestLogger,
private smartshellInstance: any, // SmartShell instance from @push.rocks/smartshell
private timeoutSeconds: number | null,
private filterTags: string[]
) {
super();
}
/**
* Check if Node.js and tsrun are available
*/
async checkAvailable(): Promise<RuntimeAvailability> {
try {
// Check Node.js version
const nodeVersion = process.version;
// Check if tsrun is available
const result = await this.smartshellInstance.exec('tsrun --version', {
cwd: process.cwd(),
onError: () => {
// Ignore error
}
});
if (result.exitCode !== 0) {
return {
available: false,
error: 'tsrun not found. Install with: pnpm install --save-dev @git.zone/tsrun',
};
}
return {
available: true,
version: nodeVersion,
};
} catch (error) {
return {
available: false,
error: error.message,
};
}
}
/**
* Create command configuration for Node.js test execution
*/
createCommand(testFile: string, options?: RuntimeOptions): RuntimeCommand {
const mergedOptions = this.mergeOptions(options);
// Build tsrun options
const args: string[] = [];
if (process.argv.includes('--web')) {
args.push('--web');
}
// Add any extra args
if (mergedOptions.extraArgs) {
args.push(...mergedOptions.extraArgs);
}
// Set environment variables
const env = { ...mergedOptions.env };
if (this.filterTags.length > 0) {
env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
}
return {
command: 'tsrun',
args: [testFile, ...args],
env,
cwd: mergedOptions.cwd,
};
}
/**
* Execute a test file in Node.js
*/
async run(
testFile: string,
index: number,
total: number,
options?: RuntimeOptions
): Promise<TapParser> {
this.logger.testFileStart(testFile, this.displayName, index, total);
const tapParser = new TapParser(testFile + ':node', this.logger);
const mergedOptions = this.mergeOptions(options);
// Build tsrun command
let tsrunOptions = '';
if (process.argv.includes('--web')) {
tsrunOptions += ' --web';
}
// Set filter tags as environment variable
if (this.filterTags.length > 0) {
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
}
// Check for 00init.ts file in test directory
const testDir = plugins.path.dirname(testFile);
const initFile = plugins.path.join(testDir, '00init.ts');
let runCommand = `tsrun ${testFile}${tsrunOptions}`;
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
// If 00init.ts exists, run it first
let loaderPath: string | null = null;
if (initFileExists) {
// Create a temporary loader file that imports both 00init.ts and the test file
const absoluteInitFile = plugins.path.resolve(initFile);
const absoluteTestFile = plugins.path.resolve(testFile);
const loaderContent = `
import '${absoluteInitFile.replace(/\\/g, '/')}';
import '${absoluteTestFile.replace(/\\/g, '/')}';
`;
loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(testFile)}`);
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
runCommand = `tsrun ${loaderPath}${tsrunOptions}`;
}
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
// If we created a loader file, clean it up after test execution
if (loaderPath) {
const cleanup = () => {
try {
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
plugins.smartfile.fs.removeSync(loaderPath);
}
} catch (e) {
// Ignore cleanup errors
}
};
execResultStreaming.childProcess.on('exit', cleanup);
execResultStreaming.childProcess.on('error', cleanup);
}
// Start warning timer if no timeout was specified
let warningTimer: NodeJS.Timeout | null = null;
if (this.timeoutSeconds === null) {
warningTimer = setTimeout(() => {
console.error('');
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
console.error(cs(` File: ${testFile}`, 'orange'));
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
console.error('');
}, 60000); // 1 minute
}
// Handle timeout if specified
if (this.timeoutSeconds !== null) {
const timeoutMs = this.timeoutSeconds * 1000;
let timeoutId: NodeJS.Timeout;
const timeoutPromise = new Promise<void>((_resolve, reject) => {
timeoutId = setTimeout(async () => {
// Use smartshell's terminate() to kill entire process tree
await execResultStreaming.terminate();
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
}, timeoutMs);
});
try {
await Promise.race([
tapParser.handleTapProcess(execResultStreaming.childProcess),
timeoutPromise
]);
// Clear timeout if test completed successfully
clearTimeout(timeoutId);
} catch (error) {
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Handle timeout error
tapParser.handleTimeout(this.timeoutSeconds);
// Ensure entire process tree is killed if still running
try {
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
} catch (killError) {
// Process tree might already be dead
}
await tapParser.evaluateFinalResult();
}
} else {
await tapParser.handleTapProcess(execResultStreaming.childProcess);
}
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
return tapParser;
}
}

View File

@@ -0,0 +1,211 @@
/**
* Runtime parser for test file naming convention
* Supports: test.runtime1+runtime2.modifier.ts
* Examples:
* - test.node.ts
* - test.chromium.ts
* - test.node+chromium.ts
* - test.deno+bun.ts
* - test.chromium.nonci.ts
*/
export type Runtime = 'node' | 'chromium' | 'deno' | 'bun';
export type Modifier = 'nonci';
export interface ParsedFilename {
baseName: string;
runtimes: Runtime[];
modifiers: Modifier[];
extension: string;
isLegacy: boolean;
original: string;
}
export interface ParserConfig {
strictUnknownRuntime?: boolean; // default: true
defaultRuntimes?: Runtime[]; // default: ['node']
}
const KNOWN_RUNTIMES: Set<string> = new Set(['node', 'chromium', 'deno', 'bun']);
const KNOWN_MODIFIERS: Set<string> = new Set(['nonci']);
const VALID_EXTENSIONS: Set<string> = new Set(['ts', 'tsx', 'mts', 'cts']);
// Legacy mappings for backwards compatibility
const LEGACY_RUNTIME_MAP: Record<string, Runtime[]> = {
browser: ['chromium'],
both: ['node', 'chromium'],
};
/**
* Parse a test filename to extract runtimes, modifiers, and detect legacy patterns
* Algorithm: Right-to-left token analysis from the extension
*/
export function parseTestFilename(
filePath: string,
config: ParserConfig = {}
): ParsedFilename {
const strictUnknownRuntime = config.strictUnknownRuntime ?? true;
const defaultRuntimes = config.defaultRuntimes ?? ['node'];
// Extract just the filename from the path
const fileName = filePath.split('/').pop() || filePath;
const original = fileName;
// Step 1: Extract and validate extension
const lastDot = fileName.lastIndexOf('.');
if (lastDot === -1) {
throw new Error(`Invalid test file: no extension found in "${fileName}"`);
}
const extension = fileName.substring(lastDot + 1);
if (!VALID_EXTENSIONS.has(extension)) {
throw new Error(
`Invalid test file extension ".${extension}" in "${fileName}". ` +
`Valid extensions: ${Array.from(VALID_EXTENSIONS).join(', ')}`
);
}
// Step 2: Split remaining basename by dots
const withoutExtension = fileName.substring(0, lastDot);
const tokens = withoutExtension.split('.');
if (tokens.length === 0) {
throw new Error(`Invalid test file: empty basename in "${fileName}"`);
}
// Step 3: Parse from right to left
let isLegacy = false;
const modifiers: Modifier[] = [];
let runtimes: Runtime[] = [];
let runtimeTokenIndex = -1;
// Scan from right to left
for (let i = tokens.length - 1; i >= 0; i--) {
const token = tokens[i];
// Check if this is a known modifier
if (KNOWN_MODIFIERS.has(token)) {
modifiers.unshift(token as Modifier);
continue;
}
// Check if this is a legacy runtime token
if (LEGACY_RUNTIME_MAP[token]) {
isLegacy = true;
runtimes = LEGACY_RUNTIME_MAP[token];
runtimeTokenIndex = i;
break;
}
// Check if this is a runtime chain (may contain + separators)
if (token.includes('+')) {
const runtimeCandidates = token.split('+').map(r => r.trim()).filter(Boolean);
const validRuntimes: Runtime[] = [];
const invalidRuntimes: string[] = [];
for (const candidate of runtimeCandidates) {
if (KNOWN_RUNTIMES.has(candidate)) {
// Dedupe: only add if not already in list
if (!validRuntimes.includes(candidate as Runtime)) {
validRuntimes.push(candidate as Runtime);
}
} else {
invalidRuntimes.push(candidate);
}
}
if (invalidRuntimes.length > 0) {
if (strictUnknownRuntime) {
throw new Error(
`Unknown runtime(s) in "${fileName}": ${invalidRuntimes.join(', ')}. ` +
`Valid runtimes: ${Array.from(KNOWN_RUNTIMES).join(', ')}`
);
} else {
console.warn(
`⚠️ Warning: Unknown runtime(s) in "${fileName}": ${invalidRuntimes.join(', ')}. ` +
`Defaulting to: ${defaultRuntimes.join('+')}`
);
runtimes = [...defaultRuntimes];
runtimeTokenIndex = i;
break;
}
}
if (validRuntimes.length > 0) {
runtimes = validRuntimes;
runtimeTokenIndex = i;
break;
}
}
// Check if this is a single runtime token
if (KNOWN_RUNTIMES.has(token)) {
runtimes = [token as Runtime];
runtimeTokenIndex = i;
break;
}
// If we've scanned past modifiers and haven't found a runtime, stop looking
if (modifiers.length > 0) {
break;
}
}
// Step 4: Determine base name
// Everything before the runtime token (if found) is the base name
const baseNameTokens = runtimeTokenIndex >= 0 ? tokens.slice(0, runtimeTokenIndex) : tokens;
const baseName = baseNameTokens.join('.');
// Step 5: Apply defaults if no runtime was detected
if (runtimes.length === 0) {
runtimes = [...defaultRuntimes];
}
return {
baseName: baseName || 'test',
runtimes,
modifiers,
extension,
isLegacy,
original,
};
}
/**
* Check if a filename uses legacy naming convention
*/
export function isLegacyFilename(fileName: string): boolean {
const tokens = fileName.split('.');
for (const token of tokens) {
if (LEGACY_RUNTIME_MAP[token]) {
return true;
}
}
return false;
}
/**
* Get the suggested new filename for a legacy filename
*/
export function getLegacyMigrationTarget(fileName: string): string | null {
const parsed = parseTestFilename(fileName, { strictUnknownRuntime: false });
if (!parsed.isLegacy) {
return null;
}
// Reconstruct filename with new naming
const parts = [parsed.baseName];
if (parsed.runtimes.length > 0) {
parts.push(parsed.runtimes.join('+'));
}
if (parsed.modifiers.length > 0) {
parts.push(...parsed.modifiers);
}
parts.push(parsed.extension);
return parts.join('.');
}

View File

@@ -10,6 +10,7 @@ import { TsTestLogger } from './tstest.logging.js';
export class TapCombinator { export class TapCombinator {
tapParserStore: TapParser[] = []; tapParserStore: TapParser[] = [];
skippedFiles: string[] = [];
private logger: TsTestLogger; private logger: TsTestLogger;
constructor(logger: TsTestLogger) { constructor(logger: TsTestLogger) {
@@ -20,9 +21,13 @@ export class TapCombinator {
this.tapParserStore.push(tapParserArg); this.tapParserStore.push(tapParserArg);
} }
addSkippedFile(filename: string) {
this.skippedFiles.push(filename);
}
evaluate() { evaluate() {
// Call the logger's summary method // Call the logger's summary method with skipped files
this.logger.summary(); this.logger.summary(this.skippedFiles);
// Check for failures // Check for failures
let failGlobal = false; let failGlobal = false;

View File

@@ -8,28 +8,59 @@ import * as plugins from './tstest.plugins.js';
import { TapTestResult } from './tstest.classes.tap.testresult.js'; import { TapTestResult } from './tstest.classes.tap.testresult.js';
import * as logPrefixes from './tstest.logprefixes.js'; import * as logPrefixes from './tstest.logprefixes.js';
import { TsTestLogger } from './tstest.logging.js'; import { TsTestLogger } from './tstest.logging.js';
import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';
import type { IProtocolMessage, ITestResult, IPlanLine, IErrorBlock, ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
export class TapParser { export class TapParser {
testStore: TapTestResult[] = []; testStore: TapTestResult[] = [];
expectedTestsRegex = /([0-9]*)\.\.([0-9]*)$/; expectedTests: number = 0;
expectedTests: number; receivedTests: number = 0;
receivedTests: number;
testStatusRegex = /(ok|not\sok)\s([0-9]+)\s-\s(.*)(\s#\s(.*))?$/;
activeTapTestResult: TapTestResult; activeTapTestResult: TapTestResult;
collectingErrorDetails: boolean = false;
currentTestError: string[] = [];
pretaskRegex = /^::__PRETASK:(.*)$/;
private logger: TsTestLogger; private logger: TsTestLogger;
private protocolParser: ProtocolParser;
private protocolVersion: string | null = null;
private startTime: number;
/** /**
* the constructor for TapParser * the constructor for TapParser
*/ */
constructor(public fileName: string, logger?: TsTestLogger) { constructor(public fileName: string, logger?: TsTestLogger) {
this.logger = logger; this.logger = logger;
this.protocolParser = new ProtocolParser();
this.startTime = Date.now();
}
/**
* Handle test file timeout
*/
public handleTimeout(timeoutSeconds: number) {
// If no tests have been defined yet, set expected to 1
if (this.expectedTests === 0) {
this.expectedTests = 1;
}
// Create a fake failing test result for timeout
this._getNewTapTestResult();
this.activeTapTestResult.testOk = false;
this.activeTapTestResult.testSettled = true;
this.testStore.push(this.activeTapTestResult);
// Log the timeout error
if (this.logger) {
// First log the test result
this.logger.testResult(
`Test file timeout`,
false,
timeoutSeconds * 1000,
`Error: Test file exceeded timeout of ${timeoutSeconds} seconds`
);
this.logger.testErrorDetails(`Test execution was terminated after ${timeoutSeconds} seconds`);
}
// Don't call evaluateFinalResult here, let the caller handle it
} }
private _getNewTapTestResult() { private _getNewTapTestResult() {
@@ -45,141 +76,298 @@ export class TapParser {
logLineArray.pop(); logLineArray.pop();
} }
// lets parse the log information // Process each line through the protocol parser
for (const logLine of logLineArray) { for (const logLine of logLineArray) {
let logLineIsTapProtocol = false; const messages = this.protocolParser.parseLine(logLine);
if (!this.expectedTests && this.expectedTestsRegex.test(logLine)) {
logLineIsTapProtocol = true;
const regexResult = this.expectedTestsRegex.exec(logLine);
this.expectedTests = parseInt(regexResult[2]);
if (this.logger) {
this.logger.tapOutput(`Expecting ${this.expectedTests} tests!`);
}
// initiating first TapResult if (messages.length > 0) {
this._getNewTapTestResult(); // Handle protocol messages
} else if (this.pretaskRegex.test(logLine)) { for (const message of messages) {
logLineIsTapProtocol = true; this._handleProtocolMessage(message, logLine);
const pretaskContentMatch = this.pretaskRegex.exec(logLine);
if (pretaskContentMatch && pretaskContentMatch[1]) {
if (this.logger) {
this.logger.tapOutput(`Pretask -> ${pretaskContentMatch[1]}: Success.`);
}
}
} else if (this.testStatusRegex.test(logLine)) {
logLineIsTapProtocol = true;
const regexResult = this.testStatusRegex.exec(logLine);
const testId = parseInt(regexResult[2]);
const testOk = (() => {
if (regexResult[1] === 'ok') {
return true;
}
return false;
})();
const testSubject = regexResult[3];
const testMetadata = regexResult[5]; // This will be either "time=XXXms" or "SKIP reason" or "TODO reason"
let testDuration = 0;
let isSkipped = false;
let isTodo = false;
if (testMetadata) {
const timeMatch = testMetadata.match(/time=(\d+)ms/);
const skipMatch = testMetadata.match(/SKIP\s*(.*)/);
const todoMatch = testMetadata.match(/TODO\s*(.*)/);
if (timeMatch) {
testDuration = parseInt(timeMatch[1]);
} else if (skipMatch) {
isSkipped = true;
} else if (todoMatch) {
isTodo = true;
}
}
// test for protocol error - disabled as it's not critical
// The test ID mismatch can occur when tests are filtered, skipped, or use todo
// if (testId !== this.activeTapTestResult.id) {
// if (this.logger) {
// this.logger.error('Something is strange! Test Ids are not equal!');
// }
// }
this.activeTapTestResult.setTestResult(testOk);
if (testOk) {
if (this.logger) {
this.logger.testResult(testSubject, true, testDuration);
} }
} else { } else {
// Start collecting error details for failed test // Not a protocol message, handle as console output
this.collectingErrorDetails = true;
this.currentTestError = [];
if (this.logger) {
this.logger.testResult(testSubject, false, testDuration);
}
}
}
if (!logLineIsTapProtocol) {
if (this.activeTapTestResult) { if (this.activeTapTestResult) {
this.activeTapTestResult.addLogLine(logLine); this.activeTapTestResult.addLogLine(logLine);
} }
// Check for snapshot communication // Check for snapshot communication (legacy)
const snapshotMatch = logLine.match(/###SNAPSHOT###(.+)###SNAPSHOT###/); const snapshotMatch = logLine.match(/###SNAPSHOT###(.+)###SNAPSHOT###/);
if (snapshotMatch) { if (snapshotMatch) {
const base64Data = snapshotMatch[1]; const base64Data = snapshotMatch[1];
try { try {
const snapshotData = JSON.parse(Buffer.from(base64Data, 'base64').toString()); const snapshotData = JSON.parse(Buffer.from(base64Data, 'base64').toString());
this.handleSnapshot(snapshotData); this.handleSnapshot(snapshotData);
} catch (error) { } catch (error: any) {
if (this.logger) { if (this.logger) {
this.logger.testConsoleOutput(`Error parsing snapshot data: ${error.message}`); this.logger.testConsoleOutput(`Error parsing snapshot data: ${error.message}`);
} }
} }
} else { } else if (this.logger) {
// Check if we're collecting error details // This is console output from the test file
if (this.collectingErrorDetails) {
// Check if this line is an error detail (starts with Error: or has stack trace characteristics)
if (logLine.trim().startsWith('Error:') || logLine.trim().match(/^\s*at\s/)) {
this.currentTestError.push(logLine);
} else if (this.currentTestError.length > 0) {
// End of error details, show the error
const errorMessage = this.currentTestError.join('\n');
if (this.logger) {
this.logger.testErrorDetails(errorMessage);
}
this.collectingErrorDetails = false;
this.currentTestError = [];
}
}
// Don't output TAP error details as console output when we're collecting them
if (!this.collectingErrorDetails || (!logLine.trim().startsWith('Error:') && !logLine.trim().match(/^\s*at\s/))) {
if (this.logger) {
// This is console output from the test file, not TAP protocol
this.logger.testConsoleOutput(logLine); this.logger.testConsoleOutput(logLine);
} }
} }
} }
} }
if (this.activeTapTestResult && this.activeTapTestResult.testSettled) { private _handleProtocolMessage(message: IProtocolMessage, originalLine: string) {
// Ensure any pending error is shown before settling the test switch (message.type) {
if (this.collectingErrorDetails && this.currentTestError.length > 0) { case 'protocol':
const errorMessage = this.currentTestError.join('\n'); this.protocolVersion = message.content.version;
if (this.logger) { if (this.logger) {
this.logger.testErrorDetails(errorMessage); this.logger.tapOutput(`Protocol version: ${this.protocolVersion}`);
} }
this.collectingErrorDetails = false; break;
this.currentTestError = [];
case 'version':
// TAP version, we can ignore this
break;
case 'plan':
const plan = message.content as IPlanLine;
this.expectedTests = plan.end - plan.start + 1;
if (plan.skipAll) {
if (this.logger) {
this.logger.tapOutput(`Skipping all tests: ${plan.skipAll}`);
}
} else {
if (this.logger) {
this.logger.tapOutput(`Expecting ${this.expectedTests} tests!`);
}
}
// Initialize first TapResult
this._getNewTapTestResult();
break;
case 'test':
const testResult = message.content as ITestResult;
// Update active test result
this.activeTapTestResult.setTestResult(testResult.ok);
// Extract test duration from metadata
let testDuration = 0;
if (testResult.metadata?.time) {
testDuration = testResult.metadata.time;
} }
// Log test result
if (this.logger) {
if (testResult.ok) {
this.logger.testResult(testResult.description, true, testDuration);
} else {
this.logger.testResult(testResult.description, false, testDuration);
// If there's error metadata, show it
if (testResult.metadata?.error) {
const error = testResult.metadata.error;
let errorDetails = error.message;
if (error.stack) {
errorDetails = error.stack;
}
this.logger.testErrorDetails(errorDetails);
}
}
}
// Handle directives (skip/todo)
if (testResult.directive) {
if (this.logger) {
if (testResult.directive.type === 'skip') {
this.logger.testConsoleOutput(`Test skipped: ${testResult.directive.reason || 'No reason given'}`);
} else if (testResult.directive.type === 'todo') {
this.logger.testConsoleOutput(`Test todo: ${testResult.directive.reason || 'No reason given'}`);
}
}
}
// Mark test as settled and move to next
this.activeTapTestResult.testSettled = true;
this.testStore.push(this.activeTapTestResult); this.testStore.push(this.activeTapTestResult);
this._getNewTapTestResult(); this._getNewTapTestResult();
break;
case 'comment':
if (this.logger) {
// Check if it's a pretask comment
const pretaskMatch = message.content.match(/^Pretask -> (.+): Success\.$/);
if (pretaskMatch) {
this.logger.tapOutput(message.content);
} else {
this.logger.testConsoleOutput(message.content);
} }
} }
break;
case 'bailout':
if (this.logger) {
this.logger.error(`Bail out! ${message.content}`);
}
break;
case 'error':
const errorBlock = message.content as IErrorBlock;
if (this.logger && errorBlock.error) {
let errorDetails = errorBlock.error.message;
if (errorBlock.error.stack) {
errorDetails = errorBlock.error.stack;
}
this.logger.testErrorDetails(errorDetails);
}
break;
case 'snapshot':
// Handle new protocol snapshot format
const snapshot = message.content;
this.handleSnapshot({
path: snapshot.name,
content: typeof snapshot.content === 'string' ? snapshot.content : JSON.stringify(snapshot.content),
action: 'compare' // Default action
});
break;
case 'event':
const event = message.content as ITestEvent;
this._handleTestEvent(event);
break;
}
}
private _handleTestEvent(event: ITestEvent) {
if (!this.logger) return;
switch (event.eventType) {
case 'test:queued':
// We can track queued tests if needed
break;
case 'test:started':
this.logger.testConsoleOutput(cs(`Test starting: ${event.data.description}`, 'cyan'));
if (event.data.retry) {
this.logger.testConsoleOutput(cs(` Retry attempt ${event.data.retry}`, 'orange'));
}
break;
case 'test:progress':
if (event.data.progress !== undefined) {
this.logger.testConsoleOutput(cs(` Progress: ${event.data.progress}%`, 'cyan'));
}
break;
case 'test:completed':
// Test completion is already handled by the test result
// This event provides additional timing info if needed
break;
case 'suite:started':
this.logger.testConsoleOutput(cs(`\nSuite: ${event.data.suiteName}`, 'blue'));
break;
case 'suite:completed':
this.logger.testConsoleOutput(cs(`Suite completed: ${event.data.suiteName}\n`, 'blue'));
break;
case 'hook:started':
this.logger.testConsoleOutput(cs(` Hook: ${event.data.hookName}`, 'cyan'));
break;
case 'hook:completed':
// Silent unless there's an error
if (event.data.error) {
this.logger.testConsoleOutput(cs(` Hook failed: ${event.data.hookName}`, 'red'));
}
break;
case 'assertion:failed':
// Enhanced assertion failure with diff
if (event.data.error) {
this._displayAssertionError(event.data.error);
}
break;
}
}
private _displayAssertionError(error: any) {
if (!this.logger) return;
// Display error message
if (error.message) {
this.logger.testErrorDetails(error.message);
}
// Display visual diff if available
if (error.diff) {
this._displayDiff(error.diff, error.expected, error.actual);
}
}
private _displayDiff(diff: any, expected: any, actual: any) {
if (!this.logger) return;
this.logger.testConsoleOutput(cs('\n Diff:', 'cyan'));
switch (diff.type) {
case 'string':
this._displayStringDiff(diff.changes);
break;
case 'object':
this._displayObjectDiff(diff.changes, expected, actual);
break;
case 'array':
this._displayArrayDiff(diff.changes, expected, actual);
break;
case 'primitive':
this._displayPrimitiveDiff(diff.changes);
break;
}
}
private _displayStringDiff(changes: any[]) {
for (const change of changes) {
const linePrefix = ` Line ${change.line + 1}: `;
if (change.type === 'add') {
this.logger.testConsoleOutput(cs(`${linePrefix}+ ${change.content}`, 'green'));
} else if (change.type === 'remove') {
this.logger.testConsoleOutput(cs(`${linePrefix}- ${change.content}`, 'red'));
}
}
}
private _displayObjectDiff(changes: any[], expected: any, actual: any) {
this.logger.testConsoleOutput(cs(' Expected:', 'red'));
this.logger.testConsoleOutput(` ${JSON.stringify(expected, null, 2)}`);
this.logger.testConsoleOutput(cs(' Actual:', 'green'));
this.logger.testConsoleOutput(` ${JSON.stringify(actual, null, 2)}`);
this.logger.testConsoleOutput(cs('\n Changes:', 'cyan'));
for (const change of changes) {
const path = change.path.join('.');
if (change.type === 'add') {
this.logger.testConsoleOutput(cs(` + ${path}: ${JSON.stringify(change.newValue)}`, 'green'));
} else if (change.type === 'remove') {
this.logger.testConsoleOutput(cs(` - ${path}: ${JSON.stringify(change.oldValue)}`, 'red'));
} else if (change.type === 'modify') {
this.logger.testConsoleOutput(cs(` ~ ${path}:`, 'cyan'));
this.logger.testConsoleOutput(cs(` - ${JSON.stringify(change.oldValue)}`, 'red'));
this.logger.testConsoleOutput(cs(` + ${JSON.stringify(change.newValue)}`, 'green'));
}
}
}
private _displayArrayDiff(changes: any[], expected: any[], actual: any[]) {
this._displayObjectDiff(changes, expected, actual);
}
private _displayPrimitiveDiff(changes: any[]) {
const change = changes[0];
if (change) {
this.logger.testConsoleOutput(cs(` Expected: ${JSON.stringify(change.oldValue)}`, 'red'));
this.logger.testConsoleOutput(cs(` Actual: ${JSON.stringify(change.newValue)}`, 'green'));
}
} }
/** /**
@@ -294,6 +482,7 @@ export class TapParser {
public async evaluateFinalResult() { public async evaluateFinalResult() {
this.receivedTests = this.testStore.length; this.receivedTests = this.testStore.length;
const duration = Date.now() - this.startTime;
// check wether all tests ran // check wether all tests ran
if (this.expectedTests === this.receivedTests) { if (this.expectedTests === this.receivedTests) {
@@ -305,23 +494,26 @@ export class TapParser {
this.logger.error(`Only ${this.receivedTests} out of ${this.expectedTests} completed!`); this.logger.error(`Only ${this.receivedTests} out of ${this.expectedTests} completed!`);
} }
} }
if (!this.expectedTests) { if (!this.expectedTests && this.receivedTests === 0) {
if (this.logger) { if (this.logger) {
this.logger.error('No tests were defined. Therefore the testfile failed!'); this.logger.error('No tests were defined. Therefore the testfile failed!');
this.logger.testFileEnd(0, 1, duration); // Count as 1 failure
} }
} else if (this.expectedTests !== this.receivedTests) { } else if (this.expectedTests !== this.receivedTests) {
if (this.logger) { if (this.logger) {
this.logger.error('The amount of received tests and expectedTests is unequal! Therefore the testfile failed'); this.logger.error('The amount of received tests and expectedTests is unequal! Therefore the testfile failed');
const errorCount = this.getErrorTests().length || 1; // At least 1 error
this.logger.testFileEnd(this.receivedTests - errorCount, errorCount, duration);
} }
} else if (this.getErrorTests().length === 0) { } else if (this.getErrorTests().length === 0) {
if (this.logger) { if (this.logger) {
this.logger.tapOutput('All tests are successfull!!!'); this.logger.tapOutput('All tests are successfull!!!');
this.logger.testFileEnd(this.receivedTests, 0, 0); this.logger.testFileEnd(this.receivedTests, 0, duration);
} }
} else { } else {
if (this.logger) { if (this.logger) {
this.logger.tapOutput(`${this.getErrorTests().length} tests threw an error!!!`, true); this.logger.tapOutput(`${this.getErrorTests().length} tests threw an error!!!`, true);
this.logger.testFileEnd(this.receivedTests - this.getErrorTests().length, this.getErrorTests().length, 0); this.logger.testFileEnd(this.receivedTests - this.getErrorTests().length, this.getErrorTests().length, duration);
} }
} }
} }

View File

@@ -1,6 +1,5 @@
import * as plugins from './tstest.plugins.js'; import * as plugins from './tstest.plugins.js';
import * as paths from './tstest.paths.js'; import * as paths from './tstest.paths.js';
import * as logPrefixes from './tstest.logprefixes.js';
import { coloredString as cs } from '@push.rocks/consolecolor'; import { coloredString as cs } from '@push.rocks/consolecolor';
@@ -11,11 +10,22 @@ import { TestExecutionMode } from './index.js';
import { TsTestLogger } from './tstest.logging.js'; import { TsTestLogger } from './tstest.logging.js';
import type { LogOptions } from './tstest.logging.js'; import type { LogOptions } from './tstest.logging.js';
// Runtime adapters
import { parseTestFilename } from './tstest.classes.runtime.parser.js';
import { RuntimeAdapterRegistry } from './tstest.classes.runtime.adapter.js';
import { NodeRuntimeAdapter } from './tstest.classes.runtime.node.js';
import { ChromiumRuntimeAdapter } from './tstest.classes.runtime.chromium.js';
import { DenoRuntimeAdapter } from './tstest.classes.runtime.deno.js';
import { BunRuntimeAdapter } from './tstest.classes.runtime.bun.js';
export class TsTest { export class TsTest {
public testDir: TestDirectory; public testDir: TestDirectory;
public executionMode: TestExecutionMode; public executionMode: TestExecutionMode;
public logger: TsTestLogger; public logger: TsTestLogger;
public filterTags: string[]; public filterTags: string[];
public startFromFile: number | null;
public stopAtFile: number | null;
public timeoutSeconds: number | null;
public smartshellInstance = new plugins.smartshell.Smartshell({ public smartshellInstance = new plugins.smartshell.Smartshell({
executor: 'bash', executor: 'bash',
@@ -26,18 +36,42 @@ export class TsTest {
public tsbundleInstance = new plugins.tsbundle.TsBundle(); public tsbundleInstance = new plugins.tsbundle.TsBundle();
constructor(cwdArg: string, testPathArg: string, executionModeArg: TestExecutionMode, logOptions: LogOptions = {}, tags: string[] = []) { public runtimeRegistry = new RuntimeAdapterRegistry();
constructor(cwdArg: string, testPathArg: string, executionModeArg: TestExecutionMode, logOptions: LogOptions = {}, tags: string[] = [], startFromFile: number | null = null, stopAtFile: number | null = null, timeoutSeconds: number | null = null) {
this.executionMode = executionModeArg; this.executionMode = executionModeArg;
this.testDir = new TestDirectory(cwdArg, testPathArg, executionModeArg); this.testDir = new TestDirectory(cwdArg, testPathArg, executionModeArg);
this.logger = new TsTestLogger(logOptions); this.logger = new TsTestLogger(logOptions);
this.filterTags = tags; this.filterTags = tags;
this.startFromFile = startFromFile;
this.stopAtFile = stopAtFile;
this.timeoutSeconds = timeoutSeconds;
// Register runtime adapters
this.runtimeRegistry.register(
new NodeRuntimeAdapter(this.logger, this.smartshellInstance, this.timeoutSeconds, this.filterTags)
);
this.runtimeRegistry.register(
new ChromiumRuntimeAdapter(this.logger, this.tsbundleInstance, this.smartbrowserInstance, this.timeoutSeconds)
);
this.runtimeRegistry.register(
new DenoRuntimeAdapter(this.logger, this.smartshellInstance, this.timeoutSeconds, this.filterTags)
);
this.runtimeRegistry.register(
new BunRuntimeAdapter(this.logger, this.smartshellInstance, this.timeoutSeconds, this.filterTags)
);
} }
async run() { async run() {
// Move previous log files if --logfile option is used
if (this.logger.options.logFile) {
await this.movePreviousLogFiles();
}
const testGroups = await this.testDir.getTestFileGroups(); const testGroups = await this.testDir.getTestFileGroups();
const allFiles = [...testGroups.serial, ...Object.values(testGroups.parallelGroups).flat()]; const allFiles = [...testGroups.serial, ...Object.values(testGroups.parallelGroups).flat()];
// Log test discovery // Log test discovery - always show full count
this.logger.testDiscovery( this.logger.testDiscovery(
allFiles.length, allFiles.length,
this.testDir.testPath, this.testDir.testPath,
@@ -50,7 +84,7 @@ export class TsTest {
// Execute serial tests first // Execute serial tests first
for (const fileNameArg of testGroups.serial) { for (const fileNameArg of testGroups.serial) {
fileIndex++; fileIndex++;
await this.runSingleTest(fileNameArg, fileIndex, allFiles.length, tapCombinator); await this.runSingleTestOrSkip(fileNameArg, fileIndex, allFiles.length, tapCombinator);
} }
// Execute parallel groups sequentially // Execute parallel groups sequentially
@@ -64,7 +98,7 @@ export class TsTest {
// Run all tests in this group in parallel // Run all tests in this group in parallel
const parallelPromises = groupFiles.map(async (fileNameArg) => { const parallelPromises = groupFiles.map(async (fileNameArg) => {
fileIndex++; fileIndex++;
return this.runSingleTest(fileNameArg, fileIndex, allFiles.length, tapCombinator); return this.runSingleTestOrSkip(fileNameArg, fileIndex, allFiles.length, tapCombinator);
}); });
await Promise.all(parallelPromises); await Promise.all(parallelPromises);
@@ -75,30 +109,140 @@ export class TsTest {
tapCombinator.evaluate(); tapCombinator.evaluate();
} }
private async runSingleTest(fileNameArg: string, fileIndex: number, totalFiles: number, tapCombinator: TapCombinator) { public async runWatch(ignorePatterns: string[] = []) {
switch (true) { const smartchokInstance = new plugins.smartchok.Smartchok([this.testDir.cwd]);
case process.env.CI && fileNameArg.includes('.nonci.'):
this.logger.tapOutput(`Skipping ${fileNameArg} - marked as non-CI`);
break;
case fileNameArg.endsWith('.browser.ts') || fileNameArg.endsWith('.browser.nonci.ts'):
const tapParserBrowser = await this.runInChrome(fileNameArg, fileIndex, totalFiles);
tapCombinator.addTapParser(tapParserBrowser);
break;
case fileNameArg.endsWith('.both.ts') || fileNameArg.endsWith('.both.nonci.ts'):
this.logger.sectionStart('Part 1: Chrome');
const tapParserBothBrowser = await this.runInChrome(fileNameArg, fileIndex, totalFiles);
tapCombinator.addTapParser(tapParserBothBrowser);
this.logger.sectionEnd();
this.logger.sectionStart('Part 2: Node'); console.clear();
const tapParserBothNode = await this.runInNode(fileNameArg, fileIndex, totalFiles); this.logger.watchModeStart();
tapCombinator.addTapParser(tapParserBothNode);
// Initial run
await this.run();
// Set up file watcher
const fileChanges = new Map<string, NodeJS.Timeout>();
const debounceTime = 300; // 300ms debounce
const runTestsAfterChange = async () => {
console.clear();
const changedFiles = Array.from(fileChanges.keys());
fileChanges.clear();
this.logger.watchModeRerun(changedFiles);
await this.run();
this.logger.watchModeWaiting();
};
// Start watching before subscribing to events
await smartchokInstance.start();
// Subscribe to file change events
const changeObservable = await smartchokInstance.getObservableFor('change');
const addObservable = await smartchokInstance.getObservableFor('add');
const unlinkObservable = await smartchokInstance.getObservableFor('unlink');
const handleFileChange = (changedPath: string) => {
// Skip if path matches ignore patterns
if (ignorePatterns.some(pattern => changedPath.includes(pattern))) {
return;
}
// Clear existing timeout for this file if any
if (fileChanges.has(changedPath)) {
clearTimeout(fileChanges.get(changedPath));
}
// Set new timeout for this file
const timeout = setTimeout(() => {
fileChanges.delete(changedPath);
if (fileChanges.size === 0) {
runTestsAfterChange();
}
}, debounceTime);
fileChanges.set(changedPath, timeout);
};
// Subscribe to all relevant events
changeObservable.subscribe(([path]) => handleFileChange(path));
addObservable.subscribe(([path]) => handleFileChange(path));
unlinkObservable.subscribe(([path]) => handleFileChange(path));
this.logger.watchModeWaiting();
// Handle Ctrl+C to exit gracefully
process.on('SIGINT', async () => {
this.logger.watchModeStop();
await smartchokInstance.stop();
process.exit(0);
});
// Keep the process running
await new Promise(() => {}); // This promise never resolves
}
private async runSingleTestOrSkip(fileNameArg: string, fileIndex: number, totalFiles: number, tapCombinator: TapCombinator) {
// Check if this file should be skipped based on range
if (this.startFromFile !== null && fileIndex < this.startFromFile) {
this.logger.testFileSkipped(fileNameArg, fileIndex, totalFiles, `before start range (${this.startFromFile})`);
tapCombinator.addSkippedFile(fileNameArg);
return;
}
if (this.stopAtFile !== null && fileIndex > this.stopAtFile) {
this.logger.testFileSkipped(fileNameArg, fileIndex, totalFiles, `after stop range (${this.stopAtFile})`);
tapCombinator.addSkippedFile(fileNameArg);
return;
}
// File is in range, run it
await this.runSingleTest(fileNameArg, fileIndex, totalFiles, tapCombinator);
}
private async runSingleTest(fileNameArg: string, fileIndex: number, totalFiles: number, tapCombinator: TapCombinator) {
// Parse the filename to determine runtimes and modifiers
const fileName = plugins.path.basename(fileNameArg);
const parsed = parseTestFilename(fileName, { strictUnknownRuntime: false });
// Check for nonci modifier in CI environment
if (process.env.CI && parsed.modifiers.includes('nonci')) {
this.logger.tapOutput(`Skipping ${fileNameArg} - marked as non-CI`);
return;
}
// Show deprecation warning for legacy naming
if (parsed.isLegacy) {
console.warn('');
console.warn(cs('⚠️ DEPRECATION WARNING', 'orange'));
console.warn(cs(` File: ${fileName}`, 'orange'));
console.warn(cs(` Legacy naming detected. Please migrate to new naming convention.`, 'orange'));
console.warn(cs(` Suggested: ${fileName.replace('.browser.', '.chromium.').replace('.both.', '.node+chromium.')}`, 'green'));
console.warn(cs(` Run: tstest migrate --dry-run`, 'cyan'));
console.warn('');
}
// Get adapters for the specified runtimes
const adapters = this.runtimeRegistry.getAdaptersForRuntimes(parsed.runtimes);
if (adapters.length === 0) {
this.logger.tapOutput(`Skipping ${fileNameArg} - no runtime adapters available`);
return;
}
// Execute tests for each runtime
if (adapters.length === 1) {
// Single runtime - no sections needed
const adapter = adapters[0];
const tapParser = await adapter.run(fileNameArg, fileIndex, totalFiles);
tapCombinator.addTapParser(tapParser);
} else {
// Multiple runtimes - use sections
for (let i = 0; i < adapters.length; i++) {
const adapter = adapters[i];
this.logger.sectionStart(`Part ${i + 1}: ${adapter.displayName}`);
const tapParser = await adapter.run(fileNameArg, fileIndex, totalFiles);
tapCombinator.addTapParser(tapParser);
this.logger.sectionEnd(); this.logger.sectionEnd();
break; }
default:
const tapParserNode = await this.runInNode(fileNameArg, fileIndex, totalFiles);
tapCombinator.addTapParser(tapParserNode);
break;
} }
} }
@@ -117,13 +261,131 @@ export class TsTest {
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(','); process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
} }
const execResultStreaming = await this.smartshellInstance.execStreamingSilent( // Check for 00init.ts file in test directory
`tsrun ${fileNameArg}${tsrunOptions}` const testDir = plugins.path.dirname(fileNameArg);
); const initFile = plugins.path.join(testDir, '00init.ts');
let runCommand = `tsrun ${fileNameArg}${tsrunOptions}`;
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
// If 00init.ts exists, run it first
if (initFileExists) {
// Create a temporary loader file that imports both 00init.ts and the test file
const absoluteInitFile = plugins.path.resolve(initFile);
const absoluteTestFile = plugins.path.resolve(fileNameArg);
const loaderContent = `
import '${absoluteInitFile.replace(/\\/g, '/')}';
import '${absoluteTestFile.replace(/\\/g, '/')}';
`;
const loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(fileNameArg)}`);
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
runCommand = `tsrun ${loaderPath}${tsrunOptions}`;
}
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
// If we created a loader file, clean it up after test execution
if (initFileExists) {
const loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(fileNameArg)}`);
const cleanup = () => {
try {
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
plugins.smartfile.fs.removeSync(loaderPath);
}
} catch (e) {
// Ignore cleanup errors
}
};
execResultStreaming.childProcess.on('exit', cleanup);
execResultStreaming.childProcess.on('error', cleanup);
}
// Start warning timer if no timeout was specified
let warningTimer: NodeJS.Timeout | null = null;
if (this.timeoutSeconds === null) {
warningTimer = setTimeout(() => {
console.error('');
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
console.error(cs(` File: ${fileNameArg}`, 'orange'));
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
console.error('');
}, 60000); // 1 minute
}
// Handle timeout if specified
if (this.timeoutSeconds !== null) {
const timeoutMs = this.timeoutSeconds * 1000;
let timeoutId: NodeJS.Timeout;
const timeoutPromise = new Promise<void>((_resolve, reject) => {
timeoutId = setTimeout(async () => {
// Use smartshell's terminate() to kill entire process tree
await execResultStreaming.terminate();
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
}, timeoutMs);
});
try {
await Promise.race([
tapParser.handleTapProcess(execResultStreaming.childProcess),
timeoutPromise
]);
// Clear timeout if test completed successfully
clearTimeout(timeoutId);
} catch (error) {
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Handle timeout error
tapParser.handleTimeout(this.timeoutSeconds);
// Ensure entire process tree is killed if still running
try {
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
} catch (killError) {
// Process tree might already be dead
}
await tapParser.evaluateFinalResult();
}
} else {
await tapParser.handleTapProcess(execResultStreaming.childProcess); await tapParser.handleTapProcess(execResultStreaming.childProcess);
}
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
return tapParser; return tapParser;
} }
private async findFreePorts(): Promise<{ httpPort: number; wsPort: number }> {
const smartnetwork = new plugins.smartnetwork.SmartNetwork();
// Find random free HTTP port in range 30000-40000 to minimize collision chance
const httpPort = await smartnetwork.findFreePort(30000, 40000, { randomize: true });
if (!httpPort) {
throw new Error('Could not find a free HTTP port in range 30000-40000');
}
// Find random free WebSocket port, excluding the HTTP port to ensure they're different
const wsPort = await smartnetwork.findFreePort(30000, 40000, {
randomize: true,
exclude: [httpPort]
});
if (!wsPort) {
throw new Error('Could not find a free WebSocket port in range 30000-40000');
}
// Log selected ports for debugging
if (!this.logger.options.quiet) {
console.log(`Selected ports - HTTP: ${httpPort}, WebSocket: ${wsPort}`);
}
return { httpPort, wsPort };
}
public async runInChrome(fileNameArg: string, index: number, total: number): Promise<TapParser> { public async runInChrome(fileNameArg: string, index: number, total: number): Promise<TapParser> {
this.logger.testFileStart(fileNameArg, 'chromium', index, total); this.logger.testFileStart(fileNameArg, 'chromium', index, total);
@@ -138,20 +400,24 @@ export class TsTest {
bundler: 'esbuild', bundler: 'esbuild',
}); });
// Find free ports for HTTP and WebSocket
const { httpPort, wsPort } = await this.findFreePorts();
// lets create a server // lets create a server
const server = new plugins.typedserver.servertools.Server({ const server = new plugins.typedserver.servertools.Server({
cors: true, cors: true,
port: 3007, port: httpPort,
}); });
server.addRoute( server.addRoute(
'/test', '/test',
new plugins.typedserver.servertools.Handler('GET', async (req, res) => { new plugins.typedserver.servertools.Handler('GET', async (_req, res) => {
res.type('.html'); res.type('.html');
res.write(` res.write(`
<html> <html>
<head> <head>
<script> <script>
globalThis.testdom = true; globalThis.testdom = true;
globalThis.wsPort = ${wsPort};
</script> </script>
</head> </head>
<body></body> <body></body>
@@ -160,12 +426,12 @@ export class TsTest {
res.end(); res.end();
}) })
); );
server.addRoute('*', new plugins.typedserver.servertools.HandlerStatic(tsbundleCacheDirPath)); server.addRoute('/*splat', new plugins.typedserver.servertools.HandlerStatic(tsbundleCacheDirPath));
await server.start(); await server.start();
// lets handle realtime comms // lets handle realtime comms
const tapParser = new TapParser(fileNameArg + ':chrome', this.logger); const tapParser = new TapParser(fileNameArg + ':chrome', this.logger);
const wss = new plugins.ws.WebSocketServer({ port: 8080 }); const wss = new plugins.ws.WebSocketServer({ port: wsPort });
wss.on('connection', (ws) => { wss.on('connection', (ws) => {
ws.on('message', (message) => { ws.on('message', (message) => {
const messageStr = message.toString(); const messageStr = message.toString();
@@ -178,13 +444,14 @@ export class TsTest {
}); });
}); });
// lets do the browser bit // lets do the browser bit with timeout handling
await this.smartbrowserInstance.start(); await this.smartbrowserInstance.start();
const evaluation = await this.smartbrowserInstance.evaluateOnPage(
`http://localhost:3007/test?bundleName=${bundleFileName}`, const evaluatePromise = this.smartbrowserInstance.evaluateOnPage(
`http://localhost:${httpPort}/test?bundleName=${bundleFileName}`,
async () => { async () => {
// lets enable real time comms // lets enable real time comms
const ws = new WebSocket('ws://localhost:8080'); const ws = new WebSocket(`ws://localhost:${globalThis.wsPort}`);
await new Promise((resolve) => (ws.onopen = resolve)); await new Promise((resolve) => (ws.onopen = resolve));
// Ensure this function is declared with 'async' // Ensure this function is declared with 'async'
@@ -193,12 +460,12 @@ export class TsTest {
const originalError = console.error; const originalError = console.error;
// Override console methods to capture the logs // Override console methods to capture the logs
console.log = (...args) => { console.log = (...args: any[]) => {
logStore.push(args.join(' ')); logStore.push(args.join(' '));
ws.send(args.join(' ')); ws.send(args.join(' '));
originalLog(...args); originalLog(...args);
}; };
console.error = (...args) => { console.error = (...args: any[]) => {
logStore.push(args.join(' ')); logStore.push(args.join(' '));
ws.send(args.join(' ')); ws.send(args.join(' '));
originalError(...args); originalError(...args);
@@ -237,16 +504,127 @@ export class TsTest {
return logStore.join('\n'); return logStore.join('\n');
} }
); );
// Start warning timer if no timeout was specified
let warningTimer: NodeJS.Timeout | null = null;
if (this.timeoutSeconds === null) {
warningTimer = setTimeout(() => {
console.error('');
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
console.error(cs(` File: ${fileNameArg}`, 'orange'));
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
console.error('');
}, 60000); // 1 minute
}
// Handle timeout if specified
if (this.timeoutSeconds !== null) {
const timeoutMs = this.timeoutSeconds * 1000;
let timeoutId: NodeJS.Timeout;
const timeoutPromise = new Promise<void>((_resolve, reject) => {
timeoutId = setTimeout(() => {
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
}, timeoutMs);
});
try {
await Promise.race([
evaluatePromise,
timeoutPromise
]);
// Clear timeout if test completed successfully
clearTimeout(timeoutId);
} catch (error) {
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Handle timeout error
tapParser.handleTimeout(this.timeoutSeconds);
}
} else {
await evaluatePromise;
}
// Clear warning timer if it was set
if (warningTimer) {
clearTimeout(warningTimer);
}
// Always clean up resources, even on timeout
try {
await this.smartbrowserInstance.stop(); await this.smartbrowserInstance.stop();
} catch (error) {
// Browser might already be stopped
}
try {
await server.stop(); await server.stop();
} catch (error) {
// Server might already be stopped
}
try {
wss.close(); wss.close();
} catch (error) {
// WebSocket server might already be closed
}
console.log( console.log(
`${cs('=> ', 'blue')} Stopped ${cs(fileNameArg, 'orange')} chromium instance and server.` `${cs('=> ', 'blue')} Stopped ${cs(fileNameArg, 'orange')} chromium instance and server.`
); );
// lets create the tap parser // Always evaluate final result (handleTimeout just sets up the test state)
await tapParser.evaluateFinalResult(); await tapParser.evaluateFinalResult();
return tapParser; return tapParser;
} }
public async runInDeno() {} public async runInDeno() {}
private async movePreviousLogFiles() {
const logDir = plugins.path.join('.nogit', 'testlogs');
const previousDir = plugins.path.join('.nogit', 'testlogs', 'previous');
const errDir = plugins.path.join('.nogit', 'testlogs', '00err');
const diffDir = plugins.path.join('.nogit', 'testlogs', '00diff');
try {
// Delete 00err and 00diff directories if they exist
if (plugins.smartfile.fs.isDirectorySync(errDir)) {
plugins.smartfile.fs.removeSync(errDir);
}
if (plugins.smartfile.fs.isDirectorySync(diffDir)) {
plugins.smartfile.fs.removeSync(diffDir);
}
// Get all .log files in log directory (not in subdirectories)
const files = await plugins.smartfile.fs.listFileTree(logDir, '*.log');
const logFiles = files.filter((file: string) => !file.includes('/'));
if (logFiles.length === 0) {
return;
}
// Ensure previous directory exists
await plugins.smartfile.fs.ensureDir(previousDir);
// Move each log file to previous directory
for (const file of logFiles) {
const filename = plugins.path.basename(file);
const sourcePath = plugins.path.join(logDir, filename);
const destPath = plugins.path.join(previousDir, filename);
try {
// Copy file to new location and remove original
await plugins.smartfile.fs.copy(sourcePath, destPath);
await plugins.smartfile.fs.remove(sourcePath);
} catch (error) {
// Silently continue if a file can't be moved
}
}
} catch (error) {
// Directory might not exist, which is fine
return;
}
}
} }

View File

@@ -30,12 +30,14 @@ export interface TestSummary {
totalTests: number; totalTests: number;
totalPassed: number; totalPassed: number;
totalFailed: number; totalFailed: number;
totalSkipped: number;
totalDuration: number; totalDuration: number;
fileResults: TestFileResult[]; fileResults: TestFileResult[];
skippedFiles: string[];
} }
export class TsTestLogger { export class TsTestLogger {
private options: LogOptions; public readonly options: LogOptions;
private startTime: number; private startTime: number;
private fileResults: TestFileResult[] = []; private fileResults: TestFileResult[] = [];
private currentFileResult: TestFileResult | null = null; private currentFileResult: TestFileResult | null = null;
@@ -153,8 +155,16 @@ export class TsTestLogger {
// Only set up test log file if --logfile option is specified // Only set up test log file if --logfile option is specified
if (this.options.logFile) { if (this.options.logFile) {
const baseFilename = path.basename(filename, '.ts'); // Create a safe filename that preserves directory structure
this.currentTestLogFile = path.join('.nogit', 'testlogs', `${baseFilename}.log`); // Convert relative path to a flat filename by replacing separators with __
const relativeFilename = path.relative(process.cwd(), filename);
const safeFilename = relativeFilename
.replace(/\\/g, '/') // Normalize Windows paths
.replace(/\//g, '__') // Replace path separators with double underscores
.replace(/\.ts$/, '') // Remove .ts extension
.replace(/^\.\.__|^\.__|^__/, ''); // Clean up leading separators from relative paths
this.currentTestLogFile = path.join('.nogit', 'testlogs', `${safeFilename}.log`);
// Ensure the directory exists // Ensure the directory exists
const logDir = path.dirname(this.currentTestLogFile); const logDir = path.dirname(this.currentTestLogFile);
@@ -232,9 +242,51 @@ export class TsTestLogger {
if (!this.options.quiet) { if (!this.options.quiet) {
const total = passed + failed; const total = passed + failed;
const status = failed === 0 ? 'PASSED' : 'FAILED'; const durationStr = duration >= 1000 ? `${(duration / 1000).toFixed(1)}s` : `${duration}ms`;
const color = failed === 0 ? 'green' : 'red';
this.log(this.format(` Summary: ${passed}/${total} ${status}`, color)); if (failed === 0) {
this.log(this.format(` Summary: ${passed}/${total} PASSED in ${durationStr}`, 'green'));
} else {
this.log(this.format(` Summary: ${passed} passed, ${failed} failed of ${total} tests in ${durationStr}`, 'red'));
}
}
// If using --logfile, handle error copy and diff detection
if (this.options.logFile && this.currentTestLogFile) {
try {
const logContent = fs.readFileSync(this.currentTestLogFile, 'utf-8');
const logDir = path.dirname(this.currentTestLogFile);
const logBasename = path.basename(this.currentTestLogFile);
// Create error copy if there were failures
if (failed > 0) {
const errorDir = path.join(logDir, '00err');
if (!fs.existsSync(errorDir)) {
fs.mkdirSync(errorDir, { recursive: true });
}
const errorLogPath = path.join(errorDir, logBasename);
fs.writeFileSync(errorLogPath, logContent);
}
// Check for previous version and create diff if changed
const previousLogPath = path.join(logDir, 'previous', logBasename);
if (fs.existsSync(previousLogPath)) {
const previousContent = fs.readFileSync(previousLogPath, 'utf-8');
// Simple check if content differs
if (previousContent !== logContent) {
const diffDir = path.join(logDir, '00diff');
if (!fs.existsSync(diffDir)) {
fs.mkdirSync(diffDir, { recursive: true });
}
const diffLogPath = path.join(diffDir, logBasename);
const diffContent = this.createDiff(previousContent, logContent, logBasename);
fs.writeFileSync(diffLogPath, diffContent);
}
}
} catch (error) {
// Silently fail to avoid disrupting the test run
}
} }
// Clear the current test log file reference only if using --logfile // Clear the current test log file reference only if using --logfile
@@ -244,7 +296,7 @@ export class TsTestLogger {
} }
// TAP output forwarding (for TAP protocol messages) // TAP output forwarding (for TAP protocol messages)
tapOutput(message: string, isError: boolean = false) { tapOutput(message: string, _isError: boolean = false) {
if (this.options.json) return; if (this.options.json) return;
// Never show raw TAP protocol messages in console // Never show raw TAP protocol messages in console
@@ -274,6 +326,19 @@ export class TsTestLogger {
} }
} }
// Skipped test file
testFileSkipped(filename: string, index: number, total: number, reason: string) {
if (this.options.json) {
this.logJson({ event: 'fileSkipped', filename, index, total, reason });
return;
}
if (this.options.quiet) return;
this.log(this.format(`\n⏭ ${filename} (${index}/${total})`, 'yellow'));
this.log(this.format(` Skipped: ${reason}`, 'dim'));
}
// Browser console // Browser console
browserConsole(message: string, level: string = 'log') { browserConsole(message: string, level: string = 'log') {
if (this.options.json) { if (this.options.json) {
@@ -309,15 +374,17 @@ export class TsTestLogger {
} }
// Final summary // Final summary
summary() { summary(skippedFiles: string[] = []) {
const totalDuration = Date.now() - this.startTime; const totalDuration = Date.now() - this.startTime;
const summary: TestSummary = { const summary: TestSummary = {
totalFiles: this.fileResults.length, totalFiles: this.fileResults.length + skippedFiles.length,
totalTests: this.fileResults.reduce((sum, r) => sum + r.total, 0), totalTests: this.fileResults.reduce((sum, r) => sum + r.total, 0),
totalPassed: this.fileResults.reduce((sum, r) => sum + r.passed, 0), totalPassed: this.fileResults.reduce((sum, r) => sum + r.passed, 0),
totalFailed: this.fileResults.reduce((sum, r) => sum + r.failed, 0), totalFailed: this.fileResults.reduce((sum, r) => sum + r.failed, 0),
totalSkipped: skippedFiles.length,
totalDuration, totalDuration,
fileResults: this.fileResults fileResults: this.fileResults,
skippedFiles
}; };
if (this.options.json) { if (this.options.json) {
@@ -327,7 +394,13 @@ export class TsTestLogger {
if (this.options.quiet) { if (this.options.quiet) {
const status = summary.totalFailed === 0 ? 'PASSED' : 'FAILED'; const status = summary.totalFailed === 0 ? 'PASSED' : 'FAILED';
this.log(`\nSummary: ${summary.totalPassed}/${summary.totalTests} | ${totalDuration}ms | ${status}`); const durationStr = totalDuration >= 1000 ? `${(totalDuration / 1000).toFixed(1)}s` : `${totalDuration}ms`;
if (summary.totalFailed === 0) {
this.log(`\nSummary: ${summary.totalPassed}/${summary.totalTests} | ${durationStr} | ${status}`);
} else {
this.log(`\nSummary: ${summary.totalPassed} passed, ${summary.totalFailed} failed of ${summary.totalTests} tests | ${durationStr} | ${status}`);
}
return; return;
} }
@@ -338,7 +411,11 @@ export class TsTestLogger {
this.log(this.format(`│ Total Tests: ${summary.totalTests.toString().padStart(14)}`, 'white')); this.log(this.format(`│ Total Tests: ${summary.totalTests.toString().padStart(14)}`, 'white'));
this.log(this.format(`│ Passed: ${summary.totalPassed.toString().padStart(14)}`, 'green')); this.log(this.format(`│ Passed: ${summary.totalPassed.toString().padStart(14)}`, 'green'));
this.log(this.format(`│ Failed: ${summary.totalFailed.toString().padStart(14)}`, summary.totalFailed > 0 ? 'red' : 'green')); this.log(this.format(`│ Failed: ${summary.totalFailed.toString().padStart(14)}`, summary.totalFailed > 0 ? 'red' : 'green'));
this.log(this.format(`│ Duration: ${totalDuration.toString().padStart(14)}ms │`, 'white')); if (summary.totalSkipped > 0) {
this.log(this.format(`│ Skipped: ${summary.totalSkipped.toString().padStart(14)}`, 'yellow'));
}
const durationStrFormatted = totalDuration >= 1000 ? `${(totalDuration / 1000).toFixed(1)}s` : `${totalDuration}ms`;
this.log(this.format(`│ Duration: ${durationStrFormatted.padStart(14)}`, 'white'));
this.log(this.format('└────────────────────────────────┘', 'dim')); this.log(this.format('└────────────────────────────────┘', 'dim'));
// File results // File results
@@ -359,15 +436,23 @@ export class TsTestLogger {
// Performance metrics // Performance metrics
if (this.options.verbose) { if (this.options.verbose) {
const avgDuration = Math.round(totalDuration / summary.totalTests); // Calculate metrics based on actual test durations
const slowestTest = this.fileResults const allTests = this.fileResults.flatMap(r => r.tests);
.flatMap(r => r.tests) const testDurations = allTests.map(t => t.duration);
.sort((a, b) => b.duration - a.duration)[0]; const sumOfTestDurations = testDurations.reduce((sum, d) => sum + d, 0);
const avgTestDuration = allTests.length > 0 ? Math.round(sumOfTestDurations / allTests.length) : 0;
// Find slowest test (exclude 0ms durations unless all are 0)
const nonZeroDurations = allTests.filter(t => t.duration > 0);
const testsToSort = nonZeroDurations.length > 0 ? nonZeroDurations : allTests;
const slowestTest = testsToSort.sort((a, b) => b.duration - a.duration)[0];
this.log(this.format('\n⏱ Performance Metrics:', 'cyan')); this.log(this.format('\n⏱ Performance Metrics:', 'cyan'));
this.log(this.format(` Average per test: ${avgDuration}ms`, 'white')); this.log(this.format(` Average per test: ${avgTestDuration}ms`, 'white'));
if (slowestTest) { if (slowestTest && slowestTest.duration > 0) {
this.log(this.format(` Slowest test: ${slowestTest.name} (${slowestTest.duration}ms)`, 'yellow')); this.log(this.format(` Slowest test: ${slowestTest.name} (${slowestTest.duration}ms)`, 'orange'));
} else if (allTests.length > 0) {
this.log(this.format(` All tests completed in <1ms`, 'dim'));
} }
} }
@@ -377,6 +462,20 @@ export class TsTestLogger {
this.log(this.format(`\n${status}`, statusColor)); this.log(this.format(`\n${status}`, statusColor));
} }
// Warning display
warning(message: string) {
if (this.options.json) {
this.logJson({ event: 'warning', message });
return;
}
if (this.options.quiet) {
console.log(`WARNING: ${message}`);
} else {
this.log(this.format(` ⚠️ ${message}`, 'orange'));
}
}
// Error display // Error display
error(message: string, file?: string, stack?: string) { error(message: string, file?: string, stack?: string) {
if (this.options.json) { if (this.options.json) {
@@ -396,4 +495,91 @@ export class TsTestLogger {
} }
} }
} }
// Create a diff between two log contents
private createDiff(previousContent: string, currentContent: string, filename: string): string {
const previousLines = previousContent.split('\n');
const currentLines = currentContent.split('\n');
let diff = `DIFF REPORT: ${filename}\n`;
diff += `Generated: ${new Date().toISOString()}\n`;
diff += '='.repeat(80) + '\n\n';
// Simple line-by-line comparison
const maxLines = Math.max(previousLines.length, currentLines.length);
let hasChanges = false;
for (let i = 0; i < maxLines; i++) {
const prevLine = previousLines[i] || '';
const currLine = currentLines[i] || '';
if (prevLine !== currLine) {
hasChanges = true;
if (i < previousLines.length && i >= currentLines.length) {
// Line was removed
diff += `- [Line ${i + 1}] ${prevLine}\n`;
} else if (i >= previousLines.length && i < currentLines.length) {
// Line was added
diff += `+ [Line ${i + 1}] ${currLine}\n`;
} else {
// Line was modified
diff += `- [Line ${i + 1}] ${prevLine}\n`;
diff += `+ [Line ${i + 1}] ${currLine}\n`;
}
}
}
if (!hasChanges) {
diff += 'No changes detected.\n';
}
diff += '\n' + '='.repeat(80) + '\n';
diff += `Previous version had ${previousLines.length} lines\n`;
diff += `Current version has ${currentLines.length} lines\n`;
return diff;
}
// Watch mode methods
watchModeStart() {
if (this.options.json) {
this.logJson({ event: 'watchModeStart' });
return;
}
this.log(this.format('\n👀 Watch Mode', 'cyan'));
this.log(this.format(' Running tests in watch mode...', 'dim'));
this.log(this.format(' Press Ctrl+C to exit\n', 'dim'));
}
watchModeWaiting() {
if (this.options.json) {
this.logJson({ event: 'watchModeWaiting' });
return;
}
this.log(this.format('\n Waiting for file changes...', 'dim'));
}
watchModeRerun(changedFiles: string[]) {
if (this.options.json) {
this.logJson({ event: 'watchModeRerun', changedFiles });
return;
}
this.log(this.format('\n🔄 File changes detected:', 'cyan'));
changedFiles.forEach(file => {
this.log(this.format(`${file}`, 'yellow'));
});
this.log(this.format('\n Re-running tests...\n', 'dim'));
}
watchModeStop() {
if (this.options.json) {
this.logJson({ event: 'watchModeStop' });
return;
}
this.log(this.format('\n\n👋 Stopping watch mode...', 'cyan'));
}
} }

View File

@@ -13,9 +13,11 @@ export {
// @push.rocks scope // @push.rocks scope
import * as consolecolor from '@push.rocks/consolecolor'; import * as consolecolor from '@push.rocks/consolecolor';
import * as smartbrowser from '@push.rocks/smartbrowser'; import * as smartbrowser from '@push.rocks/smartbrowser';
import * as smartchok from '@push.rocks/smartchok';
import * as smartdelay from '@push.rocks/smartdelay'; import * as smartdelay from '@push.rocks/smartdelay';
import * as smartfile from '@push.rocks/smartfile'; import * as smartfile from '@push.rocks/smartfile';
import * as smartlog from '@push.rocks/smartlog'; import * as smartlog from '@push.rocks/smartlog';
import * as smartnetwork from '@push.rocks/smartnetwork';
import * as smartpromise from '@push.rocks/smartpromise'; import * as smartpromise from '@push.rocks/smartpromise';
import * as smartshell from '@push.rocks/smartshell'; import * as smartshell from '@push.rocks/smartshell';
import * as tapbundle from '../dist_ts_tapbundle/index.js'; import * as tapbundle from '../dist_ts_tapbundle/index.js';
@@ -23,15 +25,17 @@ import * as tapbundle from '../dist_ts_tapbundle/index.js';
export { export {
consolecolor, consolecolor,
smartbrowser, smartbrowser,
smartchok,
smartdelay, smartdelay,
smartfile, smartfile,
smartlog, smartlog,
smartnetwork,
smartpromise, smartpromise,
smartshell, smartshell,
tapbundle, tapbundle,
}; };
// @gitzone scope // @git.zone scope
import * as tsbundle from '@git.zone/tsbundle'; import * as tsbundle from '@git.zone/tsbundle';
export { tsbundle }; export { tsbundle };

View File

@@ -3,6 +3,5 @@ export { TapWrap } from './tapbundle.classes.tapwrap.js';
export { webhelpers } from './webhelpers.js'; export { webhelpers } from './webhelpers.js';
export { TapTools } from './tapbundle.classes.taptools.js'; export { TapTools } from './tapbundle.classes.taptools.js';
import { expect } from '@push.rocks/smartexpect'; // Export enhanced expect with diff generation
export { expect, setProtocolEmitter } from './tapbundle.expect.wrapper.js';
export { expect };

389
ts_tapbundle/readme.md Normal file
View File

@@ -0,0 +1,389 @@
# @git.zone/tstest/tapbundle
> 🧪 Core TAP testing framework with enhanced assertions and lifecycle hooks
## Installation
```bash
# tapbundle is typically included as part of @git.zone/tstest
pnpm install --save-dev @git.zone/tstest
```
## Overview
`@git.zone/tstest/tapbundle` is the core testing framework module that provides the TAP (Test Anything Protocol) implementation for tstest. It offers a comprehensive API for writing and organizing tests with support for lifecycle hooks, test suites, enhanced assertions with diff generation, and flexible test configuration.
## Key Features
- 🎯 **TAP Protocol Compliant** - Full TAP version 13 support
- 🔍 **Enhanced Assertions** - Built on smartexpect with automatic diff generation
- 🏗️ **Test Suites** - Organize tests with `describe()` blocks
- 🔄 **Lifecycle Hooks** - beforeEach/afterEach at suite and global levels
- 🏷️ **Test Tagging** - Filter tests by tags for selective execution
-**Parallel Testing** - Run tests concurrently with `testParallel()`
- 🔁 **Automatic Retries** - Configure retry logic for flaky tests
- ⏱️ **Timeout Control** - Set timeouts at global, file, or test level
- 🎨 **Fluent API** - Chain test configurations with builder pattern
- 📊 **Protocol Events** - Real-time test execution events
## Basic Usage
### Simple Test File
```typescript
import { tap, expect } from '@git.zone/tstest/tapbundle';
tap.test('should add numbers correctly', async () => {
const result = 2 + 2;
expect(result).toEqual(4);
});
export default tap.start();
```
### Using Test Suites
```typescript
import { tap, expect } from '@git.zone/tstest/tapbundle';
tap.describe('Calculator', () => {
tap.beforeEach(async (tapTools) => {
// Setup before each test in this suite
});
tap.test('should add', async () => {
expect(2 + 2).toEqual(4);
});
tap.test('should subtract', async () => {
expect(5 - 3).toEqual(2);
});
tap.afterEach(async (tapTools) => {
// Cleanup after each test in this suite
});
});
export default tap.start();
```
## API Reference
### Main Test Methods
#### `tap.test(description, testFunction)`
Define a standard test that runs sequentially.
```typescript
tap.test('should validate user input', async () => {
// test code
});
```
#### `tap.testParallel(description, testFunction)`
Define a test that runs in parallel with other parallel tests.
```typescript
tap.testParallel('should fetch user data', async () => {
// test code
});
```
#### `tap.describe(description, suiteFunction)`
Create a test suite to group related tests.
```typescript
tap.describe('User Authentication', () => {
tap.test('should login', async () => { });
tap.test('should logout', async () => { });
});
```
### Test Modes
#### Skip Tests
```typescript
tap.skip.test('not ready yet', async () => {
// This test will be skipped
});
```
#### Only Mode
```typescript
tap.only.test('focus on this test', async () => {
// Only tests marked with 'only' will run
});
```
#### Todo Tests
```typescript
tap.todo.test('implement feature X');
```
### Fluent Test Builder
Chain test configurations for expressive test definitions:
```typescript
tap
.tags('integration', 'database')
.priority('high')
.retry(3)
.timeout(5000)
.test('should handle database connection', async () => {
// test with configured settings
});
```
### Lifecycle Hooks
#### Suite-Level Hooks
```typescript
tap.describe('Database Tests', () => {
tap.beforeEach(async (tapTools) => {
// Runs before each test in this suite
});
tap.afterEach(async (tapTools) => {
// Runs after each test in this suite
});
tap.test('test 1', async () => { });
tap.test('test 2', async () => { });
});
```
#### Global Hooks
```typescript
tap.settings({
beforeAll: async () => {
// Runs once before all tests
},
afterAll: async () => {
// Runs once after all tests
},
beforeEach: async (testName) => {
// Runs before every test
},
afterEach: async (testName, passed) => {
// Runs after every test
}
});
```
### Global Settings
Configure test behavior at the file level:
```typescript
tap.settings({
timeout: 10000, // Default timeout for all tests
retries: 2, // Retry failed tests
retryDelay: 1000, // Delay between retries
bail: false, // Stop on first failure
suppressConsole: false, // Hide console output
verboseErrors: true, // Show full stack traces
showTestDuration: true, // Display test durations
maxConcurrency: 4, // Max parallel tests
});
```
### Enhanced Assertions
The `expect` function is an enhanced wrapper around [@push.rocks/smartexpect](https://code.foss.global/push.rocks/smartexpect) that automatically generates diffs for failed assertions.
```typescript
import { expect } from '@git.zone/tstest/tapbundle';
tap.test('should compare objects', async () => {
const actual = { name: 'John', age: 30 };
const expected = { name: 'John', age: 31 };
// Will show a detailed diff of the differences
expect(actual).toEqual(expected);
});
```
#### Available Assertions
```typescript
// Equality
expect(value).toEqual(expected);
expect(value).toBe(expected);
// Truthiness
expect(value).toBeTruthy();
expect(value).toBeFalsy();
// Type checks
expect(value).toBeType('string');
// Strings
expect(string).toMatch(/pattern/);
expect(string).toContain('substring');
// Arrays
expect(array).toContain(item);
// Exceptions
expect(fn).toThrow();
expect(fn).toThrow('error message');
// Async
await expect(promise).toResolve();
await expect(promise).toReject();
```
### Test Tagging and Filtering
Tag tests for selective execution:
```typescript
// Define tests with tags
tap.tags('integration', 'slow').test('complex test', async () => {
// test code
});
tap.tags('unit').test('fast test', async () => {
// test code
});
```
Filter tests by setting the environment variable:
```bash
TSTEST_FILTER_TAGS=unit tstest test/mytest.node.ts
```
### TapTools
Each test receives a `tapTools` instance with utilities:
```typescript
tap.test('should have utilities', async (tapTools) => {
// Mark test as skipped
tapTools.markAsSkipped('reason');
// Mark as todo
tapTools.todo('not implemented');
// Configure retries
tapTools.retry(3);
// Log test output
tapTools.log('debug message');
});
```
## Advanced Features
### Pre-Tasks
Run setup tasks before any tests execute:
```typescript
tap.preTask('setup database', async () => {
// Runs before any tests
});
tap.test('first test', async () => {
// Database is ready
});
```
### Test Priority
Organize tests by priority level:
```typescript
tap.priority('high').test('critical test', async () => { });
tap.priority('medium').test('normal test', async () => { });
tap.priority('low').test('optional test', async () => { });
```
### Nested Suites
Create deeply nested test organization:
```typescript
tap.describe('API', () => {
tap.describe('Users', () => {
tap.describe('GET /users', () => {
tap.test('should return all users', async () => { });
});
});
});
```
### Protocol Events
Access real-time test events for custom tooling:
```typescript
import { setProtocolEmitter } from '@git.zone/tstest/tapbundle';
// Get access to protocol emitter for custom event handling
// Events: test:started, test:completed, assertion:failed, suite:started, suite:completed
```
## Best Practices
1. **Always export `tap.start()`** at the end of test files:
```typescript
export default tap.start();
```
2. **Use descriptive test names** that explain what is being tested:
```typescript
tap.test('should return 404 when user does not exist', async () => { });
```
3. **Group related tests** with `describe()` blocks:
```typescript
tap.describe('User validation', () => {
// All user validation tests
});
```
4. **Leverage lifecycle hooks** to reduce duplication:
```typescript
tap.beforeEach(async () => {
// Common setup
});
```
5. **Tag tests appropriately** for flexible test execution:
```typescript
tap.tags('integration', 'database').test('...', async () => { });
```
## TypeScript Support
tapbundle is written in TypeScript and provides full type definitions. The `Tap` class accepts a generic type for shared context:
```typescript
interface MyTestContext {
db: DatabaseConnection;
user: User;
}
const tap = new Tap<MyTestContext>();
tap.test('should use context', async (tapTools) => {
// tapTools is typed with MyTestContext
});
```
## Legal
This project is licensed under MIT.
© 2025 Task Venture Capital GmbH. All rights reserved.

View File

@@ -0,0 +1,117 @@
import type { ITapSettings, ISettingsManager } from './tapbundle.interfaces.js';
export class SettingsManager implements ISettingsManager {
private globalSettings: ITapSettings = {};
private fileSettings: ITapSettings = {};
private testSettings: Map<string, ITapSettings> = new Map();
// Default settings
private defaultSettings: ITapSettings = {
timeout: undefined, // No timeout by default
slowThreshold: 1000, // 1 second
bail: false,
retries: 0,
retryDelay: 0,
suppressConsole: false,
verboseErrors: true,
showTestDuration: true,
maxConcurrency: 5,
isolateTests: false,
enableSnapshots: true,
snapshotDirectory: '.snapshots',
updateSnapshots: false,
};
/**
* Get merged settings for current context
*/
public getSettings(): ITapSettings {
return this.mergeSettings(
this.defaultSettings,
this.globalSettings,
this.fileSettings
);
}
/**
* Set global settings (from 00init.ts or tap.settings())
*/
public setGlobalSettings(settings: ITapSettings): void {
this.globalSettings = { ...this.globalSettings, ...settings };
}
/**
* Set file-level settings
*/
public setFileSettings(settings: ITapSettings): void {
this.fileSettings = { ...this.fileSettings, ...settings };
}
/**
* Set test-specific settings
*/
public setTestSettings(testId: string, settings: ITapSettings): void {
const existingSettings = this.testSettings.get(testId) || {};
this.testSettings.set(testId, { ...existingSettings, ...settings });
}
/**
* Get settings for specific test
*/
public getTestSettings(testId: string): ITapSettings {
const testSpecificSettings = this.testSettings.get(testId) || {};
return this.mergeSettings(
this.defaultSettings,
this.globalSettings,
this.fileSettings,
testSpecificSettings
);
}
/**
* Merge settings with proper inheritance
* Later settings override earlier ones
*/
private mergeSettings(...settingsArray: ITapSettings[]): ITapSettings {
const result: ITapSettings = {};
for (const settings of settingsArray) {
// Simple properties - later values override
if (settings.timeout !== undefined) result.timeout = settings.timeout;
if (settings.slowThreshold !== undefined) result.slowThreshold = settings.slowThreshold;
if (settings.bail !== undefined) result.bail = settings.bail;
if (settings.retries !== undefined) result.retries = settings.retries;
if (settings.retryDelay !== undefined) result.retryDelay = settings.retryDelay;
if (settings.suppressConsole !== undefined) result.suppressConsole = settings.suppressConsole;
if (settings.verboseErrors !== undefined) result.verboseErrors = settings.verboseErrors;
if (settings.showTestDuration !== undefined) result.showTestDuration = settings.showTestDuration;
if (settings.maxConcurrency !== undefined) result.maxConcurrency = settings.maxConcurrency;
if (settings.isolateTests !== undefined) result.isolateTests = settings.isolateTests;
if (settings.enableSnapshots !== undefined) result.enableSnapshots = settings.enableSnapshots;
if (settings.snapshotDirectory !== undefined) result.snapshotDirectory = settings.snapshotDirectory;
if (settings.updateSnapshots !== undefined) result.updateSnapshots = settings.updateSnapshots;
// Lifecycle hooks - later ones override
if (settings.beforeAll !== undefined) result.beforeAll = settings.beforeAll;
if (settings.afterAll !== undefined) result.afterAll = settings.afterAll;
if (settings.beforeEach !== undefined) result.beforeEach = settings.beforeEach;
if (settings.afterEach !== undefined) result.afterEach = settings.afterEach;
// Environment variables - merge
if (settings.env) {
result.env = { ...result.env, ...settings.env };
}
}
return result;
}
/**
* Clear all settings (useful for testing)
*/
public clearSettings(): void {
this.globalSettings = {};
this.fileSettings = {};
this.testSettings.clear();
}
}

View File

@@ -2,6 +2,9 @@ import * as plugins from './tapbundle.plugins.js';
import { type IPreTaskFunction, PreTask } from './tapbundle.classes.pretask.js'; import { type IPreTaskFunction, PreTask } from './tapbundle.classes.pretask.js';
import { TapTest, type ITestFunction } from './tapbundle.classes.taptest.js'; import { TapTest, type ITestFunction } from './tapbundle.classes.taptest.js';
import { ProtocolEmitter, type ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
import type { ITapSettings } from './tapbundle.interfaces.js';
import { SettingsManager } from './tapbundle.classes.settingsmanager.js';
export interface ITestSuite { export interface ITestSuite {
description: string; description: string;
@@ -102,6 +105,8 @@ class TestBuilder<T> {
} }
export class Tap<T> { export class Tap<T> {
private protocolEmitter = new ProtocolEmitter();
private settingsManager = new SettingsManager();
private _skipCount = 0; private _skipCount = 0;
private _filterTags: string[] = []; private _filterTags: string[] = [];
@@ -139,12 +144,27 @@ export class Tap<T> {
*/ */
public skip = { public skip = {
test: (descriptionArg: string, functionArg: ITestFunction<T>) => { test: (descriptionArg: string, functionArg: ITestFunction<T>) => {
console.log(`skipped test: ${descriptionArg}`); const skippedTest = this.test(descriptionArg, functionArg, 'skip');
this._skipCount++; return skippedTest;
}, },
testParallel: (descriptionArg: string, functionArg: ITestFunction<T>) => { testParallel: (descriptionArg: string, functionArg: ITestFunction<T>) => {
console.log(`skipped test: ${descriptionArg}`); const skippedTest = new TapTest<T>({
this._skipCount++; description: descriptionArg,
testFunction: functionArg,
parallel: true,
});
// Mark as skip mode
skippedTest.tapTools.markAsSkipped('Marked as skip');
// Add to appropriate test list
if (this._currentSuite) {
this._currentSuite.tests.push(skippedTest);
} else {
this._tapTests.push(skippedTest);
}
return skippedTest;
}, },
}; };
@@ -153,7 +173,65 @@ export class Tap<T> {
*/ */
public only = { public only = {
test: (descriptionArg: string, testFunctionArg: ITestFunction<T>) => { test: (descriptionArg: string, testFunctionArg: ITestFunction<T>) => {
this.test(descriptionArg, testFunctionArg, 'only'); return this.test(descriptionArg, testFunctionArg, 'only');
},
testParallel: (descriptionArg: string, testFunctionArg: ITestFunction<T>) => {
const onlyTest = new TapTest<T>({
description: descriptionArg,
testFunction: testFunctionArg,
parallel: true,
});
// Add to only tests list
this._tapTestsOnly.push(onlyTest);
return onlyTest;
},
};
/**
* mark a test as todo (not yet implemented)
*/
public todo = {
test: (descriptionArg: string, functionArg?: ITestFunction<T>) => {
const defaultFunc = (async () => {}) as ITestFunction<T>;
const todoTest = new TapTest<T>({
description: descriptionArg,
testFunction: functionArg || defaultFunc,
parallel: false,
});
// Mark as todo
todoTest.tapTools.todo('Marked as todo');
// Add to appropriate test list
if (this._currentSuite) {
this._currentSuite.tests.push(todoTest);
} else {
this._tapTests.push(todoTest);
}
return todoTest;
},
testParallel: (descriptionArg: string, functionArg?: ITestFunction<T>) => {
const defaultFunc = (async () => {}) as ITestFunction<T>;
const todoTest = new TapTest<T>({
description: descriptionArg,
testFunction: functionArg || defaultFunc,
parallel: true,
});
// Mark as todo
todoTest.tapTools.todo('Marked as todo');
// Add to appropriate test list
if (this._currentSuite) {
this._currentSuite.tests.push(todoTest);
} else {
this._tapTests.push(todoTest);
}
return todoTest;
}, },
}; };
@@ -163,6 +241,21 @@ export class Tap<T> {
private _currentSuite: ITestSuite | null = null; private _currentSuite: ITestSuite | null = null;
private _rootSuites: ITestSuite[] = []; private _rootSuites: ITestSuite[] = [];
/**
* Configure global test settings
*/
public settings(settings: ITapSettings): this {
this.settingsManager.setGlobalSettings(settings);
return this;
}
/**
* Get current test settings
*/
public getSettings(): ITapSettings {
return this.settingsManager.getSettings();
}
/** /**
* Normal test function, will run one by one * Normal test function, will run one by one
* @param testDescription - A description of what the test does * @param testDescription - A description of what the test does
@@ -179,14 +272,26 @@ export class Tap<T> {
parallel: false, parallel: false,
}); });
// No options applied here - use the fluent builder syntax instead // Apply default settings from settings manager
const settings = this.settingsManager.getSettings();
if (settings.timeout !== undefined) {
localTest.timeoutMs = settings.timeout;
}
if (settings.retries !== undefined) {
localTest.tapTools.retry(settings.retries);
}
// Handle skip mode
if (modeArg === 'skip') {
localTest.tapTools.markAsSkipped('Marked as skip');
}
// If we're in a suite, add test to the suite // If we're in a suite, add test to the suite
if (this._currentSuite) { if (this._currentSuite) {
this._currentSuite.tests.push(localTest); this._currentSuite.tests.push(localTest);
} else { } else {
// Otherwise add to global test list // Otherwise add to global test list
if (modeArg === 'normal') { if (modeArg === 'normal' || modeArg === 'skip') {
this._tapTests.push(localTest); this._tapTests.push(localTest);
} else if (modeArg === 'only') { } else if (modeArg === 'only') {
this._tapTestsOnly.push(localTest); this._tapTestsOnly.push(localTest);
@@ -211,6 +316,15 @@ export class Tap<T> {
parallel: true, parallel: true,
}); });
// Apply default settings from settings manager
const settings = this.settingsManager.getSettings();
if (settings.timeout !== undefined) {
localTest.timeoutMs = settings.timeout;
}
if (settings.retries !== undefined) {
localTest.tapTools.retry(settings.retries);
}
if (this._currentSuite) { if (this._currentSuite) {
this._currentSuite.tests.push(localTest); this._currentSuite.tests.push(localTest);
} else { } else {
@@ -336,8 +450,27 @@ export class Tap<T> {
await preTask.run(); await preTask.run();
} }
// Count actual tests that will be run // Emit protocol header and TAP version
console.log(`1..${concerningTests.length}`); console.log(this.protocolEmitter.emitProtocolHeader());
console.log(this.protocolEmitter.emitTapVersion(13));
// Emit test plan
const plan = {
start: 1,
end: concerningTests.length
};
console.log(this.protocolEmitter.emitPlan(plan));
// Run global beforeAll hook if configured
const settings = this.settingsManager.getSettings();
if (settings.beforeAll) {
try {
await settings.beforeAll();
} catch (error) {
console.error('Error in beforeAll hook:', error);
throw error;
}
}
// Run tests from suites with lifecycle hooks // Run tests from suites with lifecycle hooks
let testKey = 0; let testKey = 0;
@@ -365,6 +498,33 @@ export class Tap<T> {
}); });
for (const currentTest of nonSuiteTests) { for (const currentTest of nonSuiteTests) {
// Wrap test function with global lifecycle hooks
const originalFunction = currentTest.testFunction;
const testName = currentTest.description;
currentTest.testFunction = async (tapTools) => {
// Run global beforeEach if configured
if (settings.beforeEach) {
await settings.beforeEach(testName);
}
// Run the actual test
let testPassed = true;
let result: any;
try {
result = await originalFunction(tapTools);
} catch (error) {
testPassed = false;
throw error;
} finally {
// Run global afterEach if configured
if (settings.afterEach) {
await settings.afterEach(testName, testPassed);
}
}
return result;
};
const testPromise = currentTest.run(testKey++); const testPromise = currentTest.run(testKey++);
if (currentTest.parallel) { if (currentTest.parallel) {
promiseArray.push(testPromise); promiseArray.push(testPromise);
@@ -394,6 +554,16 @@ export class Tap<T> {
console.log(failReason); console.log(failReason);
} }
// Run global afterAll hook if configured
if (settings.afterAll) {
try {
await settings.afterAll();
} catch (error) {
console.error('Error in afterAll hook:', error);
// Don't throw here, we want to complete the test run
}
}
if (optionsArg && optionsArg.throwOnError && failReasons.length > 0) { if (optionsArg && optionsArg.throwOnError && failReasons.length > 0) {
if (!smartenvInstance.isBrowser && typeof process !== 'undefined') process.exit(1); if (!smartenvInstance.isBrowser && typeof process !== 'undefined') process.exit(1);
} }
@@ -402,6 +572,13 @@ export class Tap<T> {
} }
} }
/**
* Emit an event
*/
private emitEvent(event: ITestEvent) {
console.log(this.protocolEmitter.emitEvent(event));
}
/** /**
* Run tests in a suite with lifecycle hooks * Run tests in a suite with lifecycle hooks
*/ */
@@ -412,6 +589,14 @@ export class Tap<T> {
context: { testKey: number } context: { testKey: number }
) { ) {
for (const suite of suites) { for (const suite of suites) {
// Emit suite:started event
this.emitEvent({
eventType: 'suite:started',
timestamp: Date.now(),
data: {
suiteName: suite.description
}
});
// Run beforeEach from parent suites // Run beforeEach from parent suites
const beforeEachFunctions: ITestFunction<any>[] = []; const beforeEachFunctions: ITestFunction<any>[] = [];
let currentSuite: ITestSuite | null = suite; let currentSuite: ITestSuite | null = suite;
@@ -426,15 +611,28 @@ export class Tap<T> {
for (const test of suite.tests) { for (const test of suite.tests) {
// Create wrapper test function that includes lifecycle hooks // Create wrapper test function that includes lifecycle hooks
const originalFunction = test.testFunction; const originalFunction = test.testFunction;
const testName = test.description;
test.testFunction = async (tapTools) => { test.testFunction = async (tapTools) => {
// Run all beforeEach hooks // Run global beforeEach if configured
const settings = this.settingsManager.getSettings();
if (settings.beforeEach) {
await settings.beforeEach(testName);
}
// Run all suite beforeEach hooks
for (const beforeEach of beforeEachFunctions) { for (const beforeEach of beforeEachFunctions) {
await beforeEach(tapTools); await beforeEach(tapTools);
} }
// Run the actual test // Run the actual test
const result = await originalFunction(tapTools); let testPassed = true;
let result: any;
try {
result = await originalFunction(tapTools);
} catch (error) {
testPassed = false;
throw error;
} finally {
// Run afterEach hooks in reverse order // Run afterEach hooks in reverse order
const afterEachFunctions: ITestFunction<any>[] = []; const afterEachFunctions: ITestFunction<any>[] = [];
currentSuite = suite; currentSuite = suite;
@@ -449,6 +647,12 @@ export class Tap<T> {
await afterEach(tapTools); await afterEach(tapTools);
} }
// Run global afterEach if configured
if (settings.afterEach) {
await settings.afterEach(testName, testPassed);
}
}
return result; return result;
}; };
@@ -462,6 +666,15 @@ export class Tap<T> {
// Recursively run child suites // Recursively run child suites
await this._runSuite(suite, suite.children, promiseArray, context); await this._runSuite(suite, suite.children, promiseArray, context);
// Emit suite:completed event
this.emitEvent({
eventType: 'suite:completed',
timestamp: Date.now(),
data: {
suiteName: suite.description
}
});
} }
} }

View File

@@ -1,6 +1,8 @@
import * as plugins from './tapbundle.plugins.js'; import * as plugins from './tapbundle.plugins.js';
import { tapCreator } from './tapbundle.tapcreator.js'; import { tapCreator } from './tapbundle.tapcreator.js';
import { TapTools, SkipError } from './tapbundle.classes.taptools.js'; import { TapTools, SkipError } from './tapbundle.classes.taptools.js';
import { ProtocolEmitter, type ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
import { setProtocolEmitter } from './tapbundle.expect.wrapper.js';
// imported interfaces // imported interfaces
import { Deferred } from '@push.rocks/smartpromise'; import { Deferred } from '@push.rocks/smartpromise';
@@ -9,9 +11,9 @@ import { HrtMeasurement } from '@push.rocks/smarttime';
// interfaces // interfaces
export type TTestStatus = 'success' | 'error' | 'pending' | 'errorAfterSuccess' | 'timeout' | 'skipped'; export type TTestStatus = 'success' | 'error' | 'pending' | 'errorAfterSuccess' | 'timeout' | 'skipped';
export interface ITestFunction<T> { export type ITestFunction<T> =
(tapTools?: TapTools): Promise<T>; | ((tapTools: TapTools) => Promise<T>)
} | (() => Promise<T>);
export class TapTest<T = unknown> { export class TapTest<T = unknown> {
public description: string; public description: string;
@@ -32,6 +34,7 @@ export class TapTest<T = unknown> {
public testPromise: Promise<TapTest<T>> = this.testDeferred.promise; public testPromise: Promise<TapTest<T>> = this.testDeferred.promise;
private testResultDeferred: Deferred<T> = plugins.smartpromise.defer(); private testResultDeferred: Deferred<T> = plugins.smartpromise.defer();
public testResultPromise: Promise<T> = this.testResultDeferred.promise; public testResultPromise: Promise<T> = this.testResultDeferred.promise;
private protocolEmitter = new ProtocolEmitter();
/** /**
* constructor * constructor
*/ */
@@ -48,6 +51,13 @@ export class TapTest<T = unknown> {
this.testFunction = optionsArg.testFunction; this.testFunction = optionsArg.testFunction;
} }
/**
* Emit an event
*/
private emitEvent(event: ITestEvent) {
console.log(this.protocolEmitter.emitEvent(event));
}
/** /**
* run the test * run the test
*/ */
@@ -55,11 +65,74 @@ export class TapTest<T = unknown> {
this.testKey = testKeyArg; this.testKey = testKeyArg;
const testNumber = testKeyArg + 1; const testNumber = testKeyArg + 1;
// Emit test:queued event
this.emitEvent({
eventType: 'test:queued',
timestamp: Date.now(),
data: {
testNumber,
description: this.description
}
});
// Handle todo tests // Handle todo tests
if (this.isTodo) { if (this.isTodo) {
const todoText = this.todoReason ? `# TODO ${this.todoReason}` : '# TODO'; const testResult = {
console.log(`ok ${testNumber} - ${this.description} ${todoText}`); ok: true,
testNumber,
description: this.description,
directive: {
type: 'todo' as const,
reason: this.todoReason
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'success'; this.status = 'success';
// Emit test:completed event for todo test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: 0,
error: undefined
}
});
this.testDeferred.resolve(this);
return;
}
// Handle pre-marked skip tests
if (this.tapTools.isSkipped) {
const testResult = {
ok: true,
testNumber,
description: this.description,
directive: {
type: 'skip' as const,
reason: this.tapTools.skipReason || 'Marked as skip'
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'skipped';
// Emit test:completed event for skipped test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: 0,
error: undefined
}
});
this.testDeferred.resolve(this); this.testDeferred.resolve(this);
return; return;
} }
@@ -71,6 +144,20 @@ export class TapTest<T = unknown> {
for (let attempt = 0; attempt <= maxRetries; attempt++) { for (let attempt = 0; attempt <= maxRetries; attempt++) {
this.hrtMeasurement.start(); this.hrtMeasurement.start();
// Emit test:started event
this.emitEvent({
eventType: 'test:started',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
retry: attempt > 0 ? attempt : undefined
}
});
// Set protocol emitter for enhanced expect
setProtocolEmitter(this.protocolEmitter);
try { try {
// Set up timeout if specified // Set up timeout if specified
let timeoutHandle: any; let timeoutHandle: any;
@@ -86,7 +173,9 @@ export class TapTest<T = unknown> {
} }
// Run the test function with potential timeout // Run the test function with potential timeout
const testPromise = this.testFunction(this.tapTools); const testPromise = this.testFunction.length === 0
? (this.testFunction as () => Promise<T>)()
: (this.testFunction as (tapTools: TapTools) => Promise<T>)(this.tapTools);
const testReturnValue = timeoutPromise const testReturnValue = timeoutPromise
? await Promise.race([testPromise, timeoutPromise]) ? await Promise.race([testPromise, timeoutPromise])
: await testPromise; : await testPromise;
@@ -97,10 +186,32 @@ export class TapTest<T = unknown> {
} }
this.hrtMeasurement.stop(); this.hrtMeasurement.stop();
console.log( const testResult = {
`ok ${testNumber} - ${this.description} # time=${this.hrtMeasurement.milliSeconds}ms`, ok: true,
); testNumber,
description: this.description,
metadata: {
time: this.hrtMeasurement.milliSeconds,
tags: this.tags.length > 0 ? this.tags : undefined,
file: this.fileName
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'success'; this.status = 'success';
// Emit test:completed event
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: this.hrtMeasurement.milliSeconds,
error: undefined
}
});
this.testDeferred.resolve(this); this.testDeferred.resolve(this);
this.testResultDeferred.resolve(testReturnValue); this.testResultDeferred.resolve(testReturnValue);
return; // Success, exit retry loop return; // Success, exit retry loop
@@ -110,8 +221,31 @@ export class TapTest<T = unknown> {
// Handle skip // Handle skip
if (err instanceof SkipError || err.name === 'SkipError') { if (err instanceof SkipError || err.name === 'SkipError') {
console.log(`ok ${testNumber} - ${this.description} # SKIP ${err.message.replace('Skipped: ', '')}`); const testResult = {
ok: true,
testNumber,
description: this.description,
directive: {
type: 'skip' as const,
reason: err.message.replace('Skipped: ', '')
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
this.status = 'skipped'; this.status = 'skipped';
// Emit test:completed event for skipped test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: this.hrtMeasurement.milliSeconds,
error: undefined
}
});
this.testDeferred.resolve(this); this.testDeferred.resolve(this);
return; return;
} }
@@ -120,17 +254,48 @@ export class TapTest<T = unknown> {
// If we have retries left, try again // If we have retries left, try again
if (attempt < maxRetries) { if (attempt < maxRetries) {
console.log( console.log(this.protocolEmitter.emitComment(`Retry ${attempt + 1}/${maxRetries} for test: ${this.description}`));
`# Retry ${attempt + 1}/${maxRetries} for test: ${this.description}`,
);
this.tapTools._incrementRetryCount(); this.tapTools._incrementRetryCount();
continue; continue;
} }
// Final failure // Final failure
console.log( const testResult = {
`not ok ${testNumber} - ${this.description} # time=${this.hrtMeasurement.milliSeconds}ms`, ok: false,
); testNumber,
description: this.description,
metadata: {
time: this.hrtMeasurement.milliSeconds,
retry: this.tapTools.retryCount,
maxRetries: maxRetries > 0 ? maxRetries : undefined,
error: {
message: lastError.message || String(lastError),
stack: lastError.stack,
code: lastError.code
},
tags: this.tags.length > 0 ? this.tags : undefined,
file: this.fileName
}
};
const lines = this.protocolEmitter.emitTest(testResult);
lines.forEach((line: string) => console.log(line));
// Emit test:completed event for failed test
this.emitEvent({
eventType: 'test:completed',
timestamp: Date.now(),
data: {
testNumber,
description: this.description,
duration: this.hrtMeasurement.milliSeconds,
error: {
message: lastError.message || String(lastError),
stack: lastError.stack,
type: 'runtime' as const
}
}
});
this.testDeferred.resolve(this); this.testDeferred.resolve(this);
this.testResultDeferred.resolve(err); this.testResultDeferred.resolve(err);

View File

@@ -23,6 +23,10 @@ export class TapTools {
private static _sharedContext = new Map<string, any>(); private static _sharedContext = new Map<string, any>();
private _snapshotPath: string = ''; private _snapshotPath: string = '';
// Flags for skip/todo
private _isSkipped = false;
private _skipReason?: string;
constructor(TapTestArg: TapTest<any>) { constructor(TapTestArg: TapTest<any>) {
this._tapTest = TapTestArg; this._tapTest = TapTestArg;
// Generate snapshot path based on test file and test name // Generate snapshot path based on test file and test name
@@ -45,10 +49,34 @@ export class TapTools {
* skip the rest of the test * skip the rest of the test
*/ */
public skip(reason?: string): never { public skip(reason?: string): never {
this._isSkipped = true;
this._skipReason = reason;
const skipMessage = reason ? `Skipped: ${reason}` : 'Skipped'; const skipMessage = reason ? `Skipped: ${reason}` : 'Skipped';
throw new SkipError(skipMessage); throw new SkipError(skipMessage);
} }
/**
* Mark test as skipped without throwing (for pre-marking)
*/
public markAsSkipped(reason?: string): void {
this._isSkipped = true;
this._skipReason = reason;
}
/**
* Check if test is marked as skipped
*/
public get isSkipped(): boolean {
return this._isSkipped;
}
/**
* Get skip reason
*/
public get skipReason(): string | undefined {
return this._skipReason;
}
/** /**
* conditionally skip the rest of the test * conditionally skip the rest of the test
*/ */

View File

@@ -0,0 +1,81 @@
import { expect as smartExpect } from '@push.rocks/smartexpect';
import { generateDiff } from './tapbundle.utilities.diff.js';
import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';
import type { IEnhancedError } from '../dist_ts_tapbundle_protocol/index.js';
// Store the protocol emitter for event emission
let protocolEmitter: ProtocolEmitter | null = null;
/**
* Set the protocol emitter for enhanced error reporting
*/
export function setProtocolEmitter(emitter: ProtocolEmitter) {
protocolEmitter = emitter;
}
/**
* Enhanced expect wrapper that captures assertion failures and generates diffs
*/
export function createEnhancedExpect() {
return new Proxy(smartExpect, {
apply(target, thisArg, argumentsList: any[]) {
const expectation = target.apply(thisArg, argumentsList);
// Wrap common assertion methods
const wrappedExpectation = new Proxy(expectation, {
get(target, prop, receiver) {
const originalValue = Reflect.get(target, prop, receiver);
// Wrap assertion methods that compare values
if (typeof prop === 'string' && typeof originalValue === 'function' && ['toEqual', 'toBe', 'toMatch', 'toContain'].includes(prop)) {
return function(expected: any) {
try {
return originalValue.apply(target, arguments);
} catch (error: any) {
// Enhance the error with diff information
const actual = argumentsList[0];
const enhancedError: IEnhancedError = {
message: error.message,
stack: error.stack,
actual,
expected,
type: 'assertion'
};
// Generate diff if applicable
if (prop === 'toEqual' || prop === 'toBe') {
const diff = generateDiff(expected, actual);
if (diff) {
enhancedError.diff = diff;
}
}
// Emit assertion:failed event if protocol emitter is available
if (protocolEmitter) {
const event = {
eventType: 'assertion:failed' as const,
timestamp: Date.now(),
data: {
error: enhancedError
}
};
console.log(protocolEmitter.emitEvent(event));
}
// Re-throw the enhanced error
throw error;
}
};
}
return originalValue;
}
});
return wrappedExpectation;
}
});
}
// Create the enhanced expect function
export const expect = createEnhancedExpect();

View File

@@ -0,0 +1,46 @@
export interface ITapSettings {
// Timing
timeout?: number; // Default timeout for all tests (ms)
slowThreshold?: number; // Mark tests as slow if they exceed this (ms)
// Execution Control
bail?: boolean; // Stop on first test failure
retries?: number; // Number of retries for failed tests
retryDelay?: number; // Delay between retries (ms)
// Output Control
suppressConsole?: boolean; // Suppress console output in passing tests
verboseErrors?: boolean; // Show full stack traces
showTestDuration?: boolean; // Show duration for each test
// Parallel Execution
maxConcurrency?: number; // Max parallel tests (for .para files)
isolateTests?: boolean; // Run each test in fresh context
// Lifecycle Hooks
beforeAll?: () => Promise<void> | void;
afterAll?: () => Promise<void> | void;
beforeEach?: (testName: string) => Promise<void> | void;
afterEach?: (testName: string, passed: boolean) => Promise<void> | void;
// Environment
env?: Record<string, string>; // Additional environment variables
// Features
enableSnapshots?: boolean; // Enable snapshot testing
snapshotDirectory?: string; // Custom snapshot directory
updateSnapshots?: boolean; // Update snapshots instead of comparing
}
export interface ISettingsManager {
// Get merged settings for current context
getSettings(): ITapSettings;
// Apply settings at different levels
setGlobalSettings(settings: ITapSettings): void;
setFileSettings(settings: ITapSettings): void;
setTestSettings(testId: string, settings: ITapSettings): void;
// Get settings for specific test
getTestSettings(testId: string): ITapSettings;
}

View File

@@ -0,0 +1,188 @@
import type { IDiffResult, IDiffChange } from '../dist_ts_tapbundle_protocol/index.js';
/**
* Generate a diff between two values
*/
export function generateDiff(expected: any, actual: any, context: number = 3): IDiffResult | null {
// Handle same values
if (expected === actual) {
return null;
}
// Determine diff type based on values
if (typeof expected === 'string' && typeof actual === 'string') {
return generateStringDiff(expected, actual, context);
} else if (Array.isArray(expected) && Array.isArray(actual)) {
return generateArrayDiff(expected, actual);
} else if (expected && actual && typeof expected === 'object' && typeof actual === 'object') {
return generateObjectDiff(expected, actual);
} else {
return generatePrimitiveDiff(expected, actual);
}
}
/**
* Generate diff for primitive values
*/
function generatePrimitiveDiff(expected: any, actual: any): IDiffResult {
return {
type: 'primitive',
changes: [{
type: 'modify',
oldValue: expected,
newValue: actual
}]
};
}
/**
* Generate diff for strings (line-by-line)
*/
function generateStringDiff(expected: string, actual: string, context: number): IDiffResult {
const expectedLines = expected.split('\n');
const actualLines = actual.split('\n');
const changes: IDiffChange[] = [];
// Simple line-by-line diff
const maxLines = Math.max(expectedLines.length, actualLines.length);
for (let i = 0; i < maxLines; i++) {
const expectedLine = expectedLines[i];
const actualLine = actualLines[i];
if (expectedLine === undefined) {
changes.push({
type: 'add',
line: i,
content: actualLine
});
} else if (actualLine === undefined) {
changes.push({
type: 'remove',
line: i,
content: expectedLine
});
} else if (expectedLine !== actualLine) {
changes.push({
type: 'remove',
line: i,
content: expectedLine
});
changes.push({
type: 'add',
line: i,
content: actualLine
});
}
}
return {
type: 'string',
changes,
context
};
}
/**
* Generate diff for arrays
*/
function generateArrayDiff(expected: any[], actual: any[]): IDiffResult {
const changes: IDiffChange[] = [];
const maxLength = Math.max(expected.length, actual.length);
for (let i = 0; i < maxLength; i++) {
const expectedItem = expected[i];
const actualItem = actual[i];
if (i >= expected.length) {
changes.push({
type: 'add',
path: [String(i)],
newValue: actualItem
});
} else if (i >= actual.length) {
changes.push({
type: 'remove',
path: [String(i)],
oldValue: expectedItem
});
} else if (!deepEqual(expectedItem, actualItem)) {
changes.push({
type: 'modify',
path: [String(i)],
oldValue: expectedItem,
newValue: actualItem
});
}
}
return {
type: 'array',
changes
};
}
/**
* Generate diff for objects
*/
function generateObjectDiff(expected: any, actual: any): IDiffResult {
const changes: IDiffChange[] = [];
const allKeys = new Set([...Object.keys(expected), ...Object.keys(actual)]);
for (const key of allKeys) {
const expectedValue = expected[key];
const actualValue = actual[key];
if (!(key in expected)) {
changes.push({
type: 'add',
path: [key],
newValue: actualValue
});
} else if (!(key in actual)) {
changes.push({
type: 'remove',
path: [key],
oldValue: expectedValue
});
} else if (!deepEqual(expectedValue, actualValue)) {
changes.push({
type: 'modify',
path: [key],
oldValue: expectedValue,
newValue: actualValue
});
}
}
return {
type: 'object',
changes
};
}
/**
* Deep equality check
*/
function deepEqual(a: any, b: any): boolean {
if (a === b) return true;
if (a === null || b === null) return false;
if (typeof a !== typeof b) return false;
if (typeof a === 'object') {
if (Array.isArray(a) && Array.isArray(b)) {
if (a.length !== b.length) return false;
return a.every((item, index) => deepEqual(item, b[index]));
}
const keysA = Object.keys(a);
const keysB = Object.keys(b);
if (keysA.length !== keysB.length) return false;
return keysA.every(key => deepEqual(a[key], b[key]));
}
return false;
}

View File

@@ -1,3 +1,3 @@
{ {
"order": 1 "order": 2
} }

View File

@@ -9,9 +9,12 @@ export class TestFileProvider {
public async getDockerAlpineImageAsLocalTarball(): Promise<string> { public async getDockerAlpineImageAsLocalTarball(): Promise<string> {
const filePath = plugins.path.join(paths.testFilesDir, 'alpine.tar') const filePath = plugins.path.join(paths.testFilesDir, 'alpine.tar')
// fetch the docker alpine image // fetch the docker alpine image
const response = await plugins.smartrequest.getBinary(fileUrls.dockerAlpineImage); const response = await plugins.smartrequest.SmartRequest.create()
.url(fileUrls.dockerAlpineImage)
.get();
await plugins.smartfile.fs.ensureDir(paths.testFilesDir); await plugins.smartfile.fs.ensureDir(paths.testFilesDir);
await plugins.smartfile.memory.toFs(response.body, filePath); const buffer = Buffer.from(await response.arrayBuffer());
await plugins.smartfile.memory.toFs(buffer, filePath);
return filePath; return filePath;
} }
} }

367
ts_tapbundle_node/readme.md Normal file
View File

@@ -0,0 +1,367 @@
# @git.zone/tstest/tapbundle_node
> 🔧 Node.js-specific testing utilities for enhanced test capabilities
## Installation
```bash
# tapbundle_node is included as part of @git.zone/tstest
pnpm install --save-dev @git.zone/tstest
```
## Overview
`@git.zone/tstest/tapbundle_node` provides Node.js-specific utilities for testing. These tools are only available when running tests in Node.js runtime and provide functionality for working with environment variables, shell commands, test databases, storage systems, and HTTPS certificates.
## Key Features
- 🔐 **Environment Variables** - On-demand environment variable loading with qenv
- 💻 **Shell Commands** - Execute bash commands during tests
- 🔒 **HTTPS Certificates** - Generate self-signed certificates for testing
- 🗄️ **MongoDB Testing** - Create ephemeral MongoDB instances
- 📦 **S3 Storage Testing** - Create local S3-compatible storage for tests
- 📁 **Test File Management** - Download and manage test assets
## Basic Usage
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_node';
import { tap } from '@git.zone/tstest/tapbundle';
tap.test('should use node-specific tools', async () => {
// Use Node.js-specific utilities
const result = await tapNodeTools.runCommand('echo "hello"');
console.log(result);
});
export default tap.start();
```
## API Reference
### tapNodeTools
The main singleton instance providing all Node.js-specific utilities.
#### Environment Variables
##### `getQenv()`
Get the qenv instance for managing environment variables from `.nogit/` directory.
```typescript
const qenv = await tapNodeTools.getQenv();
// qenv will load from .env files in .nogit/ directory
```
##### `getEnvVarOnDemand(envVarName)`
Request an environment variable. If not available, qenv will prompt for it and store it securely.
```typescript
tap.test('should get API key', async () => {
const apiKey = await tapNodeTools.getEnvVarOnDemand('GITHUB_API_KEY');
// If GITHUB_API_KEY is not set, qenv will prompt for it
// The value is stored in .nogit/.env for future use
});
```
**Use Cases:**
- API keys for integration tests
- Database credentials
- Service endpoints
- Any sensitive configuration needed for testing
#### Shell Commands
##### `runCommand(command)`
Execute a bash command and return the result.
```typescript
tap.test('should execute shell commands', async () => {
const result = await tapNodeTools.runCommand('ls -la');
console.log(result.stdout);
});
```
**Use Cases:**
- Setup test environment
- Execute CLI tools
- File system operations
- Process management
#### HTTPS Certificates
##### `createHttpsCert(commonName?, allowSelfSigned?)`
Generate a self-signed HTTPS certificate for testing secure connections.
```typescript
tap.test('should create HTTPS server', async () => {
const { key, cert } = await tapNodeTools.createHttpsCert('localhost', true);
// Use with Node.js https module
const server = https.createServer({ key, cert }, (req, res) => {
res.end('Hello Secure World');
});
server.listen(3000);
});
```
**Parameters:**
- `commonName` (optional): Certificate common name, default: 'localhost'
- `allowSelfSigned` (optional): Allow self-signed certificates by setting `NODE_TLS_REJECT_UNAUTHORIZED=0`, default: true
**Returns:**
- `key`: PEM-encoded private key
- `cert`: PEM-encoded certificate
**Use Cases:**
- Testing HTTPS servers
- Testing secure WebSocket connections
- Testing certificate validation logic
- Mocking secure external services
#### Database Testing
##### `createSmartmongo()`
Create an ephemeral MongoDB instance for testing. Automatically started and ready to use.
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_node';
tap.test('should use MongoDB', async () => {
const mongoInstance = await tapNodeTools.createSmartmongo();
// Use the MongoDB instance
const db = await mongoInstance.getDatabase('testdb');
const collection = await db.getCollection('users');
await collection.insertOne({ name: 'Alice', age: 30 });
const user = await collection.findOne({ name: 'Alice' });
expect(user.age).toEqual(30);
// Cleanup (optional - instance will be cleaned up automatically)
await mongoInstance.stop();
});
export default tap.start();
```
**Features:**
- Ephemeral instance (starts fresh)
- Automatic cleanup
- Full MongoDB API via [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo)
**Use Cases:**
- Testing database operations
- Integration tests with MongoDB
- Testing data models
- Schema validation tests
#### Storage Testing
##### `createSmarts3()`
Create a local S3-compatible storage instance for testing object storage operations.
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_node';
tap.test('should use S3 storage', async () => {
const s3Instance = await tapNodeTools.createSmarts3();
// Use the S3 instance (MinIO-compatible API)
const bucket = await s3Instance.createBucket('test-bucket');
await bucket.putObject('file.txt', Buffer.from('Hello S3'));
const file = await bucket.getObject('file.txt');
expect(file.toString()).toEqual('Hello S3');
// Cleanup
await s3Instance.stop();
});
export default tap.start();
```
**Configuration:**
- Port: 3003 (default)
- Clean slate: true (starts fresh each time)
- Full S3-compatible API via [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3)
**Use Cases:**
- Testing file uploads/downloads
- Testing object storage operations
- Testing backup/restore logic
- Mocking cloud storage
### TestFileProvider
Utility for downloading and managing test assets.
#### `getDockerAlpineImageAsLocalTarball()`
Download the Alpine Linux Docker image as a tarball for testing.
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_node';
tap.test('should provide docker image', async () => {
const tarballPath = await tapNodeTools.testFileProvider.getDockerAlpineImageAsLocalTarball();
// Use the tarball path
// Path: ./.nogit/testfiles/alpine.tar
expect(tarballPath).toMatch(/alpine\.tar$/);
});
export default tap.start();
```
**Features:**
- Downloads from https://code.foss.global/testassets/docker
- Caches in `.nogit/testfiles/` directory
- Returns local file path
**Use Cases:**
- Testing Docker operations
- Testing container deployment
- Testing image handling logic
### Path Utilities
The module exports useful path constants:
```typescript
import * as paths from '@git.zone/tstest/tapbundle_node/paths';
console.log(paths.cwd); // Current working directory
console.log(paths.testFilesDir); // ./.nogit/testfiles/
```
## Patterns and Best Practices
### Testing with External Services
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_node';
import { tap, expect } from '@git.zone/tstest/tapbundle';
tap.describe('User Service Integration', () => {
let mongoInstance;
let db;
tap.beforeEach(async () => {
mongoInstance = await tapNodeTools.createSmartmongo();
db = await mongoInstance.getDatabase('testdb');
});
tap.test('should create user', async () => {
const users = await db.getCollection('users');
await users.insertOne({ name: 'Bob', email: 'bob@example.com' });
const user = await users.findOne({ name: 'Bob' });
expect(user.email).toEqual('bob@example.com');
});
tap.afterEach(async () => {
await mongoInstance.stop();
});
});
export default tap.start();
```
### Testing HTTPS Servers
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_node';
import { tap, expect } from '@git.zone/tstest/tapbundle';
import * as https from 'https';
tap.test('should serve over HTTPS', async () => {
const { key, cert } = await tapNodeTools.createHttpsCert();
const server = https.createServer({ key, cert }, (req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Secure Response');
});
await new Promise((resolve) => {
server.listen(8443, () => resolve(undefined));
});
// Test the server
const response = await fetch('https://localhost:8443');
const text = await response.text();
expect(text).toEqual('Secure Response');
// Cleanup
server.close();
});
export default tap.start();
```
### Environment-Dependent Tests
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_node';
import { tap, expect } from '@git.zone/tstest/tapbundle';
tap.test('should authenticate with GitHub', async () => {
const githubToken = await tapNodeTools.getEnvVarOnDemand('GITHUB_TOKEN');
// Use the token for API calls
const response = await fetch('https://api.github.com/user', {
headers: {
Authorization: `Bearer ${githubToken}`
}
});
expect(response.ok).toBeTruthy();
});
export default tap.start();
```
## Runtime Requirements
⚠️ **Node.js Only**: All utilities in this module require Node.js runtime. They will not work in:
- Browser environments
- Deno runtime
- Bun runtime
For multi-runtime tests, use these utilities only in `.node.ts` test files.
## File Naming
Use Node.js-specific file naming when using these utilities:
```
test/mytest.node.ts ✅ Node.js only
test/mytest.node+deno.ts ❌ Will fail in Deno
test/mytest.browser+node.ts ⚠️ Browser won't have access to these tools
```
## Dependencies
This module uses the following packages:
- [@push.rocks/qenv](https://code.foss.global/push.rocks/qenv) - Environment variable management
- [@push.rocks/smartshell](https://code.foss.global/push.rocks/smartshell) - Shell command execution
- [@push.rocks/smartcrypto](https://code.foss.global/push.rocks/smartcrypto) - Certificate generation
- [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo) - MongoDB testing
- [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3) - S3 storage testing
- [@push.rocks/smartfile](https://code.foss.global/push.rocks/smartfile) - File operations
- [@push.rocks/smartrequest](https://code.foss.global/push.rocks/smartrequest) - HTTP requests
## Legal
This project is licensed under MIT.
© 2025 Task Venture Capital GmbH. All rights reserved.

View File

@@ -0,0 +1,3 @@
{
"order": 3
}

View File

@@ -0,0 +1,13 @@
// Protocol V2 - Isomorphic implementation for improved TAP protocol
// This module is designed to work in both browser and Node.js environments
export * from './protocol.types.js';
export * from './protocol.emitter.js';
export * from './protocol.parser.js';
// Re-export main classes for convenience
export { ProtocolEmitter } from './protocol.emitter.js';
export { ProtocolParser } from './protocol.parser.js';
// Re-export constants
export { PROTOCOL_MARKERS, PROTOCOL_VERSION } from './protocol.types.js';

View File

@@ -0,0 +1,196 @@
import type {
ITestResult,
ITestMetadata,
IPlanLine,
ISnapshotData,
IErrorBlock,
ITestEvent
} from './protocol.types.js';
import {
PROTOCOL_MARKERS,
PROTOCOL_VERSION
} from './protocol.types.js';
/**
* ProtocolEmitter generates Protocol V2 messages
* This class is used by tapbundle to emit test results in the new protocol format
*/
export class ProtocolEmitter {
/**
* Emit protocol version header
*/
public emitProtocolHeader(): string {
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.PROTOCOL_PREFIX}${PROTOCOL_VERSION}${PROTOCOL_MARKERS.END}`;
}
/**
* Emit TAP version line
*/
public emitTapVersion(version: number = 13): string {
return `TAP version ${version}`;
}
/**
* Emit test plan
*/
public emitPlan(plan: IPlanLine): string {
if (plan.skipAll) {
return `1..0 # Skipped: ${plan.skipAll}`;
}
return `${plan.start}..${plan.end}`;
}
/**
* Emit a test result
*/
public emitTest(result: ITestResult): string[] {
const lines: string[] = [];
// Build the basic TAP line
let tapLine = result.ok ? 'ok' : 'not ok';
tapLine += ` ${result.testNumber}`;
tapLine += ` - ${result.description}`;
// Add directive if present
if (result.directive) {
tapLine += ` # ${result.directive.type.toUpperCase()}`;
if (result.directive.reason) {
tapLine += ` ${result.directive.reason}`;
}
}
// Add inline metadata for simple cases
if (result.metadata && this.shouldUseInlineMetadata(result.metadata)) {
const metaStr = this.createInlineMetadata(result.metadata);
if (metaStr) {
tapLine += ` ${metaStr}`;
}
}
lines.push(tapLine);
// Add block metadata for complex cases
if (result.metadata && !this.shouldUseInlineMetadata(result.metadata)) {
lines.push(...this.createBlockMetadata(result.metadata, result.testNumber));
}
return lines;
}
/**
* Emit a comment line
*/
public emitComment(comment: string): string {
return `# ${comment}`;
}
/**
* Emit bailout
*/
public emitBailout(reason: string): string {
return `Bail out! ${reason}`;
}
/**
* Emit snapshot data
*/
public emitSnapshot(snapshot: ISnapshotData): string[] {
const lines: string[] = [];
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_PREFIX}${snapshot.name}${PROTOCOL_MARKERS.END}`);
if (snapshot.format === 'json') {
lines.push(JSON.stringify(snapshot.content, null, 2));
} else {
lines.push(String(snapshot.content));
}
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_END}${PROTOCOL_MARKERS.END}`);
return lines;
}
/**
* Emit error block
*/
public emitError(error: IErrorBlock): string[] {
const lines: string[] = [];
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_PREFIX}${PROTOCOL_MARKERS.END}`);
lines.push(JSON.stringify(error, null, 2));
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_END}${PROTOCOL_MARKERS.END}`);
return lines;
}
/**
* Emit test event
*/
public emitEvent(event: ITestEvent): string {
const eventJson = JSON.stringify(event);
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.EVENT_PREFIX}${eventJson}${PROTOCOL_MARKERS.END}`;
}
/**
* Check if metadata should be inline
*/
private shouldUseInlineMetadata(metadata: ITestMetadata): boolean {
// Use inline for simple metadata (time, retry, simple skip/todo)
const hasComplexData = metadata.error ||
metadata.custom ||
(metadata.tags && metadata.tags.length > 0) ||
metadata.file ||
metadata.line;
return !hasComplexData;
}
/**
* Create inline metadata string
*/
private createInlineMetadata(metadata: ITestMetadata): string {
const parts: string[] = [];
if (metadata.time !== undefined) {
parts.push(`time:${metadata.time}`);
}
if (metadata.retry !== undefined) {
parts.push(`retry:${metadata.retry}`);
}
if (metadata.skip) {
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SKIP_PREFIX}${metadata.skip}${PROTOCOL_MARKERS.END}`;
}
if (metadata.todo) {
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.TODO_PREFIX}${metadata.todo}${PROTOCOL_MARKERS.END}`;
}
if (parts.length > 0) {
return `${PROTOCOL_MARKERS.START}${parts.join(',')}${PROTOCOL_MARKERS.END}`;
}
return '';
}
/**
* Create block metadata lines
*/
private createBlockMetadata(metadata: ITestMetadata, testNumber?: number): string[] {
const lines: string[] = [];
// Create a clean metadata object without skip/todo (handled inline)
const blockMeta = { ...metadata };
delete blockMeta.skip;
delete blockMeta.todo;
// Emit metadata block
const metaJson = JSON.stringify(blockMeta);
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.META_PREFIX}${metaJson}${PROTOCOL_MARKERS.END}`);
// Emit separate error block if present
if (metadata.error) {
lines.push(...this.emitError({ testNumber, error: metadata.error }));
}
return lines;
}
}

View File

@@ -0,0 +1,407 @@
import type {
ITestResult,
ITestMetadata,
IPlanLine,
IProtocolMessage,
ISnapshotData,
IErrorBlock,
ITestEvent
} from './protocol.types.js';
import {
PROTOCOL_MARKERS
} from './protocol.types.js';
/**
* ProtocolParser parses Protocol V2 messages
* This class is used by tstest to parse test results from the new protocol format
*/
export class ProtocolParser {
private protocolVersion: string | null = null;
private inBlock = false;
private blockType: string | null = null;
private blockContent: string[] = [];
/**
* Parse a single line and return protocol messages
*/
public parseLine(line: string): IProtocolMessage[] {
const messages: IProtocolMessage[] = [];
// Handle block content
if (this.inBlock) {
if (this.isBlockEnd(line)) {
messages.push(this.finalizeBlock());
this.inBlock = false;
this.blockType = null;
this.blockContent = [];
} else {
this.blockContent.push(line);
}
return messages;
}
// Check for block start
if (this.isBlockStart(line)) {
this.inBlock = true;
this.blockType = this.extractBlockType(line);
return messages;
}
// Check for protocol version
const protocolVersion = this.parseProtocolVersion(line);
if (protocolVersion) {
this.protocolVersion = protocolVersion;
messages.push({
type: 'protocol',
content: { version: protocolVersion }
});
return messages;
}
// Parse TAP version
const tapVersion = this.parseTapVersion(line);
if (tapVersion !== null) {
messages.push({
type: 'version',
content: tapVersion
});
return messages;
}
// Parse plan
const plan = this.parsePlan(line);
if (plan) {
messages.push({
type: 'plan',
content: plan
});
return messages;
}
// Parse bailout
const bailout = this.parseBailout(line);
if (bailout) {
messages.push({
type: 'bailout',
content: bailout
});
return messages;
}
// Parse comment
if (this.isComment(line)) {
messages.push({
type: 'comment',
content: line.substring(2) // Remove "# "
});
return messages;
}
// Parse test result
const testResult = this.parseTestResult(line);
if (testResult) {
messages.push({
type: 'test',
content: testResult
});
return messages;
}
// Parse event
const event = this.parseEvent(line);
if (event) {
messages.push({
type: 'event',
content: event
});
return messages;
}
return messages;
}
/**
* Parse protocol version header
*/
private parseProtocolVersion(line: string): string | null {
const match = this.extractProtocolData(line, PROTOCOL_MARKERS.PROTOCOL_PREFIX);
return match;
}
/**
* Parse TAP version line
*/
private parseTapVersion(line: string): number | null {
const match = line.match(/^TAP version (\d+)$/);
if (match) {
return parseInt(match[1], 10);
}
return null;
}
/**
* Parse plan line
*/
private parsePlan(line: string): IPlanLine | null {
// Skip all plan
const skipMatch = line.match(/^1\.\.0\s*#\s*Skipped:\s*(.*)$/);
if (skipMatch) {
return {
start: 1,
end: 0,
skipAll: skipMatch[1]
};
}
// Normal plan
const match = line.match(/^(\d+)\.\.(\d+)$/);
if (match) {
return {
start: parseInt(match[1], 10),
end: parseInt(match[2], 10)
};
}
return null;
}
/**
* Parse bailout
*/
private parseBailout(line: string): string | null {
const match = line.match(/^Bail out!\s*(.*)$/);
return match ? match[1] : null;
}
/**
* Parse event
*/
private parseEvent(line: string): ITestEvent | null {
const eventData = this.extractProtocolData(line, PROTOCOL_MARKERS.EVENT_PREFIX);
if (eventData) {
try {
return JSON.parse(eventData) as ITestEvent;
} catch (e) {
// Invalid JSON, ignore
return null;
}
}
return null;
}
/**
* Check if line is a comment
*/
private isComment(line: string): boolean {
return line.startsWith('# ');
}
/**
* Parse test result line
*/
private parseTestResult(line: string): ITestResult | null {
// First extract any inline metadata
const metadata = this.extractInlineMetadata(line);
const cleanLine = this.removeInlineMetadata(line);
// Parse the TAP part
const tapMatch = cleanLine.match(/^(ok|not ok)\s+(\d+)\s*-?\s*(.*)$/);
if (!tapMatch) {
return null;
}
const result: ITestResult = {
ok: tapMatch[1] === 'ok',
testNumber: parseInt(tapMatch[2], 10),
description: tapMatch[3].trim()
};
// Parse directive
const directiveMatch = result.description.match(/^(.*?)\s*#\s*(SKIP|TODO)\s*(.*)$/i);
if (directiveMatch) {
result.description = directiveMatch[1].trim();
result.directive = {
type: directiveMatch[2].toLowerCase() as 'skip' | 'todo',
reason: directiveMatch[3] || undefined
};
}
// Add metadata if found
if (metadata) {
result.metadata = metadata;
}
return result;
}
/**
* Extract inline metadata from line
*/
private extractInlineMetadata(line: string): ITestMetadata | null {
const metadata: ITestMetadata = {};
let hasData = false;
// Extract skip reason
const skipData = this.extractProtocolData(line, PROTOCOL_MARKERS.SKIP_PREFIX);
if (skipData) {
metadata.skip = skipData;
hasData = true;
}
// Extract todo reason
const todoData = this.extractProtocolData(line, PROTOCOL_MARKERS.TODO_PREFIX);
if (todoData) {
metadata.todo = todoData;
hasData = true;
}
// Extract META JSON
const metaData = this.extractProtocolData(line, PROTOCOL_MARKERS.META_PREFIX);
if (metaData) {
try {
Object.assign(metadata, JSON.parse(metaData));
hasData = true;
} catch (e) {
// Invalid JSON, ignore
}
}
// Extract simple key:value pairs
const simpleMatch = line.match(new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]+)${this.escapeRegex(PROTOCOL_MARKERS.END)}`));
if (simpleMatch && simpleMatch[1].includes(':') && !simpleMatch[1].includes('META:') && !simpleMatch[1].includes('SKIP:') && !simpleMatch[1].includes('TODO:') && !simpleMatch[1].includes('EVENT:')) {
// This is a simple key:value format (not a prefixed format)
const pairs = simpleMatch[1].split(',');
for (const pair of pairs) {
const [key, value] = pair.split(':');
if (key && value) {
if (key === 'time') {
metadata.time = parseInt(value, 10);
hasData = true;
} else if (key === 'retry') {
metadata.retry = parseInt(value, 10);
hasData = true;
}
}
}
}
return hasData ? metadata : null;
}
/**
* Remove inline metadata from line
*/
private removeInlineMetadata(line: string): string {
// Remove all protocol markers
const regex = new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}[^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*${this.escapeRegex(PROTOCOL_MARKERS.END)}`, 'g');
return line.replace(regex, '').trim();
}
/**
* Extract protocol data with specific prefix
*/
private extractProtocolData(line: string, prefix: string): string | null {
const regex = new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}${this.escapeRegex(prefix)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*)${this.escapeRegex(PROTOCOL_MARKERS.END)}`);
const match = line.match(regex);
return match ? match[1] : null;
}
/**
* Check if line starts a block
*/
private isBlockStart(line: string): boolean {
// Only match if the line is exactly the block marker (after trimming)
const trimmed = line.trim();
return trimmed === `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_PREFIX}${PROTOCOL_MARKERS.END}` ||
(trimmed.startsWith(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_PREFIX}`) &&
trimmed.endsWith(PROTOCOL_MARKERS.END) &&
!trimmed.includes(' '));
}
/**
* Check if line ends a block
*/
private isBlockEnd(line: string): boolean {
return line.includes(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_END}${PROTOCOL_MARKERS.END}`) ||
line.includes(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_END}${PROTOCOL_MARKERS.END}`);
}
/**
* Extract block type from start line
*/
private extractBlockType(line: string): string | null {
if (line.includes(PROTOCOL_MARKERS.ERROR_PREFIX)) {
return 'error';
}
if (line.includes(PROTOCOL_MARKERS.SNAPSHOT_PREFIX)) {
const match = line.match(new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}${this.escapeRegex(PROTOCOL_MARKERS.SNAPSHOT_PREFIX)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*)${this.escapeRegex(PROTOCOL_MARKERS.END)}`));
return match ? `snapshot:${match[1]}` : 'snapshot';
}
return null;
}
/**
* Finalize current block
*/
private finalizeBlock(): IProtocolMessage {
const content = this.blockContent.join('\n');
if (this.blockType === 'error') {
try {
const errorData = JSON.parse(content) as IErrorBlock;
return {
type: 'error',
content: errorData
};
} catch (e) {
return {
type: 'error',
content: { error: { message: content } }
};
}
}
if (this.blockType?.startsWith('snapshot:')) {
const name = this.blockType.substring(9);
let parsedContent = content;
let format: 'json' | 'text' = 'text';
try {
parsedContent = JSON.parse(content);
format = 'json';
} catch (e) {
// Not JSON, keep as text
}
return {
type: 'snapshot',
content: {
name,
content: parsedContent,
format
} as ISnapshotData
};
}
// Fallback
return {
type: 'comment',
content: content
};
}
/**
* Escape regex special characters
*/
private escapeRegex(str: string): string {
return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
/**
* Get protocol version
*/
public getProtocolVersion(): string | null {
return this.protocolVersion;
}
}

View File

@@ -0,0 +1,148 @@
// Protocol V2 Types and Interfaces
// This file contains all type definitions for the improved TAP protocol
export interface ITestMetadata {
// Timing
time?: number; // milliseconds
startTime?: number; // Unix timestamp
endTime?: number; // Unix timestamp
// Status
skip?: string; // skip reason
todo?: string; // todo reason
retry?: number; // retry attempt
maxRetries?: number; // max retries allowed
// Error details
error?: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
code?: string;
};
// Test context
file?: string; // source file
line?: number; // line number
column?: number; // column number
// Custom data
tags?: string[]; // test tags
custom?: Record<string, any>;
}
export interface ITestResult {
ok: boolean;
testNumber: number;
description: string;
directive?: {
type: 'skip' | 'todo';
reason?: string;
};
metadata?: ITestMetadata;
}
export interface IPlanLine {
start: number;
end: number;
skipAll?: string;
}
export interface IProtocolMessage {
type: 'test' | 'plan' | 'comment' | 'version' | 'bailout' | 'protocol' | 'snapshot' | 'error' | 'event';
content: any;
}
export interface IProtocolVersion {
version: string;
features?: string[];
}
export interface ISnapshotData {
name: string;
content: any;
format?: 'json' | 'text' | 'binary';
}
export interface IErrorBlock {
testNumber?: number;
error: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
};
}
// Enhanced Communication Types
export type EventType =
| 'test:queued'
| 'test:started'
| 'test:progress'
| 'test:completed'
| 'suite:started'
| 'suite:completed'
| 'hook:started'
| 'hook:completed'
| 'assertion:failed';
export interface ITestEvent {
eventType: EventType;
timestamp: number;
data: {
testNumber?: number;
description?: string;
suiteName?: string;
hookName?: string;
progress?: number; // 0-100
duration?: number;
error?: IEnhancedError;
[key: string]: any;
};
}
export interface IEnhancedError {
message: string;
stack?: string;
diff?: IDiffResult;
actual?: any;
expected?: any;
code?: string;
type?: 'assertion' | 'timeout' | 'uncaught' | 'syntax' | 'runtime';
}
export interface IDiffResult {
type: 'string' | 'object' | 'array' | 'primitive';
changes: IDiffChange[];
context?: number; // lines of context
}
export interface IDiffChange {
type: 'add' | 'remove' | 'modify';
path?: string[]; // for object/array diffs
oldValue?: any;
newValue?: any;
line?: number; // for string diffs
content?: string;
}
// Protocol markers
export const PROTOCOL_MARKERS = {
START: '⟦TSTEST:',
END: '⟧',
META_PREFIX: 'META:',
ERROR_PREFIX: 'ERROR',
ERROR_END: '/ERROR',
SNAPSHOT_PREFIX: 'SNAPSHOT:',
SNAPSHOT_END: '/SNAPSHOT',
PROTOCOL_PREFIX: 'PROTOCOL:',
SKIP_PREFIX: 'SKIP:',
TODO_PREFIX: 'TODO:',
EVENT_PREFIX: 'EVENT:',
} as const;
// Protocol version
export const PROTOCOL_VERSION = '2.0.0';

View File

@@ -0,0 +1,587 @@
# @git.zone/tstest/tapbundle_protocol
> 📡 Enhanced TAP Protocol V2 implementation for structured test communication
## Installation
```bash
# tapbundle_protocol is included as part of @git.zone/tstest
pnpm install --save-dev @git.zone/tstest
```
## Overview
`@git.zone/tstest/tapbundle_protocol` implements Protocol V2, an enhanced version of the Test Anything Protocol (TAP) with support for structured metadata, real-time events, error diffs, and isomorphic operation. This protocol enables rich communication between test runners and test consumers while maintaining backward compatibility with standard TAP parsers.
## Key Features
- 📋 **TAP v13 Compliant** - Fully compatible with standard TAP consumers
- 🎯 **Enhanced Metadata** - Timing, tags, errors, diffs, and custom data
- 🔄 **Real-time Events** - Live test execution updates
- 🔍 **Structured Errors** - JSON error blocks with stack traces and diffs
- 📸 **Snapshot Support** - Built-in snapshot testing protocol
- 🌐 **Isomorphic** - Works in Node.js, browsers, Deno, and Bun
- 🏷️ **Protocol Markers** - Structured data using Unicode delimiters
## Protocol V2 Format
### Protocol Markers
Protocol V2 uses special Unicode markers to embed structured data within TAP output:
- `⟦TSTEST:` - Start marker
- `⟧` - End marker
These markers allow structured data to coexist with standard TAP without breaking compatibility.
### Example Output
```tap
⟦TSTEST:PROTOCOL:2.0.0⟧
TAP version 13
1..3
ok 1 - should add numbers ⟦TSTEST:time:42⟧
not ok 2 - should validate input
⟦TSTEST:META:{"time":156,"file":"test.ts","line":42}⟧
⟦TSTEST:ERROR⟧
{
"error": {
"message": "Expected 5 to equal 6",
"diff": {...}
}
}
⟦TSTEST:/ERROR⟧
ok 3 - should handle edge cases # SKIP not implemented ⟦TSTEST:SKIP:not implemented⟧
```
## API Reference
### ProtocolEmitter
Generates Protocol V2 messages. Used by tapbundle to emit test results.
#### `emitProtocolHeader()`
Emit the protocol version header.
```typescript
import { ProtocolEmitter } from '@git.zone/tstest/tapbundle_protocol';
const emitter = new ProtocolEmitter();
console.log(emitter.emitProtocolHeader());
// Output: ⟦TSTEST:PROTOCOL:2.0.0⟧
```
#### `emitTapVersion(version?)`
Emit TAP version line.
```typescript
console.log(emitter.emitTapVersion(13));
// Output: TAP version 13
```
#### `emitPlan(plan)`
Emit test plan.
```typescript
console.log(emitter.emitPlan({ start: 1, end: 5 }));
// Output: 1..5
console.log(emitter.emitPlan({ start: 1, end: 0, skipAll: 'Not ready' }));
// Output: 1..0 # Skipped: Not ready
```
#### `emitTest(result)`
Emit a test result with optional metadata.
```typescript
const lines = emitter.emitTest({
ok: true,
testNumber: 1,
description: 'should work correctly',
metadata: {
time: 45,
tags: ['unit', 'fast']
}
});
lines.forEach(line => console.log(line));
// Output:
// ok 1 - should work correctly ⟦TSTEST:time:45⟧
// ⟦TSTEST:META:{"tags":["unit","fast"]}⟧
```
#### `emitComment(comment)`
Emit a comment line.
```typescript
console.log(emitter.emitComment('Setup complete'));
// Output: # Setup complete
```
#### `emitBailout(reason)`
Emit a bailout (abort all tests).
```typescript
console.log(emitter.emitBailout('Database connection failed'));
// Output: Bail out! Database connection failed
```
#### `emitError(error)`
Emit a structured error block.
```typescript
const lines = emitter.emitError({
testNumber: 2,
error: {
message: 'Expected 5 to equal 6',
stack: 'Error: ...',
actual: 5,
expected: 6,
diff: '...'
}
});
lines.forEach(line => console.log(line));
// Output:
// ⟦TSTEST:ERROR⟧
// {
// "testNumber": 2,
// "error": { ... }
// }
// ⟦TSTEST:/ERROR⟧
```
#### `emitEvent(event)`
Emit a real-time test event.
```typescript
console.log(emitter.emitEvent({
eventType: 'test:started',
timestamp: Date.now(),
data: {
testNumber: 1,
description: 'should work'
}
}));
// Output: ⟦TSTEST:EVENT:{"eventType":"test:started",...}⟧
```
#### `emitSnapshot(snapshot)`
Emit snapshot data.
```typescript
const lines = emitter.emitSnapshot({
name: 'user-data',
content: { name: 'Alice', age: 30 },
format: 'json'
});
lines.forEach(line => console.log(line));
// Output:
// ⟦TSTEST:SNAPSHOT:user-data⟧
// {
// "name": "Alice",
// "age": 30
// }
// ⟦TSTEST:/SNAPSHOT⟧
```
### ProtocolParser
Parses Protocol V2 messages. Used by tstest to consume test results.
#### `parseLine(line)`
Parse a single line and return protocol messages.
```typescript
import { ProtocolParser } from '@git.zone/tstest/tapbundle_protocol';
const parser = new ProtocolParser();
const messages = parser.parseLine('ok 1 - test passed ⟦TSTEST:time:42⟧');
console.log(messages);
// Output:
// [{
// type: 'test',
// content: {
// ok: true,
// testNumber: 1,
// description: 'test passed',
// metadata: { time: 42 }
// }
// }]
```
#### Message Types
The parser returns different message types:
```typescript
interface IProtocolMessage {
type: 'test' | 'plan' | 'comment' | 'version' | 'bailout' | 'protocol' | 'snapshot' | 'error' | 'event';
content: any;
}
```
**Examples:**
```typescript
// Test result
{
type: 'test',
content: {
ok: true,
testNumber: 1,
description: 'test name',
metadata: { ... }
}
}
// Plan
{
type: 'plan',
content: {
start: 1,
end: 5
}
}
// Event
{
type: 'event',
content: {
eventType: 'test:started',
timestamp: 1234567890,
data: { ... }
}
}
// Error
{
type: 'error',
content: {
testNumber: 2,
error: {
message: '...',
stack: '...',
diff: '...'
}
}
}
```
#### `getProtocolVersion()`
Get the detected protocol version.
```typescript
const version = parser.getProtocolVersion();
console.log(version); // "2.0.0" or null
```
## TypeScript Types
### ITestResult
```typescript
interface ITestResult {
ok: boolean;
testNumber: number;
description: string;
directive?: {
type: 'skip' | 'todo';
reason?: string;
};
metadata?: ITestMetadata;
}
```
### ITestMetadata
```typescript
interface ITestMetadata {
// Timing
time?: number; // Test duration in milliseconds
startTime?: number; // Unix timestamp
endTime?: number; // Unix timestamp
// Status
skip?: string; // Skip reason
todo?: string; // Todo reason
retry?: number; // Current retry attempt
maxRetries?: number; // Max retries allowed
// Error details
error?: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
code?: string;
};
// Test context
file?: string; // Source file path
line?: number; // Line number
column?: number; // Column number
// Custom data
tags?: string[]; // Test tags
custom?: Record<string, any>;
}
```
### ITestEvent
```typescript
interface ITestEvent {
eventType: EventType;
timestamp: number;
data: {
testNumber?: number;
description?: string;
suiteName?: string;
hookName?: string;
progress?: number; // 0-100
duration?: number;
error?: IEnhancedError;
[key: string]: any;
};
}
type EventType =
| 'test:queued'
| 'test:started'
| 'test:progress'
| 'test:completed'
| 'suite:started'
| 'suite:completed'
| 'hook:started'
| 'hook:completed'
| 'assertion:failed';
```
### IEnhancedError
```typescript
interface IEnhancedError {
message: string;
stack?: string;
diff?: IDiffResult;
actual?: any;
expected?: any;
code?: string;
type?: 'assertion' | 'timeout' | 'uncaught' | 'syntax' | 'runtime';
}
```
### IDiffResult
```typescript
interface IDiffResult {
type: 'string' | 'object' | 'array' | 'primitive';
changes: IDiffChange[];
context?: number; // Lines of context
}
interface IDiffChange {
type: 'add' | 'remove' | 'modify';
path?: string[]; // For object/array diffs
oldValue?: any;
newValue?: any;
line?: number; // For string diffs
content?: string;
}
```
## Protocol Constants
```typescript
import { PROTOCOL_MARKERS, PROTOCOL_VERSION } from '@git.zone/tstest/tapbundle_protocol';
console.log(PROTOCOL_VERSION); // "2.0.0"
console.log(PROTOCOL_MARKERS.START); // "⟦TSTEST:"
console.log(PROTOCOL_MARKERS.END); // "⟧"
```
### Available Markers
```typescript
const PROTOCOL_MARKERS = {
START: '⟦TSTEST:',
END: '⟧',
META_PREFIX: 'META:',
ERROR_PREFIX: 'ERROR',
ERROR_END: '/ERROR',
SNAPSHOT_PREFIX: 'SNAPSHOT:',
SNAPSHOT_END: '/SNAPSHOT',
PROTOCOL_PREFIX: 'PROTOCOL:',
SKIP_PREFIX: 'SKIP:',
TODO_PREFIX: 'TODO:',
EVENT_PREFIX: 'EVENT:',
};
```
## Usage Patterns
### Creating a Custom Test Runner
```typescript
import { ProtocolEmitter } from '@git.zone/tstest/tapbundle_protocol';
const emitter = new ProtocolEmitter();
// Emit protocol header
console.log(emitter.emitProtocolHeader());
console.log(emitter.emitTapVersion(13));
// Emit plan
console.log(emitter.emitPlan({ start: 1, end: 2 }));
// Run test 1
emitter.emitEvent({
eventType: 'test:started',
timestamp: Date.now(),
data: { testNumber: 1 }
}).split('\n').forEach(line => console.log(line));
const result1 = emitter.emitTest({
ok: true,
testNumber: 1,
description: 'first test',
metadata: { time: 45 }
});
result1.forEach(line => console.log(line));
// Run test 2
const result2 = emitter.emitTest({
ok: false,
testNumber: 2,
description: 'second test',
metadata: {
time: 120,
error: {
message: 'Assertion failed',
actual: 5,
expected: 6
}
}
});
result2.forEach(line => console.log(line));
```
### Parsing Test Output
```typescript
import { ProtocolParser } from '@git.zone/tstest/tapbundle_protocol';
import * as readline from 'readline';
const parser = new ProtocolParser();
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
rl.on('line', (line) => {
const messages = parser.parseLine(line);
messages.forEach(message => {
switch (message.type) {
case 'test':
console.log(`Test ${message.content.testNumber}: ${message.content.ok ? 'PASS' : 'FAIL'}`);
break;
case 'event':
console.log(`Event: ${message.content.eventType}`);
break;
case 'error':
console.error(`Error: ${message.content.error.message}`);
break;
}
});
});
```
### Building Test Dashboards
Real-time events enable building live test dashboards:
```typescript
const parser = new ProtocolParser();
parser.parseLine(line).forEach(message => {
if (message.type === 'event') {
const event = message.content;
switch (event.eventType) {
case 'test:started':
updateUI({ status: 'running', test: event.data.description });
break;
case 'test:completed':
updateUI({ status: 'done', duration: event.data.duration });
break;
case 'suite:started':
createSuiteCard(event.data.suiteName);
break;
}
}
});
```
## Backward Compatibility
Protocol V2 is fully backward compatible with standard TAP parsers:
- Protocol markers use Unicode characters that TAP parsers ignore
- Standard TAP output (ok/not ok, plan, comments) works everywhere
- Enhanced features gracefully degrade in standard TAP consumers
**Standard TAP View:**
```tap
TAP version 13
1..3
ok 1 - should add numbers
not ok 2 - should validate input
ok 3 - should handle edge cases # SKIP not implemented
```
**Protocol V2 View (same output):**
```tap
⟦TSTEST:PROTOCOL:2.0.0⟧
TAP version 13
1..3
ok 1 - should add numbers ⟦TSTEST:time:42⟧
not ok 2 - should validate input
⟦TSTEST:META:{"time":156}⟧
ok 3 - should handle edge cases # SKIP not implemented ⟦TSTEST:SKIP:not implemented⟧
```
## Isomorphic Design
This module works in all JavaScript environments:
- ✅ Node.js
- ✅ Browsers (via tapbundle)
- ✅ Deno
- ✅ Bun
- ✅ Web Workers
- ✅ Service Workers
No runtime-specific APIs are used, making it truly portable.
## Legal
This project is licensed under MIT.
© 2025 Task Venture Capital GmbH. All rights reserved.

View File

@@ -0,0 +1,3 @@
{
"order": 1
}