Compare commits
98 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 8fd114334f | |||
| c630a171b5 | |||
| 1a4eb5b6d9 | |||
| 41d7c1ce49 | |||
| 3ab5550cb8 | |||
| ee7b387534 | |||
| 7e67b64a6e | |||
| 1ce730d4f2 | |||
| 9357d6e7ef | |||
| 973ce771d2 | |||
| 8441881d92 | |||
| 16ca3b6374 | |||
| b94089652e | |||
| ef6f21fc9c | |||
| 592a4f33c0 | |||
| 1ea3b37d18 | |||
| 062c6e384b | |||
| b1f2eceb75 | |||
| de53b3f00c | |||
| b3f8a28766 | |||
| 86db2491a3 | |||
| b9fd8c7b02 | |||
| d6842326ad | |||
| 175b4463fa | |||
| 6a8417e400 | |||
| 4714d5e8ad | |||
| ff6aae7159 | |||
| d05ec21b73 | |||
| 956a880a4a | |||
| ee11b1ac17 | |||
| 054cbb6b3c | |||
| ecf11efb4c | |||
| 1de674e91d | |||
| 9fa2c23ab2 | |||
| 36715c9139 | |||
| ee0aca9ff7 | |||
| aaebe75326 | |||
| 265ed702ee | |||
| efbaded1f3 | |||
| 799a60188f | |||
| 3c38a53d9d | |||
| cca01b51ec | |||
| 84843ad359 | |||
| 7a8ae95be2 | |||
| 133e0eda8b | |||
| 14e32b06de | |||
| 48aebb1eac | |||
| 733b2249d0 | |||
| 008844a9e2 | |||
| e4fc6623ea | |||
| 70435cce45 | |||
| c26145205f | |||
| 82fc22653b | |||
| 3d85f54be0 | |||
| 9464c17c15 | |||
| 91b99ce304 | |||
| 899045e6aa | |||
| 845f146e91 | |||
| d1f8652fc7 | |||
| f717078558 | |||
| d2c0e533b5 | |||
| d3c7fce595 | |||
| 570e2d6b3b | |||
| b7f4b7b3b8 | |||
| 424046b0de | |||
| 0f762f2063 | |||
| 82757c4abc | |||
| 7aaeed0dc6 | |||
| c98bd85829 | |||
| 33d2ff1d4f | |||
| 91880f8d42 | |||
| 7b1732abcc | |||
| 7d09b39f2b | |||
| 96efba5903 | |||
| 3c535a8a77 | |||
| 0954265095 | |||
| e1d90589bc | |||
| 33f705d961 | |||
| 13b11ab1bf | |||
| 63280e4a9a | |||
| 23addc2d2f | |||
| 3649114c8d | |||
| 2841aba8a4 | |||
| 31bf090410 | |||
| b525754035 | |||
| aa10fc4ab3 | |||
| 3eb8ef22e5 | |||
| 763dc89f59 | |||
| e0d8ede450 | |||
| 27c950c1a1 | |||
| 83b324b09f | |||
| 63a2879cb4 | |||
| 1a375fa689 | |||
| c48887a820 | |||
| 02aeb8195e | |||
| 53d3dc55e6 | |||
| a82fdc0f26 | |||
| cfcb99de76 |
1
.serena/.gitignore
vendored
Normal file
1
.serena/.gitignore
vendored
Normal file
@@ -0,0 +1 @@
|
||||
/cache
|
||||
68
.serena/project.yml
Normal file
68
.serena/project.yml
Normal file
@@ -0,0 +1,68 @@
|
||||
# language of the project (csharp, python, rust, java, typescript, go, cpp, or ruby)
|
||||
# * For C, use cpp
|
||||
# * For JavaScript, use typescript
|
||||
# Special requirements:
|
||||
# * csharp: Requires the presence of a .sln file in the project folder.
|
||||
language: typescript
|
||||
|
||||
# whether to use the project's gitignore file to ignore files
|
||||
# Added on 2025-04-07
|
||||
ignore_all_files_in_gitignore: true
|
||||
# list of additional paths to ignore
|
||||
# same syntax as gitignore, so you can use * and **
|
||||
# Was previously called `ignored_dirs`, please update your config if you are using that.
|
||||
# Added (renamed) on 2025-04-07
|
||||
ignored_paths: []
|
||||
|
||||
# whether the project is in read-only mode
|
||||
# If set to true, all editing tools will be disabled and attempts to use them will result in an error
|
||||
# Added on 2025-04-18
|
||||
read_only: false
|
||||
|
||||
|
||||
# list of tool names to exclude. We recommend not excluding any tools, see the readme for more details.
|
||||
# Below is the complete list of tools for convenience.
|
||||
# To make sure you have the latest list of tools, and to view their descriptions,
|
||||
# execute `uv run scripts/print_tool_overview.py`.
|
||||
#
|
||||
# * `activate_project`: Activates a project by name.
|
||||
# * `check_onboarding_performed`: Checks whether project onboarding was already performed.
|
||||
# * `create_text_file`: Creates/overwrites a file in the project directory.
|
||||
# * `delete_lines`: Deletes a range of lines within a file.
|
||||
# * `delete_memory`: Deletes a memory from Serena's project-specific memory store.
|
||||
# * `execute_shell_command`: Executes a shell command.
|
||||
# * `find_referencing_code_snippets`: Finds code snippets in which the symbol at the given location is referenced.
|
||||
# * `find_referencing_symbols`: Finds symbols that reference the symbol at the given location (optionally filtered by type).
|
||||
# * `find_symbol`: Performs a global (or local) search for symbols with/containing a given name/substring (optionally filtered by type).
|
||||
# * `get_current_config`: Prints the current configuration of the agent, including the active and available projects, tools, contexts, and modes.
|
||||
# * `get_symbols_overview`: Gets an overview of the top-level symbols defined in a given file.
|
||||
# * `initial_instructions`: Gets the initial instructions for the current project.
|
||||
# Should only be used in settings where the system prompt cannot be set,
|
||||
# e.g. in clients you have no control over, like Claude Desktop.
|
||||
# * `insert_after_symbol`: Inserts content after the end of the definition of a given symbol.
|
||||
# * `insert_at_line`: Inserts content at a given line in a file.
|
||||
# * `insert_before_symbol`: Inserts content before the beginning of the definition of a given symbol.
|
||||
# * `list_dir`: Lists files and directories in the given directory (optionally with recursion).
|
||||
# * `list_memories`: Lists memories in Serena's project-specific memory store.
|
||||
# * `onboarding`: Performs onboarding (identifying the project structure and essential tasks, e.g. for testing or building).
|
||||
# * `prepare_for_new_conversation`: Provides instructions for preparing for a new conversation (in order to continue with the necessary context).
|
||||
# * `read_file`: Reads a file within the project directory.
|
||||
# * `read_memory`: Reads the memory with the given name from Serena's project-specific memory store.
|
||||
# * `remove_project`: Removes a project from the Serena configuration.
|
||||
# * `replace_lines`: Replaces a range of lines within a file with new content.
|
||||
# * `replace_symbol_body`: Replaces the full definition of a symbol.
|
||||
# * `restart_language_server`: Restarts the language server, may be necessary when edits not through Serena happen.
|
||||
# * `search_for_pattern`: Performs a search for a pattern in the project.
|
||||
# * `summarize_changes`: Provides instructions for summarizing the changes made to the codebase.
|
||||
# * `switch_modes`: Activates modes by providing a list of their names
|
||||
# * `think_about_collected_information`: Thinking tool for pondering the completeness of collected information.
|
||||
# * `think_about_task_adherence`: Thinking tool for determining whether the agent is still on track with the current task.
|
||||
# * `think_about_whether_you_are_done`: Thinking tool for determining whether the task is truly completed.
|
||||
# * `write_memory`: Writes a named memory (for future reference) to Serena's project-specific memory store.
|
||||
excluded_tools: []
|
||||
|
||||
# initial prompt for the project. It will always be given to the LLM upon activating the project
|
||||
# (contrary to the memories, which are loaded on demand).
|
||||
initial_prompt: ""
|
||||
|
||||
project_name: "tstest"
|
||||
3
.vscode/settings.json
vendored
3
.vscode/settings.json
vendored
@@ -22,5 +22,6 @@
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
],
|
||||
"deno.enable": false
|
||||
}
|
||||
|
||||
382
changelog.md
382
changelog.md
@@ -1,5 +1,387 @@
|
||||
# Changelog
|
||||
|
||||
## 2025-11-20 - 3.0.1 - fix(@push.rocks/smarts3)
|
||||
Bump @push.rocks/smarts3 dependency to ^2.2.7
|
||||
|
||||
- Update package.json: @push.rocks/smarts3 upgraded from ^2.2.6 to ^2.2.7
|
||||
|
||||
## 2025-11-19 - 3.0.0 - BREAKING CHANGE(tapbundle_serverside)
|
||||
Rename Node-specific tapbundle module to tapbundle_serverside and migrate server-side utilities
|
||||
|
||||
- Change public export in package.json from ./tapbundle_node to ./tapbundle_serverside — consumers must update imports to @git.zone/tstest/tapbundle_serverside
|
||||
- Move and re-create Node-only implementation files under ts_tapbundle_serverside (plugins, paths, classes.tapnodetools, classes.testfileprovider, index, tspublish.json) and remove legacy ts_tapbundle_node sources
|
||||
- Update internal imports and tests to reference the new tapbundle_serverside path (e.g. test/tapbundle/test.node.ts updated)
|
||||
- Update documentation (readme.md and readme.hints.md) to describe the new tapbundle_serverside export and its server-side utilities
|
||||
- Ensure build outputs and publish metadata reflect the new module directory (tspublish.json order preserved)
|
||||
|
||||
## 2025-11-19 - 2.8.3 - fix(dependencies)
|
||||
Update dependency versions
|
||||
|
||||
- Bump devDependency @git.zone/tsbuild to ^3.1.0
|
||||
- Upgrade @git.zone/tsrun to ^2.0.0 (major)
|
||||
- Upgrade @push.rocks/smartenv to ^6.0.0 (major)
|
||||
- Upgrade @push.rocks/smartrequest to ^5.0.1 (major/feature in dependency)
|
||||
- Patch updates: @api.global/typedserver → ^3.0.80, @git.zone/tsbundle → ^2.5.2, @push.rocks/smartmongo → ^2.0.14
|
||||
|
||||
## 2025-11-17 - 2.8.2 - fix(logging)
|
||||
Include runtime identifier in per-test logfile name and sanitize runtime string
|
||||
|
||||
- Append a sanitized runtime identifier to the per-test log filename (format: <safeFilename>__<safeRuntime>.log) so runs for different runtimes don't clash
|
||||
- Sanitize runtime names by lowercasing and removing non-alphanumeric characters to produce filesystem-safe filenames
|
||||
|
||||
## 2025-11-17 - 2.8.1 - fix(config)
|
||||
Remove Bun config file and set deno.json useDefineForClassFields to false for compatibility
|
||||
|
||||
- Removed bunfig.toml (Bun-specific TypeScript decorator configuration) — stops shipping a project-local Bun transpiler config.
|
||||
- Updated deno.json: set compilerOptions.useDefineForClassFields = false to keep legacy class field semantics and avoid runtime/emit incompatibilities in Deno.
|
||||
|
||||
## 2025-11-17 - 2.8.0 - feat(runtime-adapters)
|
||||
Enable TypeScript decorator support for Deno and Bun runtimes and add decorator tests
|
||||
|
||||
- Add bunfig.toml to enable experimentalDecorators for Bun runtime
|
||||
- Add deno.json to enable experimentalDecorators and set target/lib for Deno
|
||||
- Update Bun runtime adapter to note bunfig.toml discovery so Bun runs with decorator support
|
||||
- Update Deno runtime adapter to auto-detect deno.json / deno.jsonc and pass configPath in default options
|
||||
- Add integration tests for decorators (test/decorator.all.ts) to verify decorator support across runtimes
|
||||
|
||||
## 2025-10-26 - 2.7.0 - feat(tapbundle_protocol)
|
||||
Add package export for tapbundle_protocol to expose protocol utilities
|
||||
|
||||
- Add './tapbundle_protocol' export in package.json pointing to './dist_ts_tapbundle_protocol/index.js'.
|
||||
- Allows consumers to import protocol utilities (ProtocolEmitter, ProtocolParser, types) via '@git.zone/tstest/tapbundle_protocol'.
|
||||
- Non-breaking: only extends package exports surface.
|
||||
|
||||
## 2025-10-17 - 2.6.2 - fix(@push.rocks/smartrequest)
|
||||
Bump @push.rocks/smartrequest from ^4.3.1 to ^4.3.2
|
||||
|
||||
- Update dependency @push.rocks/smartrequest from ^4.3.1 to ^4.3.2
|
||||
|
||||
## 2025-10-17 - 2.6.1 - fix(runtime-adapters)
|
||||
Silence shell version checks for Bun and Deno; add local Claude settings
|
||||
|
||||
- Replace smartshell.exec with execSilent in ts/tstest.classes.runtime.bun.ts to suppress output when checking Bun availability
|
||||
- Replace smartshell.exec with execSilent in ts/tstest.classes.runtime.deno.ts to suppress output when checking Deno availability
|
||||
- Add .claude/settings.local.json to record local Claude agent permissions/config used for development
|
||||
|
||||
## 2025-10-17 - 2.6.0 - feat(runtime-adapters)
|
||||
Add runtime environment availability check and logger output; normalize runtime version strings
|
||||
|
||||
- Introduce checkEnvironment() in TsTest and invoke it at the start of run() to detect available runtimes before executing tests.
|
||||
- Add environmentCheck(availability) to TsTestLogger to print a human-friendly environment summary (with JSON and quiet-mode handling).
|
||||
- Normalize reported runtime version strings from adapters: prefix Deno and Bun versions with 'v' and simplify Chromium version text.
|
||||
- Display runtime availability information to the user before moving previous logs or running tests.
|
||||
- Includes addition of local .claude/settings.local.json (local dev/tooling settings).
|
||||
|
||||
## 2025-10-17 - 2.5.2 - fix(runtime.node)
|
||||
Improve Node runtime adapter to use tsrun.spawnPath, strengthen tsrun detection, and improve process lifecycle and loader handling; update tsrun dependency.
|
||||
|
||||
- Use tsrun.spawnPath to spawn Node test processes and pass structured spawn options (cwd, env, args, stdio).
|
||||
- Detect tsrun availability via plugins.tsrun and require spawnPath; provide a clearer error message when tsrun is missing or outdated.
|
||||
- Pass --web via spawn args and set TSTEST_FILTER_TAGS on the spawned process env instead of mutating the parent process.env.
|
||||
- When a 00init.ts exists, create a temporary loader that imports both 00init.ts and the test file, run the loader via tsrun.spawnPath, and clean up the loader after execution.
|
||||
- Use tsrunProcess.terminate()/kill for timeouts to ensure proper process termination and improve cleanup handling.
|
||||
- Export tsrun from ts/tstest.plugins.ts so runtime code can access tsrun APIs via the plugins object.
|
||||
- Bump dependency @git.zone/tsrun from ^1.3.4 to ^1.6.2 in package.json.
|
||||
|
||||
## 2025-10-16 - 2.5.1 - fix(deps)
|
||||
Bump dependencies and add local tooling settings
|
||||
|
||||
- Bumped @api.global/typedserver from ^3.0.78 to ^3.0.79
|
||||
- Bumped @git.zone/tsrun from ^1.3.3 to ^1.3.4
|
||||
- Bumped @push.rocks/smartjson from ^5.0.20 to ^5.2.0
|
||||
- Bumped @push.rocks/smartlog from ^3.1.9 to ^3.1.10
|
||||
- Add local settings configuration file for developer tooling
|
||||
|
||||
## 2025-10-12 - 2.5.0 - feat(tstest.classes.runtime.parser)
|
||||
Add support for "all" runtime token and update docs/tests; regenerate lockfile and add local settings
|
||||
|
||||
- Add support for the `all` runtime token (expands to node, chromium, deno, bun) in tstest filename parser (tstest.classes.runtime.parser)
|
||||
- Handle `all` with modifiers (e.g. `*.all.nonci.ts`) and mixed tokens (e.g. `node+all`) so it expands to the full runtime set
|
||||
- Add unit tests covering `all` cases in test/test.runtime.parser.node.ts
|
||||
- Update README (examples and tables) to document `.all.ts` and `.all.nonci.ts` usage and include a universal example
|
||||
- Update ts files' parser comments and constants to include ALL_RUNTIMES
|
||||
- Add deno.lock (dependency lockfile) and a local .claude/settings.local.json for project permissions / local settings
|
||||
|
||||
## 2025-10-11 - 2.4.3 - fix(docs)
|
||||
Update documentation: expand README with multi-runtime architecture, add module READMEs, and add local dev settings
|
||||
|
||||
- Expanded project README: fixed typos, clarified availability header, and added a detailed Multi-Runtime Architecture section (runtimes, naming conventions, migration tool, examples, and runtime-specific notes).
|
||||
- Inserted additional example output and adjusted JSON/example sections to reflect multi-runtime flows and updated totals/durations in examples.
|
||||
- Added dedicated README files for ts_tapbundle, ts_tapbundle_node, and ts_tapbundle_protocol modules with API overviews and usage guides.
|
||||
- Added .claude/settings.local.json to provide local development permissions/settings used by the project tooling.
|
||||
- Minor formatting and documentation cleanup (whitespace, headings, and changelog entries).
|
||||
|
||||
## 2025-10-10 - 2.4.2 - fix(deno)
|
||||
Enable additional Deno permissions for runtime adapters and add local dev settings
|
||||
|
||||
- Add --allow-sys, --allow-import and --node-modules-dir to the default Deno permission set used by the Deno runtime adapter
|
||||
- Include the new permission flags in the fallback permissions array when constructing Deno command args
|
||||
- Add .claude/settings.local.json to capture local development permissions and helper commands
|
||||
|
||||
## 2025-10-10 - 2.4.1 - fix(runtime/deno)
|
||||
Enable Deno runtime tests by adding required permissions and local settings
|
||||
|
||||
- ts/tstest.classes.runtime.deno.ts: expanded default Deno permissions to include --allow-net, --allow-write and --sloppy-imports to allow network access, file writes and permissive JS/TS imports
|
||||
- ts/tstest.classes.runtime.deno.ts: updated fallback permissions used when building the Deno command to match the new default set
|
||||
- Added .claude/settings.local.json with a set of allowed local commands/permissions used for local development/CI tooling
|
||||
|
||||
## 2025-10-10 - 2.4.0 - feat(runtime)
|
||||
Add runtime adapters, filename runtime parser and migration tool; integrate runtime selection into TsTest and add tests
|
||||
|
||||
- Introduce RuntimeAdapter abstraction and RuntimeAdapterRegistry to manage multiple runtimes
|
||||
- Add runtime adapters: NodeRuntimeAdapter, ChromiumRuntimeAdapter, DenoRuntimeAdapter and BunRuntimeAdapter
|
||||
- Add filename runtime parser utilities: parseTestFilename, isLegacyFilename and getLegacyMigrationTarget
|
||||
- Add Migration class to detect and (dry-run) migrate legacy test filenames to the new naming convention
|
||||
- Integrate runtime registry into TsTest and choose execution adapters based on parsed runtimes; show deprecation warnings for legacy naming
|
||||
- Add tests covering runtime parsing and migration: test/test.runtime.parser.node.ts and test/test.migration.node.ts
|
||||
|
||||
## 2025-09-12 - 2.3.8 - fix(tstest)
|
||||
Improve free port selection for Chrome runner and bump smartnetwork dependency
|
||||
|
||||
- Use randomized port selection when finding free HTTP and WebSocket ports to reduce collision probability in concurrent runs
|
||||
- Ensure WebSocket port search excludes the chosen HTTP port so the two ports will not conflict
|
||||
- Simplify failure handling: throw early if a free WebSocket port cannot be found instead of retrying with a less robust fallback
|
||||
- Bump @push.rocks/smartnetwork dependency from ^4.2.0 to ^4.4.0 to pick up new findFreePort options
|
||||
|
||||
## 2025-09-12 - 2.3.7 - fix(tests)
|
||||
Remove flaky dynamic-ports browser test and add local dev tool settings
|
||||
|
||||
- Removed test/tapbundle/test.dynamicports.ts — deletes a browser test that relied on injected dynamic WebSocket ports (reduces flaky CI/browser runs).
|
||||
- Added .claude/settings.local.json — local development settings for the CLAUDE helper (grants allowed dev/automation commands and webfetch permissions).
|
||||
|
||||
## 2025-09-03 - 2.3.6 - fix(tstest)
|
||||
Update deps, fix chrome server route for static bundles, add local tool settings and CI ignore
|
||||
|
||||
- Bump devDependency @git.zone/tsbuild to ^2.6.8
|
||||
- Bump dependencies: @api.global/typedserver to ^3.0.78, @push.rocks/smartlog to ^3.1.9, @push.rocks/smartrequest to ^4.3.1
|
||||
- Fix test server static route in ts/tstest.classes.tstest.ts: replace '(.*)' with '/*splat' so bundled test files are served correctly in Chromium runs
|
||||
- Add .claude/settings.local.json with local permissions for development tasks
|
||||
- Add .serena/.gitignore to ignore /cache
|
||||
|
||||
## 2025-08-18 - 2.3.5 - fix(core)
|
||||
Use SmartRequest with Buffer for binary downloads, tighten static route handling, bump dependencies and add workspace/config files
|
||||
|
||||
- ts_tapbundle_node/classes.testfileprovider.ts: switch to SmartRequest.create().url(...).get() and convert response to a Buffer before writing to disk to fix binary download handling for the Docker Alpine image.
|
||||
- ts/tstest.classes.tstest.ts: change server.addRoute from '*' to '(.*)' so the typedserver static handler uses a proper regex route.
|
||||
- package.json: bump several dependencies (e.g. @api.global/typedserver, @git.zone/tsbuild, @push.rocks/smartfile, @push.rocks/smartpath, @push.rocks/smartrequest, @push.rocks/smartshell) to newer patch/minor versions.
|
||||
- pnpm-workspace.yaml: add onlyBuiltDependencies list (esbuild, mongodb-memory-server, puppeteer).
|
||||
- Remove registry setting from .npmrc (cleanup).
|
||||
- Add project/agent config files: .serena/project.yml and .claude/settings.local.json for local tooling/agent configuration.
|
||||
|
||||
## 2025-08-16 - 2.3.4 - fix(ci)
|
||||
Add local Claude settings to allow required WebFetch and Bash permissions for local tooling and tests
|
||||
|
||||
- Add .claude/settings.local.json to configure allowed permissions for local assistant/automation
|
||||
- Grants WebFetch access for code.foss.global and www.npmjs.com
|
||||
- Allows various Bash commands used by local tasks and test runs (mkdir, tsbuild, pnpm, node, tsx, tstest, ls, rm, grep, cat)
|
||||
- No runtime/library code changes — configuration only
|
||||
|
||||
## 2025-08-16 - 2.3.3 - fix(dependencies)
|
||||
Bump dependency versions and add local Claude settings
|
||||
|
||||
- Bumped devDependency @git.zone/tsbuild ^2.6.3 → ^2.6.4
|
||||
- Updated @git.zone/tsbundle ^2.2.5 → ^2.5.1
|
||||
- Updated @push.rocks/consolecolor ^2.0.2 → ^2.0.3
|
||||
- Updated @push.rocks/qenv ^6.1.0 → ^6.1.3
|
||||
- Updated @push.rocks/smartchok ^1.0.34 → ^1.1.1
|
||||
- Updated @push.rocks/smartenv ^5.0.12 → ^5.0.13
|
||||
- Updated @push.rocks/smartfile ^11.2.3 → ^11.2.5
|
||||
- Updated @push.rocks/smarts3 ^2.2.5 → ^2.2.6
|
||||
- Updated @push.rocks/smartshell ^3.2.3 → ^3.2.4
|
||||
- Updated ws ^8.18.2 → ^8.18.3
|
||||
- Added .claude/settings.local.json for local Claude permissions and tooling (local-only configuration)
|
||||
|
||||
## 2025-07-24 - 2.3.2 - fix(tapbundle)
|
||||
Fix TypeScript IDE warning about tapTools parameter possibly being undefined
|
||||
|
||||
- Changed ITestFunction from interface with optional parameter to union type
|
||||
- Updated test runner to handle both function signatures (with and without tapTools)
|
||||
- Resolves IDE warnings while maintaining backward compatibility
|
||||
|
||||
## 2025-05-26 - 2.3.1 - fix(tapParser/logger)
|
||||
Fix test duration reporting and summary formatting in TAP parser and logger
|
||||
|
||||
- Introduce startTime in TapParser to capture the overall test duration
|
||||
- Pass computed duration to logger methods in evaluateFinalResult for accurate timing
|
||||
- Update summary output to format duration in a human-readable way (ms vs. s)
|
||||
- Add local permission settings configuration to .claude/settings.local.json
|
||||
|
||||
## 2025-05-26 - 2.3.0 - feat(cli)
|
||||
Add '--version' option and warn against global tstest usage in the tstest project
|
||||
|
||||
- Introduced a new '--version' CLI flag that prints the version from package.json
|
||||
- Added logic in ts/index.ts to detect if tstest is run globally within its own project and issue a warning
|
||||
- Added .claude/settings.local.json to configure allowed permissions for various commands
|
||||
|
||||
## 2025-05-26 - 2.2.6 - fix(tstest)
|
||||
Improve timeout warning timer management and summary output formatting in the test runner.
|
||||
|
||||
- Removed the global timeoutWarningTimer and replaced it with local warning timers in runInNode and runInChrome methods.
|
||||
- Added warnings when test files run for over one minute if no timeout is specified.
|
||||
- Ensured proper clearing of warning timers on successful completion or timeout.
|
||||
- Enhanced quiet mode summary output to clearly display passed and failed test counts.
|
||||
|
||||
## 2025-05-26 - 2.2.5 - fix(protocol)
|
||||
Fix inline timing metadata parsing and enhance test coverage for performance metrics and timing edge cases
|
||||
|
||||
- Updated the protocol parser to correctly parse inline key:value pairs while excluding prefixed formats (META:, SKIP:, TODO:, EVENT:)
|
||||
- Added new tests for performance metrics, timing edge cases, and protocol timing to verify accurate timing capture and retry handling
|
||||
- Expanded documentation in readme.hints.md to detail the updated timing implementation and parser fixes
|
||||
|
||||
## 2025-05-26 - 2.2.4 - fix(logging)
|
||||
Improve performance metrics reporting and add local permissions configuration
|
||||
|
||||
- Add .claude/settings.local.json to configure allowed permissions for various commands
|
||||
- Update tstest logging: compute average test duration from actual durations and adjust slowest test display formatting
|
||||
|
||||
## 2025-05-26 - 2.2.3 - fix(readme/ts/tstest.plugins)
|
||||
Update npm package scope and documentation to use '@git.zone' instead of '@gitzone', and add local settings configuration.
|
||||
|
||||
- Changed npm package links and source repository URLs in readme from '@gitzone/tstest' to '@git.zone/tstest'.
|
||||
- Updated comments in ts/tstest.plugins.ts to reflect the correct '@git.zone' scope.
|
||||
- Added .claude/settings.local.json file with local permission settings.
|
||||
|
||||
## 2025-05-26 - 2.2.2 - fix(config)
|
||||
Cleanup project configuration by adding local CLAUDE settings and removing redundant license files
|
||||
|
||||
- Added .claude/settings.local.json with updated permissions for CLI and build tasks
|
||||
- Removed license and license.md files to streamline repository content
|
||||
|
||||
## 2025-05-26 - 2.2.1 - fix(repo configuration)
|
||||
Update repository metadata to use 'git.zone' naming and add local permission settings
|
||||
|
||||
- Changed githost from 'gitlab.com' to 'code.foss.global' and gitscope from 'gitzone' to 'git.zone' in npmextra.json
|
||||
- Updated npm package name from '@gitzone/tstest' to '@git.zone/tstest' in npmextra.json and readme.md
|
||||
- Added .claude/settings.local.json with new permission configuration
|
||||
|
||||
## 2025-05-26 - 2.2.0 - feat(watch mode)
|
||||
Add watch mode support with CLI options and enhanced documentation
|
||||
|
||||
- Introduce '--watch' (or '-w') and '--watch-ignore' CLI flags for automatic test re-runs
|
||||
- Integrate @push.rocks/smartchok for file watching with 300ms debouncing
|
||||
- Update readme.md and readme.hints.md with detailed instructions and examples for watch mode
|
||||
- Add a demo test file (test/watch-demo/test.demo.ts) to illustrate the new feature
|
||||
- Add smartchok dependency in package.json
|
||||
|
||||
## 2025-05-26 - 2.1.0 - feat(core)
|
||||
Implement Protocol V2 with enhanced settings and lifecycle hooks
|
||||
|
||||
- Migrated to Protocol V2 using Unicode markers and structured metadata with new ts_tapbundle_protocol module
|
||||
- Refactored TAP parser/emitter to support improved protocol parsing and error reporting
|
||||
- Integrated global settings via tap.settings() and lifecycle hooks (beforeAll/afterAll, beforeEach/afterEach)
|
||||
- Enhanced expect wrapper with diff generation for clearer assertion failures
|
||||
- Updated test loader to automatically run 00init.ts for proper test configuration
|
||||
- Revised documentation (readme.hints.md, readme.plan.md) to reflect current implementation status and remaining work
|
||||
|
||||
## 2025-05-25 - 2.0.0 - BREAKING CHANGE(protocol)
|
||||
Introduce protocol v2 implementation and update build configuration with revised build order, new tspublish files, and enhanced documentation
|
||||
|
||||
- Added ts_tapbundle_protocol directory with isomorphic implementation for protocol v2
|
||||
- Updated readme.hints.md and readme.plan.md to explain the complete replacement of the v1 protocol and new build process
|
||||
- Revised build order in tspublish.json files across ts, ts_tapbundle, ts_tapbundle_node, and ts_tapbundle_protocol
|
||||
- Introduced .claude/settings.local.json with updated permission settings for CLI and build tools
|
||||
|
||||
## 2025-05-24 - 1.11.5 - fix(tstest)
|
||||
Fix timeout handling to correctly evaluate TAP results after killing the test process.
|
||||
|
||||
- Added call to evaluateFinalResult() after killing the process in runInNode to ensure final TAP output is processed.
|
||||
|
||||
## 2025-05-24 - 1.11.4 - fix(logging)
|
||||
Improve warning logging and add permission settings file
|
||||
|
||||
- Replace multiple logger.error calls with logger.warning for tests running over 1 minute
|
||||
- Add warning method in tstest logger to display warning messages consistently
|
||||
- Introduce .claude/settings.local.json to configure allowed permissions
|
||||
|
||||
## 2025-05-24 - 1.11.3 - fix(tstest)
|
||||
Add timeout warning for long-running tests and introduce local settings configuration
|
||||
|
||||
- Add .claude/settings.local.json with permission configuration for local development
|
||||
- Implement a timeout warning timer that notifies when tests run longer than 1 minute without an explicit timeout
|
||||
- Clear the timeout warning timer upon test completion
|
||||
- Remove unused import of logPrefixes in tstest.classes.tstest.ts
|
||||
|
||||
## 2025-05-24 - 1.11.2 - fix(tstest)
|
||||
Improve timeout and error handling in test execution along with TAP parser timeout logic improvements.
|
||||
|
||||
- In the TAP parser, ensure that expected tests are properly set when no tests are defined to avoid false negatives on timeout.
|
||||
- Use smartshell's terminate method and fallback kill to properly stop the entire process tree on timeout.
|
||||
- Clean up browser, server, and WebSocket instances reliably even when a timeout occurs.
|
||||
- Minor improvements in log file filtering and error logging for better clarity.
|
||||
|
||||
## 2025-05-24 - 1.11.1 - fix(tstest)
|
||||
Clear timeout identifiers after successful test execution and add local CLAUDE settings
|
||||
|
||||
- Ensure timeout IDs are cleared when tests complete to prevent lingering timeouts
|
||||
- Add .claude/settings.local.json with updated permission settings for CLI commands
|
||||
|
||||
## 2025-05-24 - 1.11.0 - feat(cli)
|
||||
Add new timeout and file range options with enhanced logfile diff logging
|
||||
|
||||
- Introduce --timeout <seconds> option to safeguard tests from running too long
|
||||
- Add --startFrom and --stopAt options to control the range of test files executed
|
||||
- Enhance logfile organization by automatically moving previous logs and generating diff reports for failed or changed test outputs
|
||||
- Update CLI argument parsing and internal timeout handling for both Node.js and browser tests
|
||||
|
||||
## 2025-05-24 - 1.10.2 - fix(tstest-logging)
|
||||
Improve log file handling with log rotation and diff reporting
|
||||
|
||||
- Add .claude/settings.local.json to configure allowed shell and web operations
|
||||
- Introduce movePreviousLogFiles function to archive previous log files when --logfile is used
|
||||
- Enhance logging to generate error copies and diff reports between current and previous logs
|
||||
- Add type annotations for console overrides in browser evaluations for improved stability
|
||||
|
||||
## 2025-05-23 - 1.10.1 - fix(tstest)
|
||||
Improve file range filtering and summary logging by skipping test files outside the specified range and reporting them in the final summary.
|
||||
|
||||
- Introduce runSingleTestOrSkip to check file index against startFrom/stopAt values.
|
||||
- Log skipped files with appropriate messages and add them to the summary.
|
||||
- Update the logger to include total skipped files in the test summary.
|
||||
- Add permission settings in .claude/settings.local.json to support new operations.
|
||||
|
||||
## 2025-05-23 - 1.10.0 - feat(cli)
|
||||
Add --startFrom and --stopAt options to filter test files by range
|
||||
|
||||
- Introduced CLI options --startFrom and --stopAt in ts/index.ts for selective test execution
|
||||
- Added validation to ensure provided range values are positive and startFrom is not greater than stopAt
|
||||
- Propagated file range filtering into test grouping in tstest.classes.tstest.ts, applying the range filter across serial and parallel groups
|
||||
- Updated usage messages to include the new options
|
||||
|
||||
## 2025-05-23 - 1.9.4 - fix(docs)
|
||||
Update documentation and configuration for legal notices and CI permissions. This commit adds a new local settings file for tool permissions, refines the legal and trademark sections in the readme, and improves glob test files with clearer log messages.
|
||||
|
||||
- Added .claude/settings.local.json to configure permissions for various CLI commands
|
||||
- Revised legal and trademark documentation in the readme to clarify company ownership and usage guidelines
|
||||
- Updated glob test files with improved console log messages for better clarity during test discovery
|
||||
|
||||
## 2025-05-23 - 1.9.3 - fix(tstest)
|
||||
Fix test timing display issue and update TAP protocol documentation
|
||||
|
||||
- Changed TAP parser regex to non-greedy pattern to correctly separate test timing metadata
|
||||
- Enhanced readme.hints.md with detailed explanation of test timing fix and planned protocol upgrades
|
||||
- Updated readme.md with improved usage examples for tapbundle and comprehensive test framework documentation
|
||||
- Added new protocol design document (readme.protocol.md) and improvement plan (readme.plan.md) outlining future changes
|
||||
- Introduced .claude/settings.local.json update for npm and CLI permissions
|
||||
- Exported protocol utilities and added tapbundle protocol implementation for future enhancements
|
||||
|
||||
## 2025-05-23 - 1.9.2 - fix(logging)
|
||||
Fix log file naming to prevent collisions and update logging system documentation.
|
||||
|
||||
- Enhance safe filename generation in tstest logging to preserve directory structure using double underscores.
|
||||
- Update readme.hints.md to include detailed logging system documentation and behavior.
|
||||
- Add .claude/settings.local.json with updated permissions for build tools.
|
||||
|
||||
## 2025-05-23 - 1.9.1 - fix(dependencies)
|
||||
Update dependency versions and add local configuration files
|
||||
|
||||
- Bump @git.zone/tsbuild from ^2.5.1 to ^2.6.3
|
||||
- Bump @types/node from ^22.15.18 to ^22.15.21
|
||||
- Bump @push.rocks/smartexpect from ^2.4.2 to ^2.5.0
|
||||
- Bump @push.rocks/smartfile from ^11.2.0 to ^11.2.3
|
||||
- Bump @push.rocks/smartlog from ^3.1.1 to ^3.1.8
|
||||
- Add .npmrc with npm registry configuration
|
||||
- Add .claude/settings.local.json for local permissions
|
||||
|
||||
## 2025-05-16 - 1.9.0 - feat(docs)
|
||||
Update documentation to embed tapbundle and clarify module exports for browser compatibility; also add CI permission settings.
|
||||
|
||||
|
||||
13
deno.json
Normal file
13
deno.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"experimentalDecorators": true,
|
||||
"useDefineForClassFields": false,
|
||||
"lib": [
|
||||
"ES2022",
|
||||
"DOM"
|
||||
],
|
||||
"target": "ES2022"
|
||||
},
|
||||
"nodeModulesDir": true,
|
||||
"version": "3.0.1"
|
||||
}
|
||||
19
license.md
Normal file
19
license.md
Normal file
@@ -0,0 +1,19 @@
|
||||
Copyright (c) 2014 Task Venture Capital GmbH (hello@task.vc)
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -6,11 +6,11 @@
|
||||
"gitzone": {
|
||||
"projectType": "npm",
|
||||
"module": {
|
||||
"githost": "gitlab.com",
|
||||
"gitscope": "gitzone",
|
||||
"githost": "code.foss.global",
|
||||
"gitscope": "git.zone",
|
||||
"gitrepo": "tstest",
|
||||
"description": "a test utility to run tests that match test/**/*.ts",
|
||||
"npmPackagename": "@gitzone/tstest",
|
||||
"npmPackagename": "@git.zone/tstest",
|
||||
"license": "MIT"
|
||||
}
|
||||
}
|
||||
|
||||
43
package.json
43
package.json
@@ -1,12 +1,13 @@
|
||||
{
|
||||
"name": "@git.zone/tstest",
|
||||
"version": "1.9.0",
|
||||
"version": "3.0.1",
|
||||
"private": false,
|
||||
"description": "a test utility to run tests that match test/**/*.ts",
|
||||
"exports": {
|
||||
".": "./dist_ts/index.js",
|
||||
"./tapbundle": "./dist_ts_tapbundle/index.js",
|
||||
"./tapbundle_node": "./dist_ts_tapbundle_node/index.js"
|
||||
"./tapbundle_serverside": "./dist_ts_tapbundle_serverside/index.js",
|
||||
"./tapbundle_protocol": "./dist_ts_tapbundle_protocol/index.js"
|
||||
},
|
||||
"type": "module",
|
||||
"author": "Lossless GmbH",
|
||||
@@ -24,33 +25,35 @@
|
||||
"buildDocs": "tsdoc"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@git.zone/tsbuild": "^2.5.1",
|
||||
"@types/node": "^22.15.18"
|
||||
"@git.zone/tsbuild": "^3.1.0",
|
||||
"@types/node": "^22.15.21"
|
||||
},
|
||||
"dependencies": {
|
||||
"@api.global/typedserver": "^3.0.74",
|
||||
"@git.zone/tsbundle": "^2.2.5",
|
||||
"@git.zone/tsrun": "^1.3.3",
|
||||
"@push.rocks/consolecolor": "^2.0.2",
|
||||
"@push.rocks/qenv": "^6.1.0",
|
||||
"@api.global/typedserver": "^3.0.80",
|
||||
"@git.zone/tsbundle": "^2.5.2",
|
||||
"@git.zone/tsrun": "^2.0.0",
|
||||
"@push.rocks/consolecolor": "^2.0.3",
|
||||
"@push.rocks/qenv": "^6.1.3",
|
||||
"@push.rocks/smartbrowser": "^2.0.8",
|
||||
"@push.rocks/smartchok": "^1.1.1",
|
||||
"@push.rocks/smartcrypto": "^2.0.4",
|
||||
"@push.rocks/smartdelay": "^3.0.5",
|
||||
"@push.rocks/smartenv": "^5.0.12",
|
||||
"@push.rocks/smartexpect": "^2.4.2",
|
||||
"@push.rocks/smartfile": "^11.2.0",
|
||||
"@push.rocks/smartjson": "^5.0.20",
|
||||
"@push.rocks/smartlog": "^3.1.1",
|
||||
"@push.rocks/smartmongo": "^2.0.12",
|
||||
"@push.rocks/smartpath": "^5.0.18",
|
||||
"@push.rocks/smartenv": "^6.0.0",
|
||||
"@push.rocks/smartexpect": "^2.5.0",
|
||||
"@push.rocks/smartfile": "^11.2.7",
|
||||
"@push.rocks/smartjson": "^5.2.0",
|
||||
"@push.rocks/smartlog": "^3.1.10",
|
||||
"@push.rocks/smartmongo": "^2.0.14",
|
||||
"@push.rocks/smartnetwork": "^4.4.0",
|
||||
"@push.rocks/smartpath": "^6.0.0",
|
||||
"@push.rocks/smartpromise": "^4.2.3",
|
||||
"@push.rocks/smartrequest": "^2.1.0",
|
||||
"@push.rocks/smarts3": "^2.2.5",
|
||||
"@push.rocks/smartshell": "^3.2.3",
|
||||
"@push.rocks/smartrequest": "^5.0.1",
|
||||
"@push.rocks/smarts3": "^2.2.7",
|
||||
"@push.rocks/smartshell": "^3.3.0",
|
||||
"@push.rocks/smarttime": "^4.1.1",
|
||||
"@types/ws": "^8.18.1",
|
||||
"figures": "^6.1.0",
|
||||
"ws": "^8.18.2"
|
||||
"ws": "^8.18.3"
|
||||
},
|
||||
"files": [
|
||||
"ts/**/*",
|
||||
|
||||
6218
pnpm-lock.yaml
generated
6218
pnpm-lock.yaml
generated
File diff suppressed because it is too large
Load Diff
4
pnpm-workspace.yaml
Normal file
4
pnpm-workspace.yaml
Normal file
@@ -0,0 +1,4 @@
|
||||
onlyBuiltDependencies:
|
||||
- esbuild
|
||||
- mongodb-memory-server
|
||||
- puppeteer
|
||||
275
readme.hints.md
275
readme.hints.md
@@ -6,13 +6,13 @@ This project integrates tstest with tapbundle through a modular architecture:
|
||||
|
||||
1. **tstest** (`/ts/`) - The test runner that discovers and executes test files
|
||||
2. **tapbundle** (`/ts_tapbundle/`) - The TAP testing framework for writing tests
|
||||
3. **tapbundle_node** (`/ts_tapbundle_node/`) - Node.js-specific testing utilities
|
||||
3. **tapbundle_serverside** (`/ts_tapbundle_serverside/`) - Server-side testing utilities (runCommand, env vars, HTTPS certs, MongoDB, S3, test assets)
|
||||
|
||||
## How Components Work Together
|
||||
|
||||
### Test Execution Flow
|
||||
|
||||
1. **CLI Entry Point** (`cli.js` <20> `cli.ts.js` <20> `cli.child.ts`)
|
||||
1. **CLI Entry Point** (`cli.js` <20> `cli.ts.js` <20> `cli.child.ts`)
|
||||
- The CLI uses tsx to run TypeScript files directly
|
||||
- Accepts glob patterns to find test files
|
||||
- Supports options like `--verbose`, `--quiet`, `--web`
|
||||
@@ -31,7 +31,7 @@ This project integrates tstest with tapbundle through a modular architecture:
|
||||
|
||||
1. **Import Structure**
|
||||
- Test files import from local tapbundle: `import { tap, expect } from '../../ts_tapbundle/index.js'`
|
||||
- Node-specific tests also import from tapbundle_node: `import { tapNodeTools } from '../../ts_tapbundle_node/index.js'`
|
||||
- Server-side tests also import from tapbundle_serverside for Node.js-only utilities: `import { tapNodeTools } from '../../ts_tapbundle_serverside/index.js'`
|
||||
|
||||
2. **WebHelpers**
|
||||
- Browser tests can use webhelpers for DOM manipulation
|
||||
@@ -40,9 +40,17 @@ This project integrates tstest with tapbundle through a modular architecture:
|
||||
- Automatically detects browser environment and only enables in browser context
|
||||
|
||||
3. **Build System**
|
||||
- Uses `tsbuild tsfolders` to compile TypeScript
|
||||
- Maintains separate output directories: `/dist_ts/`, `/dist_ts_tapbundle/`, `/dist_ts_tapbundle_node/`
|
||||
- Compilation order is resolved automatically based on dependencies
|
||||
- Uses `tsbuild tsfolders` to compile TypeScript (invoked by `pnpm build`)
|
||||
- Maintains separate output directories: `/dist_ts/`, `/dist_ts_tapbundle/`, `/dist_ts_tapbundle_serverside/`, `/dist_ts_tapbundle_protocol/`
|
||||
- Compilation order is resolved automatically based on dependencies in tspublish.json files
|
||||
- Protocol imports use compiled dist directories:
|
||||
```typescript
|
||||
// In ts/tstest.classes.tap.parser.ts
|
||||
import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
|
||||
// In ts_tapbundle/tapbundle.classes.tap.ts
|
||||
import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
```
|
||||
|
||||
### Test Scripts
|
||||
|
||||
@@ -59,4 +67,257 @@ The framework automatically detects the runtime environment:
|
||||
- Browser tests are compiled and served via a local server
|
||||
- WebHelpers are only enabled in browser environment
|
||||
|
||||
This architecture allows for seamless testing across both Node.js and browser environments while maintaining a clean separation of concerns.
|
||||
This architecture allows for seamless testing across both Node.js and browser environments while maintaining a clean separation of concerns.
|
||||
|
||||
## Logging System
|
||||
|
||||
### Log File Naming (Fixed in v1.9.1)
|
||||
|
||||
When using the `--logfile` flag, tstest creates log files in `.nogit/testlogs/`. The log file naming was updated to preserve directory structure and prevent collisions:
|
||||
|
||||
- **Old behavior**: `test/tapbundle/test.ts` → `.nogit/testlogs/test.log`
|
||||
- **New behavior**: `test/tapbundle/test.ts` → `.nogit/testlogs/test__tapbundle__test.log`
|
||||
|
||||
This fix ensures that test files with the same basename in different directories don't overwrite each other's logs. The implementation:
|
||||
1. Takes the relative path from the current working directory
|
||||
2. Replaces path separators (`/`) with double underscores (`__`)
|
||||
3. Removes the `.ts` extension
|
||||
4. Creates a flat filename that preserves the directory structure
|
||||
|
||||
### Test Timing Display (Fixed in v1.9.2)
|
||||
|
||||
Fixed an issue where test timing was displayed incorrectly with duplicate values like:
|
||||
- Before: `✅ test name # time=133ms (0ms)`
|
||||
- After: `✅ test name (133ms)`
|
||||
|
||||
The issue was in the TAP parser regex which was greedily capturing the entire line including the TAP timing comment. Changed the regex from `(.*)` to `(.*?)` to make it non-greedy, properly separating the test name from the timing metadata.
|
||||
|
||||
## Protocol Limitations and Improvements
|
||||
|
||||
### Current TAP Protocol Issues
|
||||
The current implementation uses standard TAP format with metadata in comments:
|
||||
```
|
||||
ok 1 - test name # time=123ms
|
||||
```
|
||||
|
||||
This has several limitations:
|
||||
1. **Delimiter Conflict**: Test descriptions containing `#` can break parsing
|
||||
2. **Regex Fragility**: Complex regex patterns that are hard to maintain
|
||||
3. **Limited Metadata**: Difficult to add rich error information or custom data
|
||||
|
||||
### Planned Protocol V2
|
||||
A new internal protocol is being designed that will:
|
||||
- Use Unicode delimiters `⟦TSTEST:⟧` that won't conflict with test content
|
||||
- Support structured JSON metadata
|
||||
- Allow rich error reporting with stack traces and diffs
|
||||
- Completely replace v1 protocol (no backwards compatibility)
|
||||
|
||||
### ts_tapbundle_protocol Directory
|
||||
The protocol v2 implementation is contained in a separate `ts_tapbundle_protocol` directory:
|
||||
- **Isomorphic Code**: All protocol code works in both browser and Node.js environments
|
||||
- **No Platform Dependencies**: No Node.js-specific imports, ensuring true cross-platform compatibility
|
||||
- **Clean Separation**: Protocol logic is isolated from platform-specific code in tstest and tapbundle
|
||||
- **Shared Implementation**: Both tstest (parser) and tapbundle (emitter) use the same protocol classes
|
||||
- **Build Process**:
|
||||
- Compiled by `pnpm build` via tsbuild to `dist_ts_tapbundle_protocol/`
|
||||
- Build order managed through tspublish.json files
|
||||
- Other modules import from the compiled dist directory, not source
|
||||
|
||||
This architectural decision ensures the protocol can be used in any JavaScript environment without modification and maintains proper build dependencies.
|
||||
|
||||
See `readme.protocol.md` for the full specification and `ts_tapbundle_protocol/` for the implementation.
|
||||
|
||||
## Protocol V2 Implementation Status
|
||||
|
||||
The Protocol V2 has been implemented to fix issues with TAP protocol parsing when test descriptions contain special characters like `#`, `###SNAPSHOT###`, or protocol markers like `⟦TSTEST:ERROR⟧`.
|
||||
|
||||
### Implementation Details:
|
||||
|
||||
1. **Protocol Components**:
|
||||
- `ProtocolEmitter` - Generates protocol v2 messages (used by tapbundle)
|
||||
- `ProtocolParser` - Parses protocol v2 messages (used by tstest)
|
||||
- Uses Unicode markers `⟦TSTEST:` and `⟧` to avoid conflicts with test content
|
||||
|
||||
2. **Current Status**:
|
||||
- ✅ Basic protocol emission and parsing works
|
||||
- ✅ Handles test descriptions with special characters correctly
|
||||
- ✅ Supports metadata for timing, tags, errors
|
||||
- ⚠️ Protocol messages sometimes appear in console output (parsing not catching all cases)
|
||||
|
||||
3. **Key Findings**:
|
||||
- `tap.skip.test()` doesn't create actual test objects, just logs and increments counter
|
||||
- `tap.todo()` method is not implemented (no `addTodo` method in Tap class)
|
||||
- Protocol parser's `isBlockStart` was fixed to only match exact block markers, not partial matches in test descriptions
|
||||
|
||||
4. **Import Paths**:
|
||||
- tstest imports from: `import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';`
|
||||
- tapbundle imports from: `import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';`
|
||||
|
||||
## Test Configuration System (Phase 2)
|
||||
|
||||
The Test Configuration System has been implemented to provide global settings and lifecycle hooks for tests.
|
||||
|
||||
### Key Features:
|
||||
|
||||
1. **00init.ts Discovery**:
|
||||
- Automatically detects `00init.ts` files in the same directory as test files
|
||||
- Creates a temporary loader file that imports both `00init.ts` and the test file
|
||||
- Loader files are cleaned up automatically after test execution
|
||||
|
||||
2. **Settings Inheritance**:
|
||||
- Global settings from `00init.ts` → File-level settings → Test-level settings
|
||||
- Settings include: timeout, retries, retryDelay, bail, concurrency
|
||||
- Lifecycle hooks: beforeAll, afterAll, beforeEach, afterEach
|
||||
|
||||
3. **Implementation Details**:
|
||||
- `SettingsManager` class handles settings inheritance and merging
|
||||
- `tap.settings()` API allows configuration at any level
|
||||
- Lifecycle hooks are integrated into test execution flow
|
||||
|
||||
### Important Development Notes:
|
||||
|
||||
1. **Local Development**: When developing tstest itself, use `node cli.js` instead of globally installed `tstest` to test changes
|
||||
|
||||
2. **Console Output Buffering**: Console output from tests is buffered and only displayed for failing tests. TAP-compliant comments (lines starting with `#`) are always shown.
|
||||
|
||||
3. **TypeScript Warnings**: Fixed async/await warnings in `movePreviousLogFiles()` by using sync versions of file operations
|
||||
|
||||
## Enhanced Communication Features (Phase 3)
|
||||
|
||||
The Enhanced Communication system has been implemented to provide rich, real-time feedback during test execution.
|
||||
|
||||
### Key Features:
|
||||
|
||||
1. **Event-Based Test Lifecycle Reporting**:
|
||||
- `test:queued` - Test is ready to run
|
||||
- `test:started` - Test execution begins
|
||||
- `test:completed` - Test finishes (with pass/fail status)
|
||||
- `suite:started` - Test suite/describe block begins
|
||||
- `suite:completed` - Test suite/describe block ends
|
||||
- `hook:started` - Lifecycle hook (beforeEach/afterEach) begins
|
||||
- `hook:completed` - Lifecycle hook finishes
|
||||
- `assertion:failed` - Assertion failure with detailed information
|
||||
|
||||
2. **Visual Diff Output for Assertion Failures**:
|
||||
- **String Diffs**: Character-by-character comparison with colored output
|
||||
- **Object/Array Diffs**: Deep property comparison showing added/removed/changed properties
|
||||
- **Primitive Diffs**: Clear display of expected vs actual values
|
||||
- **Colorized Output**: Green for expected, red for actual, yellow for differences
|
||||
- **Smart Formatting**: Multi-line strings and complex objects are formatted for readability
|
||||
|
||||
3. **Real-Time Test Progress API**:
|
||||
- Tests emit progress events as they execute
|
||||
- tstest parser processes events and updates display in real-time
|
||||
- Structured event format carries rich metadata (timing, errors, diffs)
|
||||
- Seamless integration with existing TAP protocol via Protocol V2
|
||||
|
||||
### Implementation Details:
|
||||
- Events are transmitted via Protocol V2's `EVENT` block type
|
||||
- Event data is JSON-encoded within protocol markers
|
||||
- Parser handles events asynchronously for real-time updates
|
||||
- Visual diffs are generated using custom diff algorithms for each data type
|
||||
|
||||
## Watch Mode (Phase 4)
|
||||
|
||||
tstest now supports watch mode for automatic test re-runs on file changes.
|
||||
|
||||
### Usage
|
||||
```bash
|
||||
tstest test/**/*.ts --watch
|
||||
tstest test/specific.ts -w
|
||||
```
|
||||
|
||||
### Features
|
||||
- **Automatic Re-runs**: Tests re-run when any watched file changes
|
||||
- **Debouncing**: Multiple rapid changes are batched (300ms delay)
|
||||
- **Clear Output**: Console is cleared before each run for clean results
|
||||
- **Status Updates**: Shows which files triggered the re-run
|
||||
- **Graceful Exit**: Press Ctrl+C to stop watching
|
||||
|
||||
### Options
|
||||
- `--watch` or `-w`: Enable watch mode
|
||||
- `--watch-ignore`: Comma-separated patterns to ignore (e.g., `--watch-ignore node_modules,dist`)
|
||||
|
||||
### Implementation Details
|
||||
- Uses `@push.rocks/smartchok` for cross-platform file watching
|
||||
- Watches the entire project directory from where tests are run
|
||||
- Ignores changes matching the ignore patterns
|
||||
- Shows "Waiting for file changes..." between runs
|
||||
|
||||
## Fixed Issues
|
||||
|
||||
### tap.skip.test(), tap.todo(), and tap.only.test() (Fixed)
|
||||
|
||||
Previously reported issues with these methods have been resolved:
|
||||
|
||||
1. **tap.skip.test()** - Now properly creates test objects that are counted in test results
|
||||
- Tests marked with `skip.test()` appear in the test count
|
||||
- Shows as passed with skip directive in TAP output
|
||||
- `markAsSkipped()` method added to handle pre-test skip marking
|
||||
|
||||
2. **tap.todo.test()** - Fully implemented with test object creation
|
||||
- Supports both `tap.todo.test('description')` and `tap.todo.test('description', testFunc)`
|
||||
- Todo tests are counted and marked with todo directive
|
||||
- Both regular and parallel todo tests supported
|
||||
|
||||
3. **tap.only.test()** - Works correctly for focused testing
|
||||
- When `.only` tests exist, only those tests run
|
||||
- Other tests are not executed but still counted
|
||||
- Both regular and parallel only tests supported
|
||||
|
||||
These fixes ensure accurate test counts and proper TAP-compliant output for all test states.
|
||||
|
||||
## Test Timing Implementation
|
||||
|
||||
### Timing Architecture
|
||||
|
||||
Test timing is captured using `@push.rocks/smarttime`'s `HrtMeasurement` class, which provides high-resolution timing:
|
||||
|
||||
1. **Timing Capture**:
|
||||
- Each `TapTest` instance has its own `HrtMeasurement`
|
||||
- Timer starts immediately before test function execution
|
||||
- Timer stops after test completes (or fails/times out)
|
||||
- Millisecond precision is used for reporting
|
||||
|
||||
2. **Protocol Integration**:
|
||||
- Timing is embedded in TAP output using Protocol V2 markers
|
||||
- Inline format for simple timing: `ok 1 - test name ⟦TSTEST:time:123⟧`
|
||||
- Block format for complex metadata: `⟦TSTEST:META:{"time":456,"file":"test.ts"}⟧`
|
||||
|
||||
3. **Performance Metrics Calculation**:
|
||||
- Average is calculated from sum of individual test times, not total runtime
|
||||
- Slowest test detection prefers tests with >0ms duration
|
||||
- Failed tests still contribute their execution time to metrics
|
||||
|
||||
### Edge Cases and Considerations
|
||||
|
||||
1. **Sub-millisecond Tests**:
|
||||
- Very fast tests may report 0ms due to millisecond rounding
|
||||
- Performance metrics handle this by showing "All tests completed in <1ms" when appropriate
|
||||
|
||||
2. **Special Test States**:
|
||||
- **Skipped tests**: Report 0ms (not executed)
|
||||
- **Todo tests**: Report 0ms (not executed)
|
||||
- **Failed tests**: Report actual execution time before failure
|
||||
- **Timeout tests**: Report time until timeout occurred
|
||||
|
||||
3. **Parallel Test Timing**:
|
||||
- Each parallel test tracks its own execution time independently
|
||||
- Parallel tests may have overlapping execution periods
|
||||
- Total suite time reflects wall-clock time, not sum of test times
|
||||
|
||||
4. **Hook Timing**:
|
||||
- `beforeEach`/`afterEach` hooks are not included in individual test times
|
||||
- Only the actual test function execution is measured
|
||||
|
||||
5. **Retry Timing**:
|
||||
- When tests retry, only the final attempt's duration is reported
|
||||
- Each retry attempt emits separate `test:started` events
|
||||
|
||||
### Parser Fix for Timing Metadata
|
||||
|
||||
The protocol parser was fixed to correctly handle inline timing metadata:
|
||||
- Changed condition from `!simpleMatch[1].includes(':')` to check for simple key:value pairs
|
||||
- Excludes prefixed formats (META:, SKIP:, TODO:, EVENT:) while parsing simple formats like `time:250`
|
||||
|
||||
This ensures timing metadata is correctly extracted and displayed in test results.
|
||||
361
readme.plan.md
361
readme.plan.md
@@ -2,61 +2,115 @@
|
||||
|
||||
!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
|
||||
|
||||
## 1. Enhanced Communication Between tapbundle and tstest
|
||||
## Improved Internal Protocol (NEW - Critical) ✅ COMPLETED
|
||||
|
||||
### 1.1 Real-time Test Progress API
|
||||
- Create a bidirectional communication channel between tapbundle and tstest
|
||||
- Emit events for test lifecycle stages (start, progress, completion)
|
||||
- Allow tstest to subscribe to tapbundle events for better progress reporting
|
||||
- Implement a standardized message format for test metadata
|
||||
### Current Issues ✅ RESOLVED
|
||||
- ✅ TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
|
||||
- ✅ Fragile regex parsing that breaks with special characters
|
||||
- ✅ Limited extensibility for new metadata types
|
||||
|
||||
### 1.2 Rich Error Reporting
|
||||
- Pass structured error objects from tapbundle to tstest
|
||||
- Include stack traces, code snippets, and contextual information
|
||||
- Support for error categorization (assertion failures, timeouts, uncaught exceptions)
|
||||
- Visual diff output for failed assertions
|
||||
### Proposed Solution: Protocol V2 ✅ IMPLEMENTED
|
||||
- ✅ Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
|
||||
- ✅ Structured JSON metadata format
|
||||
- ✅ Separate protocol blocks for complex data (errors, snapshots)
|
||||
- ✅ Complete replacement of v1 (no backwards compatibility needed)
|
||||
|
||||
### Implementation ✅ COMPLETED
|
||||
- ✅ Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
|
||||
- ✅ Phase 2: Replace all v1 code in both tstest and tapbundle with v2
|
||||
- ✅ Phase 3: Delete all v1 parsing and generation code
|
||||
|
||||
#### ts_tapbundle_protocol Directory
|
||||
The protocol v2 implementation will be contained in the `ts_tapbundle_protocol` directory as isomorphic TypeScript code:
|
||||
- **Isomorphic Design**: All code must work in both browser and Node.js environments
|
||||
- **No Node.js Imports**: No Node.js-specific modules allowed (no fs, path, child_process, etc.)
|
||||
- **Protocol Classes**: Contains classes implementing all sides of the protocol:
|
||||
- ✅ `ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
|
||||
- ✅ `ProtocolParser`: For parsing protocol v2 messages (used by tstest)
|
||||
- ✅ `ProtocolMessage`: Base classes for different message types
|
||||
- ✅ `ProtocolTypes`: TypeScript interfaces and types for protocol structures
|
||||
- **Pure TypeScript**: Only browser-compatible APIs and pure TypeScript/JavaScript code
|
||||
- **Build Integration**:
|
||||
- Compiled by `pnpm build` (via tsbuild) to `dist_ts_tapbundle_protocol/`
|
||||
- Build order defined in tspublish.json files
|
||||
- Imported by ts and ts_tapbundle modules from the compiled dist directory
|
||||
|
||||
See `readme.protocol.md` for detailed specification.
|
||||
|
||||
## Test Configuration System (NEW)
|
||||
|
||||
### Global Test Configuration via 00init.ts
|
||||
- **Discovery**: Check for `test/00init.ts` before running tests
|
||||
- **Execution**: Import and execute before any test files if found
|
||||
- **Purpose**: Define project-wide default test settings
|
||||
|
||||
### tap.settings() API
|
||||
```typescript
|
||||
interface TapSettings {
|
||||
// Timing
|
||||
timeout?: number; // Default timeout for all tests (ms)
|
||||
slowThreshold?: number; // Mark tests as slow if they exceed this (ms)
|
||||
|
||||
// Execution Control
|
||||
bail?: boolean; // Stop on first test failure
|
||||
retries?: number; // Number of retries for failed tests
|
||||
retryDelay?: number; // Delay between retries (ms)
|
||||
|
||||
// Output Control
|
||||
suppressConsole?: boolean; // Suppress console output in passing tests
|
||||
verboseErrors?: boolean; // Show full stack traces
|
||||
showTestDuration?: boolean; // Show duration for each test
|
||||
|
||||
// Parallel Execution
|
||||
maxConcurrency?: number; // Max parallel tests (for .para files)
|
||||
isolateTests?: boolean; // Run each test in fresh context
|
||||
|
||||
// Lifecycle Hooks
|
||||
beforeAll?: () => Promise<void> | void;
|
||||
afterAll?: () => Promise<void> | void;
|
||||
beforeEach?: (testName: string) => Promise<void> | void;
|
||||
afterEach?: (testName: string, passed: boolean) => Promise<void> | void;
|
||||
|
||||
// Environment
|
||||
env?: Record<string, string>; // Additional environment variables
|
||||
|
||||
// Features
|
||||
enableSnapshots?: boolean; // Enable snapshot testing
|
||||
snapshotDirectory?: string; // Custom snapshot directory
|
||||
updateSnapshots?: boolean; // Update snapshots instead of comparing
|
||||
}
|
||||
```
|
||||
|
||||
### Settings Inheritance
|
||||
- Global (00init.ts) → File level → Test level
|
||||
- More specific settings override less specific ones
|
||||
- Arrays/objects are merged, primitives are replaced
|
||||
|
||||
### Implementation Phases
|
||||
1. **Core Infrastructure**: Settings storage and merge logic
|
||||
2. **Discovery**: 00init.ts loading mechanism
|
||||
3. **Application**: Apply settings to test execution
|
||||
4. **Advanced**: Parallel execution and snapshot configuration
|
||||
|
||||
## 1. Enhanced Communication Between tapbundle and tstest ✅ COMPLETED
|
||||
|
||||
### 1.1 Real-time Test Progress API ✅ COMPLETED
|
||||
- ✅ Create a bidirectional communication channel between tapbundle and tstest
|
||||
- ✅ Emit events for test lifecycle stages (start, progress, completion)
|
||||
- ✅ Allow tstest to subscribe to tapbundle events for better progress reporting
|
||||
- ✅ Implement a standardized message format for test metadata
|
||||
|
||||
### 1.2 Rich Error Reporting ✅ COMPLETED
|
||||
- ✅ Pass structured error objects from tapbundle to tstest
|
||||
- ✅ Include stack traces, code snippets, and contextual information
|
||||
- ✅ Support for error categorization (assertion failures, timeouts, uncaught exceptions)
|
||||
- ✅ Visual diff output for failed assertions
|
||||
|
||||
## 2. Enhanced toolsArg Functionality
|
||||
|
||||
### 2.1 Test Flow Control ✅
|
||||
```typescript
|
||||
tap.test('conditional test', async (toolsArg) => {
|
||||
const result = await someOperation();
|
||||
|
||||
// Skip the rest of the test
|
||||
if (!result) {
|
||||
return toolsArg.skip('Precondition not met');
|
||||
}
|
||||
|
||||
// Conditional skipping
|
||||
await toolsArg.skipIf(condition, 'Reason for skipping');
|
||||
|
||||
// Mark test as todo
|
||||
await toolsArg.todo('Not implemented yet');
|
||||
});
|
||||
```
|
||||
|
||||
### 2.2 Test Metadata and Configuration ✅
|
||||
```typescript
|
||||
// Fluent syntax ✅
|
||||
tap.tags('slow', 'integration')
|
||||
.priority('high')
|
||||
.timeout(5000)
|
||||
.retry(3)
|
||||
.test('configurable test', async (toolsArg) => {
|
||||
// Test implementation
|
||||
});
|
||||
```
|
||||
|
||||
### 2.3 Test Data and Context Sharing ✅
|
||||
### 2.3 Test Data and Context Sharing (Partial)
|
||||
```typescript
|
||||
tap.test('data-driven test', async (toolsArg) => {
|
||||
// Access shared context ✅
|
||||
const sharedData = toolsArg.context.get('sharedData');
|
||||
|
||||
// Set data for other tests ✅
|
||||
toolsArg.context.set('resultData', computedValue);
|
||||
|
||||
// Parameterized test data (not yet implemented)
|
||||
const testData = toolsArg.data<TestInput>();
|
||||
expect(processData(testData)).toEqual(expected);
|
||||
@@ -65,32 +119,7 @@ tap.test('data-driven test', async (toolsArg) => {
|
||||
|
||||
## 3. Nested Tests and Test Suites
|
||||
|
||||
### 3.1 Test Grouping with describe() ✅
|
||||
```typescript
|
||||
tap.describe('User Authentication', () => {
|
||||
tap.beforeEach(async (toolsArg) => {
|
||||
// Setup for each test in this suite
|
||||
await toolsArg.context.set('db', await createTestDatabase());
|
||||
});
|
||||
|
||||
tap.afterEach(async (toolsArg) => {
|
||||
// Cleanup after each test
|
||||
await toolsArg.context.get('db').cleanup();
|
||||
});
|
||||
|
||||
tap.test('should login with valid credentials', async (toolsArg) => {
|
||||
// Test implementation
|
||||
});
|
||||
|
||||
tap.describe('Password Reset', () => {
|
||||
tap.test('should send reset email', async (toolsArg) => {
|
||||
// Nested test
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### 3.2 Hierarchical Test Organization
|
||||
### 3.2 Hierarchical Test Organization (Not yet implemented)
|
||||
- Support for multiple levels of nesting
|
||||
- Inherited context and configuration from parent suites
|
||||
- Aggregated reporting for test suites
|
||||
@@ -98,15 +127,7 @@ tap.describe('User Authentication', () => {
|
||||
|
||||
## 4. Advanced Test Features
|
||||
|
||||
### 4.1 Snapshot Testing
|
||||
```typescript
|
||||
tap.test('component render', async (toolsArg) => {
|
||||
const output = renderComponent(props);
|
||||
|
||||
// Compare with stored snapshot
|
||||
await toolsArg.matchSnapshot(output, 'component-output');
|
||||
});
|
||||
```
|
||||
### 4.1 Snapshot Testing ✅ (Basic implementation complete)
|
||||
|
||||
### 4.2 Performance Benchmarking
|
||||
```typescript
|
||||
@@ -124,42 +145,20 @@ tap.test('performance test', async (toolsArg) => {
|
||||
});
|
||||
```
|
||||
|
||||
### 4.3 Test Fixtures and Factories ✅
|
||||
```typescript
|
||||
tap.test('with fixtures', async (toolsArg) => {
|
||||
// Create test fixtures
|
||||
const user = await toolsArg.fixture('user', { name: 'Test User' });
|
||||
const post = await toolsArg.fixture('post', { author: user });
|
||||
|
||||
// Use factory functions
|
||||
const users = await toolsArg.factory('user').createMany(5);
|
||||
});
|
||||
```
|
||||
|
||||
## 5. Test Execution Improvements
|
||||
|
||||
### 5.1 Parallel Test Execution ✅
|
||||
- Run independent tests concurrently ✅
|
||||
- Configurable concurrency limits (via file naming convention)
|
||||
- Resource pooling for shared resources
|
||||
- Proper isolation between parallel tests ✅
|
||||
|
||||
Implementation:
|
||||
- Tests with `para__<groupNumber>` in filename run in parallel
|
||||
- Different groups run sequentially
|
||||
- Tests without `para__` run serially
|
||||
|
||||
### 5.2 Watch Mode
|
||||
### 5.2 Watch Mode ✅ COMPLETED
|
||||
- Automatically re-run tests on file changes
|
||||
- Intelligent test selection based on changed files
|
||||
- Fast feedback loop for development
|
||||
- Integration with IDE/editor plugins
|
||||
- Debounced file change detection (300ms)
|
||||
- Clear console output between runs
|
||||
- Shows which files triggered re-runs
|
||||
- Graceful exit with Ctrl+C
|
||||
- `--watch-ignore` option for excluding patterns
|
||||
|
||||
### 5.3 Advanced Test Filtering ✅ (partially)
|
||||
### 5.3 Advanced Test Filtering (Partial) ⚠️
|
||||
```typescript
|
||||
// Run tests by tags ✅
|
||||
tstest --tags "unit,fast"
|
||||
|
||||
// Exclude tests by pattern (not yet implemented)
|
||||
tstest --exclude "**/slow/**"
|
||||
|
||||
@@ -198,58 +197,52 @@ tstest --changed
|
||||
- Links to documentation
|
||||
- Code examples in error output
|
||||
|
||||
### 7.2 Interactive Mode (Needs Detailed Specification)
|
||||
- REPL for exploring test failures
|
||||
- Need to define: How to enter interactive mode? When tests fail?
|
||||
- What commands/features should be available in the REPL?
|
||||
- Debugging integration
|
||||
- Node.js inspector protocol integration?
|
||||
- Breakpoint support?
|
||||
- Step-through test execution
|
||||
- Pause between tests?
|
||||
- Step into/over/out functionality?
|
||||
- Interactive test data manipulation
|
||||
- Modify test inputs on the fly?
|
||||
- Inspect intermediate values?
|
||||
|
||||
### 7.3 ~~VS Code Extension~~ (Scratched)
|
||||
- ~~Test explorer integration~~
|
||||
- ~~Inline test results~~
|
||||
- ~~CodeLens for running individual tests~~
|
||||
- ~~Debugging support~~
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Core Enhancements (Priority: High) ✅
|
||||
1. Implement enhanced toolsArg methods (skip, skipIf, timeout, retry) ✅
|
||||
2. Add basic test grouping with describe() ✅
|
||||
3. Improve error reporting between tapbundle and tstest ✅
|
||||
### Phase 1: Improved Internal Protocol (Priority: Critical) ✅ COMPLETED
|
||||
1. ✅ Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
|
||||
- ✅ Implement ProtocolEmitter class for message generation
|
||||
- ✅ Implement ProtocolParser class for message parsing
|
||||
- ✅ Define ProtocolMessage types and interfaces
|
||||
- ✅ Ensure all code is browser and Node.js compatible
|
||||
- ✅ Add tspublish.json to configure build order
|
||||
2. ✅ Update build configuration to compile ts_tapbundle_protocol first
|
||||
3. ✅ Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
|
||||
4. ✅ Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
|
||||
5. ✅ Delete all v1 TAP parsing code from tstest
|
||||
6. ✅ Delete all v1 TAP generation code from tapbundle
|
||||
7. ✅ Test with real-world test suites containing special characters
|
||||
|
||||
### Phase 2: Advanced Features (Priority: Medium)
|
||||
1. Implement nested test suites ✅ (basic describe support)
|
||||
2. Add snapshot testing ✅
|
||||
3. Create test fixture system ✅
|
||||
4. Implement parallel test execution ✅
|
||||
### Phase 2: Test Configuration System (Priority: High) ✅ COMPLETED
|
||||
1. ✅ Implement tap.settings() API with TypeScript interfaces
|
||||
2. ✅ Add 00init.ts discovery and loading mechanism
|
||||
3. ✅ Implement settings inheritance and merge logic
|
||||
4. ✅ Apply settings to test execution (timeouts, retries, etc.)
|
||||
|
||||
### Phase 3: Developer Experience (Priority: Medium)
|
||||
### Phase 3: Enhanced Communication (Priority: High) ✅ COMPLETED
|
||||
1. ✅ Build on Protocol V2 for richer communication
|
||||
2. ✅ Implement real-time test progress API
|
||||
3. ✅ Add structured error reporting with diffs and traces
|
||||
|
||||
### Phase 4: Developer Experience (Priority: Medium) ❌ NOT STARTED
|
||||
1. Add watch mode
|
||||
2. Implement custom reporters
|
||||
3. ~~Create VS Code extension~~ (Scratched)
|
||||
4. Add interactive debugging (Needs detailed spec first)
|
||||
3. Complete advanced test filtering options
|
||||
4. Add performance benchmarking API
|
||||
|
||||
### Phase 4: Analytics and Performance (Priority: Low)
|
||||
### Phase 5: Analytics and Performance (Priority: Low) ❌ NOT STARTED
|
||||
1. Build test analytics dashboard
|
||||
2. Add performance benchmarking
|
||||
3. Implement coverage integration
|
||||
4. Create trend analysis tools
|
||||
2. Implement coverage integration
|
||||
3. Create trend analysis tools
|
||||
4. Add test impact analysis
|
||||
|
||||
## Technical Considerations
|
||||
|
||||
### API Design Principles
|
||||
- Maintain backward compatibility
|
||||
- Clean, modern API design without legacy constraints
|
||||
- Progressive enhancement approach
|
||||
- Opt-in features to avoid breaking changes
|
||||
- Clear migration paths for new features
|
||||
- Well-documented features and APIs
|
||||
- Clear, simple interfaces
|
||||
|
||||
### Performance Goals
|
||||
- Minimal overhead for test execution
|
||||
@@ -261,4 +254,68 @@ tstest --changed
|
||||
- Clean interfaces between tstest and tapbundle
|
||||
- Extensible plugin architecture
|
||||
- Standard test result format
|
||||
- Compatible with existing CI/CD tools
|
||||
- Compatible with existing CI/CD tools
|
||||
|
||||
## Summary of Remaining Work
|
||||
|
||||
### ✅ Completed
|
||||
- **Protocol V2**: Full implementation with Unicode delimiters, structured metadata, and special character handling
|
||||
- **Test Configuration System**: tap.settings() API, 00init.ts discovery, settings inheritance, lifecycle hooks
|
||||
- **Enhanced Communication**: Event-based test lifecycle reporting, visual diff output for assertion failures, real-time test progress API
|
||||
- **Rich Error Reporting**: Stack traces, error metadata, and visual diffs through protocol
|
||||
- **Tags Filtering**: `--tags` option for running specific tagged tests
|
||||
|
||||
### ✅ Existing Features (Not in Plan)
|
||||
- **Timeout Support**: `--timeout` option and per-test timeouts
|
||||
- **Test Retries**: `tap.retry()` for flaky test handling
|
||||
- **Parallel Tests**: `.testParallel()` for concurrent execution
|
||||
- **Snapshot Testing**: Basic implementation with `toMatchSnapshot()`
|
||||
- **Test Lifecycle**: `describe()` blocks with `beforeEach`/`afterEach`
|
||||
- **Skip Tests**: `tap.skip.test()` (though it doesn't create test objects)
|
||||
- **Log Files**: `--logfile` option saves output to `.nogit/testlogs/`
|
||||
- **Test Range**: `--startFrom` and `--stopAt` for partial runs
|
||||
|
||||
### ⚠️ Partially Completed
|
||||
- **Advanced Test Filtering**: Have `--tags` but missing `--exclude`, `--failed`, `--changed`
|
||||
|
||||
### ❌ Not Started
|
||||
|
||||
#### High Priority
|
||||
|
||||
#### Medium Priority
|
||||
2. **Developer Experience**
|
||||
- Watch mode for file changes
|
||||
- Custom reporters (JSON, JUnit, HTML, Markdown)
|
||||
- Performance benchmarking API
|
||||
- Better error messages with suggestions
|
||||
|
||||
3. **Enhanced toolsArg**
|
||||
- Test data injection
|
||||
- Context sharing between tests
|
||||
- Parameterized tests
|
||||
|
||||
4. **Test Organization**
|
||||
- Hierarchical test suites
|
||||
- Nested describe blocks
|
||||
- Suite-level lifecycle hooks
|
||||
|
||||
#### Low Priority
|
||||
5. **Analytics and Performance**
|
||||
- Test analytics dashboard
|
||||
- Code coverage integration
|
||||
- Trend analysis
|
||||
- Flaky test detection
|
||||
|
||||
### Recently Fixed Issues ✅
|
||||
- **tap.todo()**: Now fully implemented with test object creation
|
||||
- **tap.skip.test()**: Now creates test objects and maintains accurate test count
|
||||
- **tap.only.test()**: Works correctly - when .only tests exist, only those run
|
||||
|
||||
### Remaining Minor Issues
|
||||
- **Protocol Output**: Some protocol messages still appear in console output
|
||||
|
||||
### Next Recommended Steps
|
||||
1. Add Watch Mode (Phase 4) - high developer value for fast feedback
|
||||
2. Implement Custom Reporters - important for CI/CD integration
|
||||
3. Implement performance benchmarking API
|
||||
4. Add better error messages with suggestions
|
||||
287
readme.protocol.md
Normal file
287
readme.protocol.md
Normal file
@@ -0,0 +1,287 @@
|
||||
# Improved Internal Protocol Design
|
||||
|
||||
## Current Issues with TAP Protocol
|
||||
|
||||
1. **Delimiter Conflict**: Using `#` for metadata conflicts with test descriptions containing `#`
|
||||
2. **Ambiguous Parsing**: No clear boundary between test name and metadata
|
||||
3. **Limited Extensibility**: Adding new metadata requires regex changes
|
||||
4. **Mixed Concerns**: Protocol data mixed with human-readable output
|
||||
|
||||
## Proposed Internal Protocol v2
|
||||
|
||||
### Design Principles
|
||||
|
||||
1. **Clear Separation**: Protocol data must be unambiguously separated from user content
|
||||
2. **Extensibility**: Easy to add new metadata without breaking parsers
|
||||
3. **Backwards Compatible**: Can coexist with standard TAP for gradual migration
|
||||
4. **Machine Readable**: Structured format for reliable parsing
|
||||
5. **Human Friendly**: Still readable in raw form
|
||||
|
||||
### Protocol Options
|
||||
|
||||
#### Option 1: Special Delimiters
|
||||
```
|
||||
ok 1 - test description ::TSTEST:: {"time":123,"retry":0}
|
||||
not ok 2 - another test ::TSTEST:: {"time":45,"error":"timeout"}
|
||||
ok 3 - skipped test ::TSTEST:: {"time":0,"skip":"not ready"}
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
- Simple to implement
|
||||
- Backwards compatible with TAP parsers (they ignore the suffix)
|
||||
- Easy to parse with split()
|
||||
|
||||
**Cons**:
|
||||
- Still could conflict if test name contains `::TSTEST::`
|
||||
- Not standard TAP
|
||||
|
||||
#### Option 2: Separate Metadata Lines
|
||||
```
|
||||
ok 1 - test description
|
||||
::METADATA:: {"test":1,"time":123,"retry":0}
|
||||
not ok 2 - another test
|
||||
::METADATA:: {"test":2,"time":45,"error":"timeout"}
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
- Complete separation of concerns
|
||||
- No chance of conflicts
|
||||
- Can include arbitrary metadata
|
||||
|
||||
**Cons**:
|
||||
- Requires correlation between lines
|
||||
- More complex parsing
|
||||
|
||||
#### Option 3: YAML Blocks (TAP 13 Compatible)
|
||||
```
|
||||
ok 1 - test description
|
||||
---
|
||||
time: 123
|
||||
retry: 0
|
||||
...
|
||||
not ok 2 - another test
|
||||
---
|
||||
time: 45
|
||||
error: timeout
|
||||
stack: |
|
||||
Error: timeout
|
||||
at Test.run (test.js:10:5)
|
||||
...
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
- Standard TAP 13 feature
|
||||
- Structured data format
|
||||
- Human readable
|
||||
- Extensible
|
||||
|
||||
**Cons**:
|
||||
- More verbose
|
||||
- YAML parsing overhead
|
||||
|
||||
#### Option 4: Binary Protocol Markers (Recommended)
|
||||
```
|
||||
ok 1 - test description
|
||||
␛[TSTEST:eyJ0aW1lIjoxMjMsInJldHJ5IjowfQ==]␛
|
||||
not ok 2 - another test
|
||||
␛[TSTEST:eyJ0aW1lIjo0NSwiZXJyb3IiOiJ0aW1lb3V0In0=]␛
|
||||
```
|
||||
|
||||
Using ASCII escape character (␛ = \x1B) with base64 encoded JSON.
|
||||
|
||||
**Pros**:
|
||||
- Zero chance of accidental conflicts
|
||||
- Compact
|
||||
- Fast to parse
|
||||
- Invisible in most terminals
|
||||
|
||||
**Cons**:
|
||||
- Not human readable in raw form
|
||||
- Requires base64 encoding/decoding
|
||||
|
||||
### Recommended Implementation: Hybrid Approach
|
||||
|
||||
Use multiple strategies based on context:
|
||||
|
||||
1. **For timing and basic metadata**: Use structured delimiters
|
||||
```
|
||||
ok 1 - test name ⟦time:123,retry:0⟧
|
||||
```
|
||||
|
||||
2. **For complex data (errors, snapshots)**: Use separate protocol lines
|
||||
```
|
||||
ok 1 - test failed
|
||||
⟦TSTEST:ERROR⟧
|
||||
{"message":"Assertion failed","stack":"...","diff":"..."}
|
||||
⟦/TSTEST:ERROR⟧
|
||||
```
|
||||
|
||||
3. **For human-readable output**: Keep standard TAP comments
|
||||
```
|
||||
# Test suite: User Authentication
|
||||
ok 1 - should login
|
||||
```
|
||||
|
||||
### Implementation Plan
|
||||
|
||||
#### Phase 1: Parser Enhancement
|
||||
1. Add new protocol parser alongside existing TAP parser
|
||||
2. Support both old and new formats during transition
|
||||
3. Add protocol version negotiation
|
||||
|
||||
#### Phase 2: Metadata Structure
|
||||
```typescript
|
||||
interface TestMetadata {
|
||||
// Timing
|
||||
time: number; // milliseconds
|
||||
startTime?: number; // Unix timestamp
|
||||
endTime?: number; // Unix timestamp
|
||||
|
||||
// Status
|
||||
skip?: string; // skip reason
|
||||
todo?: string; // todo reason
|
||||
retry?: number; // retry attempt
|
||||
maxRetries?: number; // max retries allowed
|
||||
|
||||
// Error details
|
||||
error?: {
|
||||
message: string;
|
||||
stack?: string;
|
||||
diff?: string;
|
||||
actual?: any;
|
||||
expected?: any;
|
||||
};
|
||||
|
||||
// Test context
|
||||
file?: string; // source file
|
||||
line?: number; // line number
|
||||
column?: number; // column number
|
||||
|
||||
// Custom data
|
||||
tags?: string[]; // test tags
|
||||
custom?: Record<string, any>;
|
||||
}
|
||||
```
|
||||
|
||||
#### Phase 3: Protocol Messages
|
||||
|
||||
##### Success Message
|
||||
```
|
||||
ok 1 - user authentication works
|
||||
⟦TSTEST:META:{"time":123,"tags":["auth","unit"]}⟧
|
||||
```
|
||||
|
||||
##### Failure Message
|
||||
```
|
||||
not ok 2 - login fails with invalid password
|
||||
⟦TSTEST:META:{"time":45,"retry":1,"maxRetries":3}⟧
|
||||
⟦TSTEST:ERROR⟧
|
||||
{
|
||||
"message": "Expected 401 but got 500",
|
||||
"stack": "Error: Expected 401 but got 500\n at Test.run (auth.test.ts:25:10)",
|
||||
"actual": 500,
|
||||
"expected": 401
|
||||
}
|
||||
⟦/TSTEST:ERROR⟧
|
||||
```
|
||||
|
||||
##### Skip Message
|
||||
```
|
||||
ok 3 - database integration test ⟦TSTEST:SKIP:No database connection⟧
|
||||
```
|
||||
|
||||
##### Snapshot Communication
|
||||
```
|
||||
⟦TSTEST:SNAPSHOT:user-profile⟧
|
||||
{
|
||||
"name": "John Doe",
|
||||
"email": "john@example.com",
|
||||
"roles": ["user", "admin"]
|
||||
}
|
||||
⟦/TSTEST:SNAPSHOT⟧
|
||||
```
|
||||
|
||||
### Migration Strategy
|
||||
|
||||
1. **Version Detection**: First line indicates protocol version
|
||||
```
|
||||
⟦TSTEST:PROTOCOL:2.0⟧
|
||||
TAP version 13
|
||||
```
|
||||
|
||||
2. **Gradual Rollout**:
|
||||
- v1.10: Add protocol v2 parser, keep v1 generator
|
||||
- v1.11: Generate v2 by default, v1 with --legacy flag
|
||||
- v2.0: Remove v1 support
|
||||
|
||||
3. **Feature Flags**:
|
||||
```typescript
|
||||
tap.settings({
|
||||
protocol: 'v2', // or 'v1', 'auto'
|
||||
protocolFeatures: {
|
||||
structuredErrors: true,
|
||||
enhancedTiming: true,
|
||||
binaryMarkers: false
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### Benefits of New Protocol
|
||||
|
||||
1. **Reliability**: No more regex fragility or description conflicts
|
||||
2. **Performance**: Faster parsing with clear boundaries
|
||||
3. **Extensibility**: Easy to add new metadata fields
|
||||
4. **Debugging**: Rich error information with stack traces and diffs
|
||||
5. **Integration**: Better IDE and CI/CD tool integration
|
||||
6. **Forward Compatible**: Room for future enhancements
|
||||
|
||||
### Example Parser Implementation
|
||||
|
||||
```typescript
|
||||
class ProtocolV2Parser {
|
||||
private readonly MARKER_START = '⟦TSTEST:';
|
||||
private readonly MARKER_END = '⟧';
|
||||
|
||||
parseMetadata(line: string): TestMetadata | null {
|
||||
const start = line.lastIndexOf(this.MARKER_START);
|
||||
if (start === -1) return null;
|
||||
|
||||
const end = line.indexOf(this.MARKER_END, start);
|
||||
if (end === -1) return null;
|
||||
|
||||
const content = line.substring(start + this.MARKER_START.length, end);
|
||||
const [type, data] = content.split(':', 2);
|
||||
|
||||
switch (type) {
|
||||
case 'META':
|
||||
return JSON.parse(data);
|
||||
case 'SKIP':
|
||||
return { skip: data };
|
||||
case 'TODO':
|
||||
return { todo: data };
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
parseTestLine(line: string): ParsedTest {
|
||||
// First extract any metadata
|
||||
const metadata = this.parseMetadata(line);
|
||||
|
||||
// Then parse the TAP part (without metadata)
|
||||
const cleanLine = this.removeMetadata(line);
|
||||
const tapResult = this.parseTAP(cleanLine);
|
||||
|
||||
return { ...tapResult, metadata };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Next Steps
|
||||
|
||||
1. Implement proof of concept with basic metadata support
|
||||
2. Test with real-world test suites for edge cases
|
||||
3. Benchmark parsing performance
|
||||
4. Get feedback from users
|
||||
5. Finalize protocol specification
|
||||
6. Implement in both tapbundle and tstest
|
||||
41
test/config-test/00init.ts
Normal file
41
test/config-test/00init.ts
Normal file
@@ -0,0 +1,41 @@
|
||||
import { tap } from '../../ts_tapbundle/index.js';
|
||||
|
||||
// TAP-compliant comment output
|
||||
console.log('# 🚀 00init.ts: LOADED AND EXECUTING');
|
||||
console.log('# 🚀 00init.ts: Setting up global test configuration');
|
||||
|
||||
// Add a global variable to verify 00init.ts was loaded
|
||||
(global as any).__00INIT_LOADED = true;
|
||||
|
||||
// Configure global test settings
|
||||
tap.settings({
|
||||
// Set a default timeout of 5 seconds for all tests
|
||||
timeout: 5000,
|
||||
|
||||
// Enable retries for flaky tests
|
||||
retries: 2,
|
||||
retryDelay: 1000,
|
||||
|
||||
// Show test duration
|
||||
showTestDuration: true,
|
||||
|
||||
// Global lifecycle hooks
|
||||
beforeAll: async () => {
|
||||
console.log('Global beforeAll: Initializing test environment');
|
||||
},
|
||||
|
||||
afterAll: async () => {
|
||||
console.log('Global afterAll: Cleaning up test environment');
|
||||
},
|
||||
|
||||
beforeEach: async (testName: string) => {
|
||||
console.log(`Global beforeEach: Starting test "${testName}"`);
|
||||
},
|
||||
|
||||
afterEach: async (testName: string, passed: boolean) => {
|
||||
console.log(`Global afterEach: Test "${testName}" ${passed ? 'passed' : 'failed'}`);
|
||||
}
|
||||
});
|
||||
|
||||
console.log('# 🚀 00init.ts: Configuration COMPLETE');
|
||||
console.log('# 🚀 00init.ts: tap.settings() called successfully');
|
||||
44
test/config-test/test.config.ts
Normal file
44
test/config-test/test.config.ts
Normal file
@@ -0,0 +1,44 @@
|
||||
import { tap, expect } from '../../ts_tapbundle/index.js';
|
||||
|
||||
// TAP-compliant comment output
|
||||
console.log('# 🔍 TEST FILE LOADED - test.config.ts');
|
||||
|
||||
// Check if 00init.ts was loaded
|
||||
const initLoaded = (global as any).__00INIT_LOADED;
|
||||
console.log(`# 🔍 00init.ts loaded: ${initLoaded === true}`);
|
||||
|
||||
// Test that uses the global timeout setting
|
||||
tap.test('Test with global timeout', async (toolsArg) => {
|
||||
// This test should complete within the 5 second timeout set in 00init.ts
|
||||
await toolsArg.delayFor(2000); // 2 seconds
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Test that demonstrates retries
|
||||
tap.test('Test with retries', async () => {
|
||||
// This test will use the global retry setting (2 retries)
|
||||
console.log('Running test that might be flaky');
|
||||
|
||||
// Simulate a flaky test that passes on second try
|
||||
const randomValue = Math.random();
|
||||
console.log(`Random value: ${randomValue}`);
|
||||
|
||||
// Always pass for demonstration
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Test with custom timeout that overrides global
|
||||
tap.timeout(1000).test('Test with custom timeout', async (toolsArg) => {
|
||||
// This test has a 1 second timeout, overriding the global 5 seconds
|
||||
await toolsArg.delayFor(500); // 500ms - should pass
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Test to verify lifecycle hooks are working
|
||||
tap.test('Test lifecycle hooks', async () => {
|
||||
console.log('Inside test: lifecycle hooks should have run');
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Start the test suite
|
||||
tap.start();
|
||||
22
test/config-test/test.file-settings.ts
Normal file
22
test/config-test/test.file-settings.ts
Normal file
@@ -0,0 +1,22 @@
|
||||
import { tap, expect } from '../../ts_tapbundle/index.js';
|
||||
|
||||
// Override global settings for this file
|
||||
tap.settings({
|
||||
timeout: 2000, // Override global timeout to 2 seconds
|
||||
retries: 0, // Disable retries for this file
|
||||
});
|
||||
|
||||
tap.test('Test with file-level timeout', async (toolsArg) => {
|
||||
// This should use the file-level timeout of 2 seconds
|
||||
console.log('Running with file-level timeout of 2 seconds');
|
||||
await toolsArg.delayFor(1000); // 1 second - should pass
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('Test without retries', async () => {
|
||||
// This test should not retry even if it fails
|
||||
console.log('This test has no retries (file-level setting)');
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.start();
|
||||
91
test/decorator.all.ts
Normal file
91
test/decorator.all.ts
Normal file
@@ -0,0 +1,91 @@
|
||||
import { tap, expect } from '../ts_tapbundle/index.js';
|
||||
|
||||
/**
|
||||
* Simple class decorator for testing decorator support across runtimes
|
||||
*/
|
||||
function testDecorator(target: any) {
|
||||
target.decoratorApplied = true;
|
||||
target.decoratorData = 'Decorator was applied successfully';
|
||||
return target;
|
||||
}
|
||||
|
||||
/**
|
||||
* Method decorator for testing
|
||||
*/
|
||||
function logMethod(target: any, propertyKey: string, descriptor: PropertyDescriptor) {
|
||||
const originalMethod = descriptor.value;
|
||||
descriptor.value = function (...args: any[]) {
|
||||
const result = originalMethod.apply(this, args);
|
||||
return `[logged] ${result}`;
|
||||
};
|
||||
return descriptor;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parameter decorator for testing
|
||||
*/
|
||||
function validateParam(target: any, propertyKey: string, parameterIndex: number) {
|
||||
// Mark that parameter validation decorator was applied
|
||||
if (!target.decoratedParams) {
|
||||
target.decoratedParams = {};
|
||||
}
|
||||
if (!target.decoratedParams[propertyKey]) {
|
||||
target.decoratedParams[propertyKey] = [];
|
||||
}
|
||||
target.decoratedParams[propertyKey].push(parameterIndex);
|
||||
}
|
||||
|
||||
/**
|
||||
* Test class with decorators
|
||||
*/
|
||||
@testDecorator
|
||||
class TestClass {
|
||||
public name: string = 'test';
|
||||
|
||||
@logMethod
|
||||
public greet(message: string): string {
|
||||
return `Hello, ${message}!`;
|
||||
}
|
||||
|
||||
public getValue(): number {
|
||||
return 42;
|
||||
}
|
||||
|
||||
public testParams(@validateParam value: string): string {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
||||
// Tests
|
||||
tap.test('Class decorator should be applied', async () => {
|
||||
expect((TestClass as any).decoratorApplied).toEqual(true);
|
||||
expect((TestClass as any).decoratorData).toEqual('Decorator was applied successfully');
|
||||
});
|
||||
|
||||
tap.test('Method decorator should modify method behavior', async () => {
|
||||
const instance = new TestClass();
|
||||
const result = instance.greet('World');
|
||||
expect(result).toEqual('[logged] Hello, World!');
|
||||
});
|
||||
|
||||
tap.test('Regular methods should work normally', async () => {
|
||||
const instance = new TestClass();
|
||||
expect(instance.getValue()).toEqual(42);
|
||||
expect(instance.name).toEqual('test');
|
||||
});
|
||||
|
||||
tap.test('Parameter decorator should be applied', async () => {
|
||||
const decoratedParams = (TestClass.prototype as any).decoratedParams;
|
||||
expect(decoratedParams).toBeDefined();
|
||||
expect(decoratedParams.testParams).toBeDefined();
|
||||
expect(decoratedParams.testParams).toContain(0);
|
||||
});
|
||||
|
||||
tap.test('Decorator metadata preservation', async () => {
|
||||
const instance = new TestClass();
|
||||
expect(instance instanceof TestClass).toEqual(true);
|
||||
expect(instance.constructor.name).toEqual('TestClass');
|
||||
expect(instance.testParams('hello')).toEqual('hello');
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
8
test/glob-test/another.spec.ts
Normal file
8
test/glob-test/another.spec.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
import { tap } from '../../ts_tapbundle/index.js';
|
||||
|
||||
tap.test('spec file test', async () => {
|
||||
console.log('This is a .spec.ts file that should be found by glob');
|
||||
return true;
|
||||
});
|
||||
|
||||
tap.start();
|
||||
8
test/glob-test/nested/test.nested-glob.ts
Normal file
8
test/glob-test/nested/test.nested-glob.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
import { tap } from '../../../ts_tapbundle/index.js';
|
||||
|
||||
tap.test('nested glob pattern test', async () => {
|
||||
console.log('This test file is in a nested directory');
|
||||
return true;
|
||||
});
|
||||
|
||||
tap.start();
|
||||
8
test/glob-test/test.glob-test.ts
Normal file
8
test/glob-test/test.glob-test.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
import { tap } from '../../ts_tapbundle/index.js';
|
||||
|
||||
tap.test('glob pattern test', async () => {
|
||||
console.log('This test file should be found by glob patterns');
|
||||
return true;
|
||||
});
|
||||
|
||||
tap.start();
|
||||
@@ -1,6 +1,6 @@
|
||||
import { tap, expect } from '../../ts_tapbundle/index.js';
|
||||
|
||||
import { tapNodeTools } from '../../ts_tapbundle_node/index.js';
|
||||
import { tapNodeTools } from '../../ts_tapbundle_serverside/index.js';
|
||||
|
||||
tap.test('should execure a command', async () => {
|
||||
const result = await tapNodeTools.runCommand('ls -la');
|
||||
|
||||
167
test/tapbundle/test.performance-metrics.ts
Normal file
167
test/tapbundle/test.performance-metrics.ts
Normal file
@@ -0,0 +1,167 @@
|
||||
import { tap, expect } from '../../ts_tapbundle/index.js';
|
||||
|
||||
// Create tests with known, distinct timing patterns to verify metrics calculation
|
||||
tap.test('metric test 1 - 10ms baseline', async (tools) => {
|
||||
await tools.delayFor(10);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('metric test 2 - 20ms double baseline', async (tools) => {
|
||||
await tools.delayFor(20);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('metric test 3 - 30ms triple baseline', async (tools) => {
|
||||
await tools.delayFor(30);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('metric test 4 - 40ms quadruple baseline', async (tools) => {
|
||||
await tools.delayFor(40);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('metric test 5 - 50ms quintuple baseline', async (tools) => {
|
||||
await tools.delayFor(50);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Test that should be the slowest
|
||||
tap.test('metric test slowest - 200ms intentionally slow', async (tools) => {
|
||||
await tools.delayFor(200);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Tests to verify edge cases in average calculation
|
||||
tap.test('metric test fast 1 - minimal work', async () => {
|
||||
expect(1).toEqual(1);
|
||||
});
|
||||
|
||||
tap.test('metric test fast 2 - minimal work', async () => {
|
||||
expect(2).toEqual(2);
|
||||
});
|
||||
|
||||
tap.test('metric test fast 3 - minimal work', async () => {
|
||||
expect(3).toEqual(3);
|
||||
});
|
||||
|
||||
// Test to verify that failed tests still contribute to timing metrics
|
||||
tap.test('metric test that fails - 60ms before failure', async (tools) => {
|
||||
await tools.delayFor(60);
|
||||
expect(true).toBeFalse(); // This will fail
|
||||
});
|
||||
|
||||
// Describe block with timing to test aggregation
|
||||
tap.describe('performance metrics in describe block', () => {
|
||||
tap.test('described test 1 - 15ms', async (tools) => {
|
||||
await tools.delayFor(15);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('described test 2 - 25ms', async (tools) => {
|
||||
await tools.delayFor(25);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('described test 3 - 35ms', async (tools) => {
|
||||
await tools.delayFor(35);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
});
|
||||
|
||||
// Test timing with hooks
|
||||
tap.describe('performance with hooks', () => {
|
||||
let hookTime = 0;
|
||||
|
||||
tap.beforeEach(async () => {
|
||||
// Hooks shouldn't count toward test time
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
hookTime += 10;
|
||||
});
|
||||
|
||||
tap.afterEach(async () => {
|
||||
// Hooks shouldn't count toward test time
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
hookTime += 10;
|
||||
});
|
||||
|
||||
tap.test('test with hooks 1 - should only count test time', async (tools) => {
|
||||
await tools.delayFor(30);
|
||||
expect(true).toBeTrue();
|
||||
// Test time should be ~30ms, not 50ms (including hooks)
|
||||
});
|
||||
|
||||
tap.test('test with hooks 2 - should only count test time', async (tools) => {
|
||||
await tools.delayFor(40);
|
||||
expect(true).toBeTrue();
|
||||
// Test time should be ~40ms, not 60ms (including hooks)
|
||||
});
|
||||
});
|
||||
|
||||
// Parallel tests to verify timing is captured correctly
|
||||
tap.describe('parallel timing verification', () => {
|
||||
const startTimes: Map<string, number> = new Map();
|
||||
const endTimes: Map<string, number> = new Map();
|
||||
|
||||
tap.testParallel('parallel metric 1 - 80ms', async (tools) => {
|
||||
startTimes.set('p1', Date.now());
|
||||
await tools.delayFor(80);
|
||||
endTimes.set('p1', Date.now());
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.testParallel('parallel metric 2 - 90ms', async (tools) => {
|
||||
startTimes.set('p2', Date.now());
|
||||
await tools.delayFor(90);
|
||||
endTimes.set('p2', Date.now());
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.testParallel('parallel metric 3 - 100ms', async (tools) => {
|
||||
startTimes.set('p3', Date.now());
|
||||
await tools.delayFor(100);
|
||||
endTimes.set('p3', Date.now());
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('verify parallel execution', async () => {
|
||||
// This test runs after parallel tests
|
||||
// Verify they actually ran in parallel by checking overlapping times
|
||||
if (startTimes.size === 3 && endTimes.size === 3) {
|
||||
const p1Start = startTimes.get('p1')!;
|
||||
const p2Start = startTimes.get('p2')!;
|
||||
const p3Start = startTimes.get('p3')!;
|
||||
const p1End = endTimes.get('p1')!;
|
||||
const p2End = endTimes.get('p2')!;
|
||||
const p3End = endTimes.get('p3')!;
|
||||
|
||||
// Start times should be very close (within 50ms)
|
||||
expect(Math.abs(p1Start - p2Start)).toBeLessThan(50);
|
||||
expect(Math.abs(p2Start - p3Start)).toBeLessThan(50);
|
||||
|
||||
// There should be overlap in execution
|
||||
const p1Overlaps = p1Start < p2End && p1End > p2Start;
|
||||
const p2Overlaps = p2Start < p3End && p2End > p3Start;
|
||||
|
||||
expect(p1Overlaps || p2Overlaps).toBeTrue();
|
||||
} else {
|
||||
// Skip verification if parallel tests didn't run yet
|
||||
expect(true).toBeTrue();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// Test to ensure average calculation handles mixed timing correctly
|
||||
tap.test('final metrics test - 5ms minimal', async (tools) => {
|
||||
await tools.delayFor(5);
|
||||
expect(true).toBeTrue();
|
||||
|
||||
console.log('\n📊 Expected Performance Metrics Summary:');
|
||||
console.log('- Tests include a mix of durations from <1ms to 200ms');
|
||||
console.log('- Slowest test should be "metric test slowest" at ~200ms');
|
||||
console.log('- Average should be calculated from individual test times');
|
||||
console.log('- Failed test should still contribute its 60ms to timing');
|
||||
console.log('- Parallel tests should show their individual times (80ms, 90ms, 100ms)');
|
||||
});
|
||||
|
||||
tap.start();
|
||||
214
test/tapbundle/test.timing-edge-cases.ts
Normal file
214
test/tapbundle/test.timing-edge-cases.ts
Normal file
@@ -0,0 +1,214 @@
|
||||
import { tap, expect } from '../../ts_tapbundle/index.js';
|
||||
|
||||
tap.test('ultra-fast test - should capture sub-millisecond timing', async () => {
|
||||
// This test does almost nothing, should complete in < 1ms
|
||||
const x = 1 + 1;
|
||||
expect(x).toEqual(2);
|
||||
});
|
||||
|
||||
tap.test('test with exact 1ms delay', async (tools) => {
|
||||
const start = Date.now();
|
||||
await tools.delayFor(1);
|
||||
const elapsed = Date.now() - start;
|
||||
// Should be at least 1ms but could be more due to event loop
|
||||
expect(elapsed).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
tap.test('test with 10ms delay', async (tools) => {
|
||||
await tools.delayFor(10);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('test with 100ms delay', async (tools) => {
|
||||
await tools.delayFor(100);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('test with 250ms delay', async (tools) => {
|
||||
await tools.delayFor(250);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('test with 500ms delay', async (tools) => {
|
||||
await tools.delayFor(500);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('test with variable processing time', async (tools) => {
|
||||
// Simulate variable processing
|
||||
const iterations = 1000000;
|
||||
let sum = 0;
|
||||
for (let i = 0; i < iterations; i++) {
|
||||
sum += Math.sqrt(i);
|
||||
}
|
||||
expect(sum).toBeGreaterThan(0);
|
||||
|
||||
// Add a small delay to ensure measurable time
|
||||
await tools.delayFor(5);
|
||||
});
|
||||
|
||||
tap.test('test with multiple async operations', async () => {
|
||||
// Multiple promises in parallel
|
||||
const results = await Promise.all([
|
||||
new Promise(resolve => setTimeout(() => resolve(1), 10)),
|
||||
new Promise(resolve => setTimeout(() => resolve(2), 20)),
|
||||
new Promise(resolve => setTimeout(() => resolve(3), 30))
|
||||
]);
|
||||
|
||||
expect(results).toEqual([1, 2, 3]);
|
||||
// This should take at least 30ms (the longest delay)
|
||||
});
|
||||
|
||||
tap.test('test with synchronous heavy computation', async () => {
|
||||
// Heavy synchronous computation
|
||||
const fibonacci = (n: number): number => {
|
||||
if (n <= 1) return n;
|
||||
return fibonacci(n - 1) + fibonacci(n - 2);
|
||||
};
|
||||
|
||||
// Calculate fibonacci(30) - should take measurable time
|
||||
const result = fibonacci(30);
|
||||
expect(result).toEqual(832040);
|
||||
});
|
||||
|
||||
// Test with retry to see if timing accumulates correctly
|
||||
tap.retry(2).test('test with retry - fails first then passes', async (tools) => {
|
||||
// Get or initialize retry count
|
||||
const retryCount = tools.context.get('retryCount') || 0;
|
||||
tools.context.set('retryCount', retryCount + 1);
|
||||
|
||||
await tools.delayFor(50);
|
||||
|
||||
if (retryCount === 0) {
|
||||
throw new Error('First attempt fails');
|
||||
}
|
||||
|
||||
expect(retryCount).toEqual(1);
|
||||
});
|
||||
|
||||
// Test timeout handling
|
||||
tap.timeout(100).test('test with timeout - should complete just in time', async (tools) => {
|
||||
await tools.delayFor(80); // Just under the timeout
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Skip test - should show 0ms
|
||||
tap.skip.test('skipped test - should report 0ms', async (tools) => {
|
||||
await tools.delayFor(1000); // This won't execute
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Todo test - should show 0ms
|
||||
tap.todo.test('todo test - should report 0ms', async (tools) => {
|
||||
await tools.delayFor(1000); // This won't execute
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Test with skip inside
|
||||
tap.test('test that skips conditionally - should show time until skip', async (tools) => {
|
||||
await tools.delayFor(25);
|
||||
|
||||
const shouldSkip = true;
|
||||
if (shouldSkip) {
|
||||
tools.skip('Skipping after 25ms');
|
||||
}
|
||||
|
||||
// This won't execute
|
||||
await tools.delayFor(1000);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Test with very precise timing
|
||||
tap.test('test with precise timing measurements', async (tools) => {
|
||||
const measurements: number[] = [];
|
||||
|
||||
for (let i = 0; i < 5; i++) {
|
||||
const start = process.hrtime.bigint();
|
||||
await tools.delayFor(10);
|
||||
const end = process.hrtime.bigint();
|
||||
const durationMs = Number(end - start) / 1_000_000;
|
||||
measurements.push(durationMs);
|
||||
}
|
||||
|
||||
// All measurements should be at least 10ms
|
||||
measurements.forEach(m => {
|
||||
expect(m).toBeGreaterThanOrEqual(10);
|
||||
});
|
||||
|
||||
// But not too much more (accounting for timer precision)
|
||||
measurements.forEach(m => {
|
||||
expect(m).toBeLessThan(20);
|
||||
});
|
||||
});
|
||||
|
||||
// Test that intentionally has 0 actual work
|
||||
tap.test('empty test - absolute minimum execution time', async () => {
|
||||
// Literally nothing
|
||||
});
|
||||
|
||||
// Test with promise that resolves immediately
|
||||
tap.test('test with immediate promise resolution', async () => {
|
||||
await Promise.resolve();
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
// Test with microtask queue
|
||||
tap.test('test with microtask queue processing', async () => {
|
||||
let value = 0;
|
||||
|
||||
await Promise.resolve().then(() => {
|
||||
value = 1;
|
||||
return Promise.resolve();
|
||||
}).then(() => {
|
||||
value = 2;
|
||||
return Promise.resolve();
|
||||
}).then(() => {
|
||||
value = 3;
|
||||
});
|
||||
|
||||
expect(value).toEqual(3);
|
||||
});
|
||||
|
||||
// Test to verify timing accumulation in describe blocks
|
||||
tap.describe('timing in describe blocks', () => {
|
||||
let startTime: number;
|
||||
|
||||
tap.beforeEach(async () => {
|
||||
startTime = Date.now();
|
||||
await new Promise(resolve => setTimeout(resolve, 5));
|
||||
});
|
||||
|
||||
tap.afterEach(async () => {
|
||||
await new Promise(resolve => setTimeout(resolve, 5));
|
||||
});
|
||||
|
||||
tap.test('first test in describe', async (tools) => {
|
||||
await tools.delayFor(10);
|
||||
const elapsed = Date.now() - startTime;
|
||||
expect(elapsed).toBeGreaterThanOrEqual(10);
|
||||
});
|
||||
|
||||
tap.test('second test in describe', async (tools) => {
|
||||
await tools.delayFor(20);
|
||||
const elapsed = Date.now() - startTime;
|
||||
expect(elapsed).toBeGreaterThanOrEqual(20);
|
||||
});
|
||||
});
|
||||
|
||||
// Parallel tests to see timing differences
|
||||
tap.testParallel('parallel test 1 - 100ms', async (tools) => {
|
||||
await tools.delayFor(100);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.testParallel('parallel test 2 - 50ms', async (tools) => {
|
||||
await tools.delayFor(50);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.testParallel('parallel test 3 - 150ms', async (tools) => {
|
||||
await tools.delayFor(150);
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.start();
|
||||
204
test/tapbundle/test.timing-protocol.ts
Normal file
204
test/tapbundle/test.timing-protocol.ts
Normal file
@@ -0,0 +1,204 @@
|
||||
import { tap, expect } from '../../ts_tapbundle/index.js';
|
||||
import { ProtocolParser, ProtocolEmitter } from '../../ts_tapbundle_protocol/index.js';
|
||||
|
||||
// Test the protocol's ability to emit and parse timing metadata
|
||||
tap.test('protocol should correctly emit timing metadata', async () => {
|
||||
const emitter = new ProtocolEmitter();
|
||||
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'test with timing',
|
||||
metadata: {
|
||||
time: 123
|
||||
}
|
||||
};
|
||||
|
||||
const lines = emitter.emitTest(testResult);
|
||||
|
||||
// Should have inline timing metadata
|
||||
expect(lines.length).toEqual(1);
|
||||
expect(lines[0]).toInclude('⟦TSTEST:time:123⟧');
|
||||
});
|
||||
|
||||
tap.test('protocol should correctly parse timing metadata', async () => {
|
||||
const parser = new ProtocolParser();
|
||||
|
||||
const line = 'ok 1 - test with timing ⟦TSTEST:time:456⟧';
|
||||
const messages = parser.parseLine(line);
|
||||
|
||||
expect(messages.length).toEqual(1);
|
||||
expect(messages[0].type).toEqual('test');
|
||||
|
||||
const content = messages[0].content as any;
|
||||
expect(content.metadata).toBeDefined();
|
||||
expect(content.metadata.time).toEqual(456);
|
||||
});
|
||||
|
||||
tap.test('protocol should handle 0ms timing', async () => {
|
||||
const parser = new ProtocolParser();
|
||||
|
||||
const line = 'ok 1 - ultra fast test ⟦TSTEST:time:0⟧';
|
||||
const messages = parser.parseLine(line);
|
||||
|
||||
const content = messages[0].content as any;
|
||||
expect(content.metadata.time).toEqual(0);
|
||||
});
|
||||
|
||||
tap.test('protocol should handle large timing values', async () => {
|
||||
const parser = new ProtocolParser();
|
||||
|
||||
const line = 'ok 1 - slow test ⟦TSTEST:time:999999⟧';
|
||||
const messages = parser.parseLine(line);
|
||||
|
||||
const content = messages[0].content as any;
|
||||
expect(content.metadata.time).toEqual(999999);
|
||||
});
|
||||
|
||||
tap.test('protocol should handle timing with other metadata', async () => {
|
||||
const emitter = new ProtocolEmitter();
|
||||
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'complex test',
|
||||
metadata: {
|
||||
time: 789,
|
||||
file: 'test.ts',
|
||||
tags: ['slow', 'integration']
|
||||
}
|
||||
};
|
||||
|
||||
const lines = emitter.emitTest(testResult);
|
||||
|
||||
// Should use block metadata format for complex metadata
|
||||
expect(lines.length).toBeGreaterThan(1);
|
||||
expect(lines[1]).toInclude('META:');
|
||||
expect(lines[1]).toInclude('"time":789');
|
||||
});
|
||||
|
||||
tap.test('protocol should parse timing from block metadata', async () => {
|
||||
const parser = new ProtocolParser();
|
||||
|
||||
const lines = [
|
||||
'ok 1 - complex test',
|
||||
'⟦TSTEST:META:{"time":321,"file":"test.ts"}⟧'
|
||||
];
|
||||
|
||||
let testResult: any;
|
||||
|
||||
for (const line of lines) {
|
||||
const messages = parser.parseLine(line);
|
||||
if (messages.length > 0 && messages[0].type === 'test') {
|
||||
testResult = messages[0].content;
|
||||
}
|
||||
}
|
||||
|
||||
expect(testResult).toBeDefined();
|
||||
expect(testResult.metadata).toBeUndefined(); // Metadata comes separately in block format
|
||||
});
|
||||
|
||||
tap.test('timing for skipped tests should be 0 or missing', async () => {
|
||||
const emitter = new ProtocolEmitter();
|
||||
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'skipped test',
|
||||
directive: {
|
||||
type: 'skip' as const,
|
||||
reason: 'Not ready'
|
||||
},
|
||||
metadata: {
|
||||
time: 0
|
||||
}
|
||||
};
|
||||
|
||||
const lines = emitter.emitTest(testResult);
|
||||
expect(lines[0]).toInclude('# SKIP');
|
||||
|
||||
// If time is 0, it might be included or omitted
|
||||
if (lines[0].includes('⟦TSTEST:')) {
|
||||
expect(lines[0]).toInclude('time:0');
|
||||
}
|
||||
});
|
||||
|
||||
tap.test('protocol should handle fractional milliseconds', async () => {
|
||||
const emitter = new ProtocolEmitter();
|
||||
|
||||
// Even though we use integers, test that protocol handles them correctly
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'precise test',
|
||||
metadata: {
|
||||
time: 123 // Protocol uses integers for milliseconds
|
||||
}
|
||||
};
|
||||
|
||||
const lines = emitter.emitTest(testResult);
|
||||
expect(lines[0]).toInclude('time:123');
|
||||
});
|
||||
|
||||
tap.test('protocol should handle timing in retry scenarios', async () => {
|
||||
const emitter = new ProtocolEmitter();
|
||||
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'retry test',
|
||||
metadata: {
|
||||
time: 200,
|
||||
retry: 2
|
||||
}
|
||||
};
|
||||
|
||||
const lines = emitter.emitTest(testResult);
|
||||
// Should include both time and retry
|
||||
expect(lines[0]).toMatch(/time:200.*retry:2|retry:2.*time:200/);
|
||||
});
|
||||
|
||||
// Test actual timing capture
|
||||
tap.test('HrtMeasurement should capture accurate timing', async (tools) => {
|
||||
// Import HrtMeasurement
|
||||
const { HrtMeasurement } = await import('@push.rocks/smarttime');
|
||||
|
||||
const measurement = new HrtMeasurement();
|
||||
measurement.start();
|
||||
|
||||
await tools.delayFor(50);
|
||||
|
||||
measurement.stop();
|
||||
|
||||
// Should be at least 50ms
|
||||
expect(measurement.milliSeconds).toBeGreaterThanOrEqual(50);
|
||||
// But not too much more (allow for some overhead)
|
||||
expect(measurement.milliSeconds).toBeLessThan(100);
|
||||
});
|
||||
|
||||
tap.test('multiple timing measurements should be independent', async (tools) => {
|
||||
const { HrtMeasurement } = await import('@push.rocks/smarttime');
|
||||
|
||||
const measurement1 = new HrtMeasurement();
|
||||
const measurement2 = new HrtMeasurement();
|
||||
|
||||
measurement1.start();
|
||||
await tools.delayFor(25);
|
||||
|
||||
measurement2.start();
|
||||
await tools.delayFor(25);
|
||||
|
||||
measurement1.stop();
|
||||
await tools.delayFor(25);
|
||||
measurement2.stop();
|
||||
|
||||
// measurement1 should be ~50ms (25ms + 25ms)
|
||||
expect(measurement1.milliSeconds).toBeGreaterThanOrEqual(50);
|
||||
expect(measurement1.milliSeconds).toBeLessThan(70);
|
||||
|
||||
// measurement2 should be ~50ms (25ms + 25ms)
|
||||
expect(measurement2.milliSeconds).toBeGreaterThanOrEqual(50);
|
||||
expect(measurement2.milliSeconds).toBeLessThan(70);
|
||||
});
|
||||
|
||||
tap.start();
|
||||
9
test/test.example.latest.docker.sh
Executable file
9
test/test.example.latest.docker.sh
Executable file
@@ -0,0 +1,9 @@
|
||||
#!/bin/bash
|
||||
# Sample Docker test file
|
||||
# This file demonstrates the naming pattern: test.{baseName}.{variant}.docker.sh
|
||||
# The variant "latest" maps to the Dockerfile in the project root
|
||||
|
||||
echo "TAP version 13"
|
||||
echo "1..2"
|
||||
echo "ok 1 - Sample Docker test passes"
|
||||
echo "ok 2 - Docker environment is working"
|
||||
111
test/test.migration.node.ts
Normal file
111
test/test.migration.node.ts
Normal file
@@ -0,0 +1,111 @@
|
||||
import { expect, tap } from '../ts_tapbundle/index.js';
|
||||
import { Migration } from '../ts/tstest.classes.migration.js';
|
||||
import * as plugins from '../ts/tstest.plugins.js';
|
||||
import * as paths from '../ts/tstest.paths.js';
|
||||
|
||||
tap.test('Migration - can initialize', async () => {
|
||||
const migration = new Migration({
|
||||
baseDir: process.cwd(),
|
||||
dryRun: true,
|
||||
});
|
||||
|
||||
expect(migration).toBeInstanceOf(Migration);
|
||||
});
|
||||
|
||||
tap.test('Migration - findLegacyFiles returns empty for no legacy files', async () => {
|
||||
const migration = new Migration({
|
||||
baseDir: process.cwd(),
|
||||
pattern: 'test/test.migration.node.ts', // This file itself, not legacy
|
||||
dryRun: true,
|
||||
});
|
||||
|
||||
const legacyFiles = await migration.findLegacyFiles();
|
||||
expect(legacyFiles).toEqual([]);
|
||||
});
|
||||
|
||||
tap.test('Migration - generateReport works', async () => {
|
||||
const migration = new Migration({
|
||||
baseDir: process.cwd(),
|
||||
dryRun: true,
|
||||
});
|
||||
|
||||
const report = await migration.generateReport();
|
||||
expect(report).toBeTypeOf('string');
|
||||
expect(report).toContain('Test File Migration Report');
|
||||
});
|
||||
|
||||
tap.test('Migration - detects legacy files when they exist', async () => {
|
||||
// Create a temporary legacy test file
|
||||
const tempDir = plugins.path.join(process.cwd(), '.nogit', 'test_migration');
|
||||
await plugins.smartfile.fs.ensureEmptyDir(tempDir);
|
||||
|
||||
const legacyFile = plugins.path.join(tempDir, 'test.browser.ts');
|
||||
await plugins.smartfile.memory.toFs('// Legacy test file\nexport default Promise.resolve();', legacyFile);
|
||||
|
||||
const migration = new Migration({
|
||||
baseDir: tempDir,
|
||||
pattern: '**/*.ts',
|
||||
dryRun: true,
|
||||
});
|
||||
|
||||
const legacyFiles = await migration.findLegacyFiles();
|
||||
expect(legacyFiles.length).toEqual(1);
|
||||
expect(legacyFiles[0]).toContain('test.browser.ts');
|
||||
|
||||
// Clean up
|
||||
await plugins.smartfile.fs.removeSync(tempDir);
|
||||
});
|
||||
|
||||
tap.test('Migration - detects both legacy pattern', async () => {
|
||||
// Create temporary legacy files
|
||||
const tempDir = plugins.path.join(process.cwd(), '.nogit', 'test_migration_both');
|
||||
await plugins.smartfile.fs.ensureEmptyDir(tempDir);
|
||||
|
||||
const browserFile = plugins.path.join(tempDir, 'test.browser.ts');
|
||||
const bothFile = plugins.path.join(tempDir, 'test.both.ts');
|
||||
await plugins.smartfile.memory.toFs('// Browser test\nexport default Promise.resolve();', browserFile);
|
||||
await plugins.smartfile.memory.toFs('// Both test\nexport default Promise.resolve();', bothFile);
|
||||
|
||||
const migration = new Migration({
|
||||
baseDir: tempDir,
|
||||
pattern: '**/*.ts',
|
||||
dryRun: true,
|
||||
});
|
||||
|
||||
const legacyFiles = await migration.findLegacyFiles();
|
||||
expect(legacyFiles.length).toEqual(2);
|
||||
|
||||
// Clean up
|
||||
await plugins.smartfile.fs.removeSync(tempDir);
|
||||
});
|
||||
|
||||
tap.test('Migration - dry run does not modify files', async () => {
|
||||
// Create a temporary legacy test file
|
||||
const tempDir = plugins.path.join(process.cwd(), '.nogit', 'test_migration_dryrun');
|
||||
await plugins.smartfile.fs.ensureEmptyDir(tempDir);
|
||||
|
||||
const legacyFile = plugins.path.join(tempDir, 'test.browser.ts');
|
||||
await plugins.smartfile.memory.toFs('// Legacy test file\nexport default Promise.resolve();', legacyFile);
|
||||
|
||||
const migration = new Migration({
|
||||
baseDir: tempDir,
|
||||
pattern: '**/*.ts',
|
||||
dryRun: true,
|
||||
verbose: false,
|
||||
});
|
||||
|
||||
const summary = await migration.run();
|
||||
|
||||
expect(summary.dryRun).toEqual(true);
|
||||
expect(summary.totalLegacyFiles).toEqual(1);
|
||||
expect(summary.migratedCount).toEqual(1); // Dry run still counts as "would migrate"
|
||||
|
||||
// Verify original file still exists
|
||||
const fileExists = await plugins.smartfile.fs.fileExists(legacyFile);
|
||||
expect(fileExists).toEqual(true);
|
||||
|
||||
// Clean up
|
||||
await plugins.smartfile.fs.removeSync(tempDir);
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
203
test/test.runtime.parser.node.ts
Normal file
203
test/test.runtime.parser.node.ts
Normal file
@@ -0,0 +1,203 @@
|
||||
import { expect, tap } from '../ts_tapbundle/index.js';
|
||||
import { parseTestFilename, isLegacyFilename, getLegacyMigrationTarget } from '../ts/tstest.classes.runtime.parser.js';
|
||||
|
||||
tap.test('parseTestFilename - single runtime', async () => {
|
||||
const parsed = parseTestFilename('test.node.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - chromium runtime', async () => {
|
||||
const parsed = parseTestFilename('test.chromium.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['chromium']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - multiple runtimes', async () => {
|
||||
const parsed = parseTestFilename('test.node+chromium.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - deno+bun runtime', async () => {
|
||||
const parsed = parseTestFilename('test.deno+bun.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['deno', 'bun']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - with nonci modifier', async () => {
|
||||
const parsed = parseTestFilename('test.chromium.nonci.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['chromium']);
|
||||
expect(parsed.modifiers).toEqual(['nonci']);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - multi-runtime with nonci', async () => {
|
||||
const parsed = parseTestFilename('test.node+chromium.nonci.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium']);
|
||||
expect(parsed.modifiers).toEqual(['nonci']);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - legacy browser', async () => {
|
||||
const parsed = parseTestFilename('test.browser.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['chromium']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(true);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - legacy both', async () => {
|
||||
const parsed = parseTestFilename('test.both.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(true);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - legacy browser with nonci', async () => {
|
||||
const parsed = parseTestFilename('test.browser.nonci.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['chromium']);
|
||||
expect(parsed.modifiers).toEqual(['nonci']);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(true);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - complex basename', async () => {
|
||||
const parsed = parseTestFilename('test.some.feature.node.ts');
|
||||
expect(parsed.baseName).toEqual('test.some.feature');
|
||||
expect(parsed.runtimes).toEqual(['node']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - default to node when no runtime', async () => {
|
||||
const parsed = parseTestFilename('test.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - tsx extension', async () => {
|
||||
const parsed = parseTestFilename('test.chromium.tsx');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['chromium']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('tsx');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - deduplicates runtime tokens', async () => {
|
||||
const parsed = parseTestFilename('test.node+node.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('isLegacyFilename - detects browser', async () => {
|
||||
expect(isLegacyFilename('test.browser.ts')).toEqual(true);
|
||||
});
|
||||
|
||||
tap.test('isLegacyFilename - detects both', async () => {
|
||||
expect(isLegacyFilename('test.both.ts')).toEqual(true);
|
||||
});
|
||||
|
||||
tap.test('isLegacyFilename - rejects new naming', async () => {
|
||||
expect(isLegacyFilename('test.node.ts')).toEqual(false);
|
||||
expect(isLegacyFilename('test.chromium.ts')).toEqual(false);
|
||||
expect(isLegacyFilename('test.node+chromium.ts')).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('getLegacyMigrationTarget - browser to chromium', async () => {
|
||||
const target = getLegacyMigrationTarget('test.browser.ts');
|
||||
expect(target).toEqual('test.chromium.ts');
|
||||
});
|
||||
|
||||
tap.test('getLegacyMigrationTarget - both to node+chromium', async () => {
|
||||
const target = getLegacyMigrationTarget('test.both.ts');
|
||||
expect(target).toEqual('test.node+chromium.ts');
|
||||
});
|
||||
|
||||
tap.test('getLegacyMigrationTarget - browser with nonci', async () => {
|
||||
const target = getLegacyMigrationTarget('test.browser.nonci.ts');
|
||||
expect(target).toEqual('test.chromium.nonci.ts');
|
||||
});
|
||||
|
||||
tap.test('getLegacyMigrationTarget - both with nonci', async () => {
|
||||
const target = getLegacyMigrationTarget('test.both.nonci.ts');
|
||||
expect(target).toEqual('test.node+chromium.nonci.ts');
|
||||
});
|
||||
|
||||
tap.test('getLegacyMigrationTarget - returns null for non-legacy', async () => {
|
||||
const target = getLegacyMigrationTarget('test.node.ts');
|
||||
expect(target).toEqual(null);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - handles full paths', async () => {
|
||||
const parsed = parseTestFilename('/path/to/test.node+chromium.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium']);
|
||||
expect(parsed.original).toEqual('test.node+chromium.ts');
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - all keyword expands to all runtimes', async () => {
|
||||
const parsed = parseTestFilename('test.all.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium', 'deno', 'bun']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - all keyword with nonci modifier', async () => {
|
||||
const parsed = parseTestFilename('test.all.nonci.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium', 'deno', 'bun']);
|
||||
expect(parsed.modifiers).toEqual(['nonci']);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - all keyword with complex basename', async () => {
|
||||
const parsed = parseTestFilename('test.some.feature.all.ts');
|
||||
expect(parsed.baseName).toEqual('test.some.feature');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium', 'deno', 'bun']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
tap.test('parseTestFilename - all keyword in chain expands to all runtimes', async () => {
|
||||
const parsed = parseTestFilename('test.node+all.ts');
|
||||
expect(parsed.baseName).toEqual('test');
|
||||
expect(parsed.runtimes).toEqual(['node', 'chromium', 'deno', 'bun']);
|
||||
expect(parsed.modifiers).toEqual([]);
|
||||
expect(parsed.extension).toEqual('ts');
|
||||
expect(parsed.isLegacy).toEqual(false);
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
17
test/watch-demo/test.demo.ts
Normal file
17
test/watch-demo/test.demo.ts
Normal file
@@ -0,0 +1,17 @@
|
||||
import { tap, expect } from '../../ts_tapbundle/index.js';
|
||||
|
||||
// This test file demonstrates watch mode
|
||||
// Try modifying this file while running: tstest test/watch-demo --watch
|
||||
|
||||
let counter = 1;
|
||||
|
||||
tap.test('demo test that changes', async () => {
|
||||
expect(counter).toEqual(1);
|
||||
console.log(`Test run at: ${new Date().toISOString()}`);
|
||||
});
|
||||
|
||||
tap.test('another test', async () => {
|
||||
expect('hello').toEqual('hello');
|
||||
});
|
||||
|
||||
tap.start();
|
||||
@@ -3,6 +3,6 @@
|
||||
*/
|
||||
export const commitinfo = {
|
||||
name: '@git.zone/tstest',
|
||||
version: '1.9.0',
|
||||
version: '3.0.1',
|
||||
description: 'a test utility to run tests that match test/**/*.ts'
|
||||
}
|
||||
|
||||
140
ts/index.ts
140
ts/index.ts
@@ -8,17 +8,68 @@ export enum TestExecutionMode {
|
||||
}
|
||||
|
||||
export const runCli = async () => {
|
||||
// Check if we're using global tstest in the tstest project itself
|
||||
try {
|
||||
const packageJsonPath = `${process.cwd()}/package.json`;
|
||||
const fs = await import('fs');
|
||||
if (fs.existsSync(packageJsonPath)) {
|
||||
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
|
||||
if (packageJson.name === '@git.zone/tstest') {
|
||||
// Check if we're running from a global installation
|
||||
const execPath = process.argv[1];
|
||||
// Debug: log the paths (uncomment for debugging)
|
||||
// console.log('DEBUG: Checking global tstest usage...');
|
||||
// console.log('execPath:', execPath);
|
||||
// console.log('cwd:', process.cwd());
|
||||
// console.log('process.argv:', process.argv);
|
||||
|
||||
// Check if this is running from global installation
|
||||
const isLocalCli = execPath.includes(process.cwd());
|
||||
const isGlobalPnpm = process.argv.some(arg => arg.includes('.pnpm') && !arg.includes(process.cwd()));
|
||||
const isGlobalNpm = process.argv.some(arg => arg.includes('npm/node_modules') && !arg.includes(process.cwd()));
|
||||
|
||||
if (!isLocalCli && (isGlobalPnpm || isGlobalNpm || !execPath.includes('node_modules'))) {
|
||||
console.error('\n⚠️ WARNING: You are using a globally installed tstest in the tstest project itself!');
|
||||
console.error(' This means you are NOT testing your local changes.');
|
||||
console.error(' Please use one of these commands instead:');
|
||||
console.error(' • node cli.js <test-path>');
|
||||
console.error(' • pnpm test <test-path>');
|
||||
console.error(' • ./cli.js <test-path> (if executable)\n');
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
// Silently ignore any errors in this check
|
||||
}
|
||||
|
||||
// Parse command line arguments
|
||||
const args = process.argv.slice(2);
|
||||
const logOptions: LogOptions = {};
|
||||
let testPath: string | null = null;
|
||||
let tags: string[] = [];
|
||||
let startFromFile: number | null = null;
|
||||
let stopAtFile: number | null = null;
|
||||
let timeoutSeconds: number | null = null;
|
||||
let watchMode: boolean = false;
|
||||
let watchIgnorePatterns: string[] = [];
|
||||
|
||||
// Parse options
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const arg = args[i];
|
||||
|
||||
switch (arg) {
|
||||
case '--version':
|
||||
// Get version from package.json
|
||||
try {
|
||||
const fs = await import('fs');
|
||||
const packagePath = new URL('../package.json', import.meta.url).pathname;
|
||||
const packageData = JSON.parse(await fs.promises.readFile(packagePath, 'utf8'));
|
||||
console.log(`tstest version ${packageData.version}`);
|
||||
} catch (error) {
|
||||
console.log('tstest version unknown');
|
||||
}
|
||||
process.exit(0);
|
||||
break;
|
||||
case '--quiet':
|
||||
case '-q':
|
||||
logOptions.quiet = true;
|
||||
@@ -42,6 +93,57 @@ export const runCli = async () => {
|
||||
tags = args[++i].split(',');
|
||||
}
|
||||
break;
|
||||
case '--startFrom':
|
||||
if (i + 1 < args.length) {
|
||||
const value = parseInt(args[++i], 10);
|
||||
if (isNaN(value) || value < 1) {
|
||||
console.error('Error: --startFrom must be a positive integer');
|
||||
process.exit(1);
|
||||
}
|
||||
startFromFile = value;
|
||||
} else {
|
||||
console.error('Error: --startFrom requires a number argument');
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
case '--stopAt':
|
||||
if (i + 1 < args.length) {
|
||||
const value = parseInt(args[++i], 10);
|
||||
if (isNaN(value) || value < 1) {
|
||||
console.error('Error: --stopAt must be a positive integer');
|
||||
process.exit(1);
|
||||
}
|
||||
stopAtFile = value;
|
||||
} else {
|
||||
console.error('Error: --stopAt requires a number argument');
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
case '--timeout':
|
||||
if (i + 1 < args.length) {
|
||||
const value = parseInt(args[++i], 10);
|
||||
if (isNaN(value) || value < 1) {
|
||||
console.error('Error: --timeout must be a positive integer (seconds)');
|
||||
process.exit(1);
|
||||
}
|
||||
timeoutSeconds = value;
|
||||
} else {
|
||||
console.error('Error: --timeout requires a number argument (seconds)');
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
case '--watch':
|
||||
case '-w':
|
||||
watchMode = true;
|
||||
break;
|
||||
case '--watch-ignore':
|
||||
if (i + 1 < args.length) {
|
||||
watchIgnorePatterns = args[++i].split(',');
|
||||
} else {
|
||||
console.error('Error: --watch-ignore requires a comma-separated list of patterns');
|
||||
process.exit(1);
|
||||
}
|
||||
break;
|
||||
default:
|
||||
if (!arg.startsWith('-')) {
|
||||
testPath = arg;
|
||||
@@ -49,16 +151,28 @@ export const runCli = async () => {
|
||||
}
|
||||
}
|
||||
|
||||
// Validate test file range options
|
||||
if (startFromFile !== null && stopAtFile !== null && startFromFile > stopAtFile) {
|
||||
console.error('Error: --startFrom cannot be greater than --stopAt');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (!testPath) {
|
||||
console.error('You must specify a test directory/file/pattern as argument. Please try again.');
|
||||
console.error('\nUsage: tstest <path> [options]');
|
||||
console.error('\nOptions:');
|
||||
console.error(' --quiet, -q Minimal output');
|
||||
console.error(' --verbose, -v Verbose output');
|
||||
console.error(' --no-color Disable colored output');
|
||||
console.error(' --json Output results as JSON');
|
||||
console.error(' --logfile Write logs to .nogit/testlogs/[testfile].log');
|
||||
console.error(' --tags Run only tests with specified tags (comma-separated)');
|
||||
console.error(' --version Show version information');
|
||||
console.error(' --quiet, -q Minimal output');
|
||||
console.error(' --verbose, -v Verbose output');
|
||||
console.error(' --no-color Disable colored output');
|
||||
console.error(' --json Output results as JSON');
|
||||
console.error(' --logfile Write logs to .nogit/testlogs/[testfile].log');
|
||||
console.error(' --tags <tags> Run only tests with specified tags (comma-separated)');
|
||||
console.error(' --startFrom <n> Start running from test file number n');
|
||||
console.error(' --stopAt <n> Stop running at test file number n');
|
||||
console.error(' --timeout <s> Timeout test files after s seconds');
|
||||
console.error(' --watch, -w Watch for file changes and re-run tests');
|
||||
console.error(' --watch-ignore Patterns to ignore in watch mode (comma-separated)');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
@@ -73,6 +187,16 @@ export const runCli = async () => {
|
||||
executionMode = TestExecutionMode.DIRECTORY;
|
||||
}
|
||||
|
||||
const tsTestInstance = new TsTest(process.cwd(), testPath, executionMode, logOptions, tags);
|
||||
await tsTestInstance.run();
|
||||
const tsTestInstance = new TsTest(process.cwd(), testPath, executionMode, logOptions, tags, startFromFile, stopAtFile, timeoutSeconds);
|
||||
|
||||
if (watchMode) {
|
||||
await tsTestInstance.runWatch(watchIgnorePatterns);
|
||||
} else {
|
||||
await tsTestInstance.run();
|
||||
}
|
||||
};
|
||||
|
||||
// Execute CLI when this file is run directly
|
||||
if (import.meta.url === `file://${process.argv[1]}`) {
|
||||
runCli();
|
||||
}
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
"order": 2
|
||||
"order": 4
|
||||
}
|
||||
316
ts/tstest.classes.migration.ts
Normal file
316
ts/tstest.classes.migration.ts
Normal file
@@ -0,0 +1,316 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import { coloredString as cs } from '@push.rocks/consolecolor';
|
||||
import { parseTestFilename, getLegacyMigrationTarget, isLegacyFilename } from './tstest.classes.runtime.parser.js';
|
||||
|
||||
/**
|
||||
* Migration result for a single file
|
||||
*/
|
||||
export interface MigrationResult {
|
||||
/**
|
||||
* Original file path
|
||||
*/
|
||||
oldPath: string;
|
||||
|
||||
/**
|
||||
* New file path after migration
|
||||
*/
|
||||
newPath: string;
|
||||
|
||||
/**
|
||||
* Whether the migration was performed
|
||||
*/
|
||||
migrated: boolean;
|
||||
|
||||
/**
|
||||
* Error message if migration failed
|
||||
*/
|
||||
error?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Migration summary
|
||||
*/
|
||||
export interface MigrationSummary {
|
||||
/**
|
||||
* Total number of legacy files found
|
||||
*/
|
||||
totalLegacyFiles: number;
|
||||
|
||||
/**
|
||||
* Number of files successfully migrated
|
||||
*/
|
||||
migratedCount: number;
|
||||
|
||||
/**
|
||||
* Number of files that failed to migrate
|
||||
*/
|
||||
errorCount: number;
|
||||
|
||||
/**
|
||||
* Individual migration results
|
||||
*/
|
||||
results: MigrationResult[];
|
||||
|
||||
/**
|
||||
* Whether this was a dry run
|
||||
*/
|
||||
dryRun: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Migration options
|
||||
*/
|
||||
export interface MigrationOptions {
|
||||
/**
|
||||
* Base directory to search for test files
|
||||
* Default: process.cwd()
|
||||
*/
|
||||
baseDir?: string;
|
||||
|
||||
/**
|
||||
* Glob pattern for finding test files
|
||||
* Default: '** /*test*.ts' (without space)
|
||||
*/
|
||||
pattern?: string;
|
||||
|
||||
/**
|
||||
* Dry run mode - don't actually rename files
|
||||
* Default: true
|
||||
*/
|
||||
dryRun?: boolean;
|
||||
|
||||
/**
|
||||
* Verbose output
|
||||
* Default: false
|
||||
*/
|
||||
verbose?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Migration class for renaming legacy test files to new naming convention
|
||||
*
|
||||
* Migrations:
|
||||
* - .browser.ts → .chromium.ts
|
||||
* - .both.ts → .node+chromium.ts
|
||||
* - .both.nonci.ts → .node+chromium.nonci.ts
|
||||
* - .browser.nonci.ts → .chromium.nonci.ts
|
||||
*/
|
||||
export class Migration {
|
||||
private options: Required<MigrationOptions>;
|
||||
|
||||
constructor(options: MigrationOptions = {}) {
|
||||
this.options = {
|
||||
baseDir: options.baseDir || process.cwd(),
|
||||
pattern: options.pattern || '**/test*.ts',
|
||||
dryRun: options.dryRun !== undefined ? options.dryRun : true,
|
||||
verbose: options.verbose || false,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find all legacy test files in the base directory
|
||||
*/
|
||||
async findLegacyFiles(): Promise<string[]> {
|
||||
const files = await plugins.smartfile.fs.listFileTree(
|
||||
this.options.baseDir,
|
||||
this.options.pattern
|
||||
);
|
||||
|
||||
const legacyFiles: string[] = [];
|
||||
|
||||
for (const file of files) {
|
||||
const fileName = plugins.path.basename(file);
|
||||
if (isLegacyFilename(fileName)) {
|
||||
const absolutePath = plugins.path.isAbsolute(file)
|
||||
? file
|
||||
: plugins.path.join(this.options.baseDir, file);
|
||||
legacyFiles.push(absolutePath);
|
||||
}
|
||||
}
|
||||
|
||||
return legacyFiles;
|
||||
}
|
||||
|
||||
/**
|
||||
* Migrate a single file
|
||||
*/
|
||||
private async migrateFile(filePath: string): Promise<MigrationResult> {
|
||||
const fileName = plugins.path.basename(filePath);
|
||||
const dirName = plugins.path.dirname(filePath);
|
||||
|
||||
try {
|
||||
// Get the new filename
|
||||
const newFileName = getLegacyMigrationTarget(fileName);
|
||||
|
||||
if (!newFileName) {
|
||||
return {
|
||||
oldPath: filePath,
|
||||
newPath: filePath,
|
||||
migrated: false,
|
||||
error: 'File is not a legacy file',
|
||||
};
|
||||
}
|
||||
|
||||
const newPath = plugins.path.join(dirName, newFileName);
|
||||
|
||||
// Check if target file already exists
|
||||
if (await plugins.smartfile.fs.fileExists(newPath)) {
|
||||
return {
|
||||
oldPath: filePath,
|
||||
newPath,
|
||||
migrated: false,
|
||||
error: `Target file already exists: ${newPath}`,
|
||||
};
|
||||
}
|
||||
|
||||
if (!this.options.dryRun) {
|
||||
// Check if we're in a git repository
|
||||
const isGitRepo = await this.isGitRepository(this.options.baseDir);
|
||||
|
||||
if (isGitRepo) {
|
||||
// Use git mv to preserve history
|
||||
const smartshell = new plugins.smartshell.Smartshell({
|
||||
executor: 'bash',
|
||||
pathDirectories: [],
|
||||
});
|
||||
const gitCommand = `cd "${this.options.baseDir}" && git mv "${filePath}" "${newPath}"`;
|
||||
const result = await smartshell.exec(gitCommand);
|
||||
|
||||
if (result.exitCode !== 0) {
|
||||
throw new Error(`git mv failed: ${result.stderr}`);
|
||||
}
|
||||
} else {
|
||||
// Not a git repository - cannot migrate without git
|
||||
throw new Error('Migration requires a git repository. We have git!');
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
oldPath: filePath,
|
||||
newPath,
|
||||
migrated: true,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
oldPath: filePath,
|
||||
newPath: filePath,
|
||||
migrated: false,
|
||||
error: error.message,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a directory is a git repository
|
||||
*/
|
||||
private async isGitRepository(dir: string): Promise<boolean> {
|
||||
try {
|
||||
const gitDir = plugins.path.join(dir, '.git');
|
||||
return await plugins.smartfile.fs.isDirectory(gitDir);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the migration
|
||||
*/
|
||||
async run(): Promise<MigrationSummary> {
|
||||
const legacyFiles = await this.findLegacyFiles();
|
||||
|
||||
console.log('');
|
||||
console.log(cs('='.repeat(60), 'blue'));
|
||||
console.log(cs('Test File Migration Tool', 'blue'));
|
||||
console.log(cs('='.repeat(60), 'blue'));
|
||||
console.log('');
|
||||
|
||||
if (this.options.dryRun) {
|
||||
console.log(cs('🔍 DRY RUN MODE - No files will be modified', 'orange'));
|
||||
console.log('');
|
||||
}
|
||||
|
||||
console.log(`Found ${legacyFiles.length} legacy test file(s)`);
|
||||
console.log('');
|
||||
|
||||
const results: MigrationResult[] = [];
|
||||
let migratedCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
for (const file of legacyFiles) {
|
||||
const result = await this.migrateFile(file);
|
||||
results.push(result);
|
||||
|
||||
if (result.migrated) {
|
||||
migratedCount++;
|
||||
const oldName = plugins.path.basename(result.oldPath);
|
||||
const newName = plugins.path.basename(result.newPath);
|
||||
|
||||
if (this.options.dryRun) {
|
||||
console.log(cs(` Would migrate:`, 'cyan'));
|
||||
} else {
|
||||
console.log(cs(` ✓ Migrated:`, 'green'));
|
||||
}
|
||||
console.log(` ${oldName}`);
|
||||
console.log(cs(` → ${newName}`, 'green'));
|
||||
console.log('');
|
||||
} else if (result.error) {
|
||||
errorCount++;
|
||||
console.log(cs(` ✗ Failed: ${plugins.path.basename(result.oldPath)}`, 'red'));
|
||||
console.log(cs(` ${result.error}`, 'red'));
|
||||
console.log('');
|
||||
}
|
||||
}
|
||||
|
||||
console.log(cs('='.repeat(60), 'blue'));
|
||||
console.log(`Summary:`);
|
||||
console.log(` Total legacy files: ${legacyFiles.length}`);
|
||||
console.log(` Successfully migrated: ${migratedCount}`);
|
||||
console.log(` Errors: ${errorCount}`);
|
||||
console.log(cs('='.repeat(60), 'blue'));
|
||||
|
||||
if (this.options.dryRun && legacyFiles.length > 0) {
|
||||
console.log('');
|
||||
console.log(cs('To apply these changes, run:', 'orange'));
|
||||
console.log(cs(' tstest migrate --write', 'orange'));
|
||||
}
|
||||
|
||||
console.log('');
|
||||
|
||||
return {
|
||||
totalLegacyFiles: legacyFiles.length,
|
||||
migratedCount,
|
||||
errorCount,
|
||||
results,
|
||||
dryRun: this.options.dryRun,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a migration report without performing the migration
|
||||
*/
|
||||
async generateReport(): Promise<string> {
|
||||
const legacyFiles = await this.findLegacyFiles();
|
||||
|
||||
let report = '';
|
||||
report += 'Test File Migration Report\n';
|
||||
report += '='.repeat(60) + '\n';
|
||||
report += '\n';
|
||||
report += `Found ${legacyFiles.length} legacy test file(s)\n`;
|
||||
report += '\n';
|
||||
|
||||
for (const file of legacyFiles) {
|
||||
const fileName = plugins.path.basename(file);
|
||||
const newFileName = getLegacyMigrationTarget(fileName);
|
||||
|
||||
if (newFileName) {
|
||||
report += `${fileName}\n`;
|
||||
report += ` → ${newFileName}\n`;
|
||||
report += '\n';
|
||||
}
|
||||
}
|
||||
|
||||
report += '='.repeat(60) + '\n';
|
||||
|
||||
return report;
|
||||
}
|
||||
}
|
||||
245
ts/tstest.classes.runtime.adapter.ts
Normal file
245
ts/tstest.classes.runtime.adapter.ts
Normal file
@@ -0,0 +1,245 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import type { Runtime } from './tstest.classes.runtime.parser.js';
|
||||
import { TapParser } from './tstest.classes.tap.parser.js';
|
||||
|
||||
/**
|
||||
* Runtime-specific configuration options
|
||||
*/
|
||||
export interface RuntimeOptions {
|
||||
/**
|
||||
* Environment variables to pass to the runtime
|
||||
*/
|
||||
env?: Record<string, string>;
|
||||
|
||||
/**
|
||||
* Additional command-line arguments
|
||||
*/
|
||||
extraArgs?: string[];
|
||||
|
||||
/**
|
||||
* Working directory for test execution
|
||||
*/
|
||||
cwd?: string;
|
||||
|
||||
/**
|
||||
* Timeout in milliseconds (0 = no timeout)
|
||||
*/
|
||||
timeout?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Deno-specific configuration options
|
||||
*/
|
||||
export interface DenoOptions extends RuntimeOptions {
|
||||
/**
|
||||
* Permissions to grant to Deno
|
||||
* Default: ['--allow-read', '--allow-env']
|
||||
*/
|
||||
permissions?: string[];
|
||||
|
||||
/**
|
||||
* Path to deno.json config file
|
||||
*/
|
||||
configPath?: string;
|
||||
|
||||
/**
|
||||
* Path to import map file
|
||||
*/
|
||||
importMap?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Chromium-specific configuration options
|
||||
*/
|
||||
export interface ChromiumOptions extends RuntimeOptions {
|
||||
/**
|
||||
* Chromium launch arguments
|
||||
*/
|
||||
launchArgs?: string[];
|
||||
|
||||
/**
|
||||
* Headless mode (default: true)
|
||||
*/
|
||||
headless?: boolean;
|
||||
|
||||
/**
|
||||
* Port range for HTTP server
|
||||
*/
|
||||
portRange?: { min: number; max: number };
|
||||
}
|
||||
|
||||
/**
|
||||
* Command configuration returned by createCommand()
|
||||
*/
|
||||
export interface RuntimeCommand {
|
||||
/**
|
||||
* The main command executable (e.g., 'node', 'deno', 'bun')
|
||||
*/
|
||||
command: string;
|
||||
|
||||
/**
|
||||
* Command-line arguments
|
||||
*/
|
||||
args: string[];
|
||||
|
||||
/**
|
||||
* Environment variables
|
||||
*/
|
||||
env?: Record<string, string>;
|
||||
|
||||
/**
|
||||
* Working directory
|
||||
*/
|
||||
cwd?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Runtime availability check result
|
||||
*/
|
||||
export interface RuntimeAvailability {
|
||||
/**
|
||||
* Whether the runtime is available
|
||||
*/
|
||||
available: boolean;
|
||||
|
||||
/**
|
||||
* Version string if available
|
||||
*/
|
||||
version?: string;
|
||||
|
||||
/**
|
||||
* Error message if not available
|
||||
*/
|
||||
error?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Abstract base class for runtime adapters
|
||||
* Each runtime (Node, Chromium, Deno, Bun) implements this interface
|
||||
*/
|
||||
export abstract class RuntimeAdapter {
|
||||
/**
|
||||
* Runtime identifier
|
||||
*/
|
||||
abstract readonly id: Runtime;
|
||||
|
||||
/**
|
||||
* Human-readable display name
|
||||
*/
|
||||
abstract readonly displayName: string;
|
||||
|
||||
/**
|
||||
* Check if this runtime is available on the system
|
||||
* @returns Availability information including version
|
||||
*/
|
||||
abstract checkAvailable(): Promise<RuntimeAvailability>;
|
||||
|
||||
/**
|
||||
* Create the command configuration for executing a test
|
||||
* @param testFile - Absolute path to the test file
|
||||
* @param options - Runtime-specific options
|
||||
* @returns Command configuration
|
||||
*/
|
||||
abstract createCommand(testFile: string, options?: RuntimeOptions): RuntimeCommand;
|
||||
|
||||
/**
|
||||
* Execute a test file and return a TAP parser
|
||||
* @param testFile - Absolute path to the test file
|
||||
* @param index - Test index (for display)
|
||||
* @param total - Total number of tests (for display)
|
||||
* @param options - Runtime-specific options
|
||||
* @returns TAP parser with test results
|
||||
*/
|
||||
abstract run(
|
||||
testFile: string,
|
||||
index: number,
|
||||
total: number,
|
||||
options?: RuntimeOptions
|
||||
): Promise<TapParser>;
|
||||
|
||||
/**
|
||||
* Get the default options for this runtime
|
||||
* Can be overridden by subclasses
|
||||
*/
|
||||
protected getDefaultOptions(): RuntimeOptions {
|
||||
return {
|
||||
timeout: 0,
|
||||
extraArgs: [],
|
||||
env: {},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge user options with defaults
|
||||
*/
|
||||
protected mergeOptions<T extends RuntimeOptions>(userOptions?: T): T {
|
||||
const defaults = this.getDefaultOptions();
|
||||
return {
|
||||
...defaults,
|
||||
...userOptions,
|
||||
env: { ...defaults.env, ...userOptions?.env },
|
||||
extraArgs: [...(defaults.extraArgs || []), ...(userOptions?.extraArgs || [])],
|
||||
} as T;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Registry for runtime adapters
|
||||
* Manages all available runtime implementations
|
||||
*/
|
||||
export class RuntimeAdapterRegistry {
|
||||
private adapters: Map<Runtime, RuntimeAdapter> = new Map();
|
||||
|
||||
/**
|
||||
* Register a runtime adapter
|
||||
*/
|
||||
register(adapter: RuntimeAdapter): void {
|
||||
this.adapters.set(adapter.id, adapter);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get an adapter by runtime ID
|
||||
*/
|
||||
get(runtime: Runtime): RuntimeAdapter | undefined {
|
||||
return this.adapters.get(runtime);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all registered adapters
|
||||
*/
|
||||
getAll(): RuntimeAdapter[] {
|
||||
return Array.from(this.adapters.values());
|
||||
}
|
||||
|
||||
/**
|
||||
* Check which runtimes are available on the system
|
||||
*/
|
||||
async checkAvailability(): Promise<Map<Runtime, RuntimeAvailability>> {
|
||||
const results = new Map<Runtime, RuntimeAvailability>();
|
||||
|
||||
for (const [runtime, adapter] of this.adapters) {
|
||||
const availability = await adapter.checkAvailable();
|
||||
results.set(runtime, availability);
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get adapters for a list of runtimes, in order
|
||||
* @param runtimes - Ordered list of runtimes
|
||||
* @returns Adapters in the same order, skipping any that aren't registered
|
||||
*/
|
||||
getAdaptersForRuntimes(runtimes: Runtime[]): RuntimeAdapter[] {
|
||||
const adapters: RuntimeAdapter[] = [];
|
||||
|
||||
for (const runtime of runtimes) {
|
||||
const adapter = this.get(runtime);
|
||||
if (adapter) {
|
||||
adapters.push(adapter);
|
||||
}
|
||||
}
|
||||
|
||||
return adapters;
|
||||
}
|
||||
}
|
||||
222
ts/tstest.classes.runtime.bun.ts
Normal file
222
ts/tstest.classes.runtime.bun.ts
Normal file
@@ -0,0 +1,222 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import { coloredString as cs } from '@push.rocks/consolecolor';
|
||||
import {
|
||||
RuntimeAdapter,
|
||||
type RuntimeOptions,
|
||||
type RuntimeCommand,
|
||||
type RuntimeAvailability,
|
||||
} from './tstest.classes.runtime.adapter.js';
|
||||
import { TapParser } from './tstest.classes.tap.parser.js';
|
||||
import { TsTestLogger } from './tstest.logging.js';
|
||||
import type { Runtime } from './tstest.classes.runtime.parser.js';
|
||||
|
||||
/**
|
||||
* Bun runtime adapter
|
||||
* Executes tests using the Bun runtime with native TypeScript support
|
||||
*/
|
||||
export class BunRuntimeAdapter extends RuntimeAdapter {
|
||||
readonly id: Runtime = 'bun';
|
||||
readonly displayName: string = 'Bun';
|
||||
|
||||
constructor(
|
||||
private logger: TsTestLogger,
|
||||
private smartshellInstance: any, // SmartShell instance from @push.rocks/smartshell
|
||||
private timeoutSeconds: number | null,
|
||||
private filterTags: string[]
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Bun is available
|
||||
*/
|
||||
async checkAvailable(): Promise<RuntimeAvailability> {
|
||||
try {
|
||||
const result = await this.smartshellInstance.execSilent('bun --version', {
|
||||
cwd: process.cwd(),
|
||||
onError: () => {
|
||||
// Ignore error
|
||||
}
|
||||
});
|
||||
|
||||
if (result.exitCode !== 0) {
|
||||
return {
|
||||
available: false,
|
||||
error: 'Bun not found. Install from: https://bun.sh/',
|
||||
};
|
||||
}
|
||||
|
||||
// Bun version is just the version number
|
||||
const version = `v${result.stdout.trim()}`;
|
||||
|
||||
return {
|
||||
available: true,
|
||||
version: version,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
available: false,
|
||||
error: error.message,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create command configuration for Bun test execution
|
||||
*/
|
||||
createCommand(testFile: string, options?: RuntimeOptions): RuntimeCommand {
|
||||
const mergedOptions = this.mergeOptions(options);
|
||||
|
||||
const args: string[] = ['run'];
|
||||
|
||||
// Note: Bun automatically discovers bunfig.toml in the current directory
|
||||
// This ensures TypeScript decorator support is enabled if bunfig.toml is present
|
||||
|
||||
// Add extra args
|
||||
if (mergedOptions.extraArgs && mergedOptions.extraArgs.length > 0) {
|
||||
args.push(...mergedOptions.extraArgs);
|
||||
}
|
||||
|
||||
// Add test file
|
||||
args.push(testFile);
|
||||
|
||||
// Set environment variables
|
||||
const env = { ...mergedOptions.env };
|
||||
|
||||
if (this.filterTags.length > 0) {
|
||||
env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
|
||||
}
|
||||
|
||||
return {
|
||||
command: 'bun',
|
||||
args,
|
||||
env,
|
||||
cwd: mergedOptions.cwd,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a test file in Bun
|
||||
*/
|
||||
async run(
|
||||
testFile: string,
|
||||
index: number,
|
||||
total: number,
|
||||
options?: RuntimeOptions
|
||||
): Promise<TapParser> {
|
||||
this.logger.testFileStart(testFile, this.displayName, index, total);
|
||||
const tapParser = new TapParser(testFile + ':bun', this.logger);
|
||||
|
||||
const mergedOptions = this.mergeOptions(options);
|
||||
|
||||
// Build Bun command
|
||||
const command = this.createCommand(testFile, mergedOptions);
|
||||
const fullCommand = `${command.command} ${command.args.join(' ')}`;
|
||||
|
||||
// Set filter tags as environment variable
|
||||
if (this.filterTags.length > 0) {
|
||||
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
|
||||
}
|
||||
|
||||
// Check for 00init.ts file in test directory
|
||||
const testDir = plugins.path.dirname(testFile);
|
||||
const initFile = plugins.path.join(testDir, '00init.ts');
|
||||
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
|
||||
|
||||
let runCommand = fullCommand;
|
||||
let loaderPath: string | null = null;
|
||||
|
||||
// If 00init.ts exists, create a loader file
|
||||
if (initFileExists) {
|
||||
const absoluteInitFile = plugins.path.resolve(initFile);
|
||||
const absoluteTestFile = plugins.path.resolve(testFile);
|
||||
const loaderContent = `
|
||||
import '${absoluteInitFile.replace(/\\/g, '/')}';
|
||||
import '${absoluteTestFile.replace(/\\/g, '/')}';
|
||||
`;
|
||||
loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(testFile)}`);
|
||||
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
|
||||
|
||||
// Rebuild command with loader file
|
||||
const loaderCommand = this.createCommand(loaderPath, mergedOptions);
|
||||
runCommand = `${loaderCommand.command} ${loaderCommand.args.join(' ')}`;
|
||||
}
|
||||
|
||||
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
|
||||
|
||||
// If we created a loader file, clean it up after test execution
|
||||
if (loaderPath) {
|
||||
const cleanup = () => {
|
||||
try {
|
||||
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
|
||||
plugins.smartfile.fs.removeSync(loaderPath);
|
||||
}
|
||||
} catch (e) {
|
||||
// Ignore cleanup errors
|
||||
}
|
||||
};
|
||||
|
||||
execResultStreaming.childProcess.on('exit', cleanup);
|
||||
execResultStreaming.childProcess.on('error', cleanup);
|
||||
}
|
||||
|
||||
// Start warning timer if no timeout was specified
|
||||
let warningTimer: NodeJS.Timeout | null = null;
|
||||
if (this.timeoutSeconds === null) {
|
||||
warningTimer = setTimeout(() => {
|
||||
console.error('');
|
||||
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
|
||||
console.error(cs(` File: ${testFile}`, 'orange'));
|
||||
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
|
||||
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
|
||||
console.error('');
|
||||
}, 60000); // 1 minute
|
||||
}
|
||||
|
||||
// Handle timeout if specified
|
||||
if (this.timeoutSeconds !== null) {
|
||||
const timeoutMs = this.timeoutSeconds * 1000;
|
||||
let timeoutId: NodeJS.Timeout;
|
||||
|
||||
const timeoutPromise = new Promise<void>((_resolve, reject) => {
|
||||
timeoutId = setTimeout(async () => {
|
||||
// Use smartshell's terminate() to kill entire process tree
|
||||
await execResultStreaming.terminate();
|
||||
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
|
||||
}, timeoutMs);
|
||||
});
|
||||
|
||||
try {
|
||||
await Promise.race([
|
||||
tapParser.handleTapProcess(execResultStreaming.childProcess),
|
||||
timeoutPromise
|
||||
]);
|
||||
// Clear timeout if test completed successfully
|
||||
clearTimeout(timeoutId);
|
||||
} catch (error) {
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
// Handle timeout error
|
||||
tapParser.handleTimeout(this.timeoutSeconds);
|
||||
// Ensure entire process tree is killed if still running
|
||||
try {
|
||||
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
|
||||
} catch (killError) {
|
||||
// Process tree might already be dead
|
||||
}
|
||||
await tapParser.evaluateFinalResult();
|
||||
}
|
||||
} else {
|
||||
await tapParser.handleTapProcess(execResultStreaming.childProcess);
|
||||
}
|
||||
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
|
||||
return tapParser;
|
||||
}
|
||||
}
|
||||
293
ts/tstest.classes.runtime.chromium.ts
Normal file
293
ts/tstest.classes.runtime.chromium.ts
Normal file
@@ -0,0 +1,293 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import * as paths from './tstest.paths.js';
|
||||
import { coloredString as cs } from '@push.rocks/consolecolor';
|
||||
import {
|
||||
RuntimeAdapter,
|
||||
type ChromiumOptions,
|
||||
type RuntimeCommand,
|
||||
type RuntimeAvailability,
|
||||
} from './tstest.classes.runtime.adapter.js';
|
||||
import { TapParser } from './tstest.classes.tap.parser.js';
|
||||
import { TsTestLogger } from './tstest.logging.js';
|
||||
import type { Runtime } from './tstest.classes.runtime.parser.js';
|
||||
|
||||
/**
|
||||
* Chromium runtime adapter
|
||||
* Executes tests in a headless Chromium browser
|
||||
*/
|
||||
export class ChromiumRuntimeAdapter extends RuntimeAdapter {
|
||||
readonly id: Runtime = 'chromium';
|
||||
readonly displayName: string = 'Chromium';
|
||||
|
||||
constructor(
|
||||
private logger: TsTestLogger,
|
||||
private tsbundleInstance: any, // TsBundle instance from @push.rocks/tsbundle
|
||||
private smartbrowserInstance: any, // SmartBrowser instance from @push.rocks/smartbrowser
|
||||
private timeoutSeconds: number | null
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Chromium is available
|
||||
*/
|
||||
async checkAvailable(): Promise<RuntimeAvailability> {
|
||||
try {
|
||||
// Check if smartbrowser is available and can start
|
||||
// The browser binary is usually handled by @push.rocks/smartbrowser
|
||||
return {
|
||||
available: true,
|
||||
version: 'via smartbrowser',
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
available: false,
|
||||
error: error.message || 'Chromium not available',
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create command configuration for Chromium test execution
|
||||
* Note: Chromium tests don't use a traditional command, but this satisfies the interface
|
||||
*/
|
||||
createCommand(testFile: string, options?: ChromiumOptions): RuntimeCommand {
|
||||
const mergedOptions = this.mergeOptions(options);
|
||||
|
||||
return {
|
||||
command: 'chromium',
|
||||
args: [],
|
||||
env: mergedOptions.env,
|
||||
cwd: mergedOptions.cwd,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find free ports for HTTP server and WebSocket
|
||||
*/
|
||||
private async findFreePorts(): Promise<{ httpPort: number; wsPort: number }> {
|
||||
const smartnetwork = new plugins.smartnetwork.SmartNetwork();
|
||||
|
||||
// Find random free HTTP port in range 30000-40000 to minimize collision chance
|
||||
const httpPort = await smartnetwork.findFreePort(30000, 40000, { randomize: true });
|
||||
if (!httpPort) {
|
||||
throw new Error('Could not find a free HTTP port in range 30000-40000');
|
||||
}
|
||||
|
||||
// Find random free WebSocket port, excluding the HTTP port to ensure they're different
|
||||
const wsPort = await smartnetwork.findFreePort(30000, 40000, {
|
||||
randomize: true,
|
||||
exclude: [httpPort]
|
||||
});
|
||||
if (!wsPort) {
|
||||
throw new Error('Could not find a free WebSocket port in range 30000-40000');
|
||||
}
|
||||
|
||||
// Log selected ports for debugging
|
||||
if (!this.logger.options.quiet) {
|
||||
console.log(`Selected ports - HTTP: ${httpPort}, WebSocket: ${wsPort}`);
|
||||
}
|
||||
return { httpPort, wsPort };
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a test file in Chromium browser
|
||||
*/
|
||||
async run(
|
||||
testFile: string,
|
||||
index: number,
|
||||
total: number,
|
||||
options?: ChromiumOptions
|
||||
): Promise<TapParser> {
|
||||
this.logger.testFileStart(testFile, this.displayName, index, total);
|
||||
|
||||
// lets get all our paths sorted
|
||||
const tsbundleCacheDirPath = plugins.path.join(paths.cwd, './.nogit/tstest_cache');
|
||||
const bundleFileName = testFile.replace('/', '__') + '.js';
|
||||
const bundleFilePath = plugins.path.join(tsbundleCacheDirPath, bundleFileName);
|
||||
|
||||
// lets bundle the test
|
||||
await plugins.smartfile.fs.ensureEmptyDir(tsbundleCacheDirPath);
|
||||
await this.tsbundleInstance.build(process.cwd(), testFile, bundleFilePath, {
|
||||
bundler: 'esbuild',
|
||||
});
|
||||
|
||||
// Find free ports for HTTP and WebSocket
|
||||
const { httpPort, wsPort } = await this.findFreePorts();
|
||||
|
||||
// lets create a server
|
||||
const server = new plugins.typedserver.servertools.Server({
|
||||
cors: true,
|
||||
port: httpPort,
|
||||
});
|
||||
server.addRoute(
|
||||
'/test',
|
||||
new plugins.typedserver.servertools.Handler('GET', async (_req, res) => {
|
||||
res.type('.html');
|
||||
res.write(`
|
||||
<html>
|
||||
<head>
|
||||
<script>
|
||||
globalThis.testdom = true;
|
||||
globalThis.wsPort = ${wsPort};
|
||||
</script>
|
||||
</head>
|
||||
<body></body>
|
||||
</html>
|
||||
`);
|
||||
res.end();
|
||||
})
|
||||
);
|
||||
server.addRoute('/*splat', new plugins.typedserver.servertools.HandlerStatic(tsbundleCacheDirPath));
|
||||
await server.start();
|
||||
|
||||
// lets handle realtime comms
|
||||
const tapParser = new TapParser(testFile + ':chrome', this.logger);
|
||||
const wss = new plugins.ws.WebSocketServer({ port: wsPort });
|
||||
wss.on('connection', (ws) => {
|
||||
ws.on('message', (message) => {
|
||||
const messageStr = message.toString();
|
||||
if (messageStr.startsWith('console:')) {
|
||||
const [, level, ...messageParts] = messageStr.split(':');
|
||||
this.logger.browserConsole(messageParts.join(':'), level);
|
||||
} else {
|
||||
tapParser.handleTapLog(messageStr);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// lets do the browser bit with timeout handling
|
||||
await this.smartbrowserInstance.start();
|
||||
|
||||
const evaluatePromise = this.smartbrowserInstance.evaluateOnPage(
|
||||
`http://localhost:${httpPort}/test?bundleName=${bundleFileName}`,
|
||||
async () => {
|
||||
// lets enable real time comms
|
||||
const ws = new WebSocket(`ws://localhost:${globalThis.wsPort}`);
|
||||
await new Promise((resolve) => (ws.onopen = resolve));
|
||||
|
||||
// Ensure this function is declared with 'async'
|
||||
const logStore = [];
|
||||
const originalLog = console.log;
|
||||
const originalError = console.error;
|
||||
|
||||
// Override console methods to capture the logs
|
||||
console.log = (...args: any[]) => {
|
||||
logStore.push(args.join(' '));
|
||||
ws.send(args.join(' '));
|
||||
originalLog(...args);
|
||||
};
|
||||
console.error = (...args: any[]) => {
|
||||
logStore.push(args.join(' '));
|
||||
ws.send(args.join(' '));
|
||||
originalError(...args);
|
||||
};
|
||||
|
||||
const bundleName = new URLSearchParams(window.location.search).get('bundleName');
|
||||
originalLog(`::TSTEST IN CHROMIUM:: Relevant Script name is: ${bundleName}`);
|
||||
|
||||
try {
|
||||
// Dynamically import the test module
|
||||
const testModule = await import(`/${bundleName}`);
|
||||
if (testModule && testModule.default && testModule.default instanceof Promise) {
|
||||
// Execute the exported test function
|
||||
await testModule.default;
|
||||
} else if (testModule && testModule.default && typeof testModule.default.then === 'function') {
|
||||
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
|
||||
console.log('Test module default export is just promiselike: Something might be messing with your Promise implementation.');
|
||||
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
|
||||
await testModule.default;
|
||||
} else if (globalThis.tapPromise && typeof globalThis.tapPromise.then === 'function') {
|
||||
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
|
||||
console.log('Using globalThis.tapPromise');
|
||||
console.log('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
|
||||
await testModule.default;
|
||||
} else {
|
||||
console.error('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
|
||||
console.error('Test module does not export a default promise.');
|
||||
console.error('!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
|
||||
console.log(`We got: ${JSON.stringify(testModule)}`);
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(err);
|
||||
}
|
||||
|
||||
return logStore.join('\n');
|
||||
}
|
||||
);
|
||||
|
||||
// Start warning timer if no timeout was specified
|
||||
let warningTimer: NodeJS.Timeout | null = null;
|
||||
if (this.timeoutSeconds === null) {
|
||||
warningTimer = setTimeout(() => {
|
||||
console.error('');
|
||||
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
|
||||
console.error(cs(` File: ${testFile}`, 'orange'));
|
||||
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
|
||||
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
|
||||
console.error('');
|
||||
}, 60000); // 1 minute
|
||||
}
|
||||
|
||||
// Handle timeout if specified
|
||||
if (this.timeoutSeconds !== null) {
|
||||
const timeoutMs = this.timeoutSeconds * 1000;
|
||||
let timeoutId: NodeJS.Timeout;
|
||||
|
||||
const timeoutPromise = new Promise<void>((_resolve, reject) => {
|
||||
timeoutId = setTimeout(() => {
|
||||
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
|
||||
}, timeoutMs);
|
||||
});
|
||||
|
||||
try {
|
||||
await Promise.race([
|
||||
evaluatePromise,
|
||||
timeoutPromise
|
||||
]);
|
||||
// Clear timeout if test completed successfully
|
||||
clearTimeout(timeoutId);
|
||||
} catch (error) {
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
// Handle timeout error
|
||||
tapParser.handleTimeout(this.timeoutSeconds);
|
||||
}
|
||||
} else {
|
||||
await evaluatePromise;
|
||||
}
|
||||
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
|
||||
// Always clean up resources, even on timeout
|
||||
try {
|
||||
await this.smartbrowserInstance.stop();
|
||||
} catch (error) {
|
||||
// Browser might already be stopped
|
||||
}
|
||||
|
||||
try {
|
||||
await server.stop();
|
||||
} catch (error) {
|
||||
// Server might already be stopped
|
||||
}
|
||||
|
||||
try {
|
||||
wss.close();
|
||||
} catch (error) {
|
||||
// WebSocket server might already be closed
|
||||
}
|
||||
|
||||
console.log(
|
||||
`${cs('=> ', 'blue')} Stopped ${cs(testFile, 'orange')} chromium instance and server.`
|
||||
);
|
||||
// Always evaluate final result (handleTimeout just sets up the test state)
|
||||
await tapParser.evaluateFinalResult();
|
||||
return tapParser;
|
||||
}
|
||||
}
|
||||
274
ts/tstest.classes.runtime.deno.ts
Normal file
274
ts/tstest.classes.runtime.deno.ts
Normal file
@@ -0,0 +1,274 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import { coloredString as cs } from '@push.rocks/consolecolor';
|
||||
import {
|
||||
RuntimeAdapter,
|
||||
type DenoOptions,
|
||||
type RuntimeCommand,
|
||||
type RuntimeAvailability,
|
||||
} from './tstest.classes.runtime.adapter.js';
|
||||
import { TapParser } from './tstest.classes.tap.parser.js';
|
||||
import { TsTestLogger } from './tstest.logging.js';
|
||||
import type { Runtime } from './tstest.classes.runtime.parser.js';
|
||||
|
||||
/**
|
||||
* Deno runtime adapter
|
||||
* Executes tests using the Deno runtime
|
||||
*/
|
||||
export class DenoRuntimeAdapter extends RuntimeAdapter {
|
||||
readonly id: Runtime = 'deno';
|
||||
readonly displayName: string = 'Deno';
|
||||
|
||||
constructor(
|
||||
private logger: TsTestLogger,
|
||||
private smartshellInstance: any, // SmartShell instance from @push.rocks/smartshell
|
||||
private timeoutSeconds: number | null,
|
||||
private filterTags: string[]
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get default Deno options
|
||||
*/
|
||||
protected getDefaultOptions(): DenoOptions {
|
||||
// Auto-detect deno.json or deno.jsonc config file for TypeScript decorator support
|
||||
let configPath: string | undefined;
|
||||
const denoJsonPath = plugins.path.join(process.cwd(), 'deno.json');
|
||||
const denoJsoncPath = plugins.path.join(process.cwd(), 'deno.jsonc');
|
||||
|
||||
if (plugins.smartfile.fs.fileExistsSync(denoJsonPath)) {
|
||||
configPath = denoJsonPath;
|
||||
} else if (plugins.smartfile.fs.fileExistsSync(denoJsoncPath)) {
|
||||
configPath = denoJsoncPath;
|
||||
}
|
||||
|
||||
return {
|
||||
...super.getDefaultOptions(),
|
||||
configPath,
|
||||
permissions: [
|
||||
'--allow-read',
|
||||
'--allow-env',
|
||||
'--allow-net',
|
||||
'--allow-write',
|
||||
'--allow-sys', // Allow system info access
|
||||
'--allow-import', // Allow npm/node imports
|
||||
'--node-modules-dir', // Enable Node.js compatibility mode
|
||||
'--sloppy-imports', // Allow .js imports to resolve to .ts files
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Deno is available
|
||||
*/
|
||||
async checkAvailable(): Promise<RuntimeAvailability> {
|
||||
try {
|
||||
const result = await this.smartshellInstance.execSilent('deno --version', {
|
||||
cwd: process.cwd(),
|
||||
onError: () => {
|
||||
// Ignore error
|
||||
}
|
||||
});
|
||||
|
||||
if (result.exitCode !== 0) {
|
||||
return {
|
||||
available: false,
|
||||
error: 'Deno not found. Install from: https://deno.land/',
|
||||
};
|
||||
}
|
||||
|
||||
// Parse Deno version from output (first line is "deno X.Y.Z")
|
||||
const versionMatch = result.stdout.match(/deno (\d+\.\d+\.\d+)/);
|
||||
const version = versionMatch ? `v${versionMatch[1]}` : 'unknown';
|
||||
|
||||
return {
|
||||
available: true,
|
||||
version: version,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
available: false,
|
||||
error: error.message,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create command configuration for Deno test execution
|
||||
*/
|
||||
createCommand(testFile: string, options?: DenoOptions): RuntimeCommand {
|
||||
const mergedOptions = this.mergeOptions(options) as DenoOptions;
|
||||
|
||||
const args: string[] = ['run'];
|
||||
|
||||
// Add permissions
|
||||
const permissions = mergedOptions.permissions || [
|
||||
'--allow-read',
|
||||
'--allow-env',
|
||||
'--allow-net',
|
||||
'--allow-write',
|
||||
'--allow-sys',
|
||||
'--allow-import',
|
||||
'--node-modules-dir',
|
||||
'--sloppy-imports',
|
||||
];
|
||||
args.push(...permissions);
|
||||
|
||||
// Add config file if specified
|
||||
if (mergedOptions.configPath) {
|
||||
args.push('--config', mergedOptions.configPath);
|
||||
}
|
||||
|
||||
// Add import map if specified
|
||||
if (mergedOptions.importMap) {
|
||||
args.push('--import-map', mergedOptions.importMap);
|
||||
}
|
||||
|
||||
// Add extra args
|
||||
if (mergedOptions.extraArgs && mergedOptions.extraArgs.length > 0) {
|
||||
args.push(...mergedOptions.extraArgs);
|
||||
}
|
||||
|
||||
// Add test file
|
||||
args.push(testFile);
|
||||
|
||||
// Set environment variables
|
||||
const env = { ...mergedOptions.env };
|
||||
|
||||
if (this.filterTags.length > 0) {
|
||||
env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
|
||||
}
|
||||
|
||||
return {
|
||||
command: 'deno',
|
||||
args,
|
||||
env,
|
||||
cwd: mergedOptions.cwd,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a test file in Deno
|
||||
*/
|
||||
async run(
|
||||
testFile: string,
|
||||
index: number,
|
||||
total: number,
|
||||
options?: DenoOptions
|
||||
): Promise<TapParser> {
|
||||
this.logger.testFileStart(testFile, this.displayName, index, total);
|
||||
const tapParser = new TapParser(testFile + ':deno', this.logger);
|
||||
|
||||
const mergedOptions = this.mergeOptions(options) as DenoOptions;
|
||||
|
||||
// Build Deno command
|
||||
const command = this.createCommand(testFile, mergedOptions);
|
||||
const fullCommand = `${command.command} ${command.args.join(' ')}`;
|
||||
|
||||
// Set filter tags as environment variable
|
||||
if (this.filterTags.length > 0) {
|
||||
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
|
||||
}
|
||||
|
||||
// Check for 00init.ts file in test directory
|
||||
const testDir = plugins.path.dirname(testFile);
|
||||
const initFile = plugins.path.join(testDir, '00init.ts');
|
||||
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
|
||||
|
||||
let runCommand = fullCommand;
|
||||
let loaderPath: string | null = null;
|
||||
|
||||
// If 00init.ts exists, create a loader file
|
||||
if (initFileExists) {
|
||||
const absoluteInitFile = plugins.path.resolve(initFile);
|
||||
const absoluteTestFile = plugins.path.resolve(testFile);
|
||||
const loaderContent = `
|
||||
import '${absoluteInitFile.replace(/\\/g, '/')}';
|
||||
import '${absoluteTestFile.replace(/\\/g, '/')}';
|
||||
`;
|
||||
loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(testFile)}`);
|
||||
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
|
||||
|
||||
// Rebuild command with loader file
|
||||
const loaderCommand = this.createCommand(loaderPath, mergedOptions);
|
||||
runCommand = `${loaderCommand.command} ${loaderCommand.args.join(' ')}`;
|
||||
}
|
||||
|
||||
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
|
||||
|
||||
// If we created a loader file, clean it up after test execution
|
||||
if (loaderPath) {
|
||||
const cleanup = () => {
|
||||
try {
|
||||
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
|
||||
plugins.smartfile.fs.removeSync(loaderPath);
|
||||
}
|
||||
} catch (e) {
|
||||
// Ignore cleanup errors
|
||||
}
|
||||
};
|
||||
|
||||
execResultStreaming.childProcess.on('exit', cleanup);
|
||||
execResultStreaming.childProcess.on('error', cleanup);
|
||||
}
|
||||
|
||||
// Start warning timer if no timeout was specified
|
||||
let warningTimer: NodeJS.Timeout | null = null;
|
||||
if (this.timeoutSeconds === null) {
|
||||
warningTimer = setTimeout(() => {
|
||||
console.error('');
|
||||
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
|
||||
console.error(cs(` File: ${testFile}`, 'orange'));
|
||||
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
|
||||
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
|
||||
console.error('');
|
||||
}, 60000); // 1 minute
|
||||
}
|
||||
|
||||
// Handle timeout if specified
|
||||
if (this.timeoutSeconds !== null) {
|
||||
const timeoutMs = this.timeoutSeconds * 1000;
|
||||
let timeoutId: NodeJS.Timeout;
|
||||
|
||||
const timeoutPromise = new Promise<void>((_resolve, reject) => {
|
||||
timeoutId = setTimeout(async () => {
|
||||
// Use smartshell's terminate() to kill entire process tree
|
||||
await execResultStreaming.terminate();
|
||||
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
|
||||
}, timeoutMs);
|
||||
});
|
||||
|
||||
try {
|
||||
await Promise.race([
|
||||
tapParser.handleTapProcess(execResultStreaming.childProcess),
|
||||
timeoutPromise
|
||||
]);
|
||||
// Clear timeout if test completed successfully
|
||||
clearTimeout(timeoutId);
|
||||
} catch (error) {
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
// Handle timeout error
|
||||
tapParser.handleTimeout(this.timeoutSeconds);
|
||||
// Ensure entire process tree is killed if still running
|
||||
try {
|
||||
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
|
||||
} catch (killError) {
|
||||
// Process tree might already be dead
|
||||
}
|
||||
await tapParser.evaluateFinalResult();
|
||||
}
|
||||
} else {
|
||||
await tapParser.handleTapProcess(execResultStreaming.childProcess);
|
||||
}
|
||||
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
|
||||
return tapParser;
|
||||
}
|
||||
}
|
||||
251
ts/tstest.classes.runtime.docker.ts
Normal file
251
ts/tstest.classes.runtime.docker.ts
Normal file
@@ -0,0 +1,251 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import { coloredString as cs } from '@push.rocks/consolecolor';
|
||||
import {
|
||||
RuntimeAdapter,
|
||||
type RuntimeOptions,
|
||||
type RuntimeCommand,
|
||||
type RuntimeAvailability,
|
||||
} from './tstest.classes.runtime.adapter.js';
|
||||
import { TapParser } from './tstest.classes.tap.parser.js';
|
||||
import { TsTestLogger } from './tstest.logging.js';
|
||||
import type { Runtime } from './tstest.classes.runtime.parser.js';
|
||||
import {
|
||||
parseDockerTestFilename,
|
||||
mapVariantToDockerfile,
|
||||
isDockerTestFile
|
||||
} from './tstest.classes.runtime.parser.js';
|
||||
|
||||
/**
|
||||
* Docker runtime adapter
|
||||
* Executes shell test files inside Docker containers
|
||||
* Pattern: test.{variant}.docker.sh
|
||||
* Variants map to Dockerfiles: latest -> Dockerfile, others -> Dockerfile_{variant}
|
||||
*/
|
||||
export class DockerRuntimeAdapter extends RuntimeAdapter {
|
||||
readonly id: Runtime = 'node'; // Using 'node' temporarily as Runtime type doesn't include 'docker'
|
||||
readonly displayName: string = 'Docker';
|
||||
|
||||
private builtImages: Set<string> = new Set(); // Track built images to avoid rebuilding
|
||||
|
||||
constructor(
|
||||
private logger: TsTestLogger,
|
||||
private smartshellInstance: any, // SmartShell instance from @push.rocks/smartshell
|
||||
private timeoutSeconds: number | null,
|
||||
private cwd: string
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Docker CLI is available
|
||||
*/
|
||||
async checkAvailable(): Promise<RuntimeAvailability> {
|
||||
try {
|
||||
const result = await this.smartshellInstance.exec('docker --version');
|
||||
|
||||
if (result.exitCode !== 0) {
|
||||
return {
|
||||
available: false,
|
||||
error: 'Docker command failed',
|
||||
};
|
||||
}
|
||||
|
||||
// Extract version from output like "Docker version 24.0.5, build ced0996"
|
||||
const versionMatch = result.stdout.match(/Docker version ([^,]+)/);
|
||||
const version = versionMatch ? versionMatch[1] : 'unknown';
|
||||
|
||||
return {
|
||||
available: true,
|
||||
version,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
available: false,
|
||||
error: `Docker not found: ${error.message}`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create command configuration for Docker test execution
|
||||
* This is used for informational purposes
|
||||
*/
|
||||
createCommand(testFile: string, options?: RuntimeOptions): RuntimeCommand {
|
||||
const parsed = parseDockerTestFilename(testFile);
|
||||
const dockerfilePath = mapVariantToDockerfile(parsed.variant, this.cwd);
|
||||
const imageName = `tstest-${parsed.variant}`;
|
||||
|
||||
return {
|
||||
command: 'docker',
|
||||
args: [
|
||||
'run',
|
||||
'--rm',
|
||||
'-v',
|
||||
`${this.cwd}/test:/test`,
|
||||
imageName,
|
||||
'taprun',
|
||||
`/test/${plugins.path.basename(testFile)}`
|
||||
],
|
||||
env: {},
|
||||
cwd: this.cwd,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a Docker image from the specified Dockerfile
|
||||
*/
|
||||
private async buildDockerImage(dockerfilePath: string, imageName: string): Promise<void> {
|
||||
// Check if image is already built
|
||||
if (this.builtImages.has(imageName)) {
|
||||
this.logger.tapOutput(`Using cached Docker image: ${imageName}`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Check if Dockerfile exists
|
||||
if (!await plugins.smartfile.fs.fileExists(dockerfilePath)) {
|
||||
throw new Error(
|
||||
`Dockerfile not found: ${dockerfilePath}\n` +
|
||||
`Expected Dockerfile for Docker test variant.`
|
||||
);
|
||||
}
|
||||
|
||||
this.logger.tapOutput(`Building Docker image: ${imageName} from ${dockerfilePath}`);
|
||||
|
||||
try {
|
||||
const buildResult = await this.smartshellInstance.exec(
|
||||
`docker build -f ${dockerfilePath} -t ${imageName} ${this.cwd}`,
|
||||
{
|
||||
cwd: this.cwd,
|
||||
}
|
||||
);
|
||||
|
||||
if (buildResult.exitCode !== 0) {
|
||||
throw new Error(`Docker build failed:\n${buildResult.stderr}`);
|
||||
}
|
||||
|
||||
this.builtImages.add(imageName);
|
||||
this.logger.tapOutput(`✅ Docker image built successfully: ${imageName}`);
|
||||
} catch (error) {
|
||||
throw new Error(`Failed to build Docker image: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a Docker test file
|
||||
*/
|
||||
async run(
|
||||
testFile: string,
|
||||
index: number,
|
||||
total: number,
|
||||
options?: RuntimeOptions
|
||||
): Promise<TapParser> {
|
||||
this.logger.testFileStart(testFile, this.displayName, index, total);
|
||||
|
||||
// Parse the Docker test filename
|
||||
const parsed = parseDockerTestFilename(testFile);
|
||||
const dockerfilePath = mapVariantToDockerfile(parsed.variant, this.cwd);
|
||||
const imageName = `tstest-${parsed.variant}`;
|
||||
|
||||
// Build the Docker image
|
||||
await this.buildDockerImage(dockerfilePath, imageName);
|
||||
|
||||
// Prepare the test file path relative to the mounted directory
|
||||
// We need to get the path relative to cwd
|
||||
const absoluteTestPath = plugins.path.isAbsolute(testFile)
|
||||
? testFile
|
||||
: plugins.path.join(this.cwd, testFile);
|
||||
|
||||
const relativeTestPath = plugins.path.relative(this.cwd, absoluteTestPath);
|
||||
|
||||
// Create TAP parser
|
||||
const tapParser = new TapParser(testFile + ':docker', this.logger);
|
||||
|
||||
try {
|
||||
// Build docker run command
|
||||
const dockerArgs = [
|
||||
'run',
|
||||
'--rm',
|
||||
'-v',
|
||||
`${this.cwd}/test:/test`,
|
||||
imageName,
|
||||
'taprun',
|
||||
`/test/${plugins.path.basename(testFile)}`
|
||||
];
|
||||
|
||||
this.logger.tapOutput(`Executing: docker ${dockerArgs.join(' ')}`);
|
||||
|
||||
// Execute the Docker container
|
||||
const execPromise = this.smartshellInstance.execStreaming(
|
||||
`docker ${dockerArgs.join(' ')}`,
|
||||
{
|
||||
cwd: this.cwd,
|
||||
}
|
||||
);
|
||||
|
||||
// Set up timeout if configured
|
||||
let timeoutHandle: NodeJS.Timeout | null = null;
|
||||
if (this.timeoutSeconds) {
|
||||
timeoutHandle = setTimeout(() => {
|
||||
this.logger.tapOutput(`⏱️ Test timeout (${this.timeoutSeconds}s) - killing container`);
|
||||
// Try to kill any running containers with this image
|
||||
this.smartshellInstance.exec(`docker ps -q --filter ancestor=${imageName} | xargs -r docker kill`);
|
||||
}, this.timeoutSeconds * 1000);
|
||||
}
|
||||
|
||||
// Stream output to TAP parser line by line
|
||||
execPromise.childProcess.stdout.on('data', (data: Buffer) => {
|
||||
const output = data.toString();
|
||||
const lines = output.split('\n');
|
||||
for (const line of lines) {
|
||||
if (line.trim()) {
|
||||
tapParser.handleTapLog(line);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
execPromise.childProcess.stderr.on('data', (data: Buffer) => {
|
||||
const output = data.toString();
|
||||
this.logger.tapOutput(cs(`[stderr] ${output}`, 'orange'));
|
||||
});
|
||||
|
||||
// Wait for completion
|
||||
const result = await execPromise;
|
||||
|
||||
// Clear timeout
|
||||
if (timeoutHandle) {
|
||||
clearTimeout(timeoutHandle);
|
||||
}
|
||||
|
||||
if (result.exitCode !== 0) {
|
||||
this.logger.tapOutput(cs(`❌ Docker test failed with exit code ${result.exitCode}`, 'red'));
|
||||
}
|
||||
|
||||
// Evaluate final result
|
||||
await tapParser.evaluateFinalResult();
|
||||
|
||||
} catch (error) {
|
||||
this.logger.tapOutput(cs(`❌ Error running Docker test: ${error.message}`, 'red'));
|
||||
// Add a failing test result to the parser
|
||||
tapParser.handleTapLog('not ok 1 - Docker test execution failed');
|
||||
await tapParser.evaluateFinalResult();
|
||||
}
|
||||
|
||||
return tapParser;
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up built Docker images (optional, can be called at end of test suite)
|
||||
*/
|
||||
async cleanup(): Promise<void> {
|
||||
for (const imageName of this.builtImages) {
|
||||
try {
|
||||
this.logger.tapOutput(`Removing Docker image: ${imageName}`);
|
||||
await this.smartshellInstance.exec(`docker rmi ${imageName}`);
|
||||
} catch (error) {
|
||||
// Ignore cleanup errors
|
||||
this.logger.tapOutput(cs(`Warning: Failed to remove image ${imageName}: ${error.message}`, 'orange'));
|
||||
}
|
||||
}
|
||||
this.builtImages.clear();
|
||||
}
|
||||
}
|
||||
224
ts/tstest.classes.runtime.node.ts
Normal file
224
ts/tstest.classes.runtime.node.ts
Normal file
@@ -0,0 +1,224 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import { coloredString as cs } from '@push.rocks/consolecolor';
|
||||
import {
|
||||
RuntimeAdapter,
|
||||
type RuntimeOptions,
|
||||
type RuntimeCommand,
|
||||
type RuntimeAvailability,
|
||||
} from './tstest.classes.runtime.adapter.js';
|
||||
import { TapParser } from './tstest.classes.tap.parser.js';
|
||||
import { TsTestLogger } from './tstest.logging.js';
|
||||
import type { Runtime } from './tstest.classes.runtime.parser.js';
|
||||
|
||||
/**
|
||||
* Node.js runtime adapter
|
||||
* Executes tests using tsrun (TypeScript runner for Node.js)
|
||||
*/
|
||||
export class NodeRuntimeAdapter extends RuntimeAdapter {
|
||||
readonly id: Runtime = 'node';
|
||||
readonly displayName: string = 'Node.js';
|
||||
|
||||
constructor(
|
||||
private logger: TsTestLogger,
|
||||
private smartshellInstance: any, // SmartShell instance from @push.rocks/smartshell
|
||||
private timeoutSeconds: number | null,
|
||||
private filterTags: string[]
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Node.js and tsrun are available
|
||||
*/
|
||||
async checkAvailable(): Promise<RuntimeAvailability> {
|
||||
try {
|
||||
// Check Node.js version
|
||||
const nodeVersion = process.version;
|
||||
|
||||
// Check if tsrun module is available (imported as dependency)
|
||||
if (!plugins.tsrun || !plugins.tsrun.spawnPath) {
|
||||
return {
|
||||
available: false,
|
||||
error: 'tsrun module not found or outdated (requires version 1.6.0+)',
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
available: true,
|
||||
version: nodeVersion,
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
available: false,
|
||||
error: error.message,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create command configuration for Node.js test execution
|
||||
*/
|
||||
createCommand(testFile: string, options?: RuntimeOptions): RuntimeCommand {
|
||||
const mergedOptions = this.mergeOptions(options);
|
||||
|
||||
// Build tsrun options
|
||||
const args: string[] = [];
|
||||
|
||||
if (process.argv.includes('--web')) {
|
||||
args.push('--web');
|
||||
}
|
||||
|
||||
// Add any extra args
|
||||
if (mergedOptions.extraArgs) {
|
||||
args.push(...mergedOptions.extraArgs);
|
||||
}
|
||||
|
||||
// Set environment variables
|
||||
const env = { ...mergedOptions.env };
|
||||
|
||||
if (this.filterTags.length > 0) {
|
||||
env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
|
||||
}
|
||||
|
||||
return {
|
||||
command: 'tsrun',
|
||||
args: [testFile, ...args],
|
||||
env,
|
||||
cwd: mergedOptions.cwd,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a test file in Node.js using tsrun's spawnPath API
|
||||
*/
|
||||
async run(
|
||||
testFile: string,
|
||||
index: number,
|
||||
total: number,
|
||||
options?: RuntimeOptions
|
||||
): Promise<TapParser> {
|
||||
this.logger.testFileStart(testFile, this.displayName, index, total);
|
||||
const tapParser = new TapParser(testFile + ':node', this.logger);
|
||||
|
||||
const mergedOptions = this.mergeOptions(options);
|
||||
|
||||
// Build spawn options
|
||||
const spawnOptions: any = {
|
||||
cwd: mergedOptions.cwd || process.cwd(),
|
||||
env: { ...mergedOptions.env },
|
||||
args: [] as string[],
|
||||
stdio: 'pipe' as const,
|
||||
};
|
||||
|
||||
// Add --web flag if needed
|
||||
if (process.argv.includes('--web')) {
|
||||
spawnOptions.args.push('--web');
|
||||
}
|
||||
|
||||
// Set filter tags as environment variable
|
||||
if (this.filterTags.length > 0) {
|
||||
spawnOptions.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
|
||||
}
|
||||
|
||||
// Check for 00init.ts file in test directory
|
||||
const testDir = plugins.path.dirname(testFile);
|
||||
const initFile = plugins.path.join(testDir, '00init.ts');
|
||||
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
|
||||
|
||||
// Determine which file to run
|
||||
let fileToRun = testFile;
|
||||
let loaderPath: string | null = null;
|
||||
|
||||
// If 00init.ts exists, create a loader file
|
||||
if (initFileExists) {
|
||||
const absoluteInitFile = plugins.path.resolve(initFile);
|
||||
const absoluteTestFile = plugins.path.resolve(testFile);
|
||||
const loaderContent = `
|
||||
import '${absoluteInitFile.replace(/\\/g, '/')}';
|
||||
import '${absoluteTestFile.replace(/\\/g, '/')}';
|
||||
`;
|
||||
loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(testFile)}`);
|
||||
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
|
||||
fileToRun = loaderPath;
|
||||
}
|
||||
|
||||
// Spawn the test process using tsrun's spawnPath API
|
||||
// Pass undefined for fromFileUrl since fileToRun is already an absolute path
|
||||
const tsrunProcess = plugins.tsrun.spawnPath(fileToRun, undefined, spawnOptions);
|
||||
|
||||
// If we created a loader file, clean it up after test execution
|
||||
if (loaderPath) {
|
||||
const cleanup = () => {
|
||||
try {
|
||||
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
|
||||
plugins.smartfile.fs.removeSync(loaderPath);
|
||||
}
|
||||
} catch (e) {
|
||||
// Ignore cleanup errors
|
||||
}
|
||||
};
|
||||
|
||||
tsrunProcess.childProcess.on('exit', cleanup);
|
||||
tsrunProcess.childProcess.on('error', cleanup);
|
||||
}
|
||||
|
||||
// Start warning timer if no timeout was specified
|
||||
let warningTimer: NodeJS.Timeout | null = null;
|
||||
if (this.timeoutSeconds === null) {
|
||||
warningTimer = setTimeout(() => {
|
||||
console.error('');
|
||||
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
|
||||
console.error(cs(` File: ${testFile}`, 'orange'));
|
||||
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
|
||||
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
|
||||
console.error('');
|
||||
}, 60000); // 1 minute
|
||||
}
|
||||
|
||||
// Handle timeout if specified
|
||||
if (this.timeoutSeconds !== null) {
|
||||
const timeoutMs = this.timeoutSeconds * 1000;
|
||||
let timeoutId: NodeJS.Timeout;
|
||||
|
||||
const timeoutPromise = new Promise<void>((_resolve, reject) => {
|
||||
timeoutId = setTimeout(async () => {
|
||||
// Use tsrun's terminate() to gracefully kill the process
|
||||
await tsrunProcess.terminate();
|
||||
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
|
||||
}, timeoutMs);
|
||||
});
|
||||
|
||||
try {
|
||||
await Promise.race([
|
||||
tapParser.handleTapProcess(tsrunProcess.childProcess),
|
||||
timeoutPromise
|
||||
]);
|
||||
// Clear timeout if test completed successfully
|
||||
clearTimeout(timeoutId);
|
||||
} catch (error) {
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
// Handle timeout error
|
||||
tapParser.handleTimeout(this.timeoutSeconds);
|
||||
// Ensure process is killed if still running
|
||||
try {
|
||||
tsrunProcess.kill('SIGKILL');
|
||||
} catch (killError) {
|
||||
// Process might already be dead
|
||||
}
|
||||
await tapParser.evaluateFinalResult();
|
||||
}
|
||||
} else {
|
||||
await tapParser.handleTapProcess(tsrunProcess.childProcess);
|
||||
}
|
||||
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
|
||||
return tapParser;
|
||||
}
|
||||
}
|
||||
308
ts/tstest.classes.runtime.parser.ts
Normal file
308
ts/tstest.classes.runtime.parser.ts
Normal file
@@ -0,0 +1,308 @@
|
||||
/**
|
||||
* Runtime parser for test file naming convention
|
||||
* Supports: test.runtime1+runtime2.modifier.ts
|
||||
* Examples:
|
||||
* - test.node.ts
|
||||
* - test.chromium.ts
|
||||
* - test.node+chromium.ts
|
||||
* - test.deno+bun.ts
|
||||
* - test.all.ts (runs on all runtimes)
|
||||
* - test.chromium.nonci.ts
|
||||
*/
|
||||
|
||||
export type Runtime = 'node' | 'chromium' | 'deno' | 'bun';
|
||||
export type Modifier = 'nonci';
|
||||
|
||||
export interface ParsedFilename {
|
||||
baseName: string;
|
||||
runtimes: Runtime[];
|
||||
modifiers: Modifier[];
|
||||
extension: string;
|
||||
isLegacy: boolean;
|
||||
original: string;
|
||||
}
|
||||
|
||||
export interface ParserConfig {
|
||||
strictUnknownRuntime?: boolean; // default: true
|
||||
defaultRuntimes?: Runtime[]; // default: ['node']
|
||||
}
|
||||
|
||||
const KNOWN_RUNTIMES: Set<string> = new Set(['node', 'chromium', 'deno', 'bun']);
|
||||
const KNOWN_MODIFIERS: Set<string> = new Set(['nonci']);
|
||||
const VALID_EXTENSIONS: Set<string> = new Set(['ts', 'tsx', 'mts', 'cts', 'sh']);
|
||||
const ALL_RUNTIMES: Runtime[] = ['node', 'chromium', 'deno', 'bun'];
|
||||
|
||||
// Legacy mappings for backwards compatibility
|
||||
const LEGACY_RUNTIME_MAP: Record<string, Runtime[]> = {
|
||||
browser: ['chromium'],
|
||||
both: ['node', 'chromium'],
|
||||
};
|
||||
|
||||
/**
|
||||
* Parse a test filename to extract runtimes, modifiers, and detect legacy patterns
|
||||
* Algorithm: Right-to-left token analysis from the extension
|
||||
*/
|
||||
export function parseTestFilename(
|
||||
filePath: string,
|
||||
config: ParserConfig = {}
|
||||
): ParsedFilename {
|
||||
const strictUnknownRuntime = config.strictUnknownRuntime ?? true;
|
||||
const defaultRuntimes = config.defaultRuntimes ?? ['node'];
|
||||
|
||||
// Extract just the filename from the path
|
||||
const fileName = filePath.split('/').pop() || filePath;
|
||||
const original = fileName;
|
||||
|
||||
// Step 1: Extract and validate extension
|
||||
const lastDot = fileName.lastIndexOf('.');
|
||||
if (lastDot === -1) {
|
||||
throw new Error(`Invalid test file: no extension found in "${fileName}"`);
|
||||
}
|
||||
|
||||
const extension = fileName.substring(lastDot + 1);
|
||||
if (!VALID_EXTENSIONS.has(extension)) {
|
||||
throw new Error(
|
||||
`Invalid test file extension ".${extension}" in "${fileName}". ` +
|
||||
`Valid extensions: ${Array.from(VALID_EXTENSIONS).join(', ')}`
|
||||
);
|
||||
}
|
||||
|
||||
// Step 2: Split remaining basename by dots
|
||||
const withoutExtension = fileName.substring(0, lastDot);
|
||||
const tokens = withoutExtension.split('.');
|
||||
|
||||
if (tokens.length === 0) {
|
||||
throw new Error(`Invalid test file: empty basename in "${fileName}"`);
|
||||
}
|
||||
|
||||
// Step 3: Parse from right to left
|
||||
let isLegacy = false;
|
||||
const modifiers: Modifier[] = [];
|
||||
let runtimes: Runtime[] = [];
|
||||
let runtimeTokenIndex = -1;
|
||||
|
||||
// Scan from right to left
|
||||
for (let i = tokens.length - 1; i >= 0; i--) {
|
||||
const token = tokens[i];
|
||||
|
||||
// Check if this is a known modifier
|
||||
if (KNOWN_MODIFIERS.has(token)) {
|
||||
modifiers.unshift(token as Modifier);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Check if this is a legacy runtime token
|
||||
if (LEGACY_RUNTIME_MAP[token]) {
|
||||
isLegacy = true;
|
||||
runtimes = LEGACY_RUNTIME_MAP[token];
|
||||
runtimeTokenIndex = i;
|
||||
break;
|
||||
}
|
||||
|
||||
// Check if this is a runtime chain (may contain + separators)
|
||||
if (token.includes('+')) {
|
||||
const runtimeCandidates = token.split('+').map(r => r.trim()).filter(Boolean);
|
||||
const validRuntimes: Runtime[] = [];
|
||||
const invalidRuntimes: string[] = [];
|
||||
let hasAllKeyword = false;
|
||||
|
||||
for (const candidate of runtimeCandidates) {
|
||||
if (candidate === 'all') {
|
||||
hasAllKeyword = true;
|
||||
} else if (KNOWN_RUNTIMES.has(candidate)) {
|
||||
// Dedupe: only add if not already in list
|
||||
if (!validRuntimes.includes(candidate as Runtime)) {
|
||||
validRuntimes.push(candidate as Runtime);
|
||||
}
|
||||
} else {
|
||||
invalidRuntimes.push(candidate);
|
||||
}
|
||||
}
|
||||
|
||||
// If 'all' keyword is present, expand to all runtimes
|
||||
if (hasAllKeyword) {
|
||||
runtimes = [...ALL_RUNTIMES];
|
||||
runtimeTokenIndex = i;
|
||||
break;
|
||||
}
|
||||
|
||||
if (invalidRuntimes.length > 0) {
|
||||
if (strictUnknownRuntime) {
|
||||
throw new Error(
|
||||
`Unknown runtime(s) in "${fileName}": ${invalidRuntimes.join(', ')}. ` +
|
||||
`Valid runtimes: ${Array.from(KNOWN_RUNTIMES).join(', ')}, all`
|
||||
);
|
||||
} else {
|
||||
console.warn(
|
||||
`⚠️ Warning: Unknown runtime(s) in "${fileName}": ${invalidRuntimes.join(', ')}. ` +
|
||||
`Defaulting to: ${defaultRuntimes.join('+')}`
|
||||
);
|
||||
runtimes = [...defaultRuntimes];
|
||||
runtimeTokenIndex = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (validRuntimes.length > 0) {
|
||||
runtimes = validRuntimes;
|
||||
runtimeTokenIndex = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Check if this is the 'all' keyword (expands to all runtimes)
|
||||
if (token === 'all') {
|
||||
runtimes = [...ALL_RUNTIMES];
|
||||
runtimeTokenIndex = i;
|
||||
break;
|
||||
}
|
||||
|
||||
// Check if this is a single runtime token
|
||||
if (KNOWN_RUNTIMES.has(token)) {
|
||||
runtimes = [token as Runtime];
|
||||
runtimeTokenIndex = i;
|
||||
break;
|
||||
}
|
||||
|
||||
// If we've scanned past modifiers and haven't found a runtime, stop looking
|
||||
if (modifiers.length > 0) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Step 4: Determine base name
|
||||
// Everything before the runtime token (if found) is the base name
|
||||
const baseNameTokens = runtimeTokenIndex >= 0 ? tokens.slice(0, runtimeTokenIndex) : tokens;
|
||||
const baseName = baseNameTokens.join('.');
|
||||
|
||||
// Step 5: Apply defaults if no runtime was detected
|
||||
if (runtimes.length === 0) {
|
||||
runtimes = [...defaultRuntimes];
|
||||
}
|
||||
|
||||
return {
|
||||
baseName: baseName || 'test',
|
||||
runtimes,
|
||||
modifiers,
|
||||
extension,
|
||||
isLegacy,
|
||||
original,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a filename uses legacy naming convention
|
||||
*/
|
||||
export function isLegacyFilename(fileName: string): boolean {
|
||||
const tokens = fileName.split('.');
|
||||
for (const token of tokens) {
|
||||
if (LEGACY_RUNTIME_MAP[token]) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the suggested new filename for a legacy filename
|
||||
*/
|
||||
export function getLegacyMigrationTarget(fileName: string): string | null {
|
||||
const parsed = parseTestFilename(fileName, { strictUnknownRuntime: false });
|
||||
|
||||
if (!parsed.isLegacy) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Reconstruct filename with new naming
|
||||
const parts = [parsed.baseName];
|
||||
|
||||
if (parsed.runtimes.length > 0) {
|
||||
parts.push(parsed.runtimes.join('+'));
|
||||
}
|
||||
|
||||
if (parsed.modifiers.length > 0) {
|
||||
parts.push(...parsed.modifiers);
|
||||
}
|
||||
|
||||
parts.push(parsed.extension);
|
||||
|
||||
return parts.join('.');
|
||||
}
|
||||
|
||||
/**
|
||||
* Docker test file information
|
||||
*/
|
||||
export interface DockerTestFileInfo {
|
||||
baseName: string;
|
||||
variant: string;
|
||||
isDockerTest: true;
|
||||
original: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a filename matches the Docker test pattern: *.{variant}.docker.sh
|
||||
* Examples: test.latest.docker.sh, test.integration.npmci.docker.sh
|
||||
*/
|
||||
export function isDockerTestFile(fileName: string): boolean {
|
||||
// Must end with .docker.sh
|
||||
if (!fileName.endsWith('.docker.sh')) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Extract filename from path if needed
|
||||
const name = fileName.split('/').pop() || fileName;
|
||||
|
||||
// Must have at least 3 parts: [baseName, variant, docker, sh]
|
||||
const parts = name.split('.');
|
||||
return parts.length >= 4 && parts[parts.length - 2] === 'docker' && parts[parts.length - 1] === 'sh';
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a Docker test filename to extract variant and base name
|
||||
* Pattern: test.{baseName}.{variant}.docker.sh
|
||||
* Examples:
|
||||
* - test.latest.docker.sh -> { baseName: 'test', variant: 'latest' }
|
||||
* - test.integration.npmci.docker.sh -> { baseName: 'test.integration', variant: 'npmci' }
|
||||
*/
|
||||
export function parseDockerTestFilename(filePath: string): DockerTestFileInfo {
|
||||
// Extract just the filename from the path
|
||||
const fileName = filePath.split('/').pop() || filePath;
|
||||
const original = fileName;
|
||||
|
||||
if (!isDockerTestFile(fileName)) {
|
||||
throw new Error(`Not a valid Docker test file: "${fileName}". Expected pattern: *.{variant}.docker.sh`);
|
||||
}
|
||||
|
||||
// Remove .docker.sh suffix
|
||||
const withoutSuffix = fileName.slice(0, -10); // Remove '.docker.sh'
|
||||
const tokens = withoutSuffix.split('.');
|
||||
|
||||
if (tokens.length === 0) {
|
||||
throw new Error(`Invalid Docker test file: empty basename in "${fileName}"`);
|
||||
}
|
||||
|
||||
// Last token before .docker.sh is the variant
|
||||
const variant = tokens[tokens.length - 1];
|
||||
|
||||
// Everything else is the base name
|
||||
const baseName = tokens.slice(0, -1).join('.');
|
||||
|
||||
return {
|
||||
baseName: baseName || 'test',
|
||||
variant,
|
||||
isDockerTest: true,
|
||||
original,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Map a Docker variant to its corresponding Dockerfile path
|
||||
* "latest" -> "Dockerfile"
|
||||
* Other variants -> "Dockerfile_{variant}"
|
||||
*/
|
||||
export function mapVariantToDockerfile(variant: string, baseDir: string): string {
|
||||
if (variant === 'latest') {
|
||||
return `${baseDir}/Dockerfile`;
|
||||
}
|
||||
return `${baseDir}/Dockerfile_${variant}`;
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import { TsTestLogger } from './tstest.logging.js';
|
||||
|
||||
export class TapCombinator {
|
||||
tapParserStore: TapParser[] = [];
|
||||
skippedFiles: string[] = [];
|
||||
private logger: TsTestLogger;
|
||||
|
||||
constructor(logger: TsTestLogger) {
|
||||
@@ -19,10 +20,14 @@ export class TapCombinator {
|
||||
addTapParser(tapParserArg: TapParser) {
|
||||
this.tapParserStore.push(tapParserArg);
|
||||
}
|
||||
|
||||
addSkippedFile(filename: string) {
|
||||
this.skippedFiles.push(filename);
|
||||
}
|
||||
|
||||
evaluate() {
|
||||
// Call the logger's summary method
|
||||
this.logger.summary();
|
||||
// Call the logger's summary method with skipped files
|
||||
this.logger.summary(this.skippedFiles);
|
||||
|
||||
// Check for failures
|
||||
let failGlobal = false;
|
||||
|
||||
@@ -8,28 +8,59 @@ import * as plugins from './tstest.plugins.js';
|
||||
import { TapTestResult } from './tstest.classes.tap.testresult.js';
|
||||
import * as logPrefixes from './tstest.logprefixes.js';
|
||||
import { TsTestLogger } from './tstest.logging.js';
|
||||
import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
import type { IProtocolMessage, ITestResult, IPlanLine, IErrorBlock, ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
|
||||
export class TapParser {
|
||||
testStore: TapTestResult[] = [];
|
||||
|
||||
expectedTestsRegex = /([0-9]*)\.\.([0-9]*)$/;
|
||||
expectedTests: number;
|
||||
receivedTests: number;
|
||||
expectedTests: number = 0;
|
||||
receivedTests: number = 0;
|
||||
|
||||
testStatusRegex = /(ok|not\sok)\s([0-9]+)\s-\s(.*)(\s#\s(.*))?$/;
|
||||
activeTapTestResult: TapTestResult;
|
||||
collectingErrorDetails: boolean = false;
|
||||
currentTestError: string[] = [];
|
||||
|
||||
pretaskRegex = /^::__PRETASK:(.*)$/;
|
||||
|
||||
private logger: TsTestLogger;
|
||||
private protocolParser: ProtocolParser;
|
||||
private protocolVersion: string | null = null;
|
||||
private startTime: number;
|
||||
|
||||
/**
|
||||
* the constructor for TapParser
|
||||
*/
|
||||
constructor(public fileName: string, logger?: TsTestLogger) {
|
||||
this.logger = logger;
|
||||
this.protocolParser = new ProtocolParser();
|
||||
this.startTime = Date.now();
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle test file timeout
|
||||
*/
|
||||
public handleTimeout(timeoutSeconds: number) {
|
||||
// If no tests have been defined yet, set expected to 1
|
||||
if (this.expectedTests === 0) {
|
||||
this.expectedTests = 1;
|
||||
}
|
||||
|
||||
// Create a fake failing test result for timeout
|
||||
this._getNewTapTestResult();
|
||||
this.activeTapTestResult.testOk = false;
|
||||
this.activeTapTestResult.testSettled = true;
|
||||
this.testStore.push(this.activeTapTestResult);
|
||||
|
||||
// Log the timeout error
|
||||
if (this.logger) {
|
||||
// First log the test result
|
||||
this.logger.testResult(
|
||||
`Test file timeout`,
|
||||
false,
|
||||
timeoutSeconds * 1000,
|
||||
`Error: Test file exceeded timeout of ${timeoutSeconds} seconds`
|
||||
);
|
||||
this.logger.testErrorDetails(`Test execution was terminated after ${timeoutSeconds} seconds`);
|
||||
}
|
||||
|
||||
// Don't call evaluateFinalResult here, let the caller handle it
|
||||
}
|
||||
|
||||
private _getNewTapTestResult() {
|
||||
@@ -45,142 +76,299 @@ export class TapParser {
|
||||
logLineArray.pop();
|
||||
}
|
||||
|
||||
// lets parse the log information
|
||||
// Process each line through the protocol parser
|
||||
for (const logLine of logLineArray) {
|
||||
let logLineIsTapProtocol = false;
|
||||
if (!this.expectedTests && this.expectedTestsRegex.test(logLine)) {
|
||||
logLineIsTapProtocol = true;
|
||||
const regexResult = this.expectedTestsRegex.exec(logLine);
|
||||
this.expectedTests = parseInt(regexResult[2]);
|
||||
if (this.logger) {
|
||||
this.logger.tapOutput(`Expecting ${this.expectedTests} tests!`);
|
||||
const messages = this.protocolParser.parseLine(logLine);
|
||||
|
||||
if (messages.length > 0) {
|
||||
// Handle protocol messages
|
||||
for (const message of messages) {
|
||||
this._handleProtocolMessage(message, logLine);
|
||||
}
|
||||
|
||||
// initiating first TapResult
|
||||
this._getNewTapTestResult();
|
||||
} else if (this.pretaskRegex.test(logLine)) {
|
||||
logLineIsTapProtocol = true;
|
||||
const pretaskContentMatch = this.pretaskRegex.exec(logLine);
|
||||
if (pretaskContentMatch && pretaskContentMatch[1]) {
|
||||
if (this.logger) {
|
||||
this.logger.tapOutput(`Pretask -> ${pretaskContentMatch[1]}: Success.`);
|
||||
}
|
||||
}
|
||||
} else if (this.testStatusRegex.test(logLine)) {
|
||||
logLineIsTapProtocol = true;
|
||||
const regexResult = this.testStatusRegex.exec(logLine);
|
||||
const testId = parseInt(regexResult[2]);
|
||||
const testOk = (() => {
|
||||
if (regexResult[1] === 'ok') {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
})();
|
||||
|
||||
const testSubject = regexResult[3];
|
||||
const testMetadata = regexResult[5]; // This will be either "time=XXXms" or "SKIP reason" or "TODO reason"
|
||||
|
||||
let testDuration = 0;
|
||||
let isSkipped = false;
|
||||
let isTodo = false;
|
||||
|
||||
if (testMetadata) {
|
||||
const timeMatch = testMetadata.match(/time=(\d+)ms/);
|
||||
const skipMatch = testMetadata.match(/SKIP\s*(.*)/);
|
||||
const todoMatch = testMetadata.match(/TODO\s*(.*)/);
|
||||
|
||||
if (timeMatch) {
|
||||
testDuration = parseInt(timeMatch[1]);
|
||||
} else if (skipMatch) {
|
||||
isSkipped = true;
|
||||
} else if (todoMatch) {
|
||||
isTodo = true;
|
||||
}
|
||||
}
|
||||
|
||||
// test for protocol error - disabled as it's not critical
|
||||
// The test ID mismatch can occur when tests are filtered, skipped, or use todo
|
||||
// if (testId !== this.activeTapTestResult.id) {
|
||||
// if (this.logger) {
|
||||
// this.logger.error('Something is strange! Test Ids are not equal!');
|
||||
// }
|
||||
// }
|
||||
this.activeTapTestResult.setTestResult(testOk);
|
||||
|
||||
if (testOk) {
|
||||
if (this.logger) {
|
||||
this.logger.testResult(testSubject, true, testDuration);
|
||||
}
|
||||
} else {
|
||||
// Start collecting error details for failed test
|
||||
this.collectingErrorDetails = true;
|
||||
this.currentTestError = [];
|
||||
if (this.logger) {
|
||||
this.logger.testResult(testSubject, false, testDuration);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!logLineIsTapProtocol) {
|
||||
} else {
|
||||
// Not a protocol message, handle as console output
|
||||
if (this.activeTapTestResult) {
|
||||
this.activeTapTestResult.addLogLine(logLine);
|
||||
}
|
||||
|
||||
// Check for snapshot communication
|
||||
// Check for snapshot communication (legacy)
|
||||
const snapshotMatch = logLine.match(/###SNAPSHOT###(.+)###SNAPSHOT###/);
|
||||
if (snapshotMatch) {
|
||||
const base64Data = snapshotMatch[1];
|
||||
try {
|
||||
const snapshotData = JSON.parse(Buffer.from(base64Data, 'base64').toString());
|
||||
this.handleSnapshot(snapshotData);
|
||||
} catch (error) {
|
||||
} catch (error: any) {
|
||||
if (this.logger) {
|
||||
this.logger.testConsoleOutput(`Error parsing snapshot data: ${error.message}`);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Check if we're collecting error details
|
||||
if (this.collectingErrorDetails) {
|
||||
// Check if this line is an error detail (starts with Error: or has stack trace characteristics)
|
||||
if (logLine.trim().startsWith('Error:') || logLine.trim().match(/^\s*at\s/)) {
|
||||
this.currentTestError.push(logLine);
|
||||
} else if (this.currentTestError.length > 0) {
|
||||
// End of error details, show the error
|
||||
const errorMessage = this.currentTestError.join('\n');
|
||||
if (this.logger) {
|
||||
this.logger.testErrorDetails(errorMessage);
|
||||
}
|
||||
this.collectingErrorDetails = false;
|
||||
this.currentTestError = [];
|
||||
}
|
||||
}
|
||||
|
||||
// Don't output TAP error details as console output when we're collecting them
|
||||
if (!this.collectingErrorDetails || (!logLine.trim().startsWith('Error:') && !logLine.trim().match(/^\s*at\s/))) {
|
||||
if (this.logger) {
|
||||
// This is console output from the test file, not TAP protocol
|
||||
this.logger.testConsoleOutput(logLine);
|
||||
}
|
||||
}
|
||||
} else if (this.logger) {
|
||||
// This is console output from the test file
|
||||
this.logger.testConsoleOutput(logLine);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (this.activeTapTestResult && this.activeTapTestResult.testSettled) {
|
||||
// Ensure any pending error is shown before settling the test
|
||||
if (this.collectingErrorDetails && this.currentTestError.length > 0) {
|
||||
const errorMessage = this.currentTestError.join('\n');
|
||||
private _handleProtocolMessage(message: IProtocolMessage, originalLine: string) {
|
||||
switch (message.type) {
|
||||
case 'protocol':
|
||||
this.protocolVersion = message.content.version;
|
||||
if (this.logger) {
|
||||
this.logger.tapOutput(`Protocol version: ${this.protocolVersion}`);
|
||||
}
|
||||
break;
|
||||
|
||||
case 'version':
|
||||
// TAP version, we can ignore this
|
||||
break;
|
||||
|
||||
case 'plan':
|
||||
const plan = message.content as IPlanLine;
|
||||
this.expectedTests = plan.end - plan.start + 1;
|
||||
if (plan.skipAll) {
|
||||
if (this.logger) {
|
||||
this.logger.testErrorDetails(errorMessage);
|
||||
this.logger.tapOutput(`Skipping all tests: ${plan.skipAll}`);
|
||||
}
|
||||
this.collectingErrorDetails = false;
|
||||
this.currentTestError = [];
|
||||
} else {
|
||||
if (this.logger) {
|
||||
this.logger.tapOutput(`Expecting ${this.expectedTests} tests!`);
|
||||
}
|
||||
}
|
||||
// Initialize first TapResult
|
||||
this._getNewTapTestResult();
|
||||
break;
|
||||
|
||||
case 'test':
|
||||
const testResult = message.content as ITestResult;
|
||||
|
||||
// Update active test result
|
||||
this.activeTapTestResult.setTestResult(testResult.ok);
|
||||
|
||||
// Extract test duration from metadata
|
||||
let testDuration = 0;
|
||||
if (testResult.metadata?.time) {
|
||||
testDuration = testResult.metadata.time;
|
||||
}
|
||||
|
||||
// Log test result
|
||||
if (this.logger) {
|
||||
if (testResult.ok) {
|
||||
this.logger.testResult(testResult.description, true, testDuration);
|
||||
} else {
|
||||
this.logger.testResult(testResult.description, false, testDuration);
|
||||
|
||||
// If there's error metadata, show it
|
||||
if (testResult.metadata?.error) {
|
||||
const error = testResult.metadata.error;
|
||||
let errorDetails = error.message;
|
||||
if (error.stack) {
|
||||
errorDetails = error.stack;
|
||||
}
|
||||
this.logger.testErrorDetails(errorDetails);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Handle directives (skip/todo)
|
||||
if (testResult.directive) {
|
||||
if (this.logger) {
|
||||
if (testResult.directive.type === 'skip') {
|
||||
this.logger.testConsoleOutput(`Test skipped: ${testResult.directive.reason || 'No reason given'}`);
|
||||
} else if (testResult.directive.type === 'todo') {
|
||||
this.logger.testConsoleOutput(`Test todo: ${testResult.directive.reason || 'No reason given'}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Mark test as settled and move to next
|
||||
this.activeTapTestResult.testSettled = true;
|
||||
this.testStore.push(this.activeTapTestResult);
|
||||
this._getNewTapTestResult();
|
||||
break;
|
||||
|
||||
case 'comment':
|
||||
if (this.logger) {
|
||||
// Check if it's a pretask comment
|
||||
const pretaskMatch = message.content.match(/^Pretask -> (.+): Success\.$/);
|
||||
if (pretaskMatch) {
|
||||
this.logger.tapOutput(message.content);
|
||||
} else {
|
||||
this.logger.testConsoleOutput(message.content);
|
||||
}
|
||||
}
|
||||
break;
|
||||
|
||||
case 'bailout':
|
||||
if (this.logger) {
|
||||
this.logger.error(`Bail out! ${message.content}`);
|
||||
}
|
||||
break;
|
||||
|
||||
case 'error':
|
||||
const errorBlock = message.content as IErrorBlock;
|
||||
if (this.logger && errorBlock.error) {
|
||||
let errorDetails = errorBlock.error.message;
|
||||
if (errorBlock.error.stack) {
|
||||
errorDetails = errorBlock.error.stack;
|
||||
}
|
||||
this.logger.testErrorDetails(errorDetails);
|
||||
}
|
||||
break;
|
||||
|
||||
case 'snapshot':
|
||||
// Handle new protocol snapshot format
|
||||
const snapshot = message.content;
|
||||
this.handleSnapshot({
|
||||
path: snapshot.name,
|
||||
content: typeof snapshot.content === 'string' ? snapshot.content : JSON.stringify(snapshot.content),
|
||||
action: 'compare' // Default action
|
||||
});
|
||||
break;
|
||||
|
||||
case 'event':
|
||||
const event = message.content as ITestEvent;
|
||||
this._handleTestEvent(event);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private _handleTestEvent(event: ITestEvent) {
|
||||
if (!this.logger) return;
|
||||
|
||||
switch (event.eventType) {
|
||||
case 'test:queued':
|
||||
// We can track queued tests if needed
|
||||
break;
|
||||
|
||||
case 'test:started':
|
||||
this.logger.testConsoleOutput(cs(`Test starting: ${event.data.description}`, 'cyan'));
|
||||
if (event.data.retry) {
|
||||
this.logger.testConsoleOutput(cs(` Retry attempt ${event.data.retry}`, 'orange'));
|
||||
}
|
||||
break;
|
||||
|
||||
case 'test:progress':
|
||||
if (event.data.progress !== undefined) {
|
||||
this.logger.testConsoleOutput(cs(` Progress: ${event.data.progress}%`, 'cyan'));
|
||||
}
|
||||
break;
|
||||
|
||||
case 'test:completed':
|
||||
// Test completion is already handled by the test result
|
||||
// This event provides additional timing info if needed
|
||||
break;
|
||||
|
||||
case 'suite:started':
|
||||
this.logger.testConsoleOutput(cs(`\nSuite: ${event.data.suiteName}`, 'blue'));
|
||||
break;
|
||||
|
||||
case 'suite:completed':
|
||||
this.logger.testConsoleOutput(cs(`Suite completed: ${event.data.suiteName}\n`, 'blue'));
|
||||
break;
|
||||
|
||||
case 'hook:started':
|
||||
this.logger.testConsoleOutput(cs(` Hook: ${event.data.hookName}`, 'cyan'));
|
||||
break;
|
||||
|
||||
case 'hook:completed':
|
||||
// Silent unless there's an error
|
||||
if (event.data.error) {
|
||||
this.logger.testConsoleOutput(cs(` Hook failed: ${event.data.hookName}`, 'red'));
|
||||
}
|
||||
break;
|
||||
|
||||
case 'assertion:failed':
|
||||
// Enhanced assertion failure with diff
|
||||
if (event.data.error) {
|
||||
this._displayAssertionError(event.data.error);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private _displayAssertionError(error: any) {
|
||||
if (!this.logger) return;
|
||||
|
||||
// Display error message
|
||||
if (error.message) {
|
||||
this.logger.testErrorDetails(error.message);
|
||||
}
|
||||
|
||||
// Display visual diff if available
|
||||
if (error.diff) {
|
||||
this._displayDiff(error.diff, error.expected, error.actual);
|
||||
}
|
||||
}
|
||||
|
||||
private _displayDiff(diff: any, expected: any, actual: any) {
|
||||
if (!this.logger) return;
|
||||
|
||||
this.logger.testConsoleOutput(cs('\n Diff:', 'cyan'));
|
||||
|
||||
switch (diff.type) {
|
||||
case 'string':
|
||||
this._displayStringDiff(diff.changes);
|
||||
break;
|
||||
|
||||
case 'object':
|
||||
this._displayObjectDiff(diff.changes, expected, actual);
|
||||
break;
|
||||
|
||||
case 'array':
|
||||
this._displayArrayDiff(diff.changes, expected, actual);
|
||||
break;
|
||||
|
||||
case 'primitive':
|
||||
this._displayPrimitiveDiff(diff.changes);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private _displayStringDiff(changes: any[]) {
|
||||
for (const change of changes) {
|
||||
const linePrefix = ` Line ${change.line + 1}: `;
|
||||
if (change.type === 'add') {
|
||||
this.logger.testConsoleOutput(cs(`${linePrefix}+ ${change.content}`, 'green'));
|
||||
} else if (change.type === 'remove') {
|
||||
this.logger.testConsoleOutput(cs(`${linePrefix}- ${change.content}`, 'red'));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private _displayObjectDiff(changes: any[], expected: any, actual: any) {
|
||||
this.logger.testConsoleOutput(cs(' Expected:', 'red'));
|
||||
this.logger.testConsoleOutput(` ${JSON.stringify(expected, null, 2)}`);
|
||||
this.logger.testConsoleOutput(cs(' Actual:', 'green'));
|
||||
this.logger.testConsoleOutput(` ${JSON.stringify(actual, null, 2)}`);
|
||||
|
||||
this.logger.testConsoleOutput(cs('\n Changes:', 'cyan'));
|
||||
for (const change of changes) {
|
||||
const path = change.path.join('.');
|
||||
if (change.type === 'add') {
|
||||
this.logger.testConsoleOutput(cs(` + ${path}: ${JSON.stringify(change.newValue)}`, 'green'));
|
||||
} else if (change.type === 'remove') {
|
||||
this.logger.testConsoleOutput(cs(` - ${path}: ${JSON.stringify(change.oldValue)}`, 'red'));
|
||||
} else if (change.type === 'modify') {
|
||||
this.logger.testConsoleOutput(cs(` ~ ${path}:`, 'cyan'));
|
||||
this.logger.testConsoleOutput(cs(` - ${JSON.stringify(change.oldValue)}`, 'red'));
|
||||
this.logger.testConsoleOutput(cs(` + ${JSON.stringify(change.newValue)}`, 'green'));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private _displayArrayDiff(changes: any[], expected: any[], actual: any[]) {
|
||||
this._displayObjectDiff(changes, expected, actual);
|
||||
}
|
||||
|
||||
private _displayPrimitiveDiff(changes: any[]) {
|
||||
const change = changes[0];
|
||||
if (change) {
|
||||
this.logger.testConsoleOutput(cs(` Expected: ${JSON.stringify(change.oldValue)}`, 'red'));
|
||||
this.logger.testConsoleOutput(cs(` Actual: ${JSON.stringify(change.newValue)}`, 'green'));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* returns all tests that are not completed
|
||||
@@ -294,6 +482,7 @@ export class TapParser {
|
||||
|
||||
public async evaluateFinalResult() {
|
||||
this.receivedTests = this.testStore.length;
|
||||
const duration = Date.now() - this.startTime;
|
||||
|
||||
// check wether all tests ran
|
||||
if (this.expectedTests === this.receivedTests) {
|
||||
@@ -305,24 +494,27 @@ export class TapParser {
|
||||
this.logger.error(`Only ${this.receivedTests} out of ${this.expectedTests} completed!`);
|
||||
}
|
||||
}
|
||||
if (!this.expectedTests) {
|
||||
if (!this.expectedTests && this.receivedTests === 0) {
|
||||
if (this.logger) {
|
||||
this.logger.error('No tests were defined. Therefore the testfile failed!');
|
||||
this.logger.testFileEnd(0, 1, duration); // Count as 1 failure
|
||||
}
|
||||
} else if (this.expectedTests !== this.receivedTests) {
|
||||
if (this.logger) {
|
||||
this.logger.error('The amount of received tests and expectedTests is unequal! Therefore the testfile failed');
|
||||
const errorCount = this.getErrorTests().length || 1; // At least 1 error
|
||||
this.logger.testFileEnd(this.receivedTests - errorCount, errorCount, duration);
|
||||
}
|
||||
} else if (this.getErrorTests().length === 0) {
|
||||
if (this.logger) {
|
||||
this.logger.tapOutput('All tests are successfull!!!');
|
||||
this.logger.testFileEnd(this.receivedTests, 0, 0);
|
||||
this.logger.testFileEnd(this.receivedTests, 0, duration);
|
||||
}
|
||||
} else {
|
||||
if (this.logger) {
|
||||
this.logger.tapOutput(`${this.getErrorTests().length} tests threw an error!!!`, true);
|
||||
this.logger.testFileEnd(this.receivedTests - this.getErrorTests().length, this.getErrorTests().length, 0);
|
||||
this.logger.testFileEnd(this.receivedTests - this.getErrorTests().length, this.getErrorTests().length, duration);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -74,12 +74,20 @@ export class TestDirectory {
|
||||
case TestExecutionMode.DIRECTORY:
|
||||
// Directory mode - now recursive with ** pattern
|
||||
const dirPath = plugins.path.join(this.cwd, this.testPath);
|
||||
const testPattern = '**/test*.ts';
|
||||
|
||||
const testFiles = await plugins.smartfile.fs.listFileTree(dirPath, testPattern);
|
||||
|
||||
|
||||
// Search for both TypeScript test files and Docker shell test files
|
||||
const tsPattern = '**/test*.ts';
|
||||
const dockerPattern = '**/*.docker.sh';
|
||||
|
||||
const [tsFiles, dockerFiles] = await Promise.all([
|
||||
plugins.smartfile.fs.listFileTree(dirPath, tsPattern),
|
||||
plugins.smartfile.fs.listFileTree(dirPath, dockerPattern),
|
||||
]);
|
||||
|
||||
const allTestFiles = [...tsFiles, ...dockerFiles];
|
||||
|
||||
this.testfileArray = await Promise.all(
|
||||
testFiles.map(async (filePath) => {
|
||||
allTestFiles.map(async (filePath) => {
|
||||
const absolutePath = plugins.path.isAbsolute(filePath)
|
||||
? filePath
|
||||
: plugins.path.join(dirPath, filePath);
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import * as plugins from './tstest.plugins.js';
|
||||
import * as paths from './tstest.paths.js';
|
||||
import * as logPrefixes from './tstest.logprefixes.js';
|
||||
|
||||
import { coloredString as cs } from '@push.rocks/consolecolor';
|
||||
|
||||
@@ -11,11 +10,23 @@ import { TestExecutionMode } from './index.js';
|
||||
import { TsTestLogger } from './tstest.logging.js';
|
||||
import type { LogOptions } from './tstest.logging.js';
|
||||
|
||||
// Runtime adapters
|
||||
import { parseTestFilename, isDockerTestFile, parseDockerTestFilename } from './tstest.classes.runtime.parser.js';
|
||||
import { RuntimeAdapterRegistry } from './tstest.classes.runtime.adapter.js';
|
||||
import { NodeRuntimeAdapter } from './tstest.classes.runtime.node.js';
|
||||
import { ChromiumRuntimeAdapter } from './tstest.classes.runtime.chromium.js';
|
||||
import { DenoRuntimeAdapter } from './tstest.classes.runtime.deno.js';
|
||||
import { BunRuntimeAdapter } from './tstest.classes.runtime.bun.js';
|
||||
import { DockerRuntimeAdapter } from './tstest.classes.runtime.docker.js';
|
||||
|
||||
export class TsTest {
|
||||
public testDir: TestDirectory;
|
||||
public executionMode: TestExecutionMode;
|
||||
public logger: TsTestLogger;
|
||||
public filterTags: string[];
|
||||
public startFromFile: number | null;
|
||||
public stopAtFile: number | null;
|
||||
public timeoutSeconds: number | null;
|
||||
|
||||
public smartshellInstance = new plugins.smartshell.Smartshell({
|
||||
executor: 'bash',
|
||||
@@ -26,18 +37,63 @@ export class TsTest {
|
||||
|
||||
public tsbundleInstance = new plugins.tsbundle.TsBundle();
|
||||
|
||||
constructor(cwdArg: string, testPathArg: string, executionModeArg: TestExecutionMode, logOptions: LogOptions = {}, tags: string[] = []) {
|
||||
public runtimeRegistry = new RuntimeAdapterRegistry();
|
||||
public dockerAdapter: DockerRuntimeAdapter | null = null;
|
||||
|
||||
constructor(cwdArg: string, testPathArg: string, executionModeArg: TestExecutionMode, logOptions: LogOptions = {}, tags: string[] = [], startFromFile: number | null = null, stopAtFile: number | null = null, timeoutSeconds: number | null = null) {
|
||||
this.executionMode = executionModeArg;
|
||||
this.testDir = new TestDirectory(cwdArg, testPathArg, executionModeArg);
|
||||
this.logger = new TsTestLogger(logOptions);
|
||||
this.filterTags = tags;
|
||||
this.startFromFile = startFromFile;
|
||||
this.stopAtFile = stopAtFile;
|
||||
this.timeoutSeconds = timeoutSeconds;
|
||||
|
||||
// Register runtime adapters
|
||||
this.runtimeRegistry.register(
|
||||
new NodeRuntimeAdapter(this.logger, this.smartshellInstance, this.timeoutSeconds, this.filterTags)
|
||||
);
|
||||
this.runtimeRegistry.register(
|
||||
new ChromiumRuntimeAdapter(this.logger, this.tsbundleInstance, this.smartbrowserInstance, this.timeoutSeconds)
|
||||
);
|
||||
this.runtimeRegistry.register(
|
||||
new DenoRuntimeAdapter(this.logger, this.smartshellInstance, this.timeoutSeconds, this.filterTags)
|
||||
);
|
||||
this.runtimeRegistry.register(
|
||||
new BunRuntimeAdapter(this.logger, this.smartshellInstance, this.timeoutSeconds, this.filterTags)
|
||||
);
|
||||
|
||||
// Initialize Docker adapter
|
||||
this.dockerAdapter = new DockerRuntimeAdapter(
|
||||
this.logger,
|
||||
this.smartshellInstance,
|
||||
this.timeoutSeconds,
|
||||
cwdArg
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check and display available runtimes
|
||||
*/
|
||||
private async checkEnvironment() {
|
||||
const availability = await this.runtimeRegistry.checkAvailability();
|
||||
this.logger.environmentCheck(availability);
|
||||
return availability;
|
||||
}
|
||||
|
||||
async run() {
|
||||
// Check and display environment
|
||||
await this.checkEnvironment();
|
||||
|
||||
// Move previous log files if --logfile option is used
|
||||
if (this.logger.options.logFile) {
|
||||
await this.movePreviousLogFiles();
|
||||
}
|
||||
|
||||
const testGroups = await this.testDir.getTestFileGroups();
|
||||
const allFiles = [...testGroups.serial, ...Object.values(testGroups.parallelGroups).flat()];
|
||||
|
||||
// Log test discovery
|
||||
// Log test discovery - always show full count
|
||||
this.logger.testDiscovery(
|
||||
allFiles.length,
|
||||
this.testDir.testPath,
|
||||
@@ -50,7 +106,7 @@ export class TsTest {
|
||||
// Execute serial tests first
|
||||
for (const fileNameArg of testGroups.serial) {
|
||||
fileIndex++;
|
||||
await this.runSingleTest(fileNameArg, fileIndex, allFiles.length, tapCombinator);
|
||||
await this.runSingleTestOrSkip(fileNameArg, fileIndex, allFiles.length, tapCombinator);
|
||||
}
|
||||
|
||||
// Execute parallel groups sequentially
|
||||
@@ -64,7 +120,7 @@ export class TsTest {
|
||||
// Run all tests in this group in parallel
|
||||
const parallelPromises = groupFiles.map(async (fileNameArg) => {
|
||||
fileIndex++;
|
||||
return this.runSingleTest(fileNameArg, fileIndex, allFiles.length, tapCombinator);
|
||||
return this.runSingleTestOrSkip(fileNameArg, fileIndex, allFiles.length, tapCombinator);
|
||||
});
|
||||
|
||||
await Promise.all(parallelPromises);
|
||||
@@ -75,30 +131,168 @@ export class TsTest {
|
||||
tapCombinator.evaluate();
|
||||
}
|
||||
|
||||
public async runWatch(ignorePatterns: string[] = []) {
|
||||
const smartchokInstance = new plugins.smartchok.Smartchok([this.testDir.cwd]);
|
||||
|
||||
console.clear();
|
||||
this.logger.watchModeStart();
|
||||
|
||||
// Initial run
|
||||
await this.run();
|
||||
|
||||
// Set up file watcher
|
||||
const fileChanges = new Map<string, NodeJS.Timeout>();
|
||||
const debounceTime = 300; // 300ms debounce
|
||||
|
||||
const runTestsAfterChange = async () => {
|
||||
console.clear();
|
||||
const changedFiles = Array.from(fileChanges.keys());
|
||||
fileChanges.clear();
|
||||
|
||||
this.logger.watchModeRerun(changedFiles);
|
||||
await this.run();
|
||||
this.logger.watchModeWaiting();
|
||||
};
|
||||
|
||||
// Start watching before subscribing to events
|
||||
await smartchokInstance.start();
|
||||
|
||||
// Subscribe to file change events
|
||||
const changeObservable = await smartchokInstance.getObservableFor('change');
|
||||
const addObservable = await smartchokInstance.getObservableFor('add');
|
||||
const unlinkObservable = await smartchokInstance.getObservableFor('unlink');
|
||||
|
||||
const handleFileChange = (changedPath: string) => {
|
||||
// Skip if path matches ignore patterns
|
||||
if (ignorePatterns.some(pattern => changedPath.includes(pattern))) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Clear existing timeout for this file if any
|
||||
if (fileChanges.has(changedPath)) {
|
||||
clearTimeout(fileChanges.get(changedPath));
|
||||
}
|
||||
|
||||
// Set new timeout for this file
|
||||
const timeout = setTimeout(() => {
|
||||
fileChanges.delete(changedPath);
|
||||
if (fileChanges.size === 0) {
|
||||
runTestsAfterChange();
|
||||
}
|
||||
}, debounceTime);
|
||||
|
||||
fileChanges.set(changedPath, timeout);
|
||||
};
|
||||
|
||||
// Subscribe to all relevant events
|
||||
changeObservable.subscribe(([path]) => handleFileChange(path));
|
||||
addObservable.subscribe(([path]) => handleFileChange(path));
|
||||
unlinkObservable.subscribe(([path]) => handleFileChange(path));
|
||||
|
||||
this.logger.watchModeWaiting();
|
||||
|
||||
// Handle Ctrl+C to exit gracefully
|
||||
process.on('SIGINT', async () => {
|
||||
this.logger.watchModeStop();
|
||||
await smartchokInstance.stop();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
// Keep the process running
|
||||
await new Promise(() => {}); // This promise never resolves
|
||||
}
|
||||
|
||||
private async runSingleTestOrSkip(fileNameArg: string, fileIndex: number, totalFiles: number, tapCombinator: TapCombinator) {
|
||||
// Check if this file should be skipped based on range
|
||||
if (this.startFromFile !== null && fileIndex < this.startFromFile) {
|
||||
this.logger.testFileSkipped(fileNameArg, fileIndex, totalFiles, `before start range (${this.startFromFile})`);
|
||||
tapCombinator.addSkippedFile(fileNameArg);
|
||||
return;
|
||||
}
|
||||
|
||||
if (this.stopAtFile !== null && fileIndex > this.stopAtFile) {
|
||||
this.logger.testFileSkipped(fileNameArg, fileIndex, totalFiles, `after stop range (${this.stopAtFile})`);
|
||||
tapCombinator.addSkippedFile(fileNameArg);
|
||||
return;
|
||||
}
|
||||
|
||||
// File is in range, run it
|
||||
await this.runSingleTest(fileNameArg, fileIndex, totalFiles, tapCombinator);
|
||||
}
|
||||
|
||||
private async runSingleTest(fileNameArg: string, fileIndex: number, totalFiles: number, tapCombinator: TapCombinator) {
|
||||
switch (true) {
|
||||
case process.env.CI && fileNameArg.includes('.nonci.'):
|
||||
this.logger.tapOutput(`Skipping ${fileNameArg} - marked as non-CI`);
|
||||
break;
|
||||
case fileNameArg.endsWith('.browser.ts') || fileNameArg.endsWith('.browser.nonci.ts'):
|
||||
const tapParserBrowser = await this.runInChrome(fileNameArg, fileIndex, totalFiles);
|
||||
tapCombinator.addTapParser(tapParserBrowser);
|
||||
break;
|
||||
case fileNameArg.endsWith('.both.ts') || fileNameArg.endsWith('.both.nonci.ts'):
|
||||
this.logger.sectionStart('Part 1: Chrome');
|
||||
const tapParserBothBrowser = await this.runInChrome(fileNameArg, fileIndex, totalFiles);
|
||||
tapCombinator.addTapParser(tapParserBothBrowser);
|
||||
const fileName = plugins.path.basename(fileNameArg);
|
||||
|
||||
// Check if this is a Docker test file
|
||||
if (isDockerTestFile(fileName)) {
|
||||
return await this.runDockerTest(fileNameArg, fileIndex, totalFiles, tapCombinator);
|
||||
}
|
||||
|
||||
// Parse the filename to determine runtimes and modifiers (for TypeScript tests)
|
||||
const parsed = parseTestFilename(fileName, { strictUnknownRuntime: false });
|
||||
|
||||
// Check for nonci modifier in CI environment
|
||||
if (process.env.CI && parsed.modifiers.includes('nonci')) {
|
||||
this.logger.tapOutput(`Skipping ${fileNameArg} - marked as non-CI`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Show deprecation warning for legacy naming
|
||||
if (parsed.isLegacy) {
|
||||
console.warn('');
|
||||
console.warn(cs('⚠️ DEPRECATION WARNING', 'orange'));
|
||||
console.warn(cs(` File: ${fileName}`, 'orange'));
|
||||
console.warn(cs(` Legacy naming detected. Please migrate to new naming convention.`, 'orange'));
|
||||
console.warn(cs(` Suggested: ${fileName.replace('.browser.', '.chromium.').replace('.both.', '.node+chromium.')}`, 'green'));
|
||||
console.warn(cs(` Run: tstest migrate --dry-run`, 'cyan'));
|
||||
console.warn('');
|
||||
}
|
||||
|
||||
// Get adapters for the specified runtimes
|
||||
const adapters = this.runtimeRegistry.getAdaptersForRuntimes(parsed.runtimes);
|
||||
|
||||
if (adapters.length === 0) {
|
||||
this.logger.tapOutput(`Skipping ${fileNameArg} - no runtime adapters available`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Execute tests for each runtime
|
||||
if (adapters.length === 1) {
|
||||
// Single runtime - no sections needed
|
||||
const adapter = adapters[0];
|
||||
const tapParser = await adapter.run(fileNameArg, fileIndex, totalFiles);
|
||||
tapCombinator.addTapParser(tapParser);
|
||||
} else {
|
||||
// Multiple runtimes - use sections
|
||||
for (let i = 0; i < adapters.length; i++) {
|
||||
const adapter = adapters[i];
|
||||
this.logger.sectionStart(`Part ${i + 1}: ${adapter.displayName}`);
|
||||
const tapParser = await adapter.run(fileNameArg, fileIndex, totalFiles);
|
||||
tapCombinator.addTapParser(tapParser);
|
||||
this.logger.sectionEnd();
|
||||
|
||||
this.logger.sectionStart('Part 2: Node');
|
||||
const tapParserBothNode = await this.runInNode(fileNameArg, fileIndex, totalFiles);
|
||||
tapCombinator.addTapParser(tapParserBothNode);
|
||||
this.logger.sectionEnd();
|
||||
break;
|
||||
default:
|
||||
const tapParserNode = await this.runInNode(fileNameArg, fileIndex, totalFiles);
|
||||
tapCombinator.addTapParser(tapParserNode);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a Docker test file
|
||||
*/
|
||||
private async runDockerTest(
|
||||
fileNameArg: string,
|
||||
fileIndex: number,
|
||||
totalFiles: number,
|
||||
tapCombinator: TapCombinator
|
||||
): Promise<void> {
|
||||
if (!this.dockerAdapter) {
|
||||
this.logger.tapOutput(cs('❌ Docker adapter not initialized', 'red'));
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const tapParser = await this.dockerAdapter.run(fileNameArg, fileIndex, totalFiles);
|
||||
tapCombinator.addTapParser(tapParser);
|
||||
} catch (error) {
|
||||
this.logger.tapOutput(cs(`❌ Docker test failed: ${error.message}`, 'red'));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -117,13 +311,131 @@ export class TsTest {
|
||||
process.env.TSTEST_FILTER_TAGS = this.filterTags.join(',');
|
||||
}
|
||||
|
||||
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(
|
||||
`tsrun ${fileNameArg}${tsrunOptions}`
|
||||
);
|
||||
await tapParser.handleTapProcess(execResultStreaming.childProcess);
|
||||
// Check for 00init.ts file in test directory
|
||||
const testDir = plugins.path.dirname(fileNameArg);
|
||||
const initFile = plugins.path.join(testDir, '00init.ts');
|
||||
let runCommand = `tsrun ${fileNameArg}${tsrunOptions}`;
|
||||
|
||||
const initFileExists = await plugins.smartfile.fs.fileExists(initFile);
|
||||
|
||||
// If 00init.ts exists, run it first
|
||||
if (initFileExists) {
|
||||
// Create a temporary loader file that imports both 00init.ts and the test file
|
||||
const absoluteInitFile = plugins.path.resolve(initFile);
|
||||
const absoluteTestFile = plugins.path.resolve(fileNameArg);
|
||||
const loaderContent = `
|
||||
import '${absoluteInitFile.replace(/\\/g, '/')}';
|
||||
import '${absoluteTestFile.replace(/\\/g, '/')}';
|
||||
`;
|
||||
const loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(fileNameArg)}`);
|
||||
await plugins.smartfile.memory.toFs(loaderContent, loaderPath);
|
||||
runCommand = `tsrun ${loaderPath}${tsrunOptions}`;
|
||||
}
|
||||
|
||||
const execResultStreaming = await this.smartshellInstance.execStreamingSilent(runCommand);
|
||||
|
||||
// If we created a loader file, clean it up after test execution
|
||||
if (initFileExists) {
|
||||
const loaderPath = plugins.path.join(testDir, `.loader_${plugins.path.basename(fileNameArg)}`);
|
||||
const cleanup = () => {
|
||||
try {
|
||||
if (plugins.smartfile.fs.fileExistsSync(loaderPath)) {
|
||||
plugins.smartfile.fs.removeSync(loaderPath);
|
||||
}
|
||||
} catch (e) {
|
||||
// Ignore cleanup errors
|
||||
}
|
||||
};
|
||||
|
||||
execResultStreaming.childProcess.on('exit', cleanup);
|
||||
execResultStreaming.childProcess.on('error', cleanup);
|
||||
}
|
||||
|
||||
// Start warning timer if no timeout was specified
|
||||
let warningTimer: NodeJS.Timeout | null = null;
|
||||
if (this.timeoutSeconds === null) {
|
||||
warningTimer = setTimeout(() => {
|
||||
console.error('');
|
||||
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
|
||||
console.error(cs(` File: ${fileNameArg}`, 'orange'));
|
||||
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
|
||||
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
|
||||
console.error('');
|
||||
}, 60000); // 1 minute
|
||||
}
|
||||
|
||||
// Handle timeout if specified
|
||||
if (this.timeoutSeconds !== null) {
|
||||
const timeoutMs = this.timeoutSeconds * 1000;
|
||||
let timeoutId: NodeJS.Timeout;
|
||||
|
||||
const timeoutPromise = new Promise<void>((_resolve, reject) => {
|
||||
timeoutId = setTimeout(async () => {
|
||||
// Use smartshell's terminate() to kill entire process tree
|
||||
await execResultStreaming.terminate();
|
||||
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
|
||||
}, timeoutMs);
|
||||
});
|
||||
|
||||
try {
|
||||
await Promise.race([
|
||||
tapParser.handleTapProcess(execResultStreaming.childProcess),
|
||||
timeoutPromise
|
||||
]);
|
||||
// Clear timeout if test completed successfully
|
||||
clearTimeout(timeoutId);
|
||||
} catch (error) {
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
// Handle timeout error
|
||||
tapParser.handleTimeout(this.timeoutSeconds);
|
||||
// Ensure entire process tree is killed if still running
|
||||
try {
|
||||
await execResultStreaming.kill(); // This kills the entire process tree with SIGKILL
|
||||
} catch (killError) {
|
||||
// Process tree might already be dead
|
||||
}
|
||||
await tapParser.evaluateFinalResult();
|
||||
}
|
||||
} else {
|
||||
await tapParser.handleTapProcess(execResultStreaming.childProcess);
|
||||
}
|
||||
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
|
||||
return tapParser;
|
||||
}
|
||||
|
||||
private async findFreePorts(): Promise<{ httpPort: number; wsPort: number }> {
|
||||
const smartnetwork = new plugins.smartnetwork.SmartNetwork();
|
||||
|
||||
// Find random free HTTP port in range 30000-40000 to minimize collision chance
|
||||
const httpPort = await smartnetwork.findFreePort(30000, 40000, { randomize: true });
|
||||
if (!httpPort) {
|
||||
throw new Error('Could not find a free HTTP port in range 30000-40000');
|
||||
}
|
||||
|
||||
// Find random free WebSocket port, excluding the HTTP port to ensure they're different
|
||||
const wsPort = await smartnetwork.findFreePort(30000, 40000, {
|
||||
randomize: true,
|
||||
exclude: [httpPort]
|
||||
});
|
||||
if (!wsPort) {
|
||||
throw new Error('Could not find a free WebSocket port in range 30000-40000');
|
||||
}
|
||||
|
||||
// Log selected ports for debugging
|
||||
if (!this.logger.options.quiet) {
|
||||
console.log(`Selected ports - HTTP: ${httpPort}, WebSocket: ${wsPort}`);
|
||||
}
|
||||
return { httpPort, wsPort };
|
||||
}
|
||||
|
||||
public async runInChrome(fileNameArg: string, index: number, total: number): Promise<TapParser> {
|
||||
this.logger.testFileStart(fileNameArg, 'chromium', index, total);
|
||||
|
||||
@@ -138,20 +450,24 @@ export class TsTest {
|
||||
bundler: 'esbuild',
|
||||
});
|
||||
|
||||
// Find free ports for HTTP and WebSocket
|
||||
const { httpPort, wsPort } = await this.findFreePorts();
|
||||
|
||||
// lets create a server
|
||||
const server = new plugins.typedserver.servertools.Server({
|
||||
cors: true,
|
||||
port: 3007,
|
||||
port: httpPort,
|
||||
});
|
||||
server.addRoute(
|
||||
'/test',
|
||||
new plugins.typedserver.servertools.Handler('GET', async (req, res) => {
|
||||
new plugins.typedserver.servertools.Handler('GET', async (_req, res) => {
|
||||
res.type('.html');
|
||||
res.write(`
|
||||
<html>
|
||||
<head>
|
||||
<script>
|
||||
globalThis.testdom = true;
|
||||
globalThis.wsPort = ${wsPort};
|
||||
</script>
|
||||
</head>
|
||||
<body></body>
|
||||
@@ -160,12 +476,12 @@ export class TsTest {
|
||||
res.end();
|
||||
})
|
||||
);
|
||||
server.addRoute('*', new plugins.typedserver.servertools.HandlerStatic(tsbundleCacheDirPath));
|
||||
server.addRoute('/*splat', new plugins.typedserver.servertools.HandlerStatic(tsbundleCacheDirPath));
|
||||
await server.start();
|
||||
|
||||
// lets handle realtime comms
|
||||
const tapParser = new TapParser(fileNameArg + ':chrome', this.logger);
|
||||
const wss = new plugins.ws.WebSocketServer({ port: 8080 });
|
||||
const wss = new plugins.ws.WebSocketServer({ port: wsPort });
|
||||
wss.on('connection', (ws) => {
|
||||
ws.on('message', (message) => {
|
||||
const messageStr = message.toString();
|
||||
@@ -178,13 +494,14 @@ export class TsTest {
|
||||
});
|
||||
});
|
||||
|
||||
// lets do the browser bit
|
||||
// lets do the browser bit with timeout handling
|
||||
await this.smartbrowserInstance.start();
|
||||
const evaluation = await this.smartbrowserInstance.evaluateOnPage(
|
||||
`http://localhost:3007/test?bundleName=${bundleFileName}`,
|
||||
|
||||
const evaluatePromise = this.smartbrowserInstance.evaluateOnPage(
|
||||
`http://localhost:${httpPort}/test?bundleName=${bundleFileName}`,
|
||||
async () => {
|
||||
// lets enable real time comms
|
||||
const ws = new WebSocket('ws://localhost:8080');
|
||||
const ws = new WebSocket(`ws://localhost:${globalThis.wsPort}`);
|
||||
await new Promise((resolve) => (ws.onopen = resolve));
|
||||
|
||||
// Ensure this function is declared with 'async'
|
||||
@@ -193,12 +510,12 @@ export class TsTest {
|
||||
const originalError = console.error;
|
||||
|
||||
// Override console methods to capture the logs
|
||||
console.log = (...args) => {
|
||||
console.log = (...args: any[]) => {
|
||||
logStore.push(args.join(' '));
|
||||
ws.send(args.join(' '));
|
||||
originalLog(...args);
|
||||
};
|
||||
console.error = (...args) => {
|
||||
console.error = (...args: any[]) => {
|
||||
logStore.push(args.join(' '));
|
||||
ws.send(args.join(' '));
|
||||
originalError(...args);
|
||||
@@ -237,16 +554,127 @@ export class TsTest {
|
||||
return logStore.join('\n');
|
||||
}
|
||||
);
|
||||
await this.smartbrowserInstance.stop();
|
||||
await server.stop();
|
||||
wss.close();
|
||||
|
||||
// Start warning timer if no timeout was specified
|
||||
let warningTimer: NodeJS.Timeout | null = null;
|
||||
if (this.timeoutSeconds === null) {
|
||||
warningTimer = setTimeout(() => {
|
||||
console.error('');
|
||||
console.error(cs('⚠️ WARNING: Test file is running for more than 1 minute', 'orange'));
|
||||
console.error(cs(` File: ${fileNameArg}`, 'orange'));
|
||||
console.error(cs(' Consider using --timeout option to set a timeout for test files.', 'orange'));
|
||||
console.error(cs(' Example: tstest test --timeout=300 (for 5 minutes)', 'orange'));
|
||||
console.error('');
|
||||
}, 60000); // 1 minute
|
||||
}
|
||||
|
||||
// Handle timeout if specified
|
||||
if (this.timeoutSeconds !== null) {
|
||||
const timeoutMs = this.timeoutSeconds * 1000;
|
||||
let timeoutId: NodeJS.Timeout;
|
||||
|
||||
const timeoutPromise = new Promise<void>((_resolve, reject) => {
|
||||
timeoutId = setTimeout(() => {
|
||||
reject(new Error(`Test file timed out after ${this.timeoutSeconds} seconds`));
|
||||
}, timeoutMs);
|
||||
});
|
||||
|
||||
try {
|
||||
await Promise.race([
|
||||
evaluatePromise,
|
||||
timeoutPromise
|
||||
]);
|
||||
// Clear timeout if test completed successfully
|
||||
clearTimeout(timeoutId);
|
||||
} catch (error) {
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
// Handle timeout error
|
||||
tapParser.handleTimeout(this.timeoutSeconds);
|
||||
}
|
||||
} else {
|
||||
await evaluatePromise;
|
||||
}
|
||||
|
||||
// Clear warning timer if it was set
|
||||
if (warningTimer) {
|
||||
clearTimeout(warningTimer);
|
||||
}
|
||||
|
||||
// Always clean up resources, even on timeout
|
||||
try {
|
||||
await this.smartbrowserInstance.stop();
|
||||
} catch (error) {
|
||||
// Browser might already be stopped
|
||||
}
|
||||
|
||||
try {
|
||||
await server.stop();
|
||||
} catch (error) {
|
||||
// Server might already be stopped
|
||||
}
|
||||
|
||||
try {
|
||||
wss.close();
|
||||
} catch (error) {
|
||||
// WebSocket server might already be closed
|
||||
}
|
||||
|
||||
console.log(
|
||||
`${cs('=> ', 'blue')} Stopped ${cs(fileNameArg, 'orange')} chromium instance and server.`
|
||||
);
|
||||
// lets create the tap parser
|
||||
// Always evaluate final result (handleTimeout just sets up the test state)
|
||||
await tapParser.evaluateFinalResult();
|
||||
return tapParser;
|
||||
}
|
||||
|
||||
public async runInDeno() {}
|
||||
|
||||
private async movePreviousLogFiles() {
|
||||
const logDir = plugins.path.join('.nogit', 'testlogs');
|
||||
const previousDir = plugins.path.join('.nogit', 'testlogs', 'previous');
|
||||
const errDir = plugins.path.join('.nogit', 'testlogs', '00err');
|
||||
const diffDir = plugins.path.join('.nogit', 'testlogs', '00diff');
|
||||
|
||||
try {
|
||||
// Delete 00err and 00diff directories if they exist
|
||||
if (plugins.smartfile.fs.isDirectorySync(errDir)) {
|
||||
plugins.smartfile.fs.removeSync(errDir);
|
||||
}
|
||||
if (plugins.smartfile.fs.isDirectorySync(diffDir)) {
|
||||
plugins.smartfile.fs.removeSync(diffDir);
|
||||
}
|
||||
|
||||
// Get all .log files in log directory (not in subdirectories)
|
||||
const files = await plugins.smartfile.fs.listFileTree(logDir, '*.log');
|
||||
const logFiles = files.filter((file: string) => !file.includes('/'));
|
||||
|
||||
if (logFiles.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Ensure previous directory exists
|
||||
await plugins.smartfile.fs.ensureDir(previousDir);
|
||||
|
||||
// Move each log file to previous directory
|
||||
for (const file of logFiles) {
|
||||
const filename = plugins.path.basename(file);
|
||||
const sourcePath = plugins.path.join(logDir, filename);
|
||||
const destPath = plugins.path.join(previousDir, filename);
|
||||
|
||||
try {
|
||||
// Copy file to new location and remove original
|
||||
await plugins.smartfile.fs.copy(sourcePath, destPath);
|
||||
await plugins.smartfile.fs.remove(sourcePath);
|
||||
} catch (error) {
|
||||
// Silently continue if a file can't be moved
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
// Directory might not exist, which is fine
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -30,12 +30,14 @@ export interface TestSummary {
|
||||
totalTests: number;
|
||||
totalPassed: number;
|
||||
totalFailed: number;
|
||||
totalSkipped: number;
|
||||
totalDuration: number;
|
||||
fileResults: TestFileResult[];
|
||||
skippedFiles: string[];
|
||||
}
|
||||
|
||||
export class TsTestLogger {
|
||||
private options: LogOptions;
|
||||
public readonly options: LogOptions;
|
||||
private startTime: number;
|
||||
private fileResults: TestFileResult[] = [];
|
||||
private currentFileResult: TestFileResult | null = null;
|
||||
@@ -135,6 +137,43 @@ export class TsTestLogger {
|
||||
this.log(this.format(` Found: ${count} test file(s)`, 'green'));
|
||||
}
|
||||
}
|
||||
|
||||
// Environment check - display available runtimes
|
||||
environmentCheck(availability: Map<string, { available: boolean; version?: string; error?: string }>) {
|
||||
if (this.options.json) {
|
||||
const runtimes: any = {};
|
||||
for (const [runtime, info] of availability) {
|
||||
runtimes[runtime] = info;
|
||||
}
|
||||
this.logJson({ event: 'environmentCheck', runtimes });
|
||||
return;
|
||||
}
|
||||
|
||||
if (this.options.quiet) return;
|
||||
|
||||
this.log(this.format('\n🌍 Test Environment', 'bold'));
|
||||
|
||||
// Define runtime display names
|
||||
const runtimeNames: Record<string, string> = {
|
||||
node: 'Node.js',
|
||||
deno: 'Deno',
|
||||
bun: 'Bun',
|
||||
chromium: 'Chrome/Chromium'
|
||||
};
|
||||
|
||||
// Display each runtime
|
||||
for (const [runtime, info] of availability) {
|
||||
const displayName = runtimeNames[runtime] || runtime;
|
||||
|
||||
if (info.available) {
|
||||
const versionStr = info.version ? ` ${info.version}` : '';
|
||||
this.log(this.format(` ✓ ${displayName}${versionStr}`, 'green'));
|
||||
} else {
|
||||
const errorStr = info.error ? ` (${info.error})` : '';
|
||||
this.log(this.format(` ✗ ${displayName}${errorStr}`, 'dim'));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Test execution
|
||||
testFileStart(filename: string, runtime: string, index: number, total: number) {
|
||||
@@ -153,15 +192,26 @@ export class TsTestLogger {
|
||||
|
||||
// Only set up test log file if --logfile option is specified
|
||||
if (this.options.logFile) {
|
||||
const baseFilename = path.basename(filename, '.ts');
|
||||
this.currentTestLogFile = path.join('.nogit', 'testlogs', `${baseFilename}.log`);
|
||||
|
||||
// Create a safe filename that preserves directory structure
|
||||
// Convert relative path to a flat filename by replacing separators with __
|
||||
const relativeFilename = path.relative(process.cwd(), filename);
|
||||
const safeFilename = relativeFilename
|
||||
.replace(/\\/g, '/') // Normalize Windows paths
|
||||
.replace(/\//g, '__') // Replace path separators with double underscores
|
||||
.replace(/\.ts$/, '') // Remove .ts extension
|
||||
.replace(/^\.\.__|^\.__|^__/, ''); // Clean up leading separators from relative paths
|
||||
|
||||
// Sanitize runtime name for use in filename (lowercase, no spaces/dots/special chars)
|
||||
const safeRuntime = runtime.toLowerCase().replace(/[^a-z0-9]/g, '');
|
||||
|
||||
this.currentTestLogFile = path.join('.nogit', 'testlogs', `${safeFilename}__${safeRuntime}.log`);
|
||||
|
||||
// Ensure the directory exists
|
||||
const logDir = path.dirname(this.currentTestLogFile);
|
||||
if (!fs.existsSync(logDir)) {
|
||||
fs.mkdirSync(logDir, { recursive: true });
|
||||
}
|
||||
|
||||
|
||||
// Clear the log file for this test
|
||||
fs.writeFileSync(this.currentTestLogFile, '');
|
||||
}
|
||||
@@ -232,9 +282,51 @@ export class TsTestLogger {
|
||||
|
||||
if (!this.options.quiet) {
|
||||
const total = passed + failed;
|
||||
const status = failed === 0 ? 'PASSED' : 'FAILED';
|
||||
const color = failed === 0 ? 'green' : 'red';
|
||||
this.log(this.format(` Summary: ${passed}/${total} ${status}`, color));
|
||||
const durationStr = duration >= 1000 ? `${(duration / 1000).toFixed(1)}s` : `${duration}ms`;
|
||||
|
||||
if (failed === 0) {
|
||||
this.log(this.format(` Summary: ${passed}/${total} PASSED in ${durationStr}`, 'green'));
|
||||
} else {
|
||||
this.log(this.format(` Summary: ${passed} passed, ${failed} failed of ${total} tests in ${durationStr}`, 'red'));
|
||||
}
|
||||
}
|
||||
|
||||
// If using --logfile, handle error copy and diff detection
|
||||
if (this.options.logFile && this.currentTestLogFile) {
|
||||
try {
|
||||
const logContent = fs.readFileSync(this.currentTestLogFile, 'utf-8');
|
||||
const logDir = path.dirname(this.currentTestLogFile);
|
||||
const logBasename = path.basename(this.currentTestLogFile);
|
||||
|
||||
// Create error copy if there were failures
|
||||
if (failed > 0) {
|
||||
const errorDir = path.join(logDir, '00err');
|
||||
if (!fs.existsSync(errorDir)) {
|
||||
fs.mkdirSync(errorDir, { recursive: true });
|
||||
}
|
||||
const errorLogPath = path.join(errorDir, logBasename);
|
||||
fs.writeFileSync(errorLogPath, logContent);
|
||||
}
|
||||
|
||||
// Check for previous version and create diff if changed
|
||||
const previousLogPath = path.join(logDir, 'previous', logBasename);
|
||||
if (fs.existsSync(previousLogPath)) {
|
||||
const previousContent = fs.readFileSync(previousLogPath, 'utf-8');
|
||||
|
||||
// Simple check if content differs
|
||||
if (previousContent !== logContent) {
|
||||
const diffDir = path.join(logDir, '00diff');
|
||||
if (!fs.existsSync(diffDir)) {
|
||||
fs.mkdirSync(diffDir, { recursive: true });
|
||||
}
|
||||
const diffLogPath = path.join(diffDir, logBasename);
|
||||
const diffContent = this.createDiff(previousContent, logContent, logBasename);
|
||||
fs.writeFileSync(diffLogPath, diffContent);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
// Silently fail to avoid disrupting the test run
|
||||
}
|
||||
}
|
||||
|
||||
// Clear the current test log file reference only if using --logfile
|
||||
@@ -244,7 +336,7 @@ export class TsTestLogger {
|
||||
}
|
||||
|
||||
// TAP output forwarding (for TAP protocol messages)
|
||||
tapOutput(message: string, isError: boolean = false) {
|
||||
tapOutput(message: string, _isError: boolean = false) {
|
||||
if (this.options.json) return;
|
||||
|
||||
// Never show raw TAP protocol messages in console
|
||||
@@ -274,6 +366,19 @@ export class TsTestLogger {
|
||||
}
|
||||
}
|
||||
|
||||
// Skipped test file
|
||||
testFileSkipped(filename: string, index: number, total: number, reason: string) {
|
||||
if (this.options.json) {
|
||||
this.logJson({ event: 'fileSkipped', filename, index, total, reason });
|
||||
return;
|
||||
}
|
||||
|
||||
if (this.options.quiet) return;
|
||||
|
||||
this.log(this.format(`\n⏭️ ${filename} (${index}/${total})`, 'yellow'));
|
||||
this.log(this.format(` Skipped: ${reason}`, 'dim'));
|
||||
}
|
||||
|
||||
// Browser console
|
||||
browserConsole(message: string, level: string = 'log') {
|
||||
if (this.options.json) {
|
||||
@@ -309,15 +414,17 @@ export class TsTestLogger {
|
||||
}
|
||||
|
||||
// Final summary
|
||||
summary() {
|
||||
summary(skippedFiles: string[] = []) {
|
||||
const totalDuration = Date.now() - this.startTime;
|
||||
const summary: TestSummary = {
|
||||
totalFiles: this.fileResults.length,
|
||||
totalFiles: this.fileResults.length + skippedFiles.length,
|
||||
totalTests: this.fileResults.reduce((sum, r) => sum + r.total, 0),
|
||||
totalPassed: this.fileResults.reduce((sum, r) => sum + r.passed, 0),
|
||||
totalFailed: this.fileResults.reduce((sum, r) => sum + r.failed, 0),
|
||||
totalSkipped: skippedFiles.length,
|
||||
totalDuration,
|
||||
fileResults: this.fileResults
|
||||
fileResults: this.fileResults,
|
||||
skippedFiles
|
||||
};
|
||||
|
||||
if (this.options.json) {
|
||||
@@ -327,7 +434,13 @@ export class TsTestLogger {
|
||||
|
||||
if (this.options.quiet) {
|
||||
const status = summary.totalFailed === 0 ? 'PASSED' : 'FAILED';
|
||||
this.log(`\nSummary: ${summary.totalPassed}/${summary.totalTests} | ${totalDuration}ms | ${status}`);
|
||||
const durationStr = totalDuration >= 1000 ? `${(totalDuration / 1000).toFixed(1)}s` : `${totalDuration}ms`;
|
||||
|
||||
if (summary.totalFailed === 0) {
|
||||
this.log(`\nSummary: ${summary.totalPassed}/${summary.totalTests} | ${durationStr} | ${status}`);
|
||||
} else {
|
||||
this.log(`\nSummary: ${summary.totalPassed} passed, ${summary.totalFailed} failed of ${summary.totalTests} tests | ${durationStr} | ${status}`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -338,7 +451,11 @@ export class TsTestLogger {
|
||||
this.log(this.format(`│ Total Tests: ${summary.totalTests.toString().padStart(14)} │`, 'white'));
|
||||
this.log(this.format(`│ Passed: ${summary.totalPassed.toString().padStart(14)} │`, 'green'));
|
||||
this.log(this.format(`│ Failed: ${summary.totalFailed.toString().padStart(14)} │`, summary.totalFailed > 0 ? 'red' : 'green'));
|
||||
this.log(this.format(`│ Duration: ${totalDuration.toString().padStart(14)}ms │`, 'white'));
|
||||
if (summary.totalSkipped > 0) {
|
||||
this.log(this.format(`│ Skipped: ${summary.totalSkipped.toString().padStart(14)} │`, 'yellow'));
|
||||
}
|
||||
const durationStrFormatted = totalDuration >= 1000 ? `${(totalDuration / 1000).toFixed(1)}s` : `${totalDuration}ms`;
|
||||
this.log(this.format(`│ Duration: ${durationStrFormatted.padStart(14)} │`, 'white'));
|
||||
this.log(this.format('└────────────────────────────────┘', 'dim'));
|
||||
|
||||
// File results
|
||||
@@ -359,15 +476,23 @@ export class TsTestLogger {
|
||||
|
||||
// Performance metrics
|
||||
if (this.options.verbose) {
|
||||
const avgDuration = Math.round(totalDuration / summary.totalTests);
|
||||
const slowestTest = this.fileResults
|
||||
.flatMap(r => r.tests)
|
||||
.sort((a, b) => b.duration - a.duration)[0];
|
||||
// Calculate metrics based on actual test durations
|
||||
const allTests = this.fileResults.flatMap(r => r.tests);
|
||||
const testDurations = allTests.map(t => t.duration);
|
||||
const sumOfTestDurations = testDurations.reduce((sum, d) => sum + d, 0);
|
||||
const avgTestDuration = allTests.length > 0 ? Math.round(sumOfTestDurations / allTests.length) : 0;
|
||||
|
||||
// Find slowest test (exclude 0ms durations unless all are 0)
|
||||
const nonZeroDurations = allTests.filter(t => t.duration > 0);
|
||||
const testsToSort = nonZeroDurations.length > 0 ? nonZeroDurations : allTests;
|
||||
const slowestTest = testsToSort.sort((a, b) => b.duration - a.duration)[0];
|
||||
|
||||
this.log(this.format('\n⏱️ Performance Metrics:', 'cyan'));
|
||||
this.log(this.format(` Average per test: ${avgDuration}ms`, 'white'));
|
||||
if (slowestTest) {
|
||||
this.log(this.format(` Slowest test: ${slowestTest.name} (${slowestTest.duration}ms)`, 'yellow'));
|
||||
this.log(this.format(` Average per test: ${avgTestDuration}ms`, 'white'));
|
||||
if (slowestTest && slowestTest.duration > 0) {
|
||||
this.log(this.format(` Slowest test: ${slowestTest.name} (${slowestTest.duration}ms)`, 'orange'));
|
||||
} else if (allTests.length > 0) {
|
||||
this.log(this.format(` All tests completed in <1ms`, 'dim'));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -377,6 +502,20 @@ export class TsTestLogger {
|
||||
this.log(this.format(`\n${status}`, statusColor));
|
||||
}
|
||||
|
||||
// Warning display
|
||||
warning(message: string) {
|
||||
if (this.options.json) {
|
||||
this.logJson({ event: 'warning', message });
|
||||
return;
|
||||
}
|
||||
|
||||
if (this.options.quiet) {
|
||||
console.log(`WARNING: ${message}`);
|
||||
} else {
|
||||
this.log(this.format(` ⚠️ ${message}`, 'orange'));
|
||||
}
|
||||
}
|
||||
|
||||
// Error display
|
||||
error(message: string, file?: string, stack?: string) {
|
||||
if (this.options.json) {
|
||||
@@ -396,4 +535,91 @@ export class TsTestLogger {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Create a diff between two log contents
|
||||
private createDiff(previousContent: string, currentContent: string, filename: string): string {
|
||||
const previousLines = previousContent.split('\n');
|
||||
const currentLines = currentContent.split('\n');
|
||||
|
||||
let diff = `DIFF REPORT: ${filename}\n`;
|
||||
diff += `Generated: ${new Date().toISOString()}\n`;
|
||||
diff += '='.repeat(80) + '\n\n';
|
||||
|
||||
// Simple line-by-line comparison
|
||||
const maxLines = Math.max(previousLines.length, currentLines.length);
|
||||
let hasChanges = false;
|
||||
|
||||
for (let i = 0; i < maxLines; i++) {
|
||||
const prevLine = previousLines[i] || '';
|
||||
const currLine = currentLines[i] || '';
|
||||
|
||||
if (prevLine !== currLine) {
|
||||
hasChanges = true;
|
||||
if (i < previousLines.length && i >= currentLines.length) {
|
||||
// Line was removed
|
||||
diff += `- [Line ${i + 1}] ${prevLine}\n`;
|
||||
} else if (i >= previousLines.length && i < currentLines.length) {
|
||||
// Line was added
|
||||
diff += `+ [Line ${i + 1}] ${currLine}\n`;
|
||||
} else {
|
||||
// Line was modified
|
||||
diff += `- [Line ${i + 1}] ${prevLine}\n`;
|
||||
diff += `+ [Line ${i + 1}] ${currLine}\n`;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!hasChanges) {
|
||||
diff += 'No changes detected.\n';
|
||||
}
|
||||
|
||||
diff += '\n' + '='.repeat(80) + '\n';
|
||||
diff += `Previous version had ${previousLines.length} lines\n`;
|
||||
diff += `Current version has ${currentLines.length} lines\n`;
|
||||
|
||||
return diff;
|
||||
}
|
||||
|
||||
// Watch mode methods
|
||||
watchModeStart() {
|
||||
if (this.options.json) {
|
||||
this.logJson({ event: 'watchModeStart' });
|
||||
return;
|
||||
}
|
||||
|
||||
this.log(this.format('\n👀 Watch Mode', 'cyan'));
|
||||
this.log(this.format(' Running tests in watch mode...', 'dim'));
|
||||
this.log(this.format(' Press Ctrl+C to exit\n', 'dim'));
|
||||
}
|
||||
|
||||
watchModeWaiting() {
|
||||
if (this.options.json) {
|
||||
this.logJson({ event: 'watchModeWaiting' });
|
||||
return;
|
||||
}
|
||||
|
||||
this.log(this.format('\n Waiting for file changes...', 'dim'));
|
||||
}
|
||||
|
||||
watchModeRerun(changedFiles: string[]) {
|
||||
if (this.options.json) {
|
||||
this.logJson({ event: 'watchModeRerun', changedFiles });
|
||||
return;
|
||||
}
|
||||
|
||||
this.log(this.format('\n🔄 File changes detected:', 'cyan'));
|
||||
changedFiles.forEach(file => {
|
||||
this.log(this.format(` • ${file}`, 'yellow'));
|
||||
});
|
||||
this.log(this.format('\n Re-running tests...\n', 'dim'));
|
||||
}
|
||||
|
||||
watchModeStop() {
|
||||
if (this.options.json) {
|
||||
this.logJson({ event: 'watchModeStop' });
|
||||
return;
|
||||
}
|
||||
|
||||
this.log(this.format('\n\n👋 Stopping watch mode...', 'cyan'));
|
||||
}
|
||||
}
|
||||
@@ -13,9 +13,11 @@ export {
|
||||
// @push.rocks scope
|
||||
import * as consolecolor from '@push.rocks/consolecolor';
|
||||
import * as smartbrowser from '@push.rocks/smartbrowser';
|
||||
import * as smartchok from '@push.rocks/smartchok';
|
||||
import * as smartdelay from '@push.rocks/smartdelay';
|
||||
import * as smartfile from '@push.rocks/smartfile';
|
||||
import * as smartlog from '@push.rocks/smartlog';
|
||||
import * as smartnetwork from '@push.rocks/smartnetwork';
|
||||
import * as smartpromise from '@push.rocks/smartpromise';
|
||||
import * as smartshell from '@push.rocks/smartshell';
|
||||
import * as tapbundle from '../dist_ts_tapbundle/index.js';
|
||||
@@ -23,18 +25,21 @@ import * as tapbundle from '../dist_ts_tapbundle/index.js';
|
||||
export {
|
||||
consolecolor,
|
||||
smartbrowser,
|
||||
smartchok,
|
||||
smartdelay,
|
||||
smartfile,
|
||||
smartlog,
|
||||
smartnetwork,
|
||||
smartpromise,
|
||||
smartshell,
|
||||
tapbundle,
|
||||
};
|
||||
|
||||
// @gitzone scope
|
||||
// @git.zone scope
|
||||
import * as tsbundle from '@git.zone/tsbundle';
|
||||
import * as tsrun from '@git.zone/tsrun';
|
||||
|
||||
export { tsbundle };
|
||||
export { tsbundle, tsrun };
|
||||
|
||||
// sindresorhus
|
||||
import figures from 'figures';
|
||||
|
||||
@@ -3,6 +3,5 @@ export { TapWrap } from './tapbundle.classes.tapwrap.js';
|
||||
export { webhelpers } from './webhelpers.js';
|
||||
export { TapTools } from './tapbundle.classes.taptools.js';
|
||||
|
||||
import { expect } from '@push.rocks/smartexpect';
|
||||
|
||||
export { expect };
|
||||
// Export enhanced expect with diff generation
|
||||
export { expect, setProtocolEmitter } from './tapbundle.expect.wrapper.js';
|
||||
|
||||
389
ts_tapbundle/readme.md
Normal file
389
ts_tapbundle/readme.md
Normal file
@@ -0,0 +1,389 @@
|
||||
# @git.zone/tstest/tapbundle
|
||||
|
||||
> 🧪 Core TAP testing framework with enhanced assertions and lifecycle hooks
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# tapbundle is typically included as part of @git.zone/tstest
|
||||
pnpm install --save-dev @git.zone/tstest
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
`@git.zone/tstest/tapbundle` is the core testing framework module that provides the TAP (Test Anything Protocol) implementation for tstest. It offers a comprehensive API for writing and organizing tests with support for lifecycle hooks, test suites, enhanced assertions with diff generation, and flexible test configuration.
|
||||
|
||||
## Key Features
|
||||
|
||||
- 🎯 **TAP Protocol Compliant** - Full TAP version 13 support
|
||||
- 🔍 **Enhanced Assertions** - Built on smartexpect with automatic diff generation
|
||||
- 🏗️ **Test Suites** - Organize tests with `describe()` blocks
|
||||
- 🔄 **Lifecycle Hooks** - beforeEach/afterEach at suite and global levels
|
||||
- 🏷️ **Test Tagging** - Filter tests by tags for selective execution
|
||||
- ⚡ **Parallel Testing** - Run tests concurrently with `testParallel()`
|
||||
- 🔁 **Automatic Retries** - Configure retry logic for flaky tests
|
||||
- ⏱️ **Timeout Control** - Set timeouts at global, file, or test level
|
||||
- 🎨 **Fluent API** - Chain test configurations with builder pattern
|
||||
- 📊 **Protocol Events** - Real-time test execution events
|
||||
|
||||
## Basic Usage
|
||||
|
||||
### Simple Test File
|
||||
|
||||
```typescript
|
||||
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||
|
||||
tap.test('should add numbers correctly', async () => {
|
||||
const result = 2 + 2;
|
||||
expect(result).toEqual(4);
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
### Using Test Suites
|
||||
|
||||
```typescript
|
||||
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||
|
||||
tap.describe('Calculator', () => {
|
||||
tap.beforeEach(async (tapTools) => {
|
||||
// Setup before each test in this suite
|
||||
});
|
||||
|
||||
tap.test('should add', async () => {
|
||||
expect(2 + 2).toEqual(4);
|
||||
});
|
||||
|
||||
tap.test('should subtract', async () => {
|
||||
expect(5 - 3).toEqual(2);
|
||||
});
|
||||
|
||||
tap.afterEach(async (tapTools) => {
|
||||
// Cleanup after each test in this suite
|
||||
});
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### Main Test Methods
|
||||
|
||||
#### `tap.test(description, testFunction)`
|
||||
|
||||
Define a standard test that runs sequentially.
|
||||
|
||||
```typescript
|
||||
tap.test('should validate user input', async () => {
|
||||
// test code
|
||||
});
|
||||
```
|
||||
|
||||
#### `tap.testParallel(description, testFunction)`
|
||||
|
||||
Define a test that runs in parallel with other parallel tests.
|
||||
|
||||
```typescript
|
||||
tap.testParallel('should fetch user data', async () => {
|
||||
// test code
|
||||
});
|
||||
```
|
||||
|
||||
#### `tap.describe(description, suiteFunction)`
|
||||
|
||||
Create a test suite to group related tests.
|
||||
|
||||
```typescript
|
||||
tap.describe('User Authentication', () => {
|
||||
tap.test('should login', async () => { });
|
||||
tap.test('should logout', async () => { });
|
||||
});
|
||||
```
|
||||
|
||||
### Test Modes
|
||||
|
||||
#### Skip Tests
|
||||
|
||||
```typescript
|
||||
tap.skip.test('not ready yet', async () => {
|
||||
// This test will be skipped
|
||||
});
|
||||
```
|
||||
|
||||
#### Only Mode
|
||||
|
||||
```typescript
|
||||
tap.only.test('focus on this test', async () => {
|
||||
// Only tests marked with 'only' will run
|
||||
});
|
||||
```
|
||||
|
||||
#### Todo Tests
|
||||
|
||||
```typescript
|
||||
tap.todo.test('implement feature X');
|
||||
```
|
||||
|
||||
### Fluent Test Builder
|
||||
|
||||
Chain test configurations for expressive test definitions:
|
||||
|
||||
```typescript
|
||||
tap
|
||||
.tags('integration', 'database')
|
||||
.priority('high')
|
||||
.retry(3)
|
||||
.timeout(5000)
|
||||
.test('should handle database connection', async () => {
|
||||
// test with configured settings
|
||||
});
|
||||
```
|
||||
|
||||
### Lifecycle Hooks
|
||||
|
||||
#### Suite-Level Hooks
|
||||
|
||||
```typescript
|
||||
tap.describe('Database Tests', () => {
|
||||
tap.beforeEach(async (tapTools) => {
|
||||
// Runs before each test in this suite
|
||||
});
|
||||
|
||||
tap.afterEach(async (tapTools) => {
|
||||
// Runs after each test in this suite
|
||||
});
|
||||
|
||||
tap.test('test 1', async () => { });
|
||||
tap.test('test 2', async () => { });
|
||||
});
|
||||
```
|
||||
|
||||
#### Global Hooks
|
||||
|
||||
```typescript
|
||||
tap.settings({
|
||||
beforeAll: async () => {
|
||||
// Runs once before all tests
|
||||
},
|
||||
afterAll: async () => {
|
||||
// Runs once after all tests
|
||||
},
|
||||
beforeEach: async (testName) => {
|
||||
// Runs before every test
|
||||
},
|
||||
afterEach: async (testName, passed) => {
|
||||
// Runs after every test
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### Global Settings
|
||||
|
||||
Configure test behavior at the file level:
|
||||
|
||||
```typescript
|
||||
tap.settings({
|
||||
timeout: 10000, // Default timeout for all tests
|
||||
retries: 2, // Retry failed tests
|
||||
retryDelay: 1000, // Delay between retries
|
||||
bail: false, // Stop on first failure
|
||||
suppressConsole: false, // Hide console output
|
||||
verboseErrors: true, // Show full stack traces
|
||||
showTestDuration: true, // Display test durations
|
||||
maxConcurrency: 4, // Max parallel tests
|
||||
});
|
||||
```
|
||||
|
||||
### Enhanced Assertions
|
||||
|
||||
The `expect` function is an enhanced wrapper around [@push.rocks/smartexpect](https://code.foss.global/push.rocks/smartexpect) that automatically generates diffs for failed assertions.
|
||||
|
||||
```typescript
|
||||
import { expect } from '@git.zone/tstest/tapbundle';
|
||||
|
||||
tap.test('should compare objects', async () => {
|
||||
const actual = { name: 'John', age: 30 };
|
||||
const expected = { name: 'John', age: 31 };
|
||||
|
||||
// Will show a detailed diff of the differences
|
||||
expect(actual).toEqual(expected);
|
||||
});
|
||||
```
|
||||
|
||||
#### Available Assertions
|
||||
|
||||
```typescript
|
||||
// Equality
|
||||
expect(value).toEqual(expected);
|
||||
expect(value).toBe(expected);
|
||||
|
||||
// Truthiness
|
||||
expect(value).toBeTruthy();
|
||||
expect(value).toBeFalsy();
|
||||
|
||||
// Type checks
|
||||
expect(value).toBeType('string');
|
||||
|
||||
// Strings
|
||||
expect(string).toMatch(/pattern/);
|
||||
expect(string).toContain('substring');
|
||||
|
||||
// Arrays
|
||||
expect(array).toContain(item);
|
||||
|
||||
// Exceptions
|
||||
expect(fn).toThrow();
|
||||
expect(fn).toThrow('error message');
|
||||
|
||||
// Async
|
||||
await expect(promise).toResolve();
|
||||
await expect(promise).toReject();
|
||||
```
|
||||
|
||||
### Test Tagging and Filtering
|
||||
|
||||
Tag tests for selective execution:
|
||||
|
||||
```typescript
|
||||
// Define tests with tags
|
||||
tap.tags('integration', 'slow').test('complex test', async () => {
|
||||
// test code
|
||||
});
|
||||
|
||||
tap.tags('unit').test('fast test', async () => {
|
||||
// test code
|
||||
});
|
||||
```
|
||||
|
||||
Filter tests by setting the environment variable:
|
||||
|
||||
```bash
|
||||
TSTEST_FILTER_TAGS=unit tstest test/mytest.node.ts
|
||||
```
|
||||
|
||||
### TapTools
|
||||
|
||||
Each test receives a `tapTools` instance with utilities:
|
||||
|
||||
```typescript
|
||||
tap.test('should have utilities', async (tapTools) => {
|
||||
// Mark test as skipped
|
||||
tapTools.markAsSkipped('reason');
|
||||
|
||||
// Mark as todo
|
||||
tapTools.todo('not implemented');
|
||||
|
||||
// Configure retries
|
||||
tapTools.retry(3);
|
||||
|
||||
// Log test output
|
||||
tapTools.log('debug message');
|
||||
});
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Pre-Tasks
|
||||
|
||||
Run setup tasks before any tests execute:
|
||||
|
||||
```typescript
|
||||
tap.preTask('setup database', async () => {
|
||||
// Runs before any tests
|
||||
});
|
||||
|
||||
tap.test('first test', async () => {
|
||||
// Database is ready
|
||||
});
|
||||
```
|
||||
|
||||
### Test Priority
|
||||
|
||||
Organize tests by priority level:
|
||||
|
||||
```typescript
|
||||
tap.priority('high').test('critical test', async () => { });
|
||||
tap.priority('medium').test('normal test', async () => { });
|
||||
tap.priority('low').test('optional test', async () => { });
|
||||
```
|
||||
|
||||
### Nested Suites
|
||||
|
||||
Create deeply nested test organization:
|
||||
|
||||
```typescript
|
||||
tap.describe('API', () => {
|
||||
tap.describe('Users', () => {
|
||||
tap.describe('GET /users', () => {
|
||||
tap.test('should return all users', async () => { });
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Protocol Events
|
||||
|
||||
Access real-time test events for custom tooling:
|
||||
|
||||
```typescript
|
||||
import { setProtocolEmitter } from '@git.zone/tstest/tapbundle';
|
||||
|
||||
// Get access to protocol emitter for custom event handling
|
||||
// Events: test:started, test:completed, assertion:failed, suite:started, suite:completed
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always export `tap.start()`** at the end of test files:
|
||||
```typescript
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
2. **Use descriptive test names** that explain what is being tested:
|
||||
```typescript
|
||||
tap.test('should return 404 when user does not exist', async () => { });
|
||||
```
|
||||
|
||||
3. **Group related tests** with `describe()` blocks:
|
||||
```typescript
|
||||
tap.describe('User validation', () => {
|
||||
// All user validation tests
|
||||
});
|
||||
```
|
||||
|
||||
4. **Leverage lifecycle hooks** to reduce duplication:
|
||||
```typescript
|
||||
tap.beforeEach(async () => {
|
||||
// Common setup
|
||||
});
|
||||
```
|
||||
|
||||
5. **Tag tests appropriately** for flexible test execution:
|
||||
```typescript
|
||||
tap.tags('integration', 'database').test('...', async () => { });
|
||||
```
|
||||
|
||||
## TypeScript Support
|
||||
|
||||
tapbundle is written in TypeScript and provides full type definitions. The `Tap` class accepts a generic type for shared context:
|
||||
|
||||
```typescript
|
||||
interface MyTestContext {
|
||||
db: DatabaseConnection;
|
||||
user: User;
|
||||
}
|
||||
|
||||
const tap = new Tap<MyTestContext>();
|
||||
|
||||
tap.test('should use context', async (tapTools) => {
|
||||
// tapTools is typed with MyTestContext
|
||||
});
|
||||
```
|
||||
|
||||
## Legal
|
||||
|
||||
This project is licensed under MIT.
|
||||
|
||||
© 2025 Task Venture Capital GmbH. All rights reserved.
|
||||
117
ts_tapbundle/tapbundle.classes.settingsmanager.ts
Normal file
117
ts_tapbundle/tapbundle.classes.settingsmanager.ts
Normal file
@@ -0,0 +1,117 @@
|
||||
import type { ITapSettings, ISettingsManager } from './tapbundle.interfaces.js';
|
||||
|
||||
export class SettingsManager implements ISettingsManager {
|
||||
private globalSettings: ITapSettings = {};
|
||||
private fileSettings: ITapSettings = {};
|
||||
private testSettings: Map<string, ITapSettings> = new Map();
|
||||
|
||||
// Default settings
|
||||
private defaultSettings: ITapSettings = {
|
||||
timeout: undefined, // No timeout by default
|
||||
slowThreshold: 1000, // 1 second
|
||||
bail: false,
|
||||
retries: 0,
|
||||
retryDelay: 0,
|
||||
suppressConsole: false,
|
||||
verboseErrors: true,
|
||||
showTestDuration: true,
|
||||
maxConcurrency: 5,
|
||||
isolateTests: false,
|
||||
enableSnapshots: true,
|
||||
snapshotDirectory: '.snapshots',
|
||||
updateSnapshots: false,
|
||||
};
|
||||
|
||||
/**
|
||||
* Get merged settings for current context
|
||||
*/
|
||||
public getSettings(): ITapSettings {
|
||||
return this.mergeSettings(
|
||||
this.defaultSettings,
|
||||
this.globalSettings,
|
||||
this.fileSettings
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Set global settings (from 00init.ts or tap.settings())
|
||||
*/
|
||||
public setGlobalSettings(settings: ITapSettings): void {
|
||||
this.globalSettings = { ...this.globalSettings, ...settings };
|
||||
}
|
||||
|
||||
/**
|
||||
* Set file-level settings
|
||||
*/
|
||||
public setFileSettings(settings: ITapSettings): void {
|
||||
this.fileSettings = { ...this.fileSettings, ...settings };
|
||||
}
|
||||
|
||||
/**
|
||||
* Set test-specific settings
|
||||
*/
|
||||
public setTestSettings(testId: string, settings: ITapSettings): void {
|
||||
const existingSettings = this.testSettings.get(testId) || {};
|
||||
this.testSettings.set(testId, { ...existingSettings, ...settings });
|
||||
}
|
||||
|
||||
/**
|
||||
* Get settings for specific test
|
||||
*/
|
||||
public getTestSettings(testId: string): ITapSettings {
|
||||
const testSpecificSettings = this.testSettings.get(testId) || {};
|
||||
return this.mergeSettings(
|
||||
this.defaultSettings,
|
||||
this.globalSettings,
|
||||
this.fileSettings,
|
||||
testSpecificSettings
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge settings with proper inheritance
|
||||
* Later settings override earlier ones
|
||||
*/
|
||||
private mergeSettings(...settingsArray: ITapSettings[]): ITapSettings {
|
||||
const result: ITapSettings = {};
|
||||
|
||||
for (const settings of settingsArray) {
|
||||
// Simple properties - later values override
|
||||
if (settings.timeout !== undefined) result.timeout = settings.timeout;
|
||||
if (settings.slowThreshold !== undefined) result.slowThreshold = settings.slowThreshold;
|
||||
if (settings.bail !== undefined) result.bail = settings.bail;
|
||||
if (settings.retries !== undefined) result.retries = settings.retries;
|
||||
if (settings.retryDelay !== undefined) result.retryDelay = settings.retryDelay;
|
||||
if (settings.suppressConsole !== undefined) result.suppressConsole = settings.suppressConsole;
|
||||
if (settings.verboseErrors !== undefined) result.verboseErrors = settings.verboseErrors;
|
||||
if (settings.showTestDuration !== undefined) result.showTestDuration = settings.showTestDuration;
|
||||
if (settings.maxConcurrency !== undefined) result.maxConcurrency = settings.maxConcurrency;
|
||||
if (settings.isolateTests !== undefined) result.isolateTests = settings.isolateTests;
|
||||
if (settings.enableSnapshots !== undefined) result.enableSnapshots = settings.enableSnapshots;
|
||||
if (settings.snapshotDirectory !== undefined) result.snapshotDirectory = settings.snapshotDirectory;
|
||||
if (settings.updateSnapshots !== undefined) result.updateSnapshots = settings.updateSnapshots;
|
||||
|
||||
// Lifecycle hooks - later ones override
|
||||
if (settings.beforeAll !== undefined) result.beforeAll = settings.beforeAll;
|
||||
if (settings.afterAll !== undefined) result.afterAll = settings.afterAll;
|
||||
if (settings.beforeEach !== undefined) result.beforeEach = settings.beforeEach;
|
||||
if (settings.afterEach !== undefined) result.afterEach = settings.afterEach;
|
||||
|
||||
// Environment variables - merge
|
||||
if (settings.env) {
|
||||
result.env = { ...result.env, ...settings.env };
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all settings (useful for testing)
|
||||
*/
|
||||
public clearSettings(): void {
|
||||
this.globalSettings = {};
|
||||
this.fileSettings = {};
|
||||
this.testSettings.clear();
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,9 @@ import * as plugins from './tapbundle.plugins.js';
|
||||
|
||||
import { type IPreTaskFunction, PreTask } from './tapbundle.classes.pretask.js';
|
||||
import { TapTest, type ITestFunction } from './tapbundle.classes.taptest.js';
|
||||
import { ProtocolEmitter, type ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
import type { ITapSettings } from './tapbundle.interfaces.js';
|
||||
import { SettingsManager } from './tapbundle.classes.settingsmanager.js';
|
||||
|
||||
export interface ITestSuite {
|
||||
description: string;
|
||||
@@ -102,6 +105,8 @@ class TestBuilder<T> {
|
||||
}
|
||||
|
||||
export class Tap<T> {
|
||||
private protocolEmitter = new ProtocolEmitter();
|
||||
private settingsManager = new SettingsManager();
|
||||
private _skipCount = 0;
|
||||
private _filterTags: string[] = [];
|
||||
|
||||
@@ -139,12 +144,27 @@ export class Tap<T> {
|
||||
*/
|
||||
public skip = {
|
||||
test: (descriptionArg: string, functionArg: ITestFunction<T>) => {
|
||||
console.log(`skipped test: ${descriptionArg}`);
|
||||
this._skipCount++;
|
||||
const skippedTest = this.test(descriptionArg, functionArg, 'skip');
|
||||
return skippedTest;
|
||||
},
|
||||
testParallel: (descriptionArg: string, functionArg: ITestFunction<T>) => {
|
||||
console.log(`skipped test: ${descriptionArg}`);
|
||||
this._skipCount++;
|
||||
const skippedTest = new TapTest<T>({
|
||||
description: descriptionArg,
|
||||
testFunction: functionArg,
|
||||
parallel: true,
|
||||
});
|
||||
|
||||
// Mark as skip mode
|
||||
skippedTest.tapTools.markAsSkipped('Marked as skip');
|
||||
|
||||
// Add to appropriate test list
|
||||
if (this._currentSuite) {
|
||||
this._currentSuite.tests.push(skippedTest);
|
||||
} else {
|
||||
this._tapTests.push(skippedTest);
|
||||
}
|
||||
|
||||
return skippedTest;
|
||||
},
|
||||
};
|
||||
|
||||
@@ -153,7 +173,65 @@ export class Tap<T> {
|
||||
*/
|
||||
public only = {
|
||||
test: (descriptionArg: string, testFunctionArg: ITestFunction<T>) => {
|
||||
this.test(descriptionArg, testFunctionArg, 'only');
|
||||
return this.test(descriptionArg, testFunctionArg, 'only');
|
||||
},
|
||||
testParallel: (descriptionArg: string, testFunctionArg: ITestFunction<T>) => {
|
||||
const onlyTest = new TapTest<T>({
|
||||
description: descriptionArg,
|
||||
testFunction: testFunctionArg,
|
||||
parallel: true,
|
||||
});
|
||||
|
||||
// Add to only tests list
|
||||
this._tapTestsOnly.push(onlyTest);
|
||||
|
||||
return onlyTest;
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* mark a test as todo (not yet implemented)
|
||||
*/
|
||||
public todo = {
|
||||
test: (descriptionArg: string, functionArg?: ITestFunction<T>) => {
|
||||
const defaultFunc = (async () => {}) as ITestFunction<T>;
|
||||
const todoTest = new TapTest<T>({
|
||||
description: descriptionArg,
|
||||
testFunction: functionArg || defaultFunc,
|
||||
parallel: false,
|
||||
});
|
||||
|
||||
// Mark as todo
|
||||
todoTest.tapTools.todo('Marked as todo');
|
||||
|
||||
// Add to appropriate test list
|
||||
if (this._currentSuite) {
|
||||
this._currentSuite.tests.push(todoTest);
|
||||
} else {
|
||||
this._tapTests.push(todoTest);
|
||||
}
|
||||
|
||||
return todoTest;
|
||||
},
|
||||
testParallel: (descriptionArg: string, functionArg?: ITestFunction<T>) => {
|
||||
const defaultFunc = (async () => {}) as ITestFunction<T>;
|
||||
const todoTest = new TapTest<T>({
|
||||
description: descriptionArg,
|
||||
testFunction: functionArg || defaultFunc,
|
||||
parallel: true,
|
||||
});
|
||||
|
||||
// Mark as todo
|
||||
todoTest.tapTools.todo('Marked as todo');
|
||||
|
||||
// Add to appropriate test list
|
||||
if (this._currentSuite) {
|
||||
this._currentSuite.tests.push(todoTest);
|
||||
} else {
|
||||
this._tapTests.push(todoTest);
|
||||
}
|
||||
|
||||
return todoTest;
|
||||
},
|
||||
};
|
||||
|
||||
@@ -163,6 +241,21 @@ export class Tap<T> {
|
||||
private _currentSuite: ITestSuite | null = null;
|
||||
private _rootSuites: ITestSuite[] = [];
|
||||
|
||||
/**
|
||||
* Configure global test settings
|
||||
*/
|
||||
public settings(settings: ITapSettings): this {
|
||||
this.settingsManager.setGlobalSettings(settings);
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current test settings
|
||||
*/
|
||||
public getSettings(): ITapSettings {
|
||||
return this.settingsManager.getSettings();
|
||||
}
|
||||
|
||||
/**
|
||||
* Normal test function, will run one by one
|
||||
* @param testDescription - A description of what the test does
|
||||
@@ -179,14 +272,26 @@ export class Tap<T> {
|
||||
parallel: false,
|
||||
});
|
||||
|
||||
// No options applied here - use the fluent builder syntax instead
|
||||
// Apply default settings from settings manager
|
||||
const settings = this.settingsManager.getSettings();
|
||||
if (settings.timeout !== undefined) {
|
||||
localTest.timeoutMs = settings.timeout;
|
||||
}
|
||||
if (settings.retries !== undefined) {
|
||||
localTest.tapTools.retry(settings.retries);
|
||||
}
|
||||
|
||||
// Handle skip mode
|
||||
if (modeArg === 'skip') {
|
||||
localTest.tapTools.markAsSkipped('Marked as skip');
|
||||
}
|
||||
|
||||
// If we're in a suite, add test to the suite
|
||||
if (this._currentSuite) {
|
||||
this._currentSuite.tests.push(localTest);
|
||||
} else {
|
||||
// Otherwise add to global test list
|
||||
if (modeArg === 'normal') {
|
||||
if (modeArg === 'normal' || modeArg === 'skip') {
|
||||
this._tapTests.push(localTest);
|
||||
} else if (modeArg === 'only') {
|
||||
this._tapTestsOnly.push(localTest);
|
||||
@@ -211,6 +316,15 @@ export class Tap<T> {
|
||||
parallel: true,
|
||||
});
|
||||
|
||||
// Apply default settings from settings manager
|
||||
const settings = this.settingsManager.getSettings();
|
||||
if (settings.timeout !== undefined) {
|
||||
localTest.timeoutMs = settings.timeout;
|
||||
}
|
||||
if (settings.retries !== undefined) {
|
||||
localTest.tapTools.retry(settings.retries);
|
||||
}
|
||||
|
||||
if (this._currentSuite) {
|
||||
this._currentSuite.tests.push(localTest);
|
||||
} else {
|
||||
@@ -336,8 +450,27 @@ export class Tap<T> {
|
||||
await preTask.run();
|
||||
}
|
||||
|
||||
// Count actual tests that will be run
|
||||
console.log(`1..${concerningTests.length}`);
|
||||
// Emit protocol header and TAP version
|
||||
console.log(this.protocolEmitter.emitProtocolHeader());
|
||||
console.log(this.protocolEmitter.emitTapVersion(13));
|
||||
|
||||
// Emit test plan
|
||||
const plan = {
|
||||
start: 1,
|
||||
end: concerningTests.length
|
||||
};
|
||||
console.log(this.protocolEmitter.emitPlan(plan));
|
||||
|
||||
// Run global beforeAll hook if configured
|
||||
const settings = this.settingsManager.getSettings();
|
||||
if (settings.beforeAll) {
|
||||
try {
|
||||
await settings.beforeAll();
|
||||
} catch (error) {
|
||||
console.error('Error in beforeAll hook:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Run tests from suites with lifecycle hooks
|
||||
let testKey = 0;
|
||||
@@ -365,6 +498,33 @@ export class Tap<T> {
|
||||
});
|
||||
|
||||
for (const currentTest of nonSuiteTests) {
|
||||
// Wrap test function with global lifecycle hooks
|
||||
const originalFunction = currentTest.testFunction;
|
||||
const testName = currentTest.description;
|
||||
currentTest.testFunction = async (tapTools) => {
|
||||
// Run global beforeEach if configured
|
||||
if (settings.beforeEach) {
|
||||
await settings.beforeEach(testName);
|
||||
}
|
||||
|
||||
// Run the actual test
|
||||
let testPassed = true;
|
||||
let result: any;
|
||||
try {
|
||||
result = await originalFunction(tapTools);
|
||||
} catch (error) {
|
||||
testPassed = false;
|
||||
throw error;
|
||||
} finally {
|
||||
// Run global afterEach if configured
|
||||
if (settings.afterEach) {
|
||||
await settings.afterEach(testName, testPassed);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
};
|
||||
|
||||
const testPromise = currentTest.run(testKey++);
|
||||
if (currentTest.parallel) {
|
||||
promiseArray.push(testPromise);
|
||||
@@ -394,6 +554,16 @@ export class Tap<T> {
|
||||
console.log(failReason);
|
||||
}
|
||||
|
||||
// Run global afterAll hook if configured
|
||||
if (settings.afterAll) {
|
||||
try {
|
||||
await settings.afterAll();
|
||||
} catch (error) {
|
||||
console.error('Error in afterAll hook:', error);
|
||||
// Don't throw here, we want to complete the test run
|
||||
}
|
||||
}
|
||||
|
||||
if (optionsArg && optionsArg.throwOnError && failReasons.length > 0) {
|
||||
if (!smartenvInstance.isBrowser && typeof process !== 'undefined') process.exit(1);
|
||||
}
|
||||
@@ -402,6 +572,13 @@ export class Tap<T> {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit an event
|
||||
*/
|
||||
private emitEvent(event: ITestEvent) {
|
||||
console.log(this.protocolEmitter.emitEvent(event));
|
||||
}
|
||||
|
||||
/**
|
||||
* Run tests in a suite with lifecycle hooks
|
||||
*/
|
||||
@@ -412,6 +589,14 @@ export class Tap<T> {
|
||||
context: { testKey: number }
|
||||
) {
|
||||
for (const suite of suites) {
|
||||
// Emit suite:started event
|
||||
this.emitEvent({
|
||||
eventType: 'suite:started',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
suiteName: suite.description
|
||||
}
|
||||
});
|
||||
// Run beforeEach from parent suites
|
||||
const beforeEachFunctions: ITestFunction<any>[] = [];
|
||||
let currentSuite: ITestSuite | null = suite;
|
||||
@@ -426,27 +611,46 @@ export class Tap<T> {
|
||||
for (const test of suite.tests) {
|
||||
// Create wrapper test function that includes lifecycle hooks
|
||||
const originalFunction = test.testFunction;
|
||||
const testName = test.description;
|
||||
test.testFunction = async (tapTools) => {
|
||||
// Run all beforeEach hooks
|
||||
// Run global beforeEach if configured
|
||||
const settings = this.settingsManager.getSettings();
|
||||
if (settings.beforeEach) {
|
||||
await settings.beforeEach(testName);
|
||||
}
|
||||
|
||||
// Run all suite beforeEach hooks
|
||||
for (const beforeEach of beforeEachFunctions) {
|
||||
await beforeEach(tapTools);
|
||||
}
|
||||
|
||||
// Run the actual test
|
||||
const result = await originalFunction(tapTools);
|
||||
|
||||
// Run afterEach hooks in reverse order
|
||||
const afterEachFunctions: ITestFunction<any>[] = [];
|
||||
currentSuite = suite;
|
||||
while (currentSuite) {
|
||||
if (currentSuite.afterEach) {
|
||||
afterEachFunctions.push(currentSuite.afterEach);
|
||||
let testPassed = true;
|
||||
let result: any;
|
||||
try {
|
||||
result = await originalFunction(tapTools);
|
||||
} catch (error) {
|
||||
testPassed = false;
|
||||
throw error;
|
||||
} finally {
|
||||
// Run afterEach hooks in reverse order
|
||||
const afterEachFunctions: ITestFunction<any>[] = [];
|
||||
currentSuite = suite;
|
||||
while (currentSuite) {
|
||||
if (currentSuite.afterEach) {
|
||||
afterEachFunctions.push(currentSuite.afterEach);
|
||||
}
|
||||
currentSuite = currentSuite.parent || null;
|
||||
}
|
||||
|
||||
for (const afterEach of afterEachFunctions) {
|
||||
await afterEach(tapTools);
|
||||
}
|
||||
|
||||
// Run global afterEach if configured
|
||||
if (settings.afterEach) {
|
||||
await settings.afterEach(testName, testPassed);
|
||||
}
|
||||
currentSuite = currentSuite.parent || null;
|
||||
}
|
||||
|
||||
for (const afterEach of afterEachFunctions) {
|
||||
await afterEach(tapTools);
|
||||
}
|
||||
|
||||
return result;
|
||||
@@ -462,6 +666,15 @@ export class Tap<T> {
|
||||
|
||||
// Recursively run child suites
|
||||
await this._runSuite(suite, suite.children, promiseArray, context);
|
||||
|
||||
// Emit suite:completed event
|
||||
this.emitEvent({
|
||||
eventType: 'suite:completed',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
suiteName: suite.description
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import * as plugins from './tapbundle.plugins.js';
|
||||
import { tapCreator } from './tapbundle.tapcreator.js';
|
||||
import { TapTools, SkipError } from './tapbundle.classes.taptools.js';
|
||||
import { ProtocolEmitter, type ITestEvent } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
import { setProtocolEmitter } from './tapbundle.expect.wrapper.js';
|
||||
|
||||
// imported interfaces
|
||||
import { Deferred } from '@push.rocks/smartpromise';
|
||||
@@ -9,9 +11,9 @@ import { HrtMeasurement } from '@push.rocks/smarttime';
|
||||
// interfaces
|
||||
export type TTestStatus = 'success' | 'error' | 'pending' | 'errorAfterSuccess' | 'timeout' | 'skipped';
|
||||
|
||||
export interface ITestFunction<T> {
|
||||
(tapTools?: TapTools): Promise<T>;
|
||||
}
|
||||
export type ITestFunction<T> =
|
||||
| ((tapTools: TapTools) => Promise<T>)
|
||||
| (() => Promise<T>);
|
||||
|
||||
export class TapTest<T = unknown> {
|
||||
public description: string;
|
||||
@@ -32,6 +34,7 @@ export class TapTest<T = unknown> {
|
||||
public testPromise: Promise<TapTest<T>> = this.testDeferred.promise;
|
||||
private testResultDeferred: Deferred<T> = plugins.smartpromise.defer();
|
||||
public testResultPromise: Promise<T> = this.testResultDeferred.promise;
|
||||
private protocolEmitter = new ProtocolEmitter();
|
||||
/**
|
||||
* constructor
|
||||
*/
|
||||
@@ -48,6 +51,13 @@ export class TapTest<T = unknown> {
|
||||
this.testFunction = optionsArg.testFunction;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit an event
|
||||
*/
|
||||
private emitEvent(event: ITestEvent) {
|
||||
console.log(this.protocolEmitter.emitEvent(event));
|
||||
}
|
||||
|
||||
/**
|
||||
* run the test
|
||||
*/
|
||||
@@ -55,11 +65,74 @@ export class TapTest<T = unknown> {
|
||||
this.testKey = testKeyArg;
|
||||
const testNumber = testKeyArg + 1;
|
||||
|
||||
// Emit test:queued event
|
||||
this.emitEvent({
|
||||
eventType: 'test:queued',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber,
|
||||
description: this.description
|
||||
}
|
||||
});
|
||||
|
||||
// Handle todo tests
|
||||
if (this.isTodo) {
|
||||
const todoText = this.todoReason ? `# TODO ${this.todoReason}` : '# TODO';
|
||||
console.log(`ok ${testNumber} - ${this.description} ${todoText}`);
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber,
|
||||
description: this.description,
|
||||
directive: {
|
||||
type: 'todo' as const,
|
||||
reason: this.todoReason
|
||||
}
|
||||
};
|
||||
const lines = this.protocolEmitter.emitTest(testResult);
|
||||
lines.forEach((line: string) => console.log(line));
|
||||
this.status = 'success';
|
||||
|
||||
// Emit test:completed event for todo test
|
||||
this.emitEvent({
|
||||
eventType: 'test:completed',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber,
|
||||
description: this.description,
|
||||
duration: 0,
|
||||
error: undefined
|
||||
}
|
||||
});
|
||||
|
||||
this.testDeferred.resolve(this);
|
||||
return;
|
||||
}
|
||||
|
||||
// Handle pre-marked skip tests
|
||||
if (this.tapTools.isSkipped) {
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber,
|
||||
description: this.description,
|
||||
directive: {
|
||||
type: 'skip' as const,
|
||||
reason: this.tapTools.skipReason || 'Marked as skip'
|
||||
}
|
||||
};
|
||||
const lines = this.protocolEmitter.emitTest(testResult);
|
||||
lines.forEach((line: string) => console.log(line));
|
||||
this.status = 'skipped';
|
||||
|
||||
// Emit test:completed event for skipped test
|
||||
this.emitEvent({
|
||||
eventType: 'test:completed',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber,
|
||||
description: this.description,
|
||||
duration: 0,
|
||||
error: undefined
|
||||
}
|
||||
});
|
||||
|
||||
this.testDeferred.resolve(this);
|
||||
return;
|
||||
}
|
||||
@@ -71,6 +144,20 @@ export class TapTest<T = unknown> {
|
||||
for (let attempt = 0; attempt <= maxRetries; attempt++) {
|
||||
this.hrtMeasurement.start();
|
||||
|
||||
// Emit test:started event
|
||||
this.emitEvent({
|
||||
eventType: 'test:started',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber,
|
||||
description: this.description,
|
||||
retry: attempt > 0 ? attempt : undefined
|
||||
}
|
||||
});
|
||||
|
||||
// Set protocol emitter for enhanced expect
|
||||
setProtocolEmitter(this.protocolEmitter);
|
||||
|
||||
try {
|
||||
// Set up timeout if specified
|
||||
let timeoutHandle: any;
|
||||
@@ -86,7 +173,9 @@ export class TapTest<T = unknown> {
|
||||
}
|
||||
|
||||
// Run the test function with potential timeout
|
||||
const testPromise = this.testFunction(this.tapTools);
|
||||
const testPromise = this.testFunction.length === 0
|
||||
? (this.testFunction as () => Promise<T>)()
|
||||
: (this.testFunction as (tapTools: TapTools) => Promise<T>)(this.tapTools);
|
||||
const testReturnValue = timeoutPromise
|
||||
? await Promise.race([testPromise, timeoutPromise])
|
||||
: await testPromise;
|
||||
@@ -97,10 +186,32 @@ export class TapTest<T = unknown> {
|
||||
}
|
||||
|
||||
this.hrtMeasurement.stop();
|
||||
console.log(
|
||||
`ok ${testNumber} - ${this.description} # time=${this.hrtMeasurement.milliSeconds}ms`,
|
||||
);
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber,
|
||||
description: this.description,
|
||||
metadata: {
|
||||
time: this.hrtMeasurement.milliSeconds,
|
||||
tags: this.tags.length > 0 ? this.tags : undefined,
|
||||
file: this.fileName
|
||||
}
|
||||
};
|
||||
const lines = this.protocolEmitter.emitTest(testResult);
|
||||
lines.forEach((line: string) => console.log(line));
|
||||
this.status = 'success';
|
||||
|
||||
// Emit test:completed event
|
||||
this.emitEvent({
|
||||
eventType: 'test:completed',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber,
|
||||
description: this.description,
|
||||
duration: this.hrtMeasurement.milliSeconds,
|
||||
error: undefined
|
||||
}
|
||||
});
|
||||
|
||||
this.testDeferred.resolve(this);
|
||||
this.testResultDeferred.resolve(testReturnValue);
|
||||
return; // Success, exit retry loop
|
||||
@@ -110,8 +221,31 @@ export class TapTest<T = unknown> {
|
||||
|
||||
// Handle skip
|
||||
if (err instanceof SkipError || err.name === 'SkipError') {
|
||||
console.log(`ok ${testNumber} - ${this.description} # SKIP ${err.message.replace('Skipped: ', '')}`);
|
||||
const testResult = {
|
||||
ok: true,
|
||||
testNumber,
|
||||
description: this.description,
|
||||
directive: {
|
||||
type: 'skip' as const,
|
||||
reason: err.message.replace('Skipped: ', '')
|
||||
}
|
||||
};
|
||||
const lines = this.protocolEmitter.emitTest(testResult);
|
||||
lines.forEach((line: string) => console.log(line));
|
||||
this.status = 'skipped';
|
||||
|
||||
// Emit test:completed event for skipped test
|
||||
this.emitEvent({
|
||||
eventType: 'test:completed',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber,
|
||||
description: this.description,
|
||||
duration: this.hrtMeasurement.milliSeconds,
|
||||
error: undefined
|
||||
}
|
||||
});
|
||||
|
||||
this.testDeferred.resolve(this);
|
||||
return;
|
||||
}
|
||||
@@ -120,17 +254,48 @@ export class TapTest<T = unknown> {
|
||||
|
||||
// If we have retries left, try again
|
||||
if (attempt < maxRetries) {
|
||||
console.log(
|
||||
`# Retry ${attempt + 1}/${maxRetries} for test: ${this.description}`,
|
||||
);
|
||||
console.log(this.protocolEmitter.emitComment(`Retry ${attempt + 1}/${maxRetries} for test: ${this.description}`));
|
||||
this.tapTools._incrementRetryCount();
|
||||
continue;
|
||||
}
|
||||
|
||||
// Final failure
|
||||
console.log(
|
||||
`not ok ${testNumber} - ${this.description} # time=${this.hrtMeasurement.milliSeconds}ms`,
|
||||
);
|
||||
const testResult = {
|
||||
ok: false,
|
||||
testNumber,
|
||||
description: this.description,
|
||||
metadata: {
|
||||
time: this.hrtMeasurement.milliSeconds,
|
||||
retry: this.tapTools.retryCount,
|
||||
maxRetries: maxRetries > 0 ? maxRetries : undefined,
|
||||
error: {
|
||||
message: lastError.message || String(lastError),
|
||||
stack: lastError.stack,
|
||||
code: lastError.code
|
||||
},
|
||||
tags: this.tags.length > 0 ? this.tags : undefined,
|
||||
file: this.fileName
|
||||
}
|
||||
};
|
||||
const lines = this.protocolEmitter.emitTest(testResult);
|
||||
lines.forEach((line: string) => console.log(line));
|
||||
|
||||
// Emit test:completed event for failed test
|
||||
this.emitEvent({
|
||||
eventType: 'test:completed',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber,
|
||||
description: this.description,
|
||||
duration: this.hrtMeasurement.milliSeconds,
|
||||
error: {
|
||||
message: lastError.message || String(lastError),
|
||||
stack: lastError.stack,
|
||||
type: 'runtime' as const
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
this.testDeferred.resolve(this);
|
||||
this.testResultDeferred.resolve(err);
|
||||
|
||||
|
||||
@@ -22,6 +22,10 @@ export class TapTools {
|
||||
public testData: any = {};
|
||||
private static _sharedContext = new Map<string, any>();
|
||||
private _snapshotPath: string = '';
|
||||
|
||||
// Flags for skip/todo
|
||||
private _isSkipped = false;
|
||||
private _skipReason?: string;
|
||||
|
||||
constructor(TapTestArg: TapTest<any>) {
|
||||
this._tapTest = TapTestArg;
|
||||
@@ -45,9 +49,33 @@ export class TapTools {
|
||||
* skip the rest of the test
|
||||
*/
|
||||
public skip(reason?: string): never {
|
||||
this._isSkipped = true;
|
||||
this._skipReason = reason;
|
||||
const skipMessage = reason ? `Skipped: ${reason}` : 'Skipped';
|
||||
throw new SkipError(skipMessage);
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark test as skipped without throwing (for pre-marking)
|
||||
*/
|
||||
public markAsSkipped(reason?: string): void {
|
||||
this._isSkipped = true;
|
||||
this._skipReason = reason;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if test is marked as skipped
|
||||
*/
|
||||
public get isSkipped(): boolean {
|
||||
return this._isSkipped;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get skip reason
|
||||
*/
|
||||
public get skipReason(): string | undefined {
|
||||
return this._skipReason;
|
||||
}
|
||||
|
||||
/**
|
||||
* conditionally skip the rest of the test
|
||||
|
||||
81
ts_tapbundle/tapbundle.expect.wrapper.ts
Normal file
81
ts_tapbundle/tapbundle.expect.wrapper.ts
Normal file
@@ -0,0 +1,81 @@
|
||||
import { expect as smartExpect } from '@push.rocks/smartexpect';
|
||||
import { generateDiff } from './tapbundle.utilities.diff.js';
|
||||
import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
import type { IEnhancedError } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
|
||||
// Store the protocol emitter for event emission
|
||||
let protocolEmitter: ProtocolEmitter | null = null;
|
||||
|
||||
/**
|
||||
* Set the protocol emitter for enhanced error reporting
|
||||
*/
|
||||
export function setProtocolEmitter(emitter: ProtocolEmitter) {
|
||||
protocolEmitter = emitter;
|
||||
}
|
||||
|
||||
/**
|
||||
* Enhanced expect wrapper that captures assertion failures and generates diffs
|
||||
*/
|
||||
export function createEnhancedExpect() {
|
||||
return new Proxy(smartExpect, {
|
||||
apply(target, thisArg, argumentsList: any[]) {
|
||||
const expectation = target.apply(thisArg, argumentsList);
|
||||
|
||||
// Wrap common assertion methods
|
||||
const wrappedExpectation = new Proxy(expectation, {
|
||||
get(target, prop, receiver) {
|
||||
const originalValue = Reflect.get(target, prop, receiver);
|
||||
|
||||
// Wrap assertion methods that compare values
|
||||
if (typeof prop === 'string' && typeof originalValue === 'function' && ['toEqual', 'toBe', 'toMatch', 'toContain'].includes(prop)) {
|
||||
return function(expected: any) {
|
||||
try {
|
||||
return originalValue.apply(target, arguments);
|
||||
} catch (error: any) {
|
||||
// Enhance the error with diff information
|
||||
const actual = argumentsList[0];
|
||||
const enhancedError: IEnhancedError = {
|
||||
message: error.message,
|
||||
stack: error.stack,
|
||||
actual,
|
||||
expected,
|
||||
type: 'assertion'
|
||||
};
|
||||
|
||||
// Generate diff if applicable
|
||||
if (prop === 'toEqual' || prop === 'toBe') {
|
||||
const diff = generateDiff(expected, actual);
|
||||
if (diff) {
|
||||
enhancedError.diff = diff;
|
||||
}
|
||||
}
|
||||
|
||||
// Emit assertion:failed event if protocol emitter is available
|
||||
if (protocolEmitter) {
|
||||
const event = {
|
||||
eventType: 'assertion:failed' as const,
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
error: enhancedError
|
||||
}
|
||||
};
|
||||
console.log(protocolEmitter.emitEvent(event));
|
||||
}
|
||||
|
||||
// Re-throw the enhanced error
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
return originalValue;
|
||||
}
|
||||
});
|
||||
|
||||
return wrappedExpectation;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Create the enhanced expect function
|
||||
export const expect = createEnhancedExpect();
|
||||
46
ts_tapbundle/tapbundle.interfaces.ts
Normal file
46
ts_tapbundle/tapbundle.interfaces.ts
Normal file
@@ -0,0 +1,46 @@
|
||||
export interface ITapSettings {
|
||||
// Timing
|
||||
timeout?: number; // Default timeout for all tests (ms)
|
||||
slowThreshold?: number; // Mark tests as slow if they exceed this (ms)
|
||||
|
||||
// Execution Control
|
||||
bail?: boolean; // Stop on first test failure
|
||||
retries?: number; // Number of retries for failed tests
|
||||
retryDelay?: number; // Delay between retries (ms)
|
||||
|
||||
// Output Control
|
||||
suppressConsole?: boolean; // Suppress console output in passing tests
|
||||
verboseErrors?: boolean; // Show full stack traces
|
||||
showTestDuration?: boolean; // Show duration for each test
|
||||
|
||||
// Parallel Execution
|
||||
maxConcurrency?: number; // Max parallel tests (for .para files)
|
||||
isolateTests?: boolean; // Run each test in fresh context
|
||||
|
||||
// Lifecycle Hooks
|
||||
beforeAll?: () => Promise<void> | void;
|
||||
afterAll?: () => Promise<void> | void;
|
||||
beforeEach?: (testName: string) => Promise<void> | void;
|
||||
afterEach?: (testName: string, passed: boolean) => Promise<void> | void;
|
||||
|
||||
// Environment
|
||||
env?: Record<string, string>; // Additional environment variables
|
||||
|
||||
// Features
|
||||
enableSnapshots?: boolean; // Enable snapshot testing
|
||||
snapshotDirectory?: string; // Custom snapshot directory
|
||||
updateSnapshots?: boolean; // Update snapshots instead of comparing
|
||||
}
|
||||
|
||||
export interface ISettingsManager {
|
||||
// Get merged settings for current context
|
||||
getSettings(): ITapSettings;
|
||||
|
||||
// Apply settings at different levels
|
||||
setGlobalSettings(settings: ITapSettings): void;
|
||||
setFileSettings(settings: ITapSettings): void;
|
||||
setTestSettings(testId: string, settings: ITapSettings): void;
|
||||
|
||||
// Get settings for specific test
|
||||
getTestSettings(testId: string): ITapSettings;
|
||||
}
|
||||
188
ts_tapbundle/tapbundle.utilities.diff.ts
Normal file
188
ts_tapbundle/tapbundle.utilities.diff.ts
Normal file
@@ -0,0 +1,188 @@
|
||||
import type { IDiffResult, IDiffChange } from '../dist_ts_tapbundle_protocol/index.js';
|
||||
|
||||
/**
|
||||
* Generate a diff between two values
|
||||
*/
|
||||
export function generateDiff(expected: any, actual: any, context: number = 3): IDiffResult | null {
|
||||
// Handle same values
|
||||
if (expected === actual) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Determine diff type based on values
|
||||
if (typeof expected === 'string' && typeof actual === 'string') {
|
||||
return generateStringDiff(expected, actual, context);
|
||||
} else if (Array.isArray(expected) && Array.isArray(actual)) {
|
||||
return generateArrayDiff(expected, actual);
|
||||
} else if (expected && actual && typeof expected === 'object' && typeof actual === 'object') {
|
||||
return generateObjectDiff(expected, actual);
|
||||
} else {
|
||||
return generatePrimitiveDiff(expected, actual);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate diff for primitive values
|
||||
*/
|
||||
function generatePrimitiveDiff(expected: any, actual: any): IDiffResult {
|
||||
return {
|
||||
type: 'primitive',
|
||||
changes: [{
|
||||
type: 'modify',
|
||||
oldValue: expected,
|
||||
newValue: actual
|
||||
}]
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate diff for strings (line-by-line)
|
||||
*/
|
||||
function generateStringDiff(expected: string, actual: string, context: number): IDiffResult {
|
||||
const expectedLines = expected.split('\n');
|
||||
const actualLines = actual.split('\n');
|
||||
const changes: IDiffChange[] = [];
|
||||
|
||||
// Simple line-by-line diff
|
||||
const maxLines = Math.max(expectedLines.length, actualLines.length);
|
||||
|
||||
for (let i = 0; i < maxLines; i++) {
|
||||
const expectedLine = expectedLines[i];
|
||||
const actualLine = actualLines[i];
|
||||
|
||||
if (expectedLine === undefined) {
|
||||
changes.push({
|
||||
type: 'add',
|
||||
line: i,
|
||||
content: actualLine
|
||||
});
|
||||
} else if (actualLine === undefined) {
|
||||
changes.push({
|
||||
type: 'remove',
|
||||
line: i,
|
||||
content: expectedLine
|
||||
});
|
||||
} else if (expectedLine !== actualLine) {
|
||||
changes.push({
|
||||
type: 'remove',
|
||||
line: i,
|
||||
content: expectedLine
|
||||
});
|
||||
changes.push({
|
||||
type: 'add',
|
||||
line: i,
|
||||
content: actualLine
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
type: 'string',
|
||||
changes,
|
||||
context
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate diff for arrays
|
||||
*/
|
||||
function generateArrayDiff(expected: any[], actual: any[]): IDiffResult {
|
||||
const changes: IDiffChange[] = [];
|
||||
const maxLength = Math.max(expected.length, actual.length);
|
||||
|
||||
for (let i = 0; i < maxLength; i++) {
|
||||
const expectedItem = expected[i];
|
||||
const actualItem = actual[i];
|
||||
|
||||
if (i >= expected.length) {
|
||||
changes.push({
|
||||
type: 'add',
|
||||
path: [String(i)],
|
||||
newValue: actualItem
|
||||
});
|
||||
} else if (i >= actual.length) {
|
||||
changes.push({
|
||||
type: 'remove',
|
||||
path: [String(i)],
|
||||
oldValue: expectedItem
|
||||
});
|
||||
} else if (!deepEqual(expectedItem, actualItem)) {
|
||||
changes.push({
|
||||
type: 'modify',
|
||||
path: [String(i)],
|
||||
oldValue: expectedItem,
|
||||
newValue: actualItem
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
type: 'array',
|
||||
changes
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate diff for objects
|
||||
*/
|
||||
function generateObjectDiff(expected: any, actual: any): IDiffResult {
|
||||
const changes: IDiffChange[] = [];
|
||||
const allKeys = new Set([...Object.keys(expected), ...Object.keys(actual)]);
|
||||
|
||||
for (const key of allKeys) {
|
||||
const expectedValue = expected[key];
|
||||
const actualValue = actual[key];
|
||||
|
||||
if (!(key in expected)) {
|
||||
changes.push({
|
||||
type: 'add',
|
||||
path: [key],
|
||||
newValue: actualValue
|
||||
});
|
||||
} else if (!(key in actual)) {
|
||||
changes.push({
|
||||
type: 'remove',
|
||||
path: [key],
|
||||
oldValue: expectedValue
|
||||
});
|
||||
} else if (!deepEqual(expectedValue, actualValue)) {
|
||||
changes.push({
|
||||
type: 'modify',
|
||||
path: [key],
|
||||
oldValue: expectedValue,
|
||||
newValue: actualValue
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
type: 'object',
|
||||
changes
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Deep equality check
|
||||
*/
|
||||
function deepEqual(a: any, b: any): boolean {
|
||||
if (a === b) return true;
|
||||
|
||||
if (a === null || b === null) return false;
|
||||
if (typeof a !== typeof b) return false;
|
||||
|
||||
if (typeof a === 'object') {
|
||||
if (Array.isArray(a) && Array.isArray(b)) {
|
||||
if (a.length !== b.length) return false;
|
||||
return a.every((item, index) => deepEqual(item, b[index]));
|
||||
}
|
||||
|
||||
const keysA = Object.keys(a);
|
||||
const keysB = Object.keys(b);
|
||||
|
||||
if (keysA.length !== keysB.length) return false;
|
||||
|
||||
return keysA.every(key => deepEqual(a[key], b[key]));
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
"order": 1
|
||||
"order": 2
|
||||
}
|
||||
13
ts_tapbundle_protocol/index.ts
Normal file
13
ts_tapbundle_protocol/index.ts
Normal file
@@ -0,0 +1,13 @@
|
||||
// Protocol V2 - Isomorphic implementation for improved TAP protocol
|
||||
// This module is designed to work in both browser and Node.js environments
|
||||
|
||||
export * from './protocol.types.js';
|
||||
export * from './protocol.emitter.js';
|
||||
export * from './protocol.parser.js';
|
||||
|
||||
// Re-export main classes for convenience
|
||||
export { ProtocolEmitter } from './protocol.emitter.js';
|
||||
export { ProtocolParser } from './protocol.parser.js';
|
||||
|
||||
// Re-export constants
|
||||
export { PROTOCOL_MARKERS, PROTOCOL_VERSION } from './protocol.types.js';
|
||||
196
ts_tapbundle_protocol/protocol.emitter.ts
Normal file
196
ts_tapbundle_protocol/protocol.emitter.ts
Normal file
@@ -0,0 +1,196 @@
|
||||
import type {
|
||||
ITestResult,
|
||||
ITestMetadata,
|
||||
IPlanLine,
|
||||
ISnapshotData,
|
||||
IErrorBlock,
|
||||
ITestEvent
|
||||
} from './protocol.types.js';
|
||||
|
||||
import {
|
||||
PROTOCOL_MARKERS,
|
||||
PROTOCOL_VERSION
|
||||
} from './protocol.types.js';
|
||||
|
||||
/**
|
||||
* ProtocolEmitter generates Protocol V2 messages
|
||||
* This class is used by tapbundle to emit test results in the new protocol format
|
||||
*/
|
||||
export class ProtocolEmitter {
|
||||
/**
|
||||
* Emit protocol version header
|
||||
*/
|
||||
public emitProtocolHeader(): string {
|
||||
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.PROTOCOL_PREFIX}${PROTOCOL_VERSION}${PROTOCOL_MARKERS.END}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit TAP version line
|
||||
*/
|
||||
public emitTapVersion(version: number = 13): string {
|
||||
return `TAP version ${version}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit test plan
|
||||
*/
|
||||
public emitPlan(plan: IPlanLine): string {
|
||||
if (plan.skipAll) {
|
||||
return `1..0 # Skipped: ${plan.skipAll}`;
|
||||
}
|
||||
return `${plan.start}..${plan.end}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit a test result
|
||||
*/
|
||||
public emitTest(result: ITestResult): string[] {
|
||||
const lines: string[] = [];
|
||||
|
||||
// Build the basic TAP line
|
||||
let tapLine = result.ok ? 'ok' : 'not ok';
|
||||
tapLine += ` ${result.testNumber}`;
|
||||
tapLine += ` - ${result.description}`;
|
||||
|
||||
// Add directive if present
|
||||
if (result.directive) {
|
||||
tapLine += ` # ${result.directive.type.toUpperCase()}`;
|
||||
if (result.directive.reason) {
|
||||
tapLine += ` ${result.directive.reason}`;
|
||||
}
|
||||
}
|
||||
|
||||
// Add inline metadata for simple cases
|
||||
if (result.metadata && this.shouldUseInlineMetadata(result.metadata)) {
|
||||
const metaStr = this.createInlineMetadata(result.metadata);
|
||||
if (metaStr) {
|
||||
tapLine += ` ${metaStr}`;
|
||||
}
|
||||
}
|
||||
|
||||
lines.push(tapLine);
|
||||
|
||||
// Add block metadata for complex cases
|
||||
if (result.metadata && !this.shouldUseInlineMetadata(result.metadata)) {
|
||||
lines.push(...this.createBlockMetadata(result.metadata, result.testNumber));
|
||||
}
|
||||
|
||||
return lines;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit a comment line
|
||||
*/
|
||||
public emitComment(comment: string): string {
|
||||
return `# ${comment}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit bailout
|
||||
*/
|
||||
public emitBailout(reason: string): string {
|
||||
return `Bail out! ${reason}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit snapshot data
|
||||
*/
|
||||
public emitSnapshot(snapshot: ISnapshotData): string[] {
|
||||
const lines: string[] = [];
|
||||
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_PREFIX}${snapshot.name}${PROTOCOL_MARKERS.END}`);
|
||||
|
||||
if (snapshot.format === 'json') {
|
||||
lines.push(JSON.stringify(snapshot.content, null, 2));
|
||||
} else {
|
||||
lines.push(String(snapshot.content));
|
||||
}
|
||||
|
||||
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_END}${PROTOCOL_MARKERS.END}`);
|
||||
return lines;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit error block
|
||||
*/
|
||||
public emitError(error: IErrorBlock): string[] {
|
||||
const lines: string[] = [];
|
||||
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_PREFIX}${PROTOCOL_MARKERS.END}`);
|
||||
lines.push(JSON.stringify(error, null, 2));
|
||||
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_END}${PROTOCOL_MARKERS.END}`);
|
||||
return lines;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit test event
|
||||
*/
|
||||
public emitEvent(event: ITestEvent): string {
|
||||
const eventJson = JSON.stringify(event);
|
||||
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.EVENT_PREFIX}${eventJson}${PROTOCOL_MARKERS.END}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if metadata should be inline
|
||||
*/
|
||||
private shouldUseInlineMetadata(metadata: ITestMetadata): boolean {
|
||||
// Use inline for simple metadata (time, retry, simple skip/todo)
|
||||
const hasComplexData = metadata.error ||
|
||||
metadata.custom ||
|
||||
(metadata.tags && metadata.tags.length > 0) ||
|
||||
metadata.file ||
|
||||
metadata.line;
|
||||
|
||||
return !hasComplexData;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create inline metadata string
|
||||
*/
|
||||
private createInlineMetadata(metadata: ITestMetadata): string {
|
||||
const parts: string[] = [];
|
||||
|
||||
if (metadata.time !== undefined) {
|
||||
parts.push(`time:${metadata.time}`);
|
||||
}
|
||||
|
||||
if (metadata.retry !== undefined) {
|
||||
parts.push(`retry:${metadata.retry}`);
|
||||
}
|
||||
|
||||
if (metadata.skip) {
|
||||
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SKIP_PREFIX}${metadata.skip}${PROTOCOL_MARKERS.END}`;
|
||||
}
|
||||
|
||||
if (metadata.todo) {
|
||||
return `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.TODO_PREFIX}${metadata.todo}${PROTOCOL_MARKERS.END}`;
|
||||
}
|
||||
|
||||
if (parts.length > 0) {
|
||||
return `${PROTOCOL_MARKERS.START}${parts.join(',')}${PROTOCOL_MARKERS.END}`;
|
||||
}
|
||||
|
||||
return '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Create block metadata lines
|
||||
*/
|
||||
private createBlockMetadata(metadata: ITestMetadata, testNumber?: number): string[] {
|
||||
const lines: string[] = [];
|
||||
|
||||
// Create a clean metadata object without skip/todo (handled inline)
|
||||
const blockMeta = { ...metadata };
|
||||
delete blockMeta.skip;
|
||||
delete blockMeta.todo;
|
||||
|
||||
// Emit metadata block
|
||||
const metaJson = JSON.stringify(blockMeta);
|
||||
lines.push(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.META_PREFIX}${metaJson}${PROTOCOL_MARKERS.END}`);
|
||||
|
||||
// Emit separate error block if present
|
||||
if (metadata.error) {
|
||||
lines.push(...this.emitError({ testNumber, error: metadata.error }));
|
||||
}
|
||||
|
||||
return lines;
|
||||
}
|
||||
}
|
||||
407
ts_tapbundle_protocol/protocol.parser.ts
Normal file
407
ts_tapbundle_protocol/protocol.parser.ts
Normal file
@@ -0,0 +1,407 @@
|
||||
import type {
|
||||
ITestResult,
|
||||
ITestMetadata,
|
||||
IPlanLine,
|
||||
IProtocolMessage,
|
||||
ISnapshotData,
|
||||
IErrorBlock,
|
||||
ITestEvent
|
||||
} from './protocol.types.js';
|
||||
|
||||
import {
|
||||
PROTOCOL_MARKERS
|
||||
} from './protocol.types.js';
|
||||
|
||||
/**
|
||||
* ProtocolParser parses Protocol V2 messages
|
||||
* This class is used by tstest to parse test results from the new protocol format
|
||||
*/
|
||||
export class ProtocolParser {
|
||||
private protocolVersion: string | null = null;
|
||||
private inBlock = false;
|
||||
private blockType: string | null = null;
|
||||
private blockContent: string[] = [];
|
||||
|
||||
/**
|
||||
* Parse a single line and return protocol messages
|
||||
*/
|
||||
public parseLine(line: string): IProtocolMessage[] {
|
||||
const messages: IProtocolMessage[] = [];
|
||||
|
||||
// Handle block content
|
||||
if (this.inBlock) {
|
||||
if (this.isBlockEnd(line)) {
|
||||
messages.push(this.finalizeBlock());
|
||||
this.inBlock = false;
|
||||
this.blockType = null;
|
||||
this.blockContent = [];
|
||||
} else {
|
||||
this.blockContent.push(line);
|
||||
}
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Check for block start
|
||||
if (this.isBlockStart(line)) {
|
||||
this.inBlock = true;
|
||||
this.blockType = this.extractBlockType(line);
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Check for protocol version
|
||||
const protocolVersion = this.parseProtocolVersion(line);
|
||||
if (protocolVersion) {
|
||||
this.protocolVersion = protocolVersion;
|
||||
messages.push({
|
||||
type: 'protocol',
|
||||
content: { version: protocolVersion }
|
||||
});
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Parse TAP version
|
||||
const tapVersion = this.parseTapVersion(line);
|
||||
if (tapVersion !== null) {
|
||||
messages.push({
|
||||
type: 'version',
|
||||
content: tapVersion
|
||||
});
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Parse plan
|
||||
const plan = this.parsePlan(line);
|
||||
if (plan) {
|
||||
messages.push({
|
||||
type: 'plan',
|
||||
content: plan
|
||||
});
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Parse bailout
|
||||
const bailout = this.parseBailout(line);
|
||||
if (bailout) {
|
||||
messages.push({
|
||||
type: 'bailout',
|
||||
content: bailout
|
||||
});
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Parse comment
|
||||
if (this.isComment(line)) {
|
||||
messages.push({
|
||||
type: 'comment',
|
||||
content: line.substring(2) // Remove "# "
|
||||
});
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Parse test result
|
||||
const testResult = this.parseTestResult(line);
|
||||
if (testResult) {
|
||||
messages.push({
|
||||
type: 'test',
|
||||
content: testResult
|
||||
});
|
||||
return messages;
|
||||
}
|
||||
|
||||
// Parse event
|
||||
const event = this.parseEvent(line);
|
||||
if (event) {
|
||||
messages.push({
|
||||
type: 'event',
|
||||
content: event
|
||||
});
|
||||
return messages;
|
||||
}
|
||||
|
||||
return messages;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse protocol version header
|
||||
*/
|
||||
private parseProtocolVersion(line: string): string | null {
|
||||
const match = this.extractProtocolData(line, PROTOCOL_MARKERS.PROTOCOL_PREFIX);
|
||||
return match;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse TAP version line
|
||||
*/
|
||||
private parseTapVersion(line: string): number | null {
|
||||
const match = line.match(/^TAP version (\d+)$/);
|
||||
if (match) {
|
||||
return parseInt(match[1], 10);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse plan line
|
||||
*/
|
||||
private parsePlan(line: string): IPlanLine | null {
|
||||
// Skip all plan
|
||||
const skipMatch = line.match(/^1\.\.0\s*#\s*Skipped:\s*(.*)$/);
|
||||
if (skipMatch) {
|
||||
return {
|
||||
start: 1,
|
||||
end: 0,
|
||||
skipAll: skipMatch[1]
|
||||
};
|
||||
}
|
||||
|
||||
// Normal plan
|
||||
const match = line.match(/^(\d+)\.\.(\d+)$/);
|
||||
if (match) {
|
||||
return {
|
||||
start: parseInt(match[1], 10),
|
||||
end: parseInt(match[2], 10)
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse bailout
|
||||
*/
|
||||
private parseBailout(line: string): string | null {
|
||||
const match = line.match(/^Bail out!\s*(.*)$/);
|
||||
return match ? match[1] : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse event
|
||||
*/
|
||||
private parseEvent(line: string): ITestEvent | null {
|
||||
const eventData = this.extractProtocolData(line, PROTOCOL_MARKERS.EVENT_PREFIX);
|
||||
if (eventData) {
|
||||
try {
|
||||
return JSON.parse(eventData) as ITestEvent;
|
||||
} catch (e) {
|
||||
// Invalid JSON, ignore
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if line is a comment
|
||||
*/
|
||||
private isComment(line: string): boolean {
|
||||
return line.startsWith('# ');
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse test result line
|
||||
*/
|
||||
private parseTestResult(line: string): ITestResult | null {
|
||||
// First extract any inline metadata
|
||||
const metadata = this.extractInlineMetadata(line);
|
||||
const cleanLine = this.removeInlineMetadata(line);
|
||||
|
||||
// Parse the TAP part
|
||||
const tapMatch = cleanLine.match(/^(ok|not ok)\s+(\d+)\s*-?\s*(.*)$/);
|
||||
if (!tapMatch) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const result: ITestResult = {
|
||||
ok: tapMatch[1] === 'ok',
|
||||
testNumber: parseInt(tapMatch[2], 10),
|
||||
description: tapMatch[3].trim()
|
||||
};
|
||||
|
||||
// Parse directive
|
||||
const directiveMatch = result.description.match(/^(.*?)\s*#\s*(SKIP|TODO)\s*(.*)$/i);
|
||||
if (directiveMatch) {
|
||||
result.description = directiveMatch[1].trim();
|
||||
result.directive = {
|
||||
type: directiveMatch[2].toLowerCase() as 'skip' | 'todo',
|
||||
reason: directiveMatch[3] || undefined
|
||||
};
|
||||
}
|
||||
|
||||
// Add metadata if found
|
||||
if (metadata) {
|
||||
result.metadata = metadata;
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract inline metadata from line
|
||||
*/
|
||||
private extractInlineMetadata(line: string): ITestMetadata | null {
|
||||
const metadata: ITestMetadata = {};
|
||||
let hasData = false;
|
||||
|
||||
// Extract skip reason
|
||||
const skipData = this.extractProtocolData(line, PROTOCOL_MARKERS.SKIP_PREFIX);
|
||||
if (skipData) {
|
||||
metadata.skip = skipData;
|
||||
hasData = true;
|
||||
}
|
||||
|
||||
// Extract todo reason
|
||||
const todoData = this.extractProtocolData(line, PROTOCOL_MARKERS.TODO_PREFIX);
|
||||
if (todoData) {
|
||||
metadata.todo = todoData;
|
||||
hasData = true;
|
||||
}
|
||||
|
||||
// Extract META JSON
|
||||
const metaData = this.extractProtocolData(line, PROTOCOL_MARKERS.META_PREFIX);
|
||||
if (metaData) {
|
||||
try {
|
||||
Object.assign(metadata, JSON.parse(metaData));
|
||||
hasData = true;
|
||||
} catch (e) {
|
||||
// Invalid JSON, ignore
|
||||
}
|
||||
}
|
||||
|
||||
// Extract simple key:value pairs
|
||||
const simpleMatch = line.match(new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]+)${this.escapeRegex(PROTOCOL_MARKERS.END)}`));
|
||||
if (simpleMatch && simpleMatch[1].includes(':') && !simpleMatch[1].includes('META:') && !simpleMatch[1].includes('SKIP:') && !simpleMatch[1].includes('TODO:') && !simpleMatch[1].includes('EVENT:')) {
|
||||
// This is a simple key:value format (not a prefixed format)
|
||||
const pairs = simpleMatch[1].split(',');
|
||||
for (const pair of pairs) {
|
||||
const [key, value] = pair.split(':');
|
||||
if (key && value) {
|
||||
if (key === 'time') {
|
||||
metadata.time = parseInt(value, 10);
|
||||
hasData = true;
|
||||
} else if (key === 'retry') {
|
||||
metadata.retry = parseInt(value, 10);
|
||||
hasData = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return hasData ? metadata : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove inline metadata from line
|
||||
*/
|
||||
private removeInlineMetadata(line: string): string {
|
||||
// Remove all protocol markers
|
||||
const regex = new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}[^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*${this.escapeRegex(PROTOCOL_MARKERS.END)}`, 'g');
|
||||
return line.replace(regex, '').trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract protocol data with specific prefix
|
||||
*/
|
||||
private extractProtocolData(line: string, prefix: string): string | null {
|
||||
const regex = new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}${this.escapeRegex(prefix)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*)${this.escapeRegex(PROTOCOL_MARKERS.END)}`);
|
||||
const match = line.match(regex);
|
||||
return match ? match[1] : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if line starts a block
|
||||
*/
|
||||
private isBlockStart(line: string): boolean {
|
||||
// Only match if the line is exactly the block marker (after trimming)
|
||||
const trimmed = line.trim();
|
||||
return trimmed === `${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_PREFIX}${PROTOCOL_MARKERS.END}` ||
|
||||
(trimmed.startsWith(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_PREFIX}`) &&
|
||||
trimmed.endsWith(PROTOCOL_MARKERS.END) &&
|
||||
!trimmed.includes(' '));
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if line ends a block
|
||||
*/
|
||||
private isBlockEnd(line: string): boolean {
|
||||
return line.includes(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.ERROR_END}${PROTOCOL_MARKERS.END}`) ||
|
||||
line.includes(`${PROTOCOL_MARKERS.START}${PROTOCOL_MARKERS.SNAPSHOT_END}${PROTOCOL_MARKERS.END}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract block type from start line
|
||||
*/
|
||||
private extractBlockType(line: string): string | null {
|
||||
if (line.includes(PROTOCOL_MARKERS.ERROR_PREFIX)) {
|
||||
return 'error';
|
||||
}
|
||||
if (line.includes(PROTOCOL_MARKERS.SNAPSHOT_PREFIX)) {
|
||||
const match = line.match(new RegExp(`${this.escapeRegex(PROTOCOL_MARKERS.START)}${this.escapeRegex(PROTOCOL_MARKERS.SNAPSHOT_PREFIX)}([^${this.escapeRegex(PROTOCOL_MARKERS.END)}]*)${this.escapeRegex(PROTOCOL_MARKERS.END)}`));
|
||||
return match ? `snapshot:${match[1]}` : 'snapshot';
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Finalize current block
|
||||
*/
|
||||
private finalizeBlock(): IProtocolMessage {
|
||||
const content = this.blockContent.join('\n');
|
||||
|
||||
if (this.blockType === 'error') {
|
||||
try {
|
||||
const errorData = JSON.parse(content) as IErrorBlock;
|
||||
return {
|
||||
type: 'error',
|
||||
content: errorData
|
||||
};
|
||||
} catch (e) {
|
||||
return {
|
||||
type: 'error',
|
||||
content: { error: { message: content } }
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
if (this.blockType?.startsWith('snapshot:')) {
|
||||
const name = this.blockType.substring(9);
|
||||
let parsedContent = content;
|
||||
let format: 'json' | 'text' = 'text';
|
||||
|
||||
try {
|
||||
parsedContent = JSON.parse(content);
|
||||
format = 'json';
|
||||
} catch (e) {
|
||||
// Not JSON, keep as text
|
||||
}
|
||||
|
||||
return {
|
||||
type: 'snapshot',
|
||||
content: {
|
||||
name,
|
||||
content: parsedContent,
|
||||
format
|
||||
} as ISnapshotData
|
||||
};
|
||||
}
|
||||
|
||||
// Fallback
|
||||
return {
|
||||
type: 'comment',
|
||||
content: content
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Escape regex special characters
|
||||
*/
|
||||
private escapeRegex(str: string): string {
|
||||
return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get protocol version
|
||||
*/
|
||||
public getProtocolVersion(): string | null {
|
||||
return this.protocolVersion;
|
||||
}
|
||||
}
|
||||
148
ts_tapbundle_protocol/protocol.types.ts
Normal file
148
ts_tapbundle_protocol/protocol.types.ts
Normal file
@@ -0,0 +1,148 @@
|
||||
// Protocol V2 Types and Interfaces
|
||||
// This file contains all type definitions for the improved TAP protocol
|
||||
|
||||
export interface ITestMetadata {
|
||||
// Timing
|
||||
time?: number; // milliseconds
|
||||
startTime?: number; // Unix timestamp
|
||||
endTime?: number; // Unix timestamp
|
||||
|
||||
// Status
|
||||
skip?: string; // skip reason
|
||||
todo?: string; // todo reason
|
||||
retry?: number; // retry attempt
|
||||
maxRetries?: number; // max retries allowed
|
||||
|
||||
// Error details
|
||||
error?: {
|
||||
message: string;
|
||||
stack?: string;
|
||||
diff?: string;
|
||||
actual?: any;
|
||||
expected?: any;
|
||||
code?: string;
|
||||
};
|
||||
|
||||
// Test context
|
||||
file?: string; // source file
|
||||
line?: number; // line number
|
||||
column?: number; // column number
|
||||
|
||||
// Custom data
|
||||
tags?: string[]; // test tags
|
||||
custom?: Record<string, any>;
|
||||
}
|
||||
|
||||
export interface ITestResult {
|
||||
ok: boolean;
|
||||
testNumber: number;
|
||||
description: string;
|
||||
directive?: {
|
||||
type: 'skip' | 'todo';
|
||||
reason?: string;
|
||||
};
|
||||
metadata?: ITestMetadata;
|
||||
}
|
||||
|
||||
export interface IPlanLine {
|
||||
start: number;
|
||||
end: number;
|
||||
skipAll?: string;
|
||||
}
|
||||
|
||||
export interface IProtocolMessage {
|
||||
type: 'test' | 'plan' | 'comment' | 'version' | 'bailout' | 'protocol' | 'snapshot' | 'error' | 'event';
|
||||
content: any;
|
||||
}
|
||||
|
||||
export interface IProtocolVersion {
|
||||
version: string;
|
||||
features?: string[];
|
||||
}
|
||||
|
||||
export interface ISnapshotData {
|
||||
name: string;
|
||||
content: any;
|
||||
format?: 'json' | 'text' | 'binary';
|
||||
}
|
||||
|
||||
export interface IErrorBlock {
|
||||
testNumber?: number;
|
||||
error: {
|
||||
message: string;
|
||||
stack?: string;
|
||||
diff?: string;
|
||||
actual?: any;
|
||||
expected?: any;
|
||||
};
|
||||
}
|
||||
|
||||
// Enhanced Communication Types
|
||||
export type EventType =
|
||||
| 'test:queued'
|
||||
| 'test:started'
|
||||
| 'test:progress'
|
||||
| 'test:completed'
|
||||
| 'suite:started'
|
||||
| 'suite:completed'
|
||||
| 'hook:started'
|
||||
| 'hook:completed'
|
||||
| 'assertion:failed';
|
||||
|
||||
export interface ITestEvent {
|
||||
eventType: EventType;
|
||||
timestamp: number;
|
||||
data: {
|
||||
testNumber?: number;
|
||||
description?: string;
|
||||
suiteName?: string;
|
||||
hookName?: string;
|
||||
progress?: number; // 0-100
|
||||
duration?: number;
|
||||
error?: IEnhancedError;
|
||||
[key: string]: any;
|
||||
};
|
||||
}
|
||||
|
||||
export interface IEnhancedError {
|
||||
message: string;
|
||||
stack?: string;
|
||||
diff?: IDiffResult;
|
||||
actual?: any;
|
||||
expected?: any;
|
||||
code?: string;
|
||||
type?: 'assertion' | 'timeout' | 'uncaught' | 'syntax' | 'runtime';
|
||||
}
|
||||
|
||||
export interface IDiffResult {
|
||||
type: 'string' | 'object' | 'array' | 'primitive';
|
||||
changes: IDiffChange[];
|
||||
context?: number; // lines of context
|
||||
}
|
||||
|
||||
export interface IDiffChange {
|
||||
type: 'add' | 'remove' | 'modify';
|
||||
path?: string[]; // for object/array diffs
|
||||
oldValue?: any;
|
||||
newValue?: any;
|
||||
line?: number; // for string diffs
|
||||
content?: string;
|
||||
}
|
||||
|
||||
// Protocol markers
|
||||
export const PROTOCOL_MARKERS = {
|
||||
START: '⟦TSTEST:',
|
||||
END: '⟧',
|
||||
META_PREFIX: 'META:',
|
||||
ERROR_PREFIX: 'ERROR',
|
||||
ERROR_END: '/ERROR',
|
||||
SNAPSHOT_PREFIX: 'SNAPSHOT:',
|
||||
SNAPSHOT_END: '/SNAPSHOT',
|
||||
PROTOCOL_PREFIX: 'PROTOCOL:',
|
||||
SKIP_PREFIX: 'SKIP:',
|
||||
TODO_PREFIX: 'TODO:',
|
||||
EVENT_PREFIX: 'EVENT:',
|
||||
} as const;
|
||||
|
||||
// Protocol version
|
||||
export const PROTOCOL_VERSION = '2.0.0';
|
||||
587
ts_tapbundle_protocol/readme.md
Normal file
587
ts_tapbundle_protocol/readme.md
Normal file
@@ -0,0 +1,587 @@
|
||||
# @git.zone/tstest/tapbundle_protocol
|
||||
|
||||
> 📡 Enhanced TAP Protocol V2 implementation for structured test communication
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# tapbundle_protocol is included as part of @git.zone/tstest
|
||||
pnpm install --save-dev @git.zone/tstest
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
`@git.zone/tstest/tapbundle_protocol` implements Protocol V2, an enhanced version of the Test Anything Protocol (TAP) with support for structured metadata, real-time events, error diffs, and isomorphic operation. This protocol enables rich communication between test runners and test consumers while maintaining backward compatibility with standard TAP parsers.
|
||||
|
||||
## Key Features
|
||||
|
||||
- 📋 **TAP v13 Compliant** - Fully compatible with standard TAP consumers
|
||||
- 🎯 **Enhanced Metadata** - Timing, tags, errors, diffs, and custom data
|
||||
- 🔄 **Real-time Events** - Live test execution updates
|
||||
- 🔍 **Structured Errors** - JSON error blocks with stack traces and diffs
|
||||
- 📸 **Snapshot Support** - Built-in snapshot testing protocol
|
||||
- 🌐 **Isomorphic** - Works in Node.js, browsers, Deno, and Bun
|
||||
- 🏷️ **Protocol Markers** - Structured data using Unicode delimiters
|
||||
|
||||
## Protocol V2 Format
|
||||
|
||||
### Protocol Markers
|
||||
|
||||
Protocol V2 uses special Unicode markers to embed structured data within TAP output:
|
||||
|
||||
- `⟦TSTEST:` - Start marker
|
||||
- `⟧` - End marker
|
||||
|
||||
These markers allow structured data to coexist with standard TAP without breaking compatibility.
|
||||
|
||||
### Example Output
|
||||
|
||||
```tap
|
||||
⟦TSTEST:PROTOCOL:2.0.0⟧
|
||||
TAP version 13
|
||||
1..3
|
||||
ok 1 - should add numbers ⟦TSTEST:time:42⟧
|
||||
not ok 2 - should validate input
|
||||
⟦TSTEST:META:{"time":156,"file":"test.ts","line":42}⟧
|
||||
⟦TSTEST:ERROR⟧
|
||||
{
|
||||
"error": {
|
||||
"message": "Expected 5 to equal 6",
|
||||
"diff": {...}
|
||||
}
|
||||
}
|
||||
⟦TSTEST:/ERROR⟧
|
||||
ok 3 - should handle edge cases # SKIP not implemented ⟦TSTEST:SKIP:not implemented⟧
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### ProtocolEmitter
|
||||
|
||||
Generates Protocol V2 messages. Used by tapbundle to emit test results.
|
||||
|
||||
#### `emitProtocolHeader()`
|
||||
|
||||
Emit the protocol version header.
|
||||
|
||||
```typescript
|
||||
import { ProtocolEmitter } from '@git.zone/tstest/tapbundle_protocol';
|
||||
|
||||
const emitter = new ProtocolEmitter();
|
||||
console.log(emitter.emitProtocolHeader());
|
||||
// Output: ⟦TSTEST:PROTOCOL:2.0.0⟧
|
||||
```
|
||||
|
||||
#### `emitTapVersion(version?)`
|
||||
|
||||
Emit TAP version line.
|
||||
|
||||
```typescript
|
||||
console.log(emitter.emitTapVersion(13));
|
||||
// Output: TAP version 13
|
||||
```
|
||||
|
||||
#### `emitPlan(plan)`
|
||||
|
||||
Emit test plan.
|
||||
|
||||
```typescript
|
||||
console.log(emitter.emitPlan({ start: 1, end: 5 }));
|
||||
// Output: 1..5
|
||||
|
||||
console.log(emitter.emitPlan({ start: 1, end: 0, skipAll: 'Not ready' }));
|
||||
// Output: 1..0 # Skipped: Not ready
|
||||
```
|
||||
|
||||
#### `emitTest(result)`
|
||||
|
||||
Emit a test result with optional metadata.
|
||||
|
||||
```typescript
|
||||
const lines = emitter.emitTest({
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'should work correctly',
|
||||
metadata: {
|
||||
time: 45,
|
||||
tags: ['unit', 'fast']
|
||||
}
|
||||
});
|
||||
|
||||
lines.forEach(line => console.log(line));
|
||||
// Output:
|
||||
// ok 1 - should work correctly ⟦TSTEST:time:45⟧
|
||||
// ⟦TSTEST:META:{"tags":["unit","fast"]}⟧
|
||||
```
|
||||
|
||||
#### `emitComment(comment)`
|
||||
|
||||
Emit a comment line.
|
||||
|
||||
```typescript
|
||||
console.log(emitter.emitComment('Setup complete'));
|
||||
// Output: # Setup complete
|
||||
```
|
||||
|
||||
#### `emitBailout(reason)`
|
||||
|
||||
Emit a bailout (abort all tests).
|
||||
|
||||
```typescript
|
||||
console.log(emitter.emitBailout('Database connection failed'));
|
||||
// Output: Bail out! Database connection failed
|
||||
```
|
||||
|
||||
#### `emitError(error)`
|
||||
|
||||
Emit a structured error block.
|
||||
|
||||
```typescript
|
||||
const lines = emitter.emitError({
|
||||
testNumber: 2,
|
||||
error: {
|
||||
message: 'Expected 5 to equal 6',
|
||||
stack: 'Error: ...',
|
||||
actual: 5,
|
||||
expected: 6,
|
||||
diff: '...'
|
||||
}
|
||||
});
|
||||
|
||||
lines.forEach(line => console.log(line));
|
||||
// Output:
|
||||
// ⟦TSTEST:ERROR⟧
|
||||
// {
|
||||
// "testNumber": 2,
|
||||
// "error": { ... }
|
||||
// }
|
||||
// ⟦TSTEST:/ERROR⟧
|
||||
```
|
||||
|
||||
#### `emitEvent(event)`
|
||||
|
||||
Emit a real-time test event.
|
||||
|
||||
```typescript
|
||||
console.log(emitter.emitEvent({
|
||||
eventType: 'test:started',
|
||||
timestamp: Date.now(),
|
||||
data: {
|
||||
testNumber: 1,
|
||||
description: 'should work'
|
||||
}
|
||||
}));
|
||||
// Output: ⟦TSTEST:EVENT:{"eventType":"test:started",...}⟧
|
||||
```
|
||||
|
||||
#### `emitSnapshot(snapshot)`
|
||||
|
||||
Emit snapshot data.
|
||||
|
||||
```typescript
|
||||
const lines = emitter.emitSnapshot({
|
||||
name: 'user-data',
|
||||
content: { name: 'Alice', age: 30 },
|
||||
format: 'json'
|
||||
});
|
||||
|
||||
lines.forEach(line => console.log(line));
|
||||
// Output:
|
||||
// ⟦TSTEST:SNAPSHOT:user-data⟧
|
||||
// {
|
||||
// "name": "Alice",
|
||||
// "age": 30
|
||||
// }
|
||||
// ⟦TSTEST:/SNAPSHOT⟧
|
||||
```
|
||||
|
||||
### ProtocolParser
|
||||
|
||||
Parses Protocol V2 messages. Used by tstest to consume test results.
|
||||
|
||||
#### `parseLine(line)`
|
||||
|
||||
Parse a single line and return protocol messages.
|
||||
|
||||
```typescript
|
||||
import { ProtocolParser } from '@git.zone/tstest/tapbundle_protocol';
|
||||
|
||||
const parser = new ProtocolParser();
|
||||
|
||||
const messages = parser.parseLine('ok 1 - test passed ⟦TSTEST:time:42⟧');
|
||||
console.log(messages);
|
||||
// Output:
|
||||
// [{
|
||||
// type: 'test',
|
||||
// content: {
|
||||
// ok: true,
|
||||
// testNumber: 1,
|
||||
// description: 'test passed',
|
||||
// metadata: { time: 42 }
|
||||
// }
|
||||
// }]
|
||||
```
|
||||
|
||||
#### Message Types
|
||||
|
||||
The parser returns different message types:
|
||||
|
||||
```typescript
|
||||
interface IProtocolMessage {
|
||||
type: 'test' | 'plan' | 'comment' | 'version' | 'bailout' | 'protocol' | 'snapshot' | 'error' | 'event';
|
||||
content: any;
|
||||
}
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
|
||||
```typescript
|
||||
// Test result
|
||||
{
|
||||
type: 'test',
|
||||
content: {
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'test name',
|
||||
metadata: { ... }
|
||||
}
|
||||
}
|
||||
|
||||
// Plan
|
||||
{
|
||||
type: 'plan',
|
||||
content: {
|
||||
start: 1,
|
||||
end: 5
|
||||
}
|
||||
}
|
||||
|
||||
// Event
|
||||
{
|
||||
type: 'event',
|
||||
content: {
|
||||
eventType: 'test:started',
|
||||
timestamp: 1234567890,
|
||||
data: { ... }
|
||||
}
|
||||
}
|
||||
|
||||
// Error
|
||||
{
|
||||
type: 'error',
|
||||
content: {
|
||||
testNumber: 2,
|
||||
error: {
|
||||
message: '...',
|
||||
stack: '...',
|
||||
diff: '...'
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### `getProtocolVersion()`
|
||||
|
||||
Get the detected protocol version.
|
||||
|
||||
```typescript
|
||||
const version = parser.getProtocolVersion();
|
||||
console.log(version); // "2.0.0" or null
|
||||
```
|
||||
|
||||
## TypeScript Types
|
||||
|
||||
### ITestResult
|
||||
|
||||
```typescript
|
||||
interface ITestResult {
|
||||
ok: boolean;
|
||||
testNumber: number;
|
||||
description: string;
|
||||
directive?: {
|
||||
type: 'skip' | 'todo';
|
||||
reason?: string;
|
||||
};
|
||||
metadata?: ITestMetadata;
|
||||
}
|
||||
```
|
||||
|
||||
### ITestMetadata
|
||||
|
||||
```typescript
|
||||
interface ITestMetadata {
|
||||
// Timing
|
||||
time?: number; // Test duration in milliseconds
|
||||
startTime?: number; // Unix timestamp
|
||||
endTime?: number; // Unix timestamp
|
||||
|
||||
// Status
|
||||
skip?: string; // Skip reason
|
||||
todo?: string; // Todo reason
|
||||
retry?: number; // Current retry attempt
|
||||
maxRetries?: number; // Max retries allowed
|
||||
|
||||
// Error details
|
||||
error?: {
|
||||
message: string;
|
||||
stack?: string;
|
||||
diff?: string;
|
||||
actual?: any;
|
||||
expected?: any;
|
||||
code?: string;
|
||||
};
|
||||
|
||||
// Test context
|
||||
file?: string; // Source file path
|
||||
line?: number; // Line number
|
||||
column?: number; // Column number
|
||||
|
||||
// Custom data
|
||||
tags?: string[]; // Test tags
|
||||
custom?: Record<string, any>;
|
||||
}
|
||||
```
|
||||
|
||||
### ITestEvent
|
||||
|
||||
```typescript
|
||||
interface ITestEvent {
|
||||
eventType: EventType;
|
||||
timestamp: number;
|
||||
data: {
|
||||
testNumber?: number;
|
||||
description?: string;
|
||||
suiteName?: string;
|
||||
hookName?: string;
|
||||
progress?: number; // 0-100
|
||||
duration?: number;
|
||||
error?: IEnhancedError;
|
||||
[key: string]: any;
|
||||
};
|
||||
}
|
||||
|
||||
type EventType =
|
||||
| 'test:queued'
|
||||
| 'test:started'
|
||||
| 'test:progress'
|
||||
| 'test:completed'
|
||||
| 'suite:started'
|
||||
| 'suite:completed'
|
||||
| 'hook:started'
|
||||
| 'hook:completed'
|
||||
| 'assertion:failed';
|
||||
```
|
||||
|
||||
### IEnhancedError
|
||||
|
||||
```typescript
|
||||
interface IEnhancedError {
|
||||
message: string;
|
||||
stack?: string;
|
||||
diff?: IDiffResult;
|
||||
actual?: any;
|
||||
expected?: any;
|
||||
code?: string;
|
||||
type?: 'assertion' | 'timeout' | 'uncaught' | 'syntax' | 'runtime';
|
||||
}
|
||||
```
|
||||
|
||||
### IDiffResult
|
||||
|
||||
```typescript
|
||||
interface IDiffResult {
|
||||
type: 'string' | 'object' | 'array' | 'primitive';
|
||||
changes: IDiffChange[];
|
||||
context?: number; // Lines of context
|
||||
}
|
||||
|
||||
interface IDiffChange {
|
||||
type: 'add' | 'remove' | 'modify';
|
||||
path?: string[]; // For object/array diffs
|
||||
oldValue?: any;
|
||||
newValue?: any;
|
||||
line?: number; // For string diffs
|
||||
content?: string;
|
||||
}
|
||||
```
|
||||
|
||||
## Protocol Constants
|
||||
|
||||
```typescript
|
||||
import { PROTOCOL_MARKERS, PROTOCOL_VERSION } from '@git.zone/tstest/tapbundle_protocol';
|
||||
|
||||
console.log(PROTOCOL_VERSION); // "2.0.0"
|
||||
console.log(PROTOCOL_MARKERS.START); // "⟦TSTEST:"
|
||||
console.log(PROTOCOL_MARKERS.END); // "⟧"
|
||||
```
|
||||
|
||||
### Available Markers
|
||||
|
||||
```typescript
|
||||
const PROTOCOL_MARKERS = {
|
||||
START: '⟦TSTEST:',
|
||||
END: '⟧',
|
||||
META_PREFIX: 'META:',
|
||||
ERROR_PREFIX: 'ERROR',
|
||||
ERROR_END: '/ERROR',
|
||||
SNAPSHOT_PREFIX: 'SNAPSHOT:',
|
||||
SNAPSHOT_END: '/SNAPSHOT',
|
||||
PROTOCOL_PREFIX: 'PROTOCOL:',
|
||||
SKIP_PREFIX: 'SKIP:',
|
||||
TODO_PREFIX: 'TODO:',
|
||||
EVENT_PREFIX: 'EVENT:',
|
||||
};
|
||||
```
|
||||
|
||||
## Usage Patterns
|
||||
|
||||
### Creating a Custom Test Runner
|
||||
|
||||
```typescript
|
||||
import { ProtocolEmitter } from '@git.zone/tstest/tapbundle_protocol';
|
||||
|
||||
const emitter = new ProtocolEmitter();
|
||||
|
||||
// Emit protocol header
|
||||
console.log(emitter.emitProtocolHeader());
|
||||
console.log(emitter.emitTapVersion(13));
|
||||
|
||||
// Emit plan
|
||||
console.log(emitter.emitPlan({ start: 1, end: 2 }));
|
||||
|
||||
// Run test 1
|
||||
emitter.emitEvent({
|
||||
eventType: 'test:started',
|
||||
timestamp: Date.now(),
|
||||
data: { testNumber: 1 }
|
||||
}).split('\n').forEach(line => console.log(line));
|
||||
|
||||
const result1 = emitter.emitTest({
|
||||
ok: true,
|
||||
testNumber: 1,
|
||||
description: 'first test',
|
||||
metadata: { time: 45 }
|
||||
});
|
||||
result1.forEach(line => console.log(line));
|
||||
|
||||
// Run test 2
|
||||
const result2 = emitter.emitTest({
|
||||
ok: false,
|
||||
testNumber: 2,
|
||||
description: 'second test',
|
||||
metadata: {
|
||||
time: 120,
|
||||
error: {
|
||||
message: 'Assertion failed',
|
||||
actual: 5,
|
||||
expected: 6
|
||||
}
|
||||
}
|
||||
});
|
||||
result2.forEach(line => console.log(line));
|
||||
```
|
||||
|
||||
### Parsing Test Output
|
||||
|
||||
```typescript
|
||||
import { ProtocolParser } from '@git.zone/tstest/tapbundle_protocol';
|
||||
import * as readline from 'readline';
|
||||
|
||||
const parser = new ProtocolParser();
|
||||
|
||||
const rl = readline.createInterface({
|
||||
input: process.stdin,
|
||||
output: process.stdout
|
||||
});
|
||||
|
||||
rl.on('line', (line) => {
|
||||
const messages = parser.parseLine(line);
|
||||
|
||||
messages.forEach(message => {
|
||||
switch (message.type) {
|
||||
case 'test':
|
||||
console.log(`Test ${message.content.testNumber}: ${message.content.ok ? 'PASS' : 'FAIL'}`);
|
||||
break;
|
||||
case 'event':
|
||||
console.log(`Event: ${message.content.eventType}`);
|
||||
break;
|
||||
case 'error':
|
||||
console.error(`Error: ${message.content.error.message}`);
|
||||
break;
|
||||
}
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Building Test Dashboards
|
||||
|
||||
Real-time events enable building live test dashboards:
|
||||
|
||||
```typescript
|
||||
const parser = new ProtocolParser();
|
||||
|
||||
parser.parseLine(line).forEach(message => {
|
||||
if (message.type === 'event') {
|
||||
const event = message.content;
|
||||
|
||||
switch (event.eventType) {
|
||||
case 'test:started':
|
||||
updateUI({ status: 'running', test: event.data.description });
|
||||
break;
|
||||
case 'test:completed':
|
||||
updateUI({ status: 'done', duration: event.data.duration });
|
||||
break;
|
||||
case 'suite:started':
|
||||
createSuiteCard(event.data.suiteName);
|
||||
break;
|
||||
}
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
Protocol V2 is fully backward compatible with standard TAP parsers:
|
||||
|
||||
- Protocol markers use Unicode characters that TAP parsers ignore
|
||||
- Standard TAP output (ok/not ok, plan, comments) works everywhere
|
||||
- Enhanced features gracefully degrade in standard TAP consumers
|
||||
|
||||
**Standard TAP View:**
|
||||
```tap
|
||||
TAP version 13
|
||||
1..3
|
||||
ok 1 - should add numbers
|
||||
not ok 2 - should validate input
|
||||
ok 3 - should handle edge cases # SKIP not implemented
|
||||
```
|
||||
|
||||
**Protocol V2 View (same output):**
|
||||
```tap
|
||||
⟦TSTEST:PROTOCOL:2.0.0⟧
|
||||
TAP version 13
|
||||
1..3
|
||||
ok 1 - should add numbers ⟦TSTEST:time:42⟧
|
||||
not ok 2 - should validate input
|
||||
⟦TSTEST:META:{"time":156}⟧
|
||||
ok 3 - should handle edge cases # SKIP not implemented ⟦TSTEST:SKIP:not implemented⟧
|
||||
```
|
||||
|
||||
## Isomorphic Design
|
||||
|
||||
This module works in all JavaScript environments:
|
||||
|
||||
- ✅ Node.js
|
||||
- ✅ Browsers (via tapbundle)
|
||||
- ✅ Deno
|
||||
- ✅ Bun
|
||||
- ✅ Web Workers
|
||||
- ✅ Service Workers
|
||||
|
||||
No runtime-specific APIs are used, making it truly portable.
|
||||
|
||||
## Legal
|
||||
|
||||
This project is licensed under MIT.
|
||||
|
||||
© 2025 Task Venture Capital GmbH. All rights reserved.
|
||||
3
ts_tapbundle_protocol/tspublish.json
Normal file
3
ts_tapbundle_protocol/tspublish.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"order": 1
|
||||
}
|
||||
@@ -9,9 +9,12 @@ export class TestFileProvider {
|
||||
public async getDockerAlpineImageAsLocalTarball(): Promise<string> {
|
||||
const filePath = plugins.path.join(paths.testFilesDir, 'alpine.tar')
|
||||
// fetch the docker alpine image
|
||||
const response = await plugins.smartrequest.getBinary(fileUrls.dockerAlpineImage);
|
||||
const response = await plugins.smartrequest.SmartRequest.create()
|
||||
.url(fileUrls.dockerAlpineImage)
|
||||
.get();
|
||||
await plugins.smartfile.fs.ensureDir(paths.testFilesDir);
|
||||
await plugins.smartfile.memory.toFs(response.body, filePath);
|
||||
const buffer = Buffer.from(await response.arrayBuffer());
|
||||
await plugins.smartfile.memory.toFs(buffer, filePath);
|
||||
return filePath;
|
||||
}
|
||||
}
|
||||
369
ts_tapbundle_serverside/readme.md
Normal file
369
ts_tapbundle_serverside/readme.md
Normal file
@@ -0,0 +1,369 @@
|
||||
# @git.zone/tstest/tapbundle_serverside
|
||||
|
||||
> 🔧 Server-side testing utilities for Node.js runtime tests
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# tapbundle_serverside is included as part of @git.zone/tstest
|
||||
pnpm install --save-dev @git.zone/tstest
|
||||
```
|
||||
|
||||
## Overview
|
||||
|
||||
`@git.zone/tstest/tapbundle_serverside` provides server-side testing utilities exclusively for Node.js runtime. These tools enable shell command execution, environment variable management, HTTPS certificate generation, database testing, object storage testing, and test asset management - all functionality that only makes sense on the server-side.
|
||||
|
||||
## Key Features
|
||||
|
||||
- 🔐 **Environment Variables** - On-demand environment variable loading with qenv
|
||||
- 💻 **Shell Commands** - Execute bash commands during tests
|
||||
- 🔒 **HTTPS Certificates** - Generate self-signed certificates for testing
|
||||
- 🗄️ **MongoDB Testing** - Create ephemeral MongoDB instances
|
||||
- 📦 **S3 Storage Testing** - Create local S3-compatible storage for tests
|
||||
- 📁 **Test File Management** - Download and manage test assets
|
||||
|
||||
## Basic Usage
|
||||
|
||||
```typescript
|
||||
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
|
||||
import { tap } from '@git.zone/tstest/tapbundle';
|
||||
|
||||
tap.test('should use server-side tools', async () => {
|
||||
// Execute shell commands on the server-side
|
||||
const result = await tapNodeTools.runCommand('echo "hello"');
|
||||
console.log(result);
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### tapNodeTools
|
||||
|
||||
The main singleton instance providing all Node.js-specific utilities.
|
||||
|
||||
#### Environment Variables
|
||||
|
||||
##### `getQenv()`
|
||||
|
||||
Get the qenv instance for managing environment variables from `.nogit/` directory.
|
||||
|
||||
```typescript
|
||||
const qenv = await tapNodeTools.getQenv();
|
||||
// qenv will load from .env files in .nogit/ directory
|
||||
```
|
||||
|
||||
##### `getEnvVarOnDemand(envVarName)`
|
||||
|
||||
Request an environment variable. If not available, qenv will prompt for it and store it securely.
|
||||
|
||||
```typescript
|
||||
tap.test('should get API key', async () => {
|
||||
const apiKey = await tapNodeTools.getEnvVarOnDemand('GITHUB_API_KEY');
|
||||
// If GITHUB_API_KEY is not set, qenv will prompt for it
|
||||
// The value is stored in .nogit/.env for future use
|
||||
});
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- API keys for integration tests
|
||||
- Database credentials
|
||||
- Service endpoints
|
||||
- Any sensitive configuration needed for testing
|
||||
|
||||
#### Shell Commands
|
||||
|
||||
##### `runCommand(command)`
|
||||
|
||||
Execute a bash command and return the result.
|
||||
|
||||
```typescript
|
||||
tap.test('should execute shell commands', async () => {
|
||||
const result = await tapNodeTools.runCommand('ls -la');
|
||||
console.log(result.stdout);
|
||||
});
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- Setup test environment
|
||||
- Execute CLI tools
|
||||
- File system operations
|
||||
- Process management
|
||||
|
||||
#### HTTPS Certificates
|
||||
|
||||
##### `createHttpsCert(commonName?, allowSelfSigned?)`
|
||||
|
||||
Generate a self-signed HTTPS certificate for testing secure connections.
|
||||
|
||||
```typescript
|
||||
tap.test('should create HTTPS server', async () => {
|
||||
const { key, cert } = await tapNodeTools.createHttpsCert('localhost', true);
|
||||
|
||||
// Use with Node.js https module
|
||||
const server = https.createServer({ key, cert }, (req, res) => {
|
||||
res.end('Hello Secure World');
|
||||
});
|
||||
|
||||
server.listen(3000);
|
||||
});
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `commonName` (optional): Certificate common name, default: 'localhost'
|
||||
- `allowSelfSigned` (optional): Allow self-signed certificates by setting `NODE_TLS_REJECT_UNAUTHORIZED=0`, default: true
|
||||
|
||||
**Returns:**
|
||||
- `key`: PEM-encoded private key
|
||||
- `cert`: PEM-encoded certificate
|
||||
|
||||
**Use Cases:**
|
||||
- Testing HTTPS servers
|
||||
- Testing secure WebSocket connections
|
||||
- Testing certificate validation logic
|
||||
- Mocking secure external services
|
||||
|
||||
#### Database Testing
|
||||
|
||||
##### `createSmartmongo()`
|
||||
|
||||
Create an ephemeral MongoDB instance for testing. Automatically started and ready to use.
|
||||
|
||||
```typescript
|
||||
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
|
||||
|
||||
tap.test('should use MongoDB', async () => {
|
||||
const mongoInstance = await tapNodeTools.createSmartmongo();
|
||||
|
||||
// Use the MongoDB instance
|
||||
const db = await mongoInstance.getDatabase('testdb');
|
||||
const collection = await db.getCollection('users');
|
||||
|
||||
await collection.insertOne({ name: 'Alice', age: 30 });
|
||||
const user = await collection.findOne({ name: 'Alice' });
|
||||
|
||||
expect(user.age).toEqual(30);
|
||||
|
||||
// Cleanup (optional - instance will be cleaned up automatically)
|
||||
await mongoInstance.stop();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Ephemeral instance (starts fresh)
|
||||
- Automatic cleanup
|
||||
- Full MongoDB API via [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo)
|
||||
|
||||
**Use Cases:**
|
||||
- Testing database operations
|
||||
- Integration tests with MongoDB
|
||||
- Testing data models
|
||||
- Schema validation tests
|
||||
|
||||
#### Storage Testing
|
||||
|
||||
##### `createSmarts3()`
|
||||
|
||||
Create a local S3-compatible storage instance for testing object storage operations.
|
||||
|
||||
```typescript
|
||||
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
|
||||
|
||||
tap.test('should use S3 storage', async () => {
|
||||
const s3Instance = await tapNodeTools.createSmarts3();
|
||||
|
||||
// Use the S3 instance (MinIO-compatible API)
|
||||
const bucket = await s3Instance.createBucket('test-bucket');
|
||||
await bucket.putObject('file.txt', Buffer.from('Hello S3'));
|
||||
const file = await bucket.getObject('file.txt');
|
||||
|
||||
expect(file.toString()).toEqual('Hello S3');
|
||||
|
||||
// Cleanup
|
||||
await s3Instance.stop();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
**Configuration:**
|
||||
- Port: 3003 (default)
|
||||
- Clean slate: true (starts fresh each time)
|
||||
- Full S3-compatible API via [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3)
|
||||
|
||||
**Use Cases:**
|
||||
- Testing file uploads/downloads
|
||||
- Testing object storage operations
|
||||
- Testing backup/restore logic
|
||||
- Mocking cloud storage
|
||||
|
||||
### TestFileProvider
|
||||
|
||||
Utility for downloading and managing test assets.
|
||||
|
||||
#### `getDockerAlpineImageAsLocalTarball()`
|
||||
|
||||
Download the Alpine Linux Docker image as a tarball for testing.
|
||||
|
||||
```typescript
|
||||
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
|
||||
|
||||
tap.test('should provide docker image', async () => {
|
||||
const tarballPath = await tapNodeTools.testFileProvider.getDockerAlpineImageAsLocalTarball();
|
||||
|
||||
// Use the tarball path
|
||||
// Path: ./.nogit/testfiles/alpine.tar
|
||||
|
||||
expect(tarballPath).toMatch(/alpine\.tar$/);
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Downloads from https://code.foss.global/testassets/docker
|
||||
- Caches in `.nogit/testfiles/` directory
|
||||
- Returns local file path
|
||||
|
||||
**Use Cases:**
|
||||
- Testing Docker operations
|
||||
- Testing container deployment
|
||||
- Testing image handling logic
|
||||
|
||||
### Path Utilities
|
||||
|
||||
The module exports useful path constants:
|
||||
|
||||
```typescript
|
||||
import * as paths from '@git.zone/tstest/tapbundle_serverside/paths';
|
||||
|
||||
console.log(paths.cwd); // Current working directory
|
||||
console.log(paths.testFilesDir); // ./.nogit/testfiles/
|
||||
```
|
||||
|
||||
## Patterns and Best Practices
|
||||
|
||||
### Testing with External Services
|
||||
|
||||
```typescript
|
||||
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
|
||||
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||
|
||||
tap.describe('User Service Integration', () => {
|
||||
let mongoInstance;
|
||||
let db;
|
||||
|
||||
tap.beforeEach(async () => {
|
||||
mongoInstance = await tapNodeTools.createSmartmongo();
|
||||
db = await mongoInstance.getDatabase('testdb');
|
||||
});
|
||||
|
||||
tap.test('should create user', async () => {
|
||||
const users = await db.getCollection('users');
|
||||
await users.insertOne({ name: 'Bob', email: 'bob@example.com' });
|
||||
|
||||
const user = await users.findOne({ name: 'Bob' });
|
||||
expect(user.email).toEqual('bob@example.com');
|
||||
});
|
||||
|
||||
tap.afterEach(async () => {
|
||||
await mongoInstance.stop();
|
||||
});
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
### Testing HTTPS Servers
|
||||
|
||||
```typescript
|
||||
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
|
||||
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||
import * as https from 'https';
|
||||
|
||||
tap.test('should serve over HTTPS', async () => {
|
||||
const { key, cert } = await tapNodeTools.createHttpsCert();
|
||||
|
||||
const server = https.createServer({ key, cert }, (req, res) => {
|
||||
res.writeHead(200, { 'Content-Type': 'text/plain' });
|
||||
res.end('Secure Response');
|
||||
});
|
||||
|
||||
await new Promise((resolve) => {
|
||||
server.listen(8443, () => resolve(undefined));
|
||||
});
|
||||
|
||||
// Test the server
|
||||
const response = await fetch('https://localhost:8443');
|
||||
const text = await response.text();
|
||||
expect(text).toEqual('Secure Response');
|
||||
|
||||
// Cleanup
|
||||
server.close();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
### Environment-Dependent Tests
|
||||
|
||||
```typescript
|
||||
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
|
||||
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||
|
||||
tap.test('should authenticate with GitHub', async () => {
|
||||
const githubToken = await tapNodeTools.getEnvVarOnDemand('GITHUB_TOKEN');
|
||||
|
||||
// Use the token for API calls
|
||||
const response = await fetch('https://api.github.com/user', {
|
||||
headers: {
|
||||
Authorization: `Bearer ${githubToken}`
|
||||
}
|
||||
});
|
||||
|
||||
expect(response.ok).toBeTruthy();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
## Runtime Requirements
|
||||
|
||||
⚠️ **Server-Side Only (Node.js)**: All utilities in this module are designed exclusively for server-side testing in Node.js runtime. They provide functionality like shell command execution, file system operations, and process management that only make sense on the server.
|
||||
|
||||
**NOT available in:**
|
||||
- Browser environments
|
||||
- Deno runtime
|
||||
- Bun runtime
|
||||
|
||||
**Important:** Import tapbundle_serverside only in tests that run exclusively on the server-side (`.node.ts` test files). For cross-runtime tests, these utilities will fail in non-Node environments.
|
||||
|
||||
## File Naming
|
||||
|
||||
Use Node.js-specific file naming when using these utilities:
|
||||
|
||||
```
|
||||
test/mytest.node.ts ✅ Node.js only
|
||||
test/mytest.node+deno.ts ❌ Will fail in Deno
|
||||
test/mytest.browser+node.ts ⚠️ Browser won't have access to these tools
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
This module uses the following packages:
|
||||
- [@push.rocks/qenv](https://code.foss.global/push.rocks/qenv) - Environment variable management
|
||||
- [@push.rocks/smartshell](https://code.foss.global/push.rocks/smartshell) - Shell command execution
|
||||
- [@push.rocks/smartcrypto](https://code.foss.global/push.rocks/smartcrypto) - Certificate generation
|
||||
- [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo) - MongoDB testing
|
||||
- [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3) - S3 storage testing
|
||||
- [@push.rocks/smartfile](https://code.foss.global/push.rocks/smartfile) - File operations
|
||||
- [@push.rocks/smartrequest](https://code.foss.global/push.rocks/smartrequest) - HTTP requests
|
||||
|
||||
## Legal
|
||||
|
||||
This project is licensed under MIT.
|
||||
|
||||
© 2025 Task Venture Capital GmbH. All rights reserved.
|
||||
3
ts_tapbundle_serverside/tspublish.json
Normal file
3
ts_tapbundle_serverside/tspublish.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"order": 3
|
||||
}
|
||||
Reference in New Issue
Block a user