Compare commits

...

14 Commits

Author SHA1 Message Date
b525754035 1.10.2 2025-05-24 00:59:30 +00:00
aa10fc4ab3 fix(tstest-logging): Improve log file handling with log rotation and diff reporting 2025-05-24 00:59:30 +00:00
3eb8ef22e5 1.10.1 2025-05-23 23:18:35 +00:00
763dc89f59 fix(tstest): Improve file range filtering and summary logging by skipping test files outside the specified range and reporting them in the final summary. 2025-05-23 23:18:35 +00:00
e0d8ede450 1.10.0 2025-05-23 23:05:38 +00:00
27c950c1a1 feat(cli): Add --startFrom and --stopAt options to filter test files by range 2025-05-23 23:05:38 +00:00
83b324b09f 1.9.4 2025-05-23 22:33:34 +00:00
63a2879cb4 fix(docs): Update documentation and configuration for legal notices and CI permissions. This commit adds a new local settings file for tool permissions, refines the legal and trademark sections in the readme, and improves glob test files with clearer log messages. 2025-05-23 22:33:34 +00:00
1a375fa689 1.9.3 2025-05-23 22:27:12 +00:00
c48887a820 fix(tstest): Fix test timing display issue and update TAP protocol documentation 2025-05-23 22:27:12 +00:00
02aeb8195e 1.9.2 2025-05-23 21:31:39 +00:00
53d3dc55e6 fix(logging): Fix log file naming to prevent collisions and update logging system documentation. 2025-05-23 21:31:39 +00:00
a82fdc0f26 1.9.1 2025-05-23 18:53:08 +00:00
cfcb99de76 fix(dependencies): Update dependency versions and add local configuration files 2025-05-23 18:53:08 +00:00
20 changed files with 1492 additions and 328 deletions

1
.npmrc Normal file
View File

@ -0,0 +1 @@
registry=https://registry.npmjs.org

View File

@ -1,5 +1,64 @@
# Changelog
## 2025-05-24 - 1.10.2 - fix(tstest-logging)
Improve log file handling with log rotation and diff reporting
- Add .claude/settings.local.json to configure allowed shell and web operations
- Introduce movePreviousLogFiles function to archive previous log files when --logfile is used
- Enhance logging to generate error copies and diff reports between current and previous logs
- Add type annotations for console overrides in browser evaluations for improved stability
## 2025-05-23 - 1.10.1 - fix(tstest)
Improve file range filtering and summary logging by skipping test files outside the specified range and reporting them in the final summary.
- Introduce runSingleTestOrSkip to check file index against startFrom/stopAt values.
- Log skipped files with appropriate messages and add them to the summary.
- Update the logger to include total skipped files in the test summary.
- Add permission settings in .claude/settings.local.json to support new operations.
## 2025-05-23 - 1.10.0 - feat(cli)
Add --startFrom and --stopAt options to filter test files by range
- Introduced CLI options --startFrom and --stopAt in ts/index.ts for selective test execution
- Added validation to ensure provided range values are positive and startFrom is not greater than stopAt
- Propagated file range filtering into test grouping in tstest.classes.tstest.ts, applying the range filter across serial and parallel groups
- Updated usage messages to include the new options
## 2025-05-23 - 1.9.4 - fix(docs)
Update documentation and configuration for legal notices and CI permissions. This commit adds a new local settings file for tool permissions, refines the legal and trademark sections in the readme, and improves glob test files with clearer log messages.
- Added .claude/settings.local.json to configure permissions for various CLI commands
- Revised legal and trademark documentation in the readme to clarify company ownership and usage guidelines
- Updated glob test files with improved console log messages for better clarity during test discovery
## 2025-05-23 - 1.9.3 - fix(tstest)
Fix test timing display issue and update TAP protocol documentation
- Changed TAP parser regex to non-greedy pattern to correctly separate test timing metadata
- Enhanced readme.hints.md with detailed explanation of test timing fix and planned protocol upgrades
- Updated readme.md with improved usage examples for tapbundle and comprehensive test framework documentation
- Added new protocol design document (readme.protocol.md) and improvement plan (readme.plan.md) outlining future changes
- Introduced .claude/settings.local.json update for npm and CLI permissions
- Exported protocol utilities and added tapbundle protocol implementation for future enhancements
## 2025-05-23 - 1.9.2 - fix(logging)
Fix log file naming to prevent collisions and update logging system documentation.
- Enhance safe filename generation in tstest logging to preserve directory structure using double underscores.
- Update readme.hints.md to include detailed logging system documentation and behavior.
- Add .claude/settings.local.json with updated permissions for build tools.
## 2025-05-23 - 1.9.1 - fix(dependencies)
Update dependency versions and add local configuration files
- Bump @git.zone/tsbuild from ^2.5.1 to ^2.6.3
- Bump @types/node from ^22.15.18 to ^22.15.21
- Bump @push.rocks/smartexpect from ^2.4.2 to ^2.5.0
- Bump @push.rocks/smartfile from ^11.2.0 to ^11.2.3
- Bump @push.rocks/smartlog from ^3.1.1 to ^3.1.8
- Add .npmrc with npm registry configuration
- Add .claude/settings.local.json for local permissions
## 2025-05-16 - 1.9.0 - feat(docs)
Update documentation to embed tapbundle and clarify module exports for browser compatibility; also add CI permission settings.

19
license Normal file
View File

@ -0,0 +1,19 @@
Copyright (c) 2014 Task Venture Capital GmbH (hello@task.vc)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -1,6 +1,6 @@
{
"name": "@git.zone/tstest",
"version": "1.9.0",
"version": "1.10.2",
"private": false,
"description": "a test utility to run tests that match test/**/*.ts",
"exports": {
@ -24,8 +24,8 @@
"buildDocs": "tsdoc"
},
"devDependencies": {
"@git.zone/tsbuild": "^2.5.1",
"@types/node": "^22.15.18"
"@git.zone/tsbuild": "^2.6.3",
"@types/node": "^22.15.21"
},
"dependencies": {
"@api.global/typedserver": "^3.0.74",
@ -37,10 +37,10 @@
"@push.rocks/smartcrypto": "^2.0.4",
"@push.rocks/smartdelay": "^3.0.5",
"@push.rocks/smartenv": "^5.0.12",
"@push.rocks/smartexpect": "^2.4.2",
"@push.rocks/smartfile": "^11.2.0",
"@push.rocks/smartexpect": "^2.5.0",
"@push.rocks/smartfile": "^11.2.3",
"@push.rocks/smartjson": "^5.0.20",
"@push.rocks/smartlog": "^3.1.1",
"@push.rocks/smartlog": "^3.1.8",
"@push.rocks/smartmongo": "^2.0.12",
"@push.rocks/smartpath": "^5.0.18",
"@push.rocks/smartpromise": "^4.2.3",

167
pnpm-lock.yaml generated
View File

@ -36,17 +36,17 @@ importers:
specifier: ^5.0.12
version: 5.0.12
'@push.rocks/smartexpect':
specifier: ^2.4.2
version: 2.4.2
specifier: ^2.5.0
version: 2.5.0
'@push.rocks/smartfile':
specifier: ^11.2.0
version: 11.2.0
specifier: ^11.2.3
version: 11.2.3
'@push.rocks/smartjson':
specifier: ^5.0.20
version: 5.0.20
'@push.rocks/smartlog':
specifier: ^3.1.1
version: 3.1.1
specifier: ^3.1.8
version: 3.1.8
'@push.rocks/smartmongo':
specifier: ^2.0.12
version: 2.0.12(@aws-sdk/credential-providers@3.810.0)(socks@2.8.4)
@ -79,11 +79,11 @@ importers:
version: 8.18.2
devDependencies:
'@git.zone/tsbuild':
specifier: ^2.5.1
version: 2.5.1
specifier: ^2.6.3
version: 2.6.3
'@types/node':
specifier: ^22.15.18
version: 22.15.18
specifier: ^22.15.21
version: 22.15.21
packages:
@ -585,8 +585,8 @@ packages:
cpu: [x64]
os: [win32]
'@git.zone/tsbuild@2.5.1':
resolution: {integrity: sha512-b1TyaNnaPCD3dvdRZ2da0MkZbH9liCrhzg57pwFIB2Gx4g8UMv8ZLN2cA1NRaNE0o8NCybf3gV1L+V0FO0DrMQ==}
'@git.zone/tsbuild@2.6.3':
resolution: {integrity: sha512-KIJYGQf9g5YibQZFWniYhESi7cWDZyRiudrYyipEQdyrv0o4VwXCdFgvsi90EZyoR2gdvz9qIWKeB1VaGx/dcQ==}
hasBin: true
'@git.zone/tsbundle@2.2.5':
@ -732,8 +732,8 @@ packages:
'@push.rocks/smartexit@1.0.23':
resolution: {integrity: sha512-WmwKYcwbHBByoABhHHB+PAjr5475AtD/xBh1mDcqPrFsOOUOZq3BBUdpq25wI3ccu/SZB5IwaimiVzadls6HkA==}
'@push.rocks/smartexpect@2.4.2':
resolution: {integrity: sha512-L+aS1n5rWhf/yOh5R3zPgwycYtDr5FfrDWgasy6ShhN6Zbn/z/AOPbWcF/OpeTmx0XabWB2h5d4xBcCKLl47cQ==}
'@push.rocks/smartexpect@2.5.0':
resolution: {integrity: sha512-yoyuCoQ3tTiAriuvF+/09fNbVfFnacudL2SwHSzPhX/ugaE7VTSWXQ9A34eKOWvil0MPyDcOY36fVZDxvrPd8A==}
'@push.rocks/smartfeed@1.0.11':
resolution: {integrity: sha512-02uhXxQamgfBo3T12FsAdfyElnpoWuDUb08B2AE60DbIaukVx/7Mi17xwobApY1flNSr5StZDt8N8vxPhBhIXw==}
@ -744,8 +744,8 @@ packages:
'@push.rocks/smartfile@10.0.41':
resolution: {integrity: sha512-xOOy0duI34M2qrJZggpk51EHGXmg9+mBL1Q55tNiQKXzfx89P3coY1EAZG8tvmep3qB712QEKe7T+u04t42Kjg==}
'@push.rocks/smartfile@11.2.0':
resolution: {integrity: sha512-0Gw6DvCQ2D/BXNN6airSC7hoSBut0p/uNWf2+rqO+D6VLhIJ/QUBvF6xm/LnpPI/zcF8YlDn/GEriInB5DUtEw==}
'@push.rocks/smartfile@11.2.3':
resolution: {integrity: sha512-gXUCwzHE6TuuzQIRGuZhJhPZJcVyc4G9nll32LHgmnBAU5ynDsGWUUbtFmpgcYLSAYFM9LGZS4b+ZrQPoDrtJw==}
'@push.rocks/smartguard@3.1.0':
resolution: {integrity: sha512-J23q84f1O+TwFGmd4lrO9XLHUh2DaLXo9PN/9VmTWYzTkQDv5JehmifXVI0esophXcCIfbdIu6hbt7/aHlDF4A==}
@ -765,8 +765,8 @@ packages:
'@push.rocks/smartlog-interfaces@3.0.2':
resolution: {integrity: sha512-8hGRTJehbsFSJxLhCQkA018mZtXVPxPTblbg9VaE/EqISRzUw+eosJ2EJV7M4Qu0eiTJZjnWnNLn8CkD77ziWw==}
'@push.rocks/smartlog@3.1.1':
resolution: {integrity: sha512-bANAjbPUty6jncut3FKHcSfrU6lm/gNW27q+BrqKH3I71qYWuh9HR0+tgJz+Ha47/sONzCmLcX/I2VejZ3njJg==}
'@push.rocks/smartlog@3.1.8':
resolution: {integrity: sha512-j4H5x4/hEmiIO7q+/LKyX3N+AhRIOj1jDE4TvZDvujZkbT/9wEWfpO1bqeMe/EQbg1eOQMlAuyrcLXUcDICpQg==}
'@push.rocks/smartmanifest@2.0.2':
resolution: {integrity: sha512-QGc5C9vunjfUbYsPGz5bynV/mVmPHkrQDkWp8ZO8VJtK1GZe+njgbrNyxn2SUHR0IhSAbSXl1j4JvBqYf5eTVg==}
@ -1353,8 +1353,8 @@ packages:
'@types/node-forge@1.3.11':
resolution: {integrity: sha512-FQx220y22OKNTqaByeBGqHWYz4cl94tpcxeFdvBo3wjG6XPBuZ0BNgNZRV5J5TFmmcsJ4IzsLkmGRiQbnYsBEQ==}
'@types/node@22.15.18':
resolution: {integrity: sha512-v1DKRfUdyW+jJhZNEI1PYy29S2YRxMV5AOO/x/SjKmW0acCIOqmbj6Haf9eHAhsPmrhlHSxEhv/1WszcLWV4cg==}
'@types/node@22.15.21':
resolution: {integrity: sha512-EV/37Td6c+MgKAbkcLG6vqZ2zEYHD7bvSrzqqs2RIhbA6w3x+Dqz8MZM3sP6kGTeLrdoOgKZe+Xja7tUB2DNkQ==}
'@types/ping@0.4.4':
resolution: {integrity: sha512-ifvo6w2f5eJYlXm+HiVx67iJe8WZp87sfa683nlqED5Vnt9Z93onkokNoWqOG21EaE8fMxyKPobE+mkPEyxsdw==}
@ -1541,7 +1541,7 @@ packages:
resolution: {integrity: sha512-InJljddsYWbEL8LBnopnCg+qMQp9KcowvYWOt4YWrjD5HmxzDYKdVbDS1w/ji5rFZdRD58V5UxJPtBdpEbEJYw==}
browserify-zlib@0.1.4:
resolution: {integrity: sha1-uzX4pRn2AOD6a4SFJByXnQFB+y0=}
resolution: {integrity: sha512-19OEpq7vWgsH6WkvkBJQDFvJS1uPcbFOQ4v9CU839dO+ZZXUZO6XpE6hNCqvlIIj+4fZvRiJ6DsAQ382GwiyTQ==}
bson@4.7.2:
resolution: {integrity: sha512-Ry9wCtIZ5kGqkJoi6aD8KjxFZEx78guTQDnpXWiNthsxzrxAK/i8E6pCHAIZTbaEFWcOCvbecMukfK7XUvyLpQ==}
@ -2103,7 +2103,7 @@ packages:
engines: {node: '>= 0.6'}
from2@2.3.0:
resolution: {integrity: sha1-i/tVAr3kpNNs/e6gB/zKIdfjgq8=}
resolution: {integrity: sha512-OMcX/4IC/uqEPVgGeyfN22LJk6AZrMkRZHxcHBMBvHScDGgwTm2GT2Wkgtocyd3JfZffjj2kYUDXXII0Fk9W0g==}
fs-constants@1.0.0:
resolution: {integrity: sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==}
@ -2309,7 +2309,7 @@ packages:
resolution: {integrity: sha512-eVRqCvVlZbuw3GrM63ovNSNAeA1K16kaR/LRY/92w0zxQ5/1YzwblUX652i4Xs9RwAGjW9d9y6X88t8OaAJfWQ==}
is-deflate@1.0.0:
resolution: {integrity: sha1-yGKQHDwWH7CdrHzcfnhPgOmPLxQ=}
resolution: {integrity: sha512-YDoFpuZWu1VRXlsnlYMzKyVRITXj7Ej/V9gXQ2/pAe7X1J7M/RNOqaIYi6qUn+B7nGyB9pDXrv02dsB58d2ZAQ==}
is-docker@2.2.1:
resolution: {integrity: sha512-F+i2BKsFrH66iaUFc0woD8sLy8getkwTwtOBjvs56Cx4CgJDeKQeqfz8wAYiSb8JOprWhHH5p77PbmYCvvUuXQ==}
@ -2325,7 +2325,7 @@ packages:
engines: {node: '>= 0.4'}
is-gzip@1.0.0:
resolution: {integrity: sha1-bKiwe5nHeZgCWQDlVc7Y7YCHmoM=}
resolution: {integrity: sha512-rcfALRIb1YewtnksfRIHGcIY93QnK8BIQ/2c9yDYcG/Y6+vRoJuTWBmmSEbyLLYtXm7q35pHOHbZFQBaLrhlWQ==}
engines: {node: '>=0.10.0'}
is-ip@4.0.0:
@ -2373,7 +2373,7 @@ packages:
engines: {node: '>=8'}
isarray@1.0.0:
resolution: {integrity: sha1-u5NdSFgsuhaMBoNJV6VKPgcSTxE=}
resolution: {integrity: sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ==}
isexe@2.0.0:
resolution: {integrity: sha1-6PvzdNxVb/iUehDcsFctYz8s+hA=}
@ -2388,8 +2388,8 @@ packages:
jackspeak@3.4.3:
resolution: {integrity: sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw==}
jackspeak@4.1.0:
resolution: {integrity: sha512-9DDdhb5j6cpeitCbvLO7n7J4IxnbM6hoF6O1g4HQ5TfhvvKN8ywDM7668ZhMHRqVmxqhps/F6syWK2KcPxYlkw==}
jackspeak@4.1.1:
resolution: {integrity: sha512-zptv57P3GpL+O0I7VdMJNBZCu+BPHVQUk55Ft8/QCJjTVxrnJHuVuX/0Bl2A6/+2oyR/ZMEuFKwmzqqZ/U5nPQ==}
engines: {node: 20 || >=22}
js-base64@3.7.7:
@ -2933,7 +2933,7 @@ packages:
engines: {node: '>=14.16'}
pako@0.2.9:
resolution: {integrity: sha1-8/dSL073gjSNqBYbrZ7P1Rv4OnU=}
resolution: {integrity: sha512-NUcwaKxUxWrZLpDG+z/xZaCgQITkA/Dv4V/T6bw7VON6l1Xz/VnrBqrYjZQ12TamKHzITTfOEIYUj48y2KXImA==}
pako@1.0.11:
resolution: {integrity: sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==}
@ -3045,7 +3045,7 @@ packages:
resolution: {integrity: sha512-TwEZ+X+yCJmYfL7TPUOcvBZ4QfoT5YenQiJuX//0th53DE6w0xxLEtfK3iyryQFddXuvkIk51EEgrJQ0WJkOmQ==}
proto-list@1.2.4:
resolution: {integrity: sha1-IS1b/hMYMGpCD2QCuOJv85ZHqEk=}
resolution: {integrity: sha512-vtK/94akxsTMhe0/cbfpR+syPuszcuwhqVjJq26CuNDgFGj682oRBXOP5MJpv2r7JtE8MsiepGIqvvOTBwn2vA==}
proxy-addr@2.0.7:
resolution: {integrity: sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg==}
@ -3350,7 +3350,7 @@ packages:
engines: {node: '>=12'}
strip-json-comments@2.0.1:
resolution: {integrity: sha1-PFMZQukIwml8DsNEhYwobHygpgo=}
resolution: {integrity: sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==}
engines: {node: '>=0.10.0'}
strnum@1.1.2:
@ -3489,6 +3489,11 @@ packages:
engines: {node: '>=14.17'}
hasBin: true
typescript@5.8.3:
resolution: {integrity: sha512-p1diW6TqL9L07nNxvRMM7hMMw4c5XOo/1ibL4aAIGmSAt9slTE1Xgw5KWuof2uTOvCg9BY7ZRi+GaF+7sfgPeQ==}
engines: {node: '>=14.17'}
hasBin: true
uglify-js@3.19.3:
resolution: {integrity: sha512-v3Xu+yuwBXisp6QYTcH4UbH+xYJXqnq2m/LtQVWKWzYc1iehYnLixoQDN9FH6/j9/oybfd6W9Ghwkl8+UMKTKQ==}
engines: {node: '>=0.8.0'}
@ -3734,9 +3739,9 @@ snapshots:
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartenv': 5.0.12
'@push.rocks/smartfeed': 1.0.11
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartjson': 5.0.20
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartlog-destination-devtools': 1.0.12
'@push.rocks/smartlog-interfaces': 3.0.2
'@push.rocks/smartmanifest': 2.0.2
@ -4536,17 +4541,17 @@ snapshots:
'@esbuild/win32-x64@0.24.2':
optional: true
'@git.zone/tsbuild@2.5.1':
'@git.zone/tsbuild@2.6.3':
dependencies:
'@git.zone/tspublish': 1.9.1
'@push.rocks/early': 4.0.4
'@push.rocks/smartcli': 4.0.11
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartpath': 5.0.18
'@push.rocks/smartpromise': 4.2.3
typescript: 5.7.3
typescript: 5.8.3
transitivePeerDependencies:
- aws-crt
@ -4555,8 +4560,8 @@ snapshots:
'@push.rocks/early': 4.0.4
'@push.rocks/smartcli': 4.0.11
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartlog-destination-local': 9.0.2
'@push.rocks/smartpath': 5.0.18
'@push.rocks/smartpromise': 4.2.3
@ -4572,8 +4577,8 @@ snapshots:
dependencies:
'@push.rocks/smartcli': 4.0.11
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartnpm': 2.0.4
'@push.rocks/smartpath': 5.0.18
'@push.rocks/smartrequest': 2.1.0
@ -4583,7 +4588,7 @@ snapshots:
'@git.zone/tsrun@1.3.3':
dependencies:
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartshell': 3.2.3
tsx: 4.19.2
@ -4687,7 +4692,7 @@ snapshots:
'@push.rocks/smartcache': 1.0.16
'@push.rocks/smartenv': 5.0.12
'@push.rocks/smartexit': 1.0.23
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartjson': 5.0.20
'@push.rocks/smartpath': 5.0.18
'@push.rocks/smartpromise': 4.2.3
@ -4725,8 +4730,8 @@ snapshots:
dependencies:
'@api.global/typedrequest': 3.1.10
'@configvault.io/interfaces': 1.0.17
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartpath': 5.0.18
'@push.rocks/smartarchive@3.0.8':
@ -4796,7 +4801,7 @@ snapshots:
'@push.rocks/smartcli@4.0.11':
dependencies:
'@push.rocks/lik': 6.2.2
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartobject': 1.0.12
'@push.rocks/smartpromise': 4.2.3
'@push.rocks/smartrx': 3.0.10
@ -4821,7 +4826,7 @@ snapshots:
dependencies:
'@push.rocks/lik': 6.2.2
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartmongo': 2.0.12(@aws-sdk/credential-providers@3.810.0)(socks@2.8.4)
'@push.rocks/smartpromise': 4.2.3
'@push.rocks/smartrx': 3.0.10
@ -4857,7 +4862,7 @@ snapshots:
'@push.rocks/smartpromise': 4.2.3
tree-kill: 1.2.2
'@push.rocks/smartexpect@2.4.2':
'@push.rocks/smartexpect@2.5.0':
dependencies:
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartpromise': 4.2.3
@ -4890,7 +4895,7 @@ snapshots:
glob: 10.4.5
js-yaml: 4.1.0
'@push.rocks/smartfile@11.2.0':
'@push.rocks/smartfile@11.2.3':
dependencies:
'@push.rocks/lik': 6.2.2
'@push.rocks/smartdelay': 3.0.5
@ -4943,13 +4948,13 @@ snapshots:
'@api.global/typedrequest-interfaces': 2.0.2
'@tsclass/tsclass': 4.4.4
'@push.rocks/smartlog@3.1.1':
'@push.rocks/smartlog@3.1.8':
dependencies:
'@api.global/typedrequest-interfaces': 3.0.19
'@push.rocks/consolecolor': 2.0.2
'@push.rocks/isounique': 1.0.5
'@push.rocks/smartclickhouse': 2.0.17
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smarthash': 3.0.4
'@push.rocks/smartpromise': 4.2.3
'@push.rocks/smarttime': 4.1.1
@ -5058,7 +5063,7 @@ snapshots:
dependencies:
'@push.rocks/smartbuffer': 3.0.4
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartnetwork': 3.0.2
'@push.rocks/smartpath': 5.0.18
'@push.rocks/smartpromise': 4.2.3
@ -5111,7 +5116,7 @@ snapshots:
'@push.rocks/smarts3@2.2.5':
dependencies:
'@push.rocks/smartbucket': 3.3.7
'@push.rocks/smartfile': 11.2.0
'@push.rocks/smartfile': 11.2.3
'@push.rocks/smartpath': 5.0.18
'@tsclass/tsclass': 4.4.4
'@types/s3rver': 3.7.4
@ -5148,7 +5153,7 @@ snapshots:
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartenv': 5.0.12
'@push.rocks/smartjson': 5.0.20
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartpromise': 4.2.3
'@push.rocks/smartrx': 3.0.10
'@push.rocks/smarttime': 4.1.1
@ -5246,7 +5251,7 @@ snapshots:
dependencies:
'@push.rocks/lik': 6.2.2
'@push.rocks/smartdelay': 3.0.5
'@push.rocks/smartlog': 3.1.1
'@push.rocks/smartlog': 3.1.8
'@push.rocks/smartpromise': 4.2.3
'@push.rocks/smartrx': 3.0.10
'@push.rocks/smarttime': 4.1.1
@ -5846,22 +5851,22 @@ snapshots:
'@types/body-parser@1.19.5':
dependencies:
'@types/connect': 3.4.38
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/buffer-json@2.0.3': {}
'@types/clean-css@4.2.11':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
source-map: 0.6.1
'@types/connect@3.4.38':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/cors@2.8.18':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/debug@4.1.12':
dependencies:
@ -5871,14 +5876,14 @@ snapshots:
'@types/express-serve-static-core@5.0.5':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/qs': 6.9.18
'@types/range-parser': 1.2.7
'@types/send': 0.17.4
'@types/express-serve-static-core@5.0.6':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/qs': 6.9.18
'@types/range-parser': 1.2.7
'@types/send': 0.17.4
@ -5902,30 +5907,30 @@ snapshots:
'@types/from2@2.3.5':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/fs-extra@11.0.4':
dependencies:
'@types/jsonfile': 6.1.4
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/fs-extra@9.0.13':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/glob@7.2.0':
dependencies:
'@types/minimatch': 5.1.2
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/glob@8.1.0':
dependencies:
'@types/minimatch': 5.1.2
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/gunzip-maybe@1.4.2':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/hast@3.0.4':
dependencies:
@ -5947,7 +5952,7 @@ snapshots:
'@types/jsonfile@6.1.4':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/mdast@4.0.4':
dependencies:
@ -5965,9 +5970,9 @@ snapshots:
'@types/node-forge@1.3.11':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/node@22.15.18':
'@types/node@22.15.21':
dependencies:
undici-types: 6.21.0
@ -5983,30 +5988,30 @@ snapshots:
'@types/s3rver@3.7.4':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/semver@7.7.0': {}
'@types/send@0.17.4':
dependencies:
'@types/mime': 1.3.5
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/serve-static@1.15.7':
dependencies:
'@types/http-errors': 2.0.4
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/send': 0.17.4
'@types/symbol-tree@3.2.5': {}
'@types/tar-stream@2.2.3':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/through2@2.0.41':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/triple-beam@1.3.5': {}
@ -6030,7 +6035,7 @@ snapshots:
'@types/whatwg-url@8.2.2':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/webidl-conversions': 7.0.3
'@types/which@2.0.2': {}
@ -6039,11 +6044,11 @@ snapshots:
'@types/ws@8.18.1':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
'@types/yauzl@2.10.3':
dependencies:
'@types/node': 22.15.18
'@types/node': 22.15.21
optional: true
'@ungap/structured-clone@1.3.0': {}
@ -6502,7 +6507,7 @@ snapshots:
engine.io@6.6.4:
dependencies:
'@types/cors': 2.8.18
'@types/node': 22.15.18
'@types/node': 22.15.21
accepts: 1.3.8
base64id: 2.0.0
cookie: 0.7.2
@ -6852,7 +6857,7 @@ snapshots:
glob@11.0.2:
dependencies:
foreground-child: 3.3.1
jackspeak: 4.1.0
jackspeak: 4.1.1
minimatch: 10.0.1
minipass: 7.1.2
package-json-from-dist: 1.0.1
@ -7104,7 +7109,7 @@ snapshots:
optionalDependencies:
'@pkgjs/parseargs': 0.11.0
jackspeak@4.1.0:
jackspeak@4.1.1:
dependencies:
'@isaacs/cliui': 8.0.2
@ -8494,6 +8499,8 @@ snapshots:
typescript@5.7.3: {}
typescript@5.8.3: {}
uglify-js@3.19.3: {}
uint8array-extras@1.4.0: {}

View File

@ -12,7 +12,7 @@ This project integrates tstest with tapbundle through a modular architecture:
### Test Execution Flow
1. **CLI Entry Point** (`cli.js` <20> `cli.ts.js` <20> `cli.child.ts`)
1. **CLI Entry Point** (`cli.js` <20> `cli.ts.js` <20> `cli.child.ts`)
- The CLI uses tsx to run TypeScript files directly
- Accepts glob patterns to find test files
- Supports options like `--verbose`, `--quiet`, `--web`
@ -59,4 +59,49 @@ The framework automatically detects the runtime environment:
- Browser tests are compiled and served via a local server
- WebHelpers are only enabled in browser environment
This architecture allows for seamless testing across both Node.js and browser environments while maintaining a clean separation of concerns.
This architecture allows for seamless testing across both Node.js and browser environments while maintaining a clean separation of concerns.
## Logging System
### Log File Naming (Fixed in v1.9.1)
When using the `--logfile` flag, tstest creates log files in `.nogit/testlogs/`. The log file naming was updated to preserve directory structure and prevent collisions:
- **Old behavior**: `test/tapbundle/test.ts``.nogit/testlogs/test.log`
- **New behavior**: `test/tapbundle/test.ts``.nogit/testlogs/test__tapbundle__test.log`
This fix ensures that test files with the same basename in different directories don't overwrite each other's logs. The implementation:
1. Takes the relative path from the current working directory
2. Replaces path separators (`/`) with double underscores (`__`)
3. Removes the `.ts` extension
4. Creates a flat filename that preserves the directory structure
### Test Timing Display (Fixed in v1.9.2)
Fixed an issue where test timing was displayed incorrectly with duplicate values like:
- Before: `✅ test name # time=133ms (0ms)`
- After: `✅ test name (133ms)`
The issue was in the TAP parser regex which was greedily capturing the entire line including the TAP timing comment. Changed the regex from `(.*)` to `(.*?)` to make it non-greedy, properly separating the test name from the timing metadata.
## Protocol Limitations and Improvements
### Current TAP Protocol Issues
The current implementation uses standard TAP format with metadata in comments:
```
ok 1 - test name # time=123ms
```
This has several limitations:
1. **Delimiter Conflict**: Test descriptions containing `#` can break parsing
2. **Regex Fragility**: Complex regex patterns that are hard to maintain
3. **Limited Metadata**: Difficult to add rich error information or custom data
### Planned Protocol V2
A new internal protocol is being designed that will:
- Use Unicode delimiters `⟦TSTEST:⟧` that won't conflict with test content
- Support structured JSON metadata
- Allow rich error reporting with stack traces and diffs
- Maintain backwards compatibility during migration
See `readme.protocol.md` for the full specification and `tapbundle.protocols.ts` for the implementation utilities.

474
readme.md
View File

@ -141,9 +141,9 @@ tstest supports different test environments through file naming:
| `*.browser.ts` | Browser environment | `test.ui.browser.ts` |
| `*.both.ts` | Both Node.js and browser | `test.isomorphic.both.ts` |
### Writing Tests
### Writing Tests with tapbundle
tstest includes a built-in TAP (Test Anything Protocol) test framework. Import it from the embedded tapbundle:
tstest includes tapbundle, a powerful TAP-based test framework. Import it from the embedded tapbundle:
```typescript
import { expect, tap } from '@git.zone/tstest/tapbundle';
@ -164,100 +164,392 @@ tstest provides multiple exports for different use cases:
- `@git.zone/tstest/tapbundle` - Browser-compatible test framework
- `@git.zone/tstest/tapbundle_node` - Node.js-specific test utilities
#### Test Features
## tapbundle Test Framework
### Basic Test Syntax
**Tag-based Test Filtering**
```typescript
tap.tags('unit', 'api')
.test('should handle API requests', async () => {
// Test code
});
import { tap, expect } from '@git.zone/tstest/tapbundle';
// Run with: tstest test/ --tags unit,api
```
**Test Lifecycle Hooks**
```typescript
tap.describe('User API Tests', () => {
let testUser;
tap.beforeEach(async () => {
testUser = await createTestUser();
});
tap.afterEach(async () => {
await deleteTestUser(testUser.id);
});
tap.test('should update user profile', async () => {
// Test code using testUser
});
// Basic test
tap.test('should perform basic arithmetic', async () => {
expect(2 + 2).toEqual(4);
});
```
**Parallel Test Execution**
```typescript
// Files with matching parallel group names run concurrently
// test.auth.para__1.ts
tap.test('authentication test', async () => { /* ... */ });
// test.user.para__1.ts
tap.test('user operations test', async () => { /* ... */ });
```
**Test Timeouts and Retries**
```typescript
tap.timeout(5000)
.retry(3)
.test('flaky network test', async (tools) => {
// This test has 5 seconds to complete and will retry up to 3 times
});
```
**Snapshot Testing**
```typescript
tap.test('should match snapshot', async (tools) => {
const result = await generateReport();
await tools.matchSnapshot(result);
// Async test with tools
tap.test('async operations', async (tools) => {
await tools.delayFor(100); // delay for 100ms
const result = await fetchData();
expect(result).toBeDefined();
});
// Start test execution
tap.start();
```
**Test Fixtures**
```typescript
// Define a reusable fixture
tap.defineFixture('testUser', async () => ({
id: 1,
name: 'Test User',
email: 'test@example.com'
}));
### Test Modifiers and Chaining
tap.test('user test', async (tools) => {
const user = tools.fixture('testUser');
expect(user.name).toEqual('Test User');
});
```
**Skipping and Todo Tests**
```typescript
tap.skip.test('work in progress', async () => {
// Skip a test
tap.skip.test('not ready yet', async () => {
// This test will be skipped
});
tap.todo('implement user deletion', async () => {
// This marks a test as todo
// Run only this test (exclusive)
tap.only.test('focus on this', async () => {
// Only this test will run
});
// Todo test
tap.todo('implement later', async () => {
// Marked as todo
});
// Chaining modifiers
tap.timeout(5000)
.retry(3)
.tags('api', 'integration')
.test('complex test', async (tools) => {
// Test with 5s timeout, 3 retries, and tags
});
```
### Test Organization with describe()
```typescript
tap.describe('User Management', () => {
let testDatabase;
tap.beforeEach(async () => {
testDatabase = await createTestDB();
});
tap.afterEach(async () => {
await testDatabase.cleanup();
});
tap.test('should create user', async () => {
const user = await testDatabase.createUser({ name: 'John' });
expect(user.id).toBeDefined();
});
tap.describe('User Permissions', () => {
tap.test('should set admin role', async () => {
// Nested describe blocks
});
});
});
```
**Browser Testing**
### Test Tools (Available in Test Function)
Every test function receives a `tools` parameter with utilities:
```typescript
tap.test('using test tools', async (tools) => {
// Delay utilities
await tools.delayFor(1000); // delay for 1000ms
await tools.delayForRandom(100, 500); // random delay between 100-500ms
// Skip test conditionally
tools.skipIf(process.env.CI === 'true', 'Skipping in CI');
// Skip test unconditionally
if (!apiKeyAvailable) {
tools.skip('API key not available');
}
// Mark as todo
tools.todo('Needs implementation');
// Retry configuration
tools.retry(3); // Set retry count
// Timeout configuration
tools.timeout(10000); // Set timeout to 10s
// Context sharing between tests
tools.context.set('userId', 12345);
const userId = tools.context.get('userId');
// Deferred promises
const deferred = tools.defer();
setTimeout(() => deferred.resolve('done'), 100);
await deferred.promise;
// Colored console output
const coloredString = await tools.coloredString('Success!', 'green');
console.log(coloredString);
// Error handling helper
const error = await tools.returnError(async () => {
throw new Error('Expected error');
});
expect(error).toBeInstanceOf(Error);
});
```
### Snapshot Testing
```typescript
tap.test('snapshot test', async (tools) => {
const output = generateComplexOutput();
// Compare with saved snapshot
await tools.matchSnapshot(output);
// Named snapshots for multiple checks in one test
await tools.matchSnapshot(output.header, 'header');
await tools.matchSnapshot(output.body, 'body');
});
// Update snapshots with: UPDATE_SNAPSHOTS=true tstest test/
```
### Test Fixtures
```typescript
// Define reusable fixtures
tap.defineFixture('testUser', async (data) => ({
id: Date.now(),
name: data?.name || 'Test User',
email: data?.email || 'test@example.com',
created: new Date()
}));
tap.defineFixture('testPost', async (data) => ({
id: Date.now(),
title: data?.title || 'Test Post',
authorId: data?.authorId || 1
}));
// Use fixtures in tests
tap.test('fixture test', async (tools) => {
const user = await tools.fixture('testUser', { name: 'John' });
const post = await tools.fixture('testPost', { authorId: user.id });
expect(post.authorId).toEqual(user.id);
// Factory pattern for multiple instances
const users = await tools.factory('testUser').createMany(5);
expect(users).toHaveLength(5);
});
```
### Parallel Test Execution
```typescript
// Parallel tests within a file
tap.testParallel('parallel test 1', async () => {
await heavyOperation();
});
tap.testParallel('parallel test 2', async () => {
await anotherHeavyOperation();
});
// File naming for parallel groups
// test.api.para__1.ts - runs in parallel with other para__1 files
// test.db.para__1.ts - runs in parallel with other para__1 files
// test.auth.para__2.ts - runs after para__1 group completes
```
### Assertions with expect()
tapbundle uses @push.rocks/smartexpect for assertions:
```typescript
// Basic assertions
expect(value).toEqual(5);
expect(value).not.toEqual(10);
expect(obj).toDeepEqual({ a: 1, b: 2 });
// Type assertions
expect('hello').toBeTypeofString();
expect(42).toBeTypeofNumber();
expect(true).toBeTypeofBoolean();
expect([]).toBeArray();
expect({}).toBeTypeOf('object');
// Comparison assertions
expect(5).toBeGreaterThan(3);
expect(3).toBeLessThan(5);
expect(5).toBeGreaterThanOrEqual(5);
expect(5).toBeLessThanOrEqual(5);
expect(0.1 + 0.2).toBeCloseTo(0.3, 10);
// Truthiness
expect(true).toBeTrue();
expect(false).toBeFalse();
expect('text').toBeTruthy();
expect(0).toBeFalsy();
expect(null).toBeNull();
expect(undefined).toBeUndefined();
expect(null).toBeNullOrUndefined();
// String assertions
expect('hello world').toStartWith('hello');
expect('hello world').toEndWith('world');
expect('hello world').toInclude('lo wo');
expect('hello world').toMatch(/^hello/);
expect('option').toBeOneOf(['choice', 'option', 'alternative']);
// Array assertions
expect([1, 2, 3]).toContain(2);
expect([1, 2, 3]).toContainAll([1, 3]);
expect([1, 2, 3]).toExclude(4);
expect([1, 2, 3]).toHaveLength(3);
expect([]).toBeEmptyArray();
expect([{ id: 1 }]).toContainEqual({ id: 1 });
// Object assertions
expect(obj).toHaveProperty('name');
expect(obj).toHaveProperty('user.email', 'test@example.com');
expect(obj).toHaveDeepProperty(['level1', 'level2']);
expect(obj).toMatchObject({ name: 'John' });
// Function assertions
expect(() => { throw new Error('test'); }).toThrow();
expect(() => { throw new Error('test'); }).toThrow(Error);
expect(() => { throw new Error('test error'); }).toThrowErrorMatching(/test/);
expect(myFunction).not.toThrow();
// Promise assertions
await expect(Promise.resolve('value')).resolves.toEqual('value');
await expect(Promise.reject(new Error('fail'))).rejects.toThrow();
// Custom assertions
expect(7).customAssertion(
value => value % 2 === 1,
'Value is not odd'
);
```
### Pre-tasks
Run setup tasks before tests start:
```typescript
tap.preTask('setup database', async () => {
await initializeTestDatabase();
console.log('Database initialized');
});
tap.preTask('load environment', async () => {
await loadTestEnvironment();
});
// Pre-tasks run in order before any tests
```
### Tag-based Test Filtering
```typescript
// Tag individual tests
tap.tags('unit', 'api')
.test('api unit test', async () => {
// Test code
});
tap.tags('integration', 'slow')
.test('database integration', async () => {
// Test code
});
// Run only tests with specific tags
// tstest test/ --tags unit,api
```
### Context Sharing
Share data between tests:
```typescript
tap.test('first test', async (tools) => {
const sessionId = await createSession();
tools.context.set('sessionId', sessionId);
});
tap.test('second test', async (tools) => {
const sessionId = tools.context.get('sessionId');
expect(sessionId).toBeDefined();
// Cleanup
tools.context.delete('sessionId');
});
```
### Browser Testing with webhelpers
For browser-specific tests:
```typescript
// test.browser.ts
import { tap, webhelpers } from '@git.zone/tstest/tapbundle';
tap.test('DOM manipulation', async () => {
// Create DOM elements from HTML strings
const element = await webhelpers.fixture(webhelpers.html`
<div>Hello World</div>
<div class="test-container">
<h1>Test Title</h1>
<button id="test-btn">Click Me</button>
</div>
`);
expect(element).toBeInstanceOf(HTMLElement);
expect(element.querySelector('h1').textContent).toEqual('Test Title');
// Simulate interactions
const button = element.querySelector('#test-btn');
button.click();
});
tap.test('CSS testing', async () => {
const styles = webhelpers.css`
.test-class {
color: red;
font-size: 16px;
}
`;
// styles is a string that can be injected into the page
expect(styles).toInclude('color: red');
});
```
### Advanced Error Handling
```typescript
tap.test('error handling', async (tools) => {
// Capture errors without failing the test
const error = await tools.returnError(async () => {
await functionThatThrows();
});
expect(error).toBeInstanceOf(Error);
expect(error.message).toEqual('Expected error message');
});
```
### Test Wrap
Create wrapped test environments:
```typescript
import { TapWrap } from '@git.zone/tstest/tapbundle';
const tapWrap = new TapWrap({
before: async () => {
console.log('Before all tests');
await globalSetup();
},
after: async () => {
console.log('After all tests');
await globalCleanup();
}
});
// Tests registered here will have the wrap lifecycle
tapWrap.tap.test('wrapped test', async () => {
// This test runs with the wrap setup/teardown
});
```
@ -330,6 +622,20 @@ tstest test/ --quiet
## Changelog
### Version 1.9.2
- 🐛 Fixed test timing display issue (removed duplicate timing in output)
- 📝 Improved internal protocol design documentation
- 🔧 Added protocol v2 utilities for future improvements
### Version 1.9.1
- 🐛 Fixed log file naming to preserve directory structure
- 📁 Log files now prevent collisions: `test__dir__file.log`
### Version 1.9.0
- 📚 Comprehensive documentation update
- 🏗️ Embedded tapbundle for better integration
- 🌐 Full browser compatibility
### Version 1.8.0
- 📦 Embedded tapbundle directly into tstest project
- 🌐 Made tapbundle fully browser-compatible
@ -342,13 +648,21 @@ tstest test/ --quiet
- 📊 Enhanced TAP parser for better test reporting
- 🐛 Fixed glob pattern handling in shell scripts
## Contribution
## License and Legal Information
We are always happy for code contributions. If you are not the code contributing type that is ok. Still, maintaining Open Source repositories takes considerable time and thought. If you like the quality of what we do and our modules are useful to you we would appreciate a little monthly contribution: You can [contribute one time](https://lossless.link/contribute-onetime) or [contribute monthly](https://lossless.link/contribute). :)
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license.md) file within this repository.
## License
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
> MIT licensed | **&copy;** [Lossless GmbH](https://lossless.gmbh)
| By using this npm module you agree to our [privacy policy](https://lossless.gmbH/privacy)
### Trademarks
[![repo-footer](https://lossless.gitlab.io/publicrelations/repofooter.svg)](https://maintainedby.lossless.com)
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
### Company Information
Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

View File

@ -2,6 +2,81 @@
!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
## Improved Internal Protocol (NEW - Critical)
### Current Issues
- TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
- Fragile regex parsing that breaks with special characters
- Limited extensibility for new metadata types
### Proposed Solution: Protocol V2
- Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
- Structured JSON metadata format
- Separate protocol blocks for complex data (errors, snapshots)
- Backwards compatible with gradual migration
### Implementation
- Phase 1: Add protocol v2 parser alongside v1
- Phase 2: Generate v2 by default with --legacy flag for v1
- Phase 3: Full migration to v2 in next major version
See `readme.protocol.md` for detailed specification.
## Test Configuration System (NEW)
### Global Test Configuration via 00init.ts
- **Discovery**: Check for `test/00init.ts` before running tests
- **Execution**: Import and execute before any test files if found
- **Purpose**: Define project-wide default test settings
### tap.settings() API
```typescript
interface TapSettings {
// Timing
timeout?: number; // Default timeout for all tests (ms)
slowThreshold?: number; // Mark tests as slow if they exceed this (ms)
// Execution Control
bail?: boolean; // Stop on first test failure
retries?: number; // Number of retries for failed tests
retryDelay?: number; // Delay between retries (ms)
// Output Control
suppressConsole?: boolean; // Suppress console output in passing tests
verboseErrors?: boolean; // Show full stack traces
showTestDuration?: boolean; // Show duration for each test
// Parallel Execution
maxConcurrency?: number; // Max parallel tests (for .para files)
isolateTests?: boolean; // Run each test in fresh context
// Lifecycle Hooks
beforeAll?: () => Promise<void> | void;
afterAll?: () => Promise<void> | void;
beforeEach?: (testName: string) => Promise<void> | void;
afterEach?: (testName: string, passed: boolean) => Promise<void> | void;
// Environment
env?: Record<string, string>; // Additional environment variables
// Features
enableSnapshots?: boolean; // Enable snapshot testing
snapshotDirectory?: string; // Custom snapshot directory
updateSnapshots?: boolean; // Update snapshots instead of comparing
}
```
### Settings Inheritance
- Global (00init.ts) → File level → Test level
- More specific settings override less specific ones
- Arrays/objects are merged, primitives are replaced
### Implementation Phases
1. **Core Infrastructure**: Settings storage and merge logic
2. **Discovery**: 00init.ts loading mechanism
3. **Application**: Apply settings to test execution
4. **Advanced**: Parallel execution and snapshot configuration
## 1. Enhanced Communication Between tapbundle and tstest
### 1.1 Real-time Test Progress API
@ -18,45 +93,9 @@
## 2. Enhanced toolsArg Functionality
### 2.1 Test Flow Control ✅
```typescript
tap.test('conditional test', async (toolsArg) => {
const result = await someOperation();
// Skip the rest of the test
if (!result) {
return toolsArg.skip('Precondition not met');
}
// Conditional skipping
await toolsArg.skipIf(condition, 'Reason for skipping');
// Mark test as todo
await toolsArg.todo('Not implemented yet');
});
```
### 2.2 Test Metadata and Configuration ✅
```typescript
// Fluent syntax ✅
tap.tags('slow', 'integration')
.priority('high')
.timeout(5000)
.retry(3)
.test('configurable test', async (toolsArg) => {
// Test implementation
});
```
### 2.3 Test Data and Context Sharing ✅
### 2.3 Test Data and Context Sharing (Partial)
```typescript
tap.test('data-driven test', async (toolsArg) => {
// Access shared context ✅
const sharedData = toolsArg.context.get('sharedData');
// Set data for other tests ✅
toolsArg.context.set('resultData', computedValue);
// Parameterized test data (not yet implemented)
const testData = toolsArg.data<TestInput>();
expect(processData(testData)).toEqual(expected);
@ -65,32 +104,7 @@ tap.test('data-driven test', async (toolsArg) => {
## 3. Nested Tests and Test Suites
### 3.1 Test Grouping with describe() ✅
```typescript
tap.describe('User Authentication', () => {
tap.beforeEach(async (toolsArg) => {
// Setup for each test in this suite
await toolsArg.context.set('db', await createTestDatabase());
});
tap.afterEach(async (toolsArg) => {
// Cleanup after each test
await toolsArg.context.get('db').cleanup();
});
tap.test('should login with valid credentials', async (toolsArg) => {
// Test implementation
});
tap.describe('Password Reset', () => {
tap.test('should send reset email', async (toolsArg) => {
// Nested test
});
});
});
```
### 3.2 Hierarchical Test Organization
### 3.2 Hierarchical Test Organization (Not yet implemented)
- Support for multiple levels of nesting
- Inherited context and configuration from parent suites
- Aggregated reporting for test suites
@ -98,15 +112,7 @@ tap.describe('User Authentication', () => {
## 4. Advanced Test Features
### 4.1 Snapshot Testing
```typescript
tap.test('component render', async (toolsArg) => {
const output = renderComponent(props);
// Compare with stored snapshot
await toolsArg.matchSnapshot(output, 'component-output');
});
```
### 4.1 Snapshot Testing ✅ (Basic implementation complete)
### 4.2 Performance Benchmarking
```typescript
@ -124,30 +130,9 @@ tap.test('performance test', async (toolsArg) => {
});
```
### 4.3 Test Fixtures and Factories ✅
```typescript
tap.test('with fixtures', async (toolsArg) => {
// Create test fixtures
const user = await toolsArg.fixture('user', { name: 'Test User' });
const post = await toolsArg.fixture('post', { author: user });
// Use factory functions
const users = await toolsArg.factory('user').createMany(5);
});
```
## 5. Test Execution Improvements
### 5.1 Parallel Test Execution ✅
- Run independent tests concurrently ✅
- Configurable concurrency limits (via file naming convention)
- Resource pooling for shared resources
- Proper isolation between parallel tests ✅
Implementation:
- Tests with `para__<groupNumber>` in filename run in parallel
- Different groups run sequentially
- Tests without `para__` run serially
### 5.2 Watch Mode
- Automatically re-run tests on file changes
@ -155,11 +140,8 @@ Implementation:
- Fast feedback loop for development
- Integration with IDE/editor plugins
### 5.3 Advanced Test Filtering ✅ (partially)
### 5.3 Advanced Test Filtering (Partial)
```typescript
// Run tests by tags ✅
tstest --tags "unit,fast"
// Exclude tests by pattern (not yet implemented)
tstest --exclude "**/slow/**"
@ -198,50 +180,36 @@ tstest --changed
- Links to documentation
- Code examples in error output
### 7.2 Interactive Mode (Needs Detailed Specification)
- REPL for exploring test failures
- Need to define: How to enter interactive mode? When tests fail?
- What commands/features should be available in the REPL?
- Debugging integration
- Node.js inspector protocol integration?
- Breakpoint support?
- Step-through test execution
- Pause between tests?
- Step into/over/out functionality?
- Interactive test data manipulation
- Modify test inputs on the fly?
- Inspect intermediate values?
### 7.3 ~~VS Code Extension~~ (Scratched)
- ~~Test explorer integration~~
- ~~Inline test results~~
- ~~CodeLens for running individual tests~~
- ~~Debugging support~~
## Implementation Phases
### Phase 1: Core Enhancements (Priority: High) ✅
1. Implement enhanced toolsArg methods (skip, skipIf, timeout, retry) ✅
2. Add basic test grouping with describe() ✅
3. Improve error reporting between tapbundle and tstest ✅
### Phase 1: Improved Internal Protocol (Priority: Critical) (NEW)
1. Implement Protocol V2 parser in tstest
2. Add protocol version negotiation
3. Update tapbundle to generate V2 format with feature flag
4. Test with real-world test suites containing special characters
### Phase 2: Advanced Features (Priority: Medium)
1. Implement nested test suites ✅ (basic describe support)
2. Add snapshot testing ✅
3. Create test fixture system ✅
4. Implement parallel test execution ✅
### Phase 2: Test Configuration System (Priority: High)
1. Implement tap.settings() API with TypeScript interfaces
2. Add 00init.ts discovery and loading mechanism
3. Implement settings inheritance and merge logic
4. Apply settings to test execution (timeouts, retries, etc.)
### Phase 3: Developer Experience (Priority: Medium)
### Phase 3: Enhanced Communication (Priority: High)
1. Build on Protocol V2 for richer communication
2. Implement real-time test progress API
3. Add structured error reporting with diffs and traces
### Phase 4: Developer Experience (Priority: Medium)
1. Add watch mode
2. Implement custom reporters
3. ~~Create VS Code extension~~ (Scratched)
4. Add interactive debugging (Needs detailed spec first)
3. Complete advanced test filtering options
4. Add performance benchmarking API
### Phase 4: Analytics and Performance (Priority: Low)
### Phase 5: Analytics and Performance (Priority: Low)
1. Build test analytics dashboard
2. Add performance benchmarking
3. Implement coverage integration
4. Create trend analysis tools
2. Implement coverage integration
3. Create trend analysis tools
4. Add test impact analysis
## Technical Considerations

287
readme.protocol.md Normal file
View File

@ -0,0 +1,287 @@
# Improved Internal Protocol Design
## Current Issues with TAP Protocol
1. **Delimiter Conflict**: Using `#` for metadata conflicts with test descriptions containing `#`
2. **Ambiguous Parsing**: No clear boundary between test name and metadata
3. **Limited Extensibility**: Adding new metadata requires regex changes
4. **Mixed Concerns**: Protocol data mixed with human-readable output
## Proposed Internal Protocol v2
### Design Principles
1. **Clear Separation**: Protocol data must be unambiguously separated from user content
2. **Extensibility**: Easy to add new metadata without breaking parsers
3. **Backwards Compatible**: Can coexist with standard TAP for gradual migration
4. **Machine Readable**: Structured format for reliable parsing
5. **Human Friendly**: Still readable in raw form
### Protocol Options
#### Option 1: Special Delimiters
```
ok 1 - test description ::TSTEST:: {"time":123,"retry":0}
not ok 2 - another test ::TSTEST:: {"time":45,"error":"timeout"}
ok 3 - skipped test ::TSTEST:: {"time":0,"skip":"not ready"}
```
**Pros**:
- Simple to implement
- Backwards compatible with TAP parsers (they ignore the suffix)
- Easy to parse with split()
**Cons**:
- Still could conflict if test name contains `::TSTEST::`
- Not standard TAP
#### Option 2: Separate Metadata Lines
```
ok 1 - test description
::METADATA:: {"test":1,"time":123,"retry":0}
not ok 2 - another test
::METADATA:: {"test":2,"time":45,"error":"timeout"}
```
**Pros**:
- Complete separation of concerns
- No chance of conflicts
- Can include arbitrary metadata
**Cons**:
- Requires correlation between lines
- More complex parsing
#### Option 3: YAML Blocks (TAP 13 Compatible)
```
ok 1 - test description
---
time: 123
retry: 0
...
not ok 2 - another test
---
time: 45
error: timeout
stack: |
Error: timeout
at Test.run (test.js:10:5)
...
```
**Pros**:
- Standard TAP 13 feature
- Structured data format
- Human readable
- Extensible
**Cons**:
- More verbose
- YAML parsing overhead
#### Option 4: Binary Protocol Markers (Recommended)
```
ok 1 - test description
␛[TSTEST:eyJ0aW1lIjoxMjMsInJldHJ5IjowfQ==]␛
not ok 2 - another test
␛[TSTEST:eyJ0aW1lIjo0NSwiZXJyb3IiOiJ0aW1lb3V0In0=]␛
```
Using ASCII escape character (␛ = \x1B) with base64 encoded JSON.
**Pros**:
- Zero chance of accidental conflicts
- Compact
- Fast to parse
- Invisible in most terminals
**Cons**:
- Not human readable in raw form
- Requires base64 encoding/decoding
### Recommended Implementation: Hybrid Approach
Use multiple strategies based on context:
1. **For timing and basic metadata**: Use structured delimiters
```
ok 1 - test name ⟦time:123,retry:0⟧
```
2. **For complex data (errors, snapshots)**: Use separate protocol lines
```
ok 1 - test failed
⟦TSTEST:ERROR⟧
{"message":"Assertion failed","stack":"...","diff":"..."}
⟦/TSTEST:ERROR⟧
```
3. **For human-readable output**: Keep standard TAP comments
```
# Test suite: User Authentication
ok 1 - should login
```
### Implementation Plan
#### Phase 1: Parser Enhancement
1. Add new protocol parser alongside existing TAP parser
2. Support both old and new formats during transition
3. Add protocol version negotiation
#### Phase 2: Metadata Structure
```typescript
interface TestMetadata {
// Timing
time: number; // milliseconds
startTime?: number; // Unix timestamp
endTime?: number; // Unix timestamp
// Status
skip?: string; // skip reason
todo?: string; // todo reason
retry?: number; // retry attempt
maxRetries?: number; // max retries allowed
// Error details
error?: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
};
// Test context
file?: string; // source file
line?: number; // line number
column?: number; // column number
// Custom data
tags?: string[]; // test tags
custom?: Record<string, any>;
}
```
#### Phase 3: Protocol Messages
##### Success Message
```
ok 1 - user authentication works
⟦TSTEST:META:{"time":123,"tags":["auth","unit"]}⟧
```
##### Failure Message
```
not ok 2 - login fails with invalid password
⟦TSTEST:META:{"time":45,"retry":1,"maxRetries":3}⟧
⟦TSTEST:ERROR⟧
{
"message": "Expected 401 but got 500",
"stack": "Error: Expected 401 but got 500\n at Test.run (auth.test.ts:25:10)",
"actual": 500,
"expected": 401
}
⟦/TSTEST:ERROR⟧
```
##### Skip Message
```
ok 3 - database integration test ⟦TSTEST:SKIP:No database connection⟧
```
##### Snapshot Communication
```
⟦TSTEST:SNAPSHOT:user-profile⟧
{
"name": "John Doe",
"email": "john@example.com",
"roles": ["user", "admin"]
}
⟦/TSTEST:SNAPSHOT⟧
```
### Migration Strategy
1. **Version Detection**: First line indicates protocol version
```
⟦TSTEST:PROTOCOL:2.0⟧
TAP version 13
```
2. **Gradual Rollout**:
- v1.10: Add protocol v2 parser, keep v1 generator
- v1.11: Generate v2 by default, v1 with --legacy flag
- v2.0: Remove v1 support
3. **Feature Flags**:
```typescript
tap.settings({
protocol: 'v2', // or 'v1', 'auto'
protocolFeatures: {
structuredErrors: true,
enhancedTiming: true,
binaryMarkers: false
}
});
```
### Benefits of New Protocol
1. **Reliability**: No more regex fragility or description conflicts
2. **Performance**: Faster parsing with clear boundaries
3. **Extensibility**: Easy to add new metadata fields
4. **Debugging**: Rich error information with stack traces and diffs
5. **Integration**: Better IDE and CI/CD tool integration
6. **Forward Compatible**: Room for future enhancements
### Example Parser Implementation
```typescript
class ProtocolV2Parser {
private readonly MARKER_START = '⟦TSTEST:';
private readonly MARKER_END = '⟧';
parseMetadata(line: string): TestMetadata | null {
const start = line.lastIndexOf(this.MARKER_START);
if (start === -1) return null;
const end = line.indexOf(this.MARKER_END, start);
if (end === -1) return null;
const content = line.substring(start + this.MARKER_START.length, end);
const [type, data] = content.split(':', 2);
switch (type) {
case 'META':
return JSON.parse(data);
case 'SKIP':
return { skip: data };
case 'TODO':
return { todo: data };
default:
return null;
}
}
parseTestLine(line: string): ParsedTest {
// First extract any metadata
const metadata = this.parseMetadata(line);
// Then parse the TAP part (without metadata)
const cleanLine = this.removeMetadata(line);
const tapResult = this.parseTAP(cleanLine);
return { ...tapResult, metadata };
}
}
```
### Next Steps
1. Implement proof of concept with basic metadata support
2. Test with real-world test suites for edge cases
3. Benchmark parsing performance
4. Get feedback from users
5. Finalize protocol specification
6. Implement in both tapbundle and tstest

View File

@ -0,0 +1,8 @@
import { tap } from '../../ts_tapbundle/index.js';
tap.test('spec file test', async () => {
console.log('This is a .spec.ts file that should be found by glob');
return true;
});
tap.start();

View File

@ -0,0 +1,8 @@
import { tap } from '../../../ts_tapbundle/index.js';
tap.test('nested glob pattern test', async () => {
console.log('This test file is in a nested directory');
return true;
});
tap.start();

View File

@ -0,0 +1,8 @@
import { tap } from '../../ts_tapbundle/index.js';
tap.test('glob pattern test', async () => {
console.log('This test file should be found by glob patterns');
return true;
});
tap.start();

View File

@ -3,6 +3,6 @@
*/
export const commitinfo = {
name: '@git.zone/tstest',
version: '1.9.0',
version: '1.10.2',
description: 'a test utility to run tests that match test/**/*.ts'
}

View File

@ -13,6 +13,8 @@ export const runCli = async () => {
const logOptions: LogOptions = {};
let testPath: string | null = null;
let tags: string[] = [];
let startFromFile: number | null = null;
let stopAtFile: number | null = null;
// Parse options
for (let i = 0; i < args.length; i++) {
@ -42,6 +44,32 @@ export const runCli = async () => {
tags = args[++i].split(',');
}
break;
case '--startFrom':
if (i + 1 < args.length) {
const value = parseInt(args[++i], 10);
if (isNaN(value) || value < 1) {
console.error('Error: --startFrom must be a positive integer');
process.exit(1);
}
startFromFile = value;
} else {
console.error('Error: --startFrom requires a number argument');
process.exit(1);
}
break;
case '--stopAt':
if (i + 1 < args.length) {
const value = parseInt(args[++i], 10);
if (isNaN(value) || value < 1) {
console.error('Error: --stopAt must be a positive integer');
process.exit(1);
}
stopAtFile = value;
} else {
console.error('Error: --stopAt requires a number argument');
process.exit(1);
}
break;
default:
if (!arg.startsWith('-')) {
testPath = arg;
@ -49,16 +77,24 @@ export const runCli = async () => {
}
}
// Validate test file range options
if (startFromFile !== null && stopAtFile !== null && startFromFile > stopAtFile) {
console.error('Error: --startFrom cannot be greater than --stopAt');
process.exit(1);
}
if (!testPath) {
console.error('You must specify a test directory/file/pattern as argument. Please try again.');
console.error('\nUsage: tstest <path> [options]');
console.error('\nOptions:');
console.error(' --quiet, -q Minimal output');
console.error(' --verbose, -v Verbose output');
console.error(' --no-color Disable colored output');
console.error(' --json Output results as JSON');
console.error(' --logfile Write logs to .nogit/testlogs/[testfile].log');
console.error(' --tags Run only tests with specified tags (comma-separated)');
console.error(' --quiet, -q Minimal output');
console.error(' --verbose, -v Verbose output');
console.error(' --no-color Disable colored output');
console.error(' --json Output results as JSON');
console.error(' --logfile Write logs to .nogit/testlogs/[testfile].log');
console.error(' --tags <tags> Run only tests with specified tags (comma-separated)');
console.error(' --startFrom <n> Start running from test file number n');
console.error(' --stopAt <n> Stop running at test file number n');
process.exit(1);
}
@ -73,6 +109,11 @@ export const runCli = async () => {
executionMode = TestExecutionMode.DIRECTORY;
}
const tsTestInstance = new TsTest(process.cwd(), testPath, executionMode, logOptions, tags);
const tsTestInstance = new TsTest(process.cwd(), testPath, executionMode, logOptions, tags, startFromFile, stopAtFile);
await tsTestInstance.run();
};
// Execute CLI when this file is run directly
if (import.meta.url === `file://${process.argv[1]}`) {
runCli();
}

View File

@ -10,6 +10,7 @@ import { TsTestLogger } from './tstest.logging.js';
export class TapCombinator {
tapParserStore: TapParser[] = [];
skippedFiles: string[] = [];
private logger: TsTestLogger;
constructor(logger: TsTestLogger) {
@ -19,10 +20,14 @@ export class TapCombinator {
addTapParser(tapParserArg: TapParser) {
this.tapParserStore.push(tapParserArg);
}
addSkippedFile(filename: string) {
this.skippedFiles.push(filename);
}
evaluate() {
// Call the logger's summary method
this.logger.summary();
// Call the logger's summary method with skipped files
this.logger.summary(this.skippedFiles);
// Check for failures
let failGlobal = false;

View File

@ -16,7 +16,7 @@ export class TapParser {
expectedTests: number;
receivedTests: number;
testStatusRegex = /(ok|not\sok)\s([0-9]+)\s-\s(.*)(\s#\s(.*))?$/;
testStatusRegex = /(ok|not\sok)\s([0-9]+)\s-\s(.*?)(\s#\s(.*))?$/;
activeTapTestResult: TapTestResult;
collectingErrorDetails: boolean = false;
currentTestError: string[] = [];
@ -77,7 +77,7 @@ export class TapParser {
return false;
})();
const testSubject = regexResult[3];
const testSubject = regexResult[3].trim();
const testMetadata = regexResult[5]; // This will be either "time=XXXms" or "SKIP reason" or "TODO reason"
let testDuration = 0;

View File

@ -16,6 +16,8 @@ export class TsTest {
public executionMode: TestExecutionMode;
public logger: TsTestLogger;
public filterTags: string[];
public startFromFile: number | null;
public stopAtFile: number | null;
public smartshellInstance = new plugins.smartshell.Smartshell({
executor: 'bash',
@ -26,18 +28,25 @@ export class TsTest {
public tsbundleInstance = new plugins.tsbundle.TsBundle();
constructor(cwdArg: string, testPathArg: string, executionModeArg: TestExecutionMode, logOptions: LogOptions = {}, tags: string[] = []) {
constructor(cwdArg: string, testPathArg: string, executionModeArg: TestExecutionMode, logOptions: LogOptions = {}, tags: string[] = [], startFromFile: number | null = null, stopAtFile: number | null = null) {
this.executionMode = executionModeArg;
this.testDir = new TestDirectory(cwdArg, testPathArg, executionModeArg);
this.logger = new TsTestLogger(logOptions);
this.filterTags = tags;
this.startFromFile = startFromFile;
this.stopAtFile = stopAtFile;
}
async run() {
// Move previous log files if --logfile option is used
if (this.logger.options.logFile) {
await this.movePreviousLogFiles();
}
const testGroups = await this.testDir.getTestFileGroups();
const allFiles = [...testGroups.serial, ...Object.values(testGroups.parallelGroups).flat()];
// Log test discovery
// Log test discovery - always show full count
this.logger.testDiscovery(
allFiles.length,
this.testDir.testPath,
@ -50,7 +59,7 @@ export class TsTest {
// Execute serial tests first
for (const fileNameArg of testGroups.serial) {
fileIndex++;
await this.runSingleTest(fileNameArg, fileIndex, allFiles.length, tapCombinator);
await this.runSingleTestOrSkip(fileNameArg, fileIndex, allFiles.length, tapCombinator);
}
// Execute parallel groups sequentially
@ -64,7 +73,7 @@ export class TsTest {
// Run all tests in this group in parallel
const parallelPromises = groupFiles.map(async (fileNameArg) => {
fileIndex++;
return this.runSingleTest(fileNameArg, fileIndex, allFiles.length, tapCombinator);
return this.runSingleTestOrSkip(fileNameArg, fileIndex, allFiles.length, tapCombinator);
});
await Promise.all(parallelPromises);
@ -75,6 +84,24 @@ export class TsTest {
tapCombinator.evaluate();
}
private async runSingleTestOrSkip(fileNameArg: string, fileIndex: number, totalFiles: number, tapCombinator: TapCombinator) {
// Check if this file should be skipped based on range
if (this.startFromFile !== null && fileIndex < this.startFromFile) {
this.logger.testFileSkipped(fileNameArg, fileIndex, totalFiles, `before start range (${this.startFromFile})`);
tapCombinator.addSkippedFile(fileNameArg);
return;
}
if (this.stopAtFile !== null && fileIndex > this.stopAtFile) {
this.logger.testFileSkipped(fileNameArg, fileIndex, totalFiles, `after stop range (${this.stopAtFile})`);
tapCombinator.addSkippedFile(fileNameArg);
return;
}
// File is in range, run it
await this.runSingleTest(fileNameArg, fileIndex, totalFiles, tapCombinator);
}
private async runSingleTest(fileNameArg: string, fileIndex: number, totalFiles: number, tapCombinator: TapCombinator) {
switch (true) {
case process.env.CI && fileNameArg.includes('.nonci.'):
@ -145,7 +172,7 @@ export class TsTest {
});
server.addRoute(
'/test',
new plugins.typedserver.servertools.Handler('GET', async (req, res) => {
new plugins.typedserver.servertools.Handler('GET', async (_req, res) => {
res.type('.html');
res.write(`
<html>
@ -180,7 +207,7 @@ export class TsTest {
// lets do the browser bit
await this.smartbrowserInstance.start();
const evaluation = await this.smartbrowserInstance.evaluateOnPage(
await this.smartbrowserInstance.evaluateOnPage(
`http://localhost:3007/test?bundleName=${bundleFileName}`,
async () => {
// lets enable real time comms
@ -193,12 +220,12 @@ export class TsTest {
const originalError = console.error;
// Override console methods to capture the logs
console.log = (...args) => {
console.log = (...args: any[]) => {
logStore.push(args.join(' '));
ws.send(args.join(' '));
originalLog(...args);
};
console.error = (...args) => {
console.error = (...args: any[]) => {
logStore.push(args.join(' '));
ws.send(args.join(' '));
originalError(...args);
@ -249,4 +276,40 @@ export class TsTest {
}
public async runInDeno() {}
private async movePreviousLogFiles() {
const logDir = plugins.path.join('.nogit', 'testlogs');
const previousDir = plugins.path.join('.nogit', 'testlogs', 'previous');
try {
// Get all files in log directory
const files = await plugins.smartfile.fs.listFileTree(logDir, '*.log');
if (files.length === 0) {
return;
}
// Ensure previous directory exists
await plugins.smartfile.fs.ensureDir(previousDir);
// Move each file to previous directory
for (const file of files) {
const filename = plugins.path.basename(file);
const sourcePath = plugins.path.join(logDir, filename);
const destPath = plugins.path.join(previousDir, filename);
try {
// Read file content and write to new location
const content = await plugins.smartfile.fs.toStringSync(sourcePath);
await plugins.smartfile.fs.toFs(content, destPath);
// Remove original file
await plugins.smartfile.fs.remove(sourcePath);
} catch (error) {
// Silently continue if a file can't be moved
}
}
} catch (error) {
// Directory might not exist, which is fine
return;
}
}
}

View File

@ -30,12 +30,14 @@ export interface TestSummary {
totalTests: number;
totalPassed: number;
totalFailed: number;
totalSkipped: number;
totalDuration: number;
fileResults: TestFileResult[];
skippedFiles: string[];
}
export class TsTestLogger {
private options: LogOptions;
public readonly options: LogOptions;
private startTime: number;
private fileResults: TestFileResult[] = [];
private currentFileResult: TestFileResult | null = null;
@ -153,8 +155,16 @@ export class TsTestLogger {
// Only set up test log file if --logfile option is specified
if (this.options.logFile) {
const baseFilename = path.basename(filename, '.ts');
this.currentTestLogFile = path.join('.nogit', 'testlogs', `${baseFilename}.log`);
// Create a safe filename that preserves directory structure
// Convert relative path to a flat filename by replacing separators with __
const relativeFilename = path.relative(process.cwd(), filename);
const safeFilename = relativeFilename
.replace(/\\/g, '/') // Normalize Windows paths
.replace(/\//g, '__') // Replace path separators with double underscores
.replace(/\.ts$/, '') // Remove .ts extension
.replace(/^\.\.__|^\.__|^__/, ''); // Clean up leading separators from relative paths
this.currentTestLogFile = path.join('.nogit', 'testlogs', `${safeFilename}.log`);
// Ensure the directory exists
const logDir = path.dirname(this.currentTestLogFile);
@ -237,6 +247,36 @@ export class TsTestLogger {
this.log(this.format(` Summary: ${passed}/${total} ${status}`, color));
}
// If using --logfile, handle error copy and diff detection
if (this.options.logFile && this.currentTestLogFile) {
try {
const logContent = fs.readFileSync(this.currentTestLogFile, 'utf-8');
const logDir = path.dirname(this.currentTestLogFile);
const logBasename = path.basename(this.currentTestLogFile);
// Create error copy if there were failures
if (failed > 0) {
const errorLogPath = path.join(logDir, `00err_${logBasename}`);
fs.writeFileSync(errorLogPath, logContent);
}
// Check for previous version and create diff if changed
const previousLogPath = path.join(logDir, 'previous', logBasename);
if (fs.existsSync(previousLogPath)) {
const previousContent = fs.readFileSync(previousLogPath, 'utf-8');
// Simple check if content differs
if (previousContent !== logContent) {
const diffLogPath = path.join(logDir, `00diff_${logBasename}`);
const diffContent = this.createDiff(previousContent, logContent, logBasename);
fs.writeFileSync(diffLogPath, diffContent);
}
}
} catch (error) {
// Silently fail to avoid disrupting the test run
}
}
// Clear the current test log file reference only if using --logfile
if (this.options.logFile) {
this.currentTestLogFile = null;
@ -244,7 +284,7 @@ export class TsTestLogger {
}
// TAP output forwarding (for TAP protocol messages)
tapOutput(message: string, isError: boolean = false) {
tapOutput(message: string, _isError: boolean = false) {
if (this.options.json) return;
// Never show raw TAP protocol messages in console
@ -274,6 +314,19 @@ export class TsTestLogger {
}
}
// Skipped test file
testFileSkipped(filename: string, index: number, total: number, reason: string) {
if (this.options.json) {
this.logJson({ event: 'fileSkipped', filename, index, total, reason });
return;
}
if (this.options.quiet) return;
this.log(this.format(`\n⏭ ${filename} (${index}/${total})`, 'yellow'));
this.log(this.format(` Skipped: ${reason}`, 'dim'));
}
// Browser console
browserConsole(message: string, level: string = 'log') {
if (this.options.json) {
@ -309,15 +362,17 @@ export class TsTestLogger {
}
// Final summary
summary() {
summary(skippedFiles: string[] = []) {
const totalDuration = Date.now() - this.startTime;
const summary: TestSummary = {
totalFiles: this.fileResults.length,
totalFiles: this.fileResults.length + skippedFiles.length,
totalTests: this.fileResults.reduce((sum, r) => sum + r.total, 0),
totalPassed: this.fileResults.reduce((sum, r) => sum + r.passed, 0),
totalFailed: this.fileResults.reduce((sum, r) => sum + r.failed, 0),
totalSkipped: skippedFiles.length,
totalDuration,
fileResults: this.fileResults
fileResults: this.fileResults,
skippedFiles
};
if (this.options.json) {
@ -338,6 +393,9 @@ export class TsTestLogger {
this.log(this.format(`│ Total Tests: ${summary.totalTests.toString().padStart(14)}`, 'white'));
this.log(this.format(`│ Passed: ${summary.totalPassed.toString().padStart(14)}`, 'green'));
this.log(this.format(`│ Failed: ${summary.totalFailed.toString().padStart(14)}`, summary.totalFailed > 0 ? 'red' : 'green'));
if (summary.totalSkipped > 0) {
this.log(this.format(`│ Skipped: ${summary.totalSkipped.toString().padStart(14)}`, 'yellow'));
}
this.log(this.format(`│ Duration: ${totalDuration.toString().padStart(14)}ms │`, 'white'));
this.log(this.format('└────────────────────────────────┘', 'dim'));
@ -396,4 +454,48 @@ export class TsTestLogger {
}
}
}
// Create a diff between two log contents
private createDiff(previousContent: string, currentContent: string, filename: string): string {
const previousLines = previousContent.split('\n');
const currentLines = currentContent.split('\n');
let diff = `DIFF REPORT: ${filename}\n`;
diff += `Generated: ${new Date().toISOString()}\n`;
diff += '='.repeat(80) + '\n\n';
// Simple line-by-line comparison
const maxLines = Math.max(previousLines.length, currentLines.length);
let hasChanges = false;
for (let i = 0; i < maxLines; i++) {
const prevLine = previousLines[i] || '';
const currLine = currentLines[i] || '';
if (prevLine !== currLine) {
hasChanges = true;
if (i < previousLines.length && i >= currentLines.length) {
// Line was removed
diff += `- [Line ${i + 1}] ${prevLine}\n`;
} else if (i >= previousLines.length && i < currentLines.length) {
// Line was added
diff += `+ [Line ${i + 1}] ${currLine}\n`;
} else {
// Line was modified
diff += `- [Line ${i + 1}] ${prevLine}\n`;
diff += `+ [Line ${i + 1}] ${currLine}\n`;
}
}
}
if (!hasChanges) {
diff += 'No changes detected.\n';
}
diff += '\n' + '='.repeat(80) + '\n';
diff += `Previous version had ${previousLines.length} lines\n`;
diff += `Current version has ${currentLines.length} lines\n`;
return diff;
}
}

View File

@ -1,6 +1,9 @@
export { tap } from './tapbundle.classes.tap.js';
export { TapWrap } from './tapbundle.classes.tapwrap.js';
export { webhelpers } from './webhelpers.js';
// Protocol utilities (for future protocol v2)
export * from './tapbundle.protocols.js';
export { TapTools } from './tapbundle.classes.taptools.js';
import { expect } from '@push.rocks/smartexpect';

View File

@ -0,0 +1,226 @@
/**
* Internal protocol constants and utilities for improved TAP communication
* between tapbundle and tstest
*/
export const PROTOCOL = {
VERSION: '2.0',
MARKERS: {
START: '⟦TSTEST:',
END: '⟧',
BLOCK_END: '⟦/TSTEST:',
},
TYPES: {
META: 'META',
ERROR: 'ERROR',
SKIP: 'SKIP',
TODO: 'TODO',
SNAPSHOT: 'SNAPSHOT',
PROTOCOL: 'PROTOCOL',
}
} as const;
export interface TestMetadata {
// Timing
time?: number; // milliseconds
startTime?: number; // Unix timestamp
endTime?: number; // Unix timestamp
// Status
skip?: string; // skip reason
todo?: string; // todo reason
retry?: number; // retry attempt
maxRetries?: number; // max retries allowed
// Error details
error?: {
message: string;
stack?: string;
diff?: string;
actual?: any;
expected?: any;
};
// Test context
file?: string; // source file
line?: number; // line number
column?: number; // column number
// Custom data
tags?: string[]; // test tags
custom?: Record<string, any>;
}
export class ProtocolEncoder {
/**
* Encode metadata for inline inclusion
*/
static encodeInline(type: string, data: any): string {
if (typeof data === 'string') {
return `${PROTOCOL.MARKERS.START}${type}:${data}${PROTOCOL.MARKERS.END}`;
}
return `${PROTOCOL.MARKERS.START}${type}:${JSON.stringify(data)}${PROTOCOL.MARKERS.END}`;
}
/**
* Encode block data for multi-line content
*/
static encodeBlock(type: string, data: any): string[] {
const lines: string[] = [];
lines.push(`${PROTOCOL.MARKERS.START}${type}${PROTOCOL.MARKERS.END}`);
if (typeof data === 'string') {
lines.push(data);
} else {
lines.push(JSON.stringify(data, null, 2));
}
lines.push(`${PROTOCOL.MARKERS.BLOCK_END}${type}${PROTOCOL.MARKERS.END}`);
return lines;
}
/**
* Create a TAP line with metadata
*/
static createTestLine(
status: 'ok' | 'not ok',
number: number,
description: string,
metadata?: TestMetadata
): string {
let line = `${status} ${number} - ${description}`;
if (metadata) {
// For skip/todo, use inline format for compatibility
if (metadata.skip) {
line += ` ${this.encodeInline(PROTOCOL.TYPES.SKIP, metadata.skip)}`;
} else if (metadata.todo) {
line += ` ${this.encodeInline(PROTOCOL.TYPES.TODO, metadata.todo)}`;
} else {
// For other metadata, append inline
const metaCopy = { ...metadata };
delete metaCopy.error; // Error details go in separate block
if (Object.keys(metaCopy).length > 0) {
line += ` ${this.encodeInline(PROTOCOL.TYPES.META, metaCopy)}`;
}
}
}
return line;
}
}
export class ProtocolDecoder {
/**
* Extract all protocol markers from a line
*/
static extractMarkers(line: string): Array<{type: string, data: any, start: number, end: number}> {
const markers: Array<{type: string, data: any, start: number, end: number}> = [];
let searchFrom = 0;
while (true) {
const start = line.indexOf(PROTOCOL.MARKERS.START, searchFrom);
if (start === -1) break;
const end = line.indexOf(PROTOCOL.MARKERS.END, start);
if (end === -1) break;
const content = line.substring(start + PROTOCOL.MARKERS.START.length, end);
const colonIndex = content.indexOf(':');
if (colonIndex !== -1) {
const type = content.substring(0, colonIndex);
const dataStr = content.substring(colonIndex + 1);
let data: any;
try {
// Try to parse as JSON first
data = JSON.parse(dataStr);
} catch {
// If not JSON, treat as string
data = dataStr;
}
markers.push({
type,
data,
start,
end: end + PROTOCOL.MARKERS.END.length
});
}
searchFrom = end + 1;
}
return markers;
}
/**
* Remove protocol markers from a line
*/
static cleanLine(line: string): string {
const markers = this.extractMarkers(line);
// Remove markers from end to start to preserve indices
let cleanedLine = line;
for (let i = markers.length - 1; i >= 0; i--) {
const marker = markers[i];
cleanedLine = cleanedLine.substring(0, marker.start) +
cleanedLine.substring(marker.end);
}
return cleanedLine.trim();
}
/**
* Parse a test line and extract metadata
*/
static parseTestLine(line: string): {
cleaned: string;
metadata: TestMetadata;
} {
const markers = this.extractMarkers(line);
const metadata: TestMetadata = {};
for (const marker of markers) {
switch (marker.type) {
case PROTOCOL.TYPES.META:
Object.assign(metadata, marker.data);
break;
case PROTOCOL.TYPES.SKIP:
metadata.skip = marker.data;
break;
case PROTOCOL.TYPES.TODO:
metadata.todo = marker.data;
break;
}
}
return {
cleaned: this.cleanLine(line),
metadata
};
}
/**
* Check if a line starts a protocol block
*/
static isBlockStart(line: string): {isBlock: boolean, type?: string} {
const trimmed = line.trim();
if (trimmed.startsWith(PROTOCOL.MARKERS.START) && trimmed.endsWith(PROTOCOL.MARKERS.END)) {
const content = trimmed.slice(PROTOCOL.MARKERS.START.length, -PROTOCOL.MARKERS.END.length);
if (!content.includes(':')) {
return { isBlock: true, type: content };
}
}
return { isBlock: false };
}
/**
* Check if a line ends a protocol block
*/
static isBlockEnd(line: string, type: string): boolean {
return line.trim() === `${PROTOCOL.MARKERS.BLOCK_END}${type}${PROTOCOL.MARKERS.END}`;
}
}