feat(logging): Display failed test console logs in default mode
This commit is contained in:
parent
553d5f0df7
commit
42cd08eb1c
@ -1,5 +1,12 @@
|
||||
# Changelog
|
||||
|
||||
## 2025-05-15 - 1.4.0 - feat(logging)
|
||||
Display failed test console logs in default mode
|
||||
|
||||
- Introduce log buffering in TsTestLogger to capture console output for failed tests
|
||||
- Enhance TapParser to collect and display error details when tests fail
|
||||
- Update README and project plan to document log improvements for debugging
|
||||
|
||||
## 2025-05-15 - 1.3.1 - fix(settings)
|
||||
Add local permissions configuration and remove obsolete test output log
|
||||
|
||||
|
||||
209
readme.md
209
readme.md
@ -1,61 +1,204 @@
|
||||
# @gitzone/tstest
|
||||
a test utility to run tests that match test/**/*.ts
|
||||
🧪 **A powerful, modern test runner for TypeScript** - making your test runs beautiful and informative!
|
||||
|
||||
## Availabililty and Links
|
||||
* [npmjs.org (npm package)](https://www.npmjs.com/package/@gitzone/tstest)
|
||||
* [gitlab.com (source)](https://gitlab.com/gitzone/tstest)
|
||||
* [github.com (source mirror)](https://github.com/gitzone/tstest)
|
||||
* [docs (typedoc)](https://gitzone.gitlab.io/tstest/)
|
||||
* [code.foss.global (source)](https://code.foss.global/gitzone/tstest)
|
||||
|
||||
## Status for master
|
||||
## Why tstest?
|
||||
|
||||
Status Category | Status Badge
|
||||
-- | --
|
||||
GitLab Pipelines | [](https://lossless.cloud)
|
||||
GitLab Pipline Test Coverage | [](https://lossless.cloud)
|
||||
npm | [](https://lossless.cloud)
|
||||
Snyk | [](https://lossless.cloud)
|
||||
TypeScript Support | [](https://lossless.cloud)
|
||||
node Support | [](https://nodejs.org/dist/latest-v10.x/docs/api/)
|
||||
Code Style | [](https://lossless.cloud)
|
||||
PackagePhobia (total standalone install weight) | [](https://lossless.cloud)
|
||||
PackagePhobia (package size on registry) | [](https://lossless.cloud)
|
||||
BundlePhobia (total size when bundled) | [](https://lossless.cloud)
|
||||
Platform support | [](https://lossless.cloud) [](https://lossless.cloud)
|
||||
**tstest** is a TypeScript test runner that makes testing delightful. It's designed for modern development workflows with beautiful output, flexible test execution, and powerful features that make debugging a breeze.
|
||||
|
||||
### ✨ Key Features
|
||||
|
||||
- 🎯 **Smart Test Execution** - Run all tests, single files, or use glob patterns
|
||||
- 🎨 **Beautiful Output** - Color-coded results with emojis and clean formatting
|
||||
- 📊 **Multiple Output Modes** - Choose from normal, quiet, verbose, or JSON output
|
||||
- 🔍 **Automatic Discovery** - Finds all your test files automatically
|
||||
- 🌐 **Cross-Environment** - Supports Node.js and browser testing
|
||||
- 📝 **Detailed Logging** - Optional file logging for debugging
|
||||
- ⚡ **Performance Metrics** - See which tests are slow
|
||||
- 🤖 **CI/CD Ready** - JSON output mode for automation
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
npm install --save-dev @gitzone/tstest
|
||||
# or with pnpm
|
||||
pnpm add -D @gitzone/tstest
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
## cli usage
|
||||
### Basic Test Execution
|
||||
|
||||
lets assume we have a directory called test/ where all our tests arae defined. Simply type
|
||||
|
||||
```
|
||||
```bash
|
||||
# Run all tests in a directory
|
||||
tstest test/
|
||||
|
||||
# Run a specific test file
|
||||
tstest test/test.mycomponent.ts
|
||||
|
||||
# Use glob patterns
|
||||
tstest "test/**/*.spec.ts"
|
||||
tstest "test/unit/*.ts"
|
||||
```
|
||||
|
||||
to run all tests.
|
||||
### Execution Modes
|
||||
|
||||
## Syntax
|
||||
**tstest** intelligently detects how you want to run your tests:
|
||||
|
||||
tstest supports tap syntax. In other words your testfiles are run in a subprocess, and the console output contains trigger messages for tstest to determine test status. Inside your testfile you should use `@pushrocks/tapbundle` for the best results.
|
||||
1. **Directory mode** - Recursively finds all test files
|
||||
2. **File mode** - Runs a single test file
|
||||
3. **Glob mode** - Uses pattern matching for flexible test selection
|
||||
|
||||
## Environments
|
||||
### Command Line Options
|
||||
|
||||
tstest supports different environments:
|
||||
| Option | Description |
|
||||
|--------|-------------|
|
||||
| `--quiet`, `-q` | Minimal output - perfect for CI environments |
|
||||
| `--verbose`, `-v` | Show all console output from tests |
|
||||
| `--no-color` | Disable colored output |
|
||||
| `--json` | Output results as JSON |
|
||||
| `--logfile` | Save detailed logs to `.nogit/testlogs/[testname].log` |
|
||||
|
||||
- a testfile called `test-something.node.ts` will be run in node
|
||||
- a testfile called `test-something.chrome.ts` will be run in chrome environment (bundled through parcel and run through puppeteer)
|
||||
- a testfile called `test-something.both.ts` will be run in node an chrome, which is good for isomorphic packages.
|
||||
### Example Outputs
|
||||
|
||||
> note: there is alpha support for the deno environment by naming a file test-something.deno.ts
|
||||
#### Normal Output (Default)
|
||||
```
|
||||
🔍 Test Discovery
|
||||
Mode: directory
|
||||
Pattern: test
|
||||
Found: 4 test file(s)
|
||||
|
||||
▶️ test/test.ts (1/4)
|
||||
Runtime: node.js
|
||||
✅ prepare test (1ms)
|
||||
Summary: 1/1 PASSED
|
||||
|
||||
📊 Test Summary
|
||||
┌────────────────────────────────┐
|
||||
│ Total Files: 4 │
|
||||
│ Total Tests: 4 │
|
||||
│ Passed: 4 │
|
||||
│ Failed: 0 │
|
||||
│ Duration: 542ms │
|
||||
└────────────────────────────────┘
|
||||
|
||||
ALL TESTS PASSED! 🎉
|
||||
```
|
||||
|
||||
#### Quiet Mode
|
||||
```
|
||||
Found 4 tests
|
||||
✅ test functionality works
|
||||
✅ api calls return expected data
|
||||
✅ error handling works correctly
|
||||
✅ performance is within limits
|
||||
|
||||
Summary: 4/4 | 542ms | PASSED
|
||||
```
|
||||
|
||||
#### Verbose Mode
|
||||
Shows all console output from your tests, making debugging easier:
|
||||
```
|
||||
▶️ test/api.test.ts (1/1)
|
||||
Runtime: node.js
|
||||
Making API call to /users...
|
||||
Response received: 200 OK
|
||||
Processing user data...
|
||||
✅ api calls return expected data (145ms)
|
||||
Summary: 1/1 PASSED
|
||||
```
|
||||
|
||||
#### JSON Mode
|
||||
Perfect for CI/CD pipelines:
|
||||
```json
|
||||
{"event":"discovery","count":4,"pattern":"test","executionMode":"directory"}
|
||||
{"event":"fileStart","filename":"test/test.ts","runtime":"node.js","index":1,"total":4}
|
||||
{"event":"testResult","testName":"prepare test","passed":true,"duration":1}
|
||||
{"event":"summary","summary":{"totalFiles":4,"totalTests":4,"totalPassed":4,"totalFailed":0,"totalDuration":542}}
|
||||
```
|
||||
|
||||
## Test File Naming Conventions
|
||||
|
||||
tstest supports different test environments through file naming:
|
||||
|
||||
| Pattern | Environment | Example |
|
||||
|---------|-------------|---------|
|
||||
| `*.ts` | Node.js (default) | `test.basic.ts` |
|
||||
| `*.node.ts` | Node.js only | `test.api.node.ts` |
|
||||
| `*.chrome.ts` | Chrome browser | `test.dom.chrome.ts` |
|
||||
| `*.browser.ts` | Browser environment | `test.ui.browser.ts` |
|
||||
| `*.both.ts` | Both Node.js and browser | `test.isomorphic.both.ts` |
|
||||
|
||||
### Writing Tests
|
||||
|
||||
tstest uses TAP (Test Anything Protocol) for test output. Use `@pushrocks/tapbundle` for the best experience:
|
||||
|
||||
```typescript
|
||||
import { expect, tap } from '@push.rocks/tapbundle';
|
||||
|
||||
tap.test('my awesome test', async () => {
|
||||
const result = await myFunction();
|
||||
expect(result).toEqual('expected value');
|
||||
});
|
||||
|
||||
tap.start();
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Glob Pattern Support
|
||||
|
||||
Run specific test patterns:
|
||||
```bash
|
||||
# Run all unit tests
|
||||
tstest "test/unit/**/*.ts"
|
||||
|
||||
# Run all integration tests
|
||||
tstest "test/integration/*.test.ts"
|
||||
|
||||
# Run multiple patterns
|
||||
tstest "test/**/*.spec.ts" "test/**/*.test.ts"
|
||||
```
|
||||
|
||||
### Automatic Logging
|
||||
|
||||
Use `--logfile` to automatically save test output:
|
||||
```bash
|
||||
tstest test/ --logfile
|
||||
```
|
||||
|
||||
This creates detailed logs in `.nogit/testlogs/[testname].log` for each test file.
|
||||
|
||||
### Performance Analysis
|
||||
|
||||
In verbose mode, see performance metrics:
|
||||
```
|
||||
⏱️ Performance Metrics:
|
||||
Average per test: 135ms
|
||||
Slowest test: api integration test (486ms)
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
For continuous integration, combine quiet and JSON modes:
|
||||
```bash
|
||||
# GitHub Actions example
|
||||
tstest test/ --json > test-results.json
|
||||
|
||||
# Or minimal output
|
||||
tstest test/ --quiet
|
||||
```
|
||||
|
||||
## Contribution
|
||||
|
||||
We are always happy for code contributions. If you are not the code contributing type that is ok. Still, maintaining Open Source repositories takes considerable time and thought. If you like the quality of what we do and our modules are useful to you we would appreciate a little monthly contribution: You can [contribute one time](https://lossless.link/contribute-onetime) or [contribute monthly](https://lossless.link/contribute). :)
|
||||
|
||||
For further information read the linked docs at the top of this readme.
|
||||
## License
|
||||
|
||||
> MIT licensed | **©** [Lossless GmbH](https://lossless.gmbh)
|
||||
| By using this npm module you agree to our [privacy policy](https://lossless.gmbH/privacy)
|
||||
|
||||
[](https://maintainedby.lossless.com)
|
||||
[](https://maintainedby.lossless.com)
|
||||
41
readme.plan.md
Normal file
41
readme.plan.md
Normal file
@ -0,0 +1,41 @@
|
||||
# Plan for showing logs for failed tests
|
||||
|
||||
!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
|
||||
|
||||
## Goal
|
||||
When a test fails, we want to display all the console logs from that failed test in the terminal, even without the --verbose flag. This makes debugging failed tests much easier.
|
||||
|
||||
## Current Behavior
|
||||
- Default mode: Only shows test results, no console logs
|
||||
- Verbose mode: Shows all console logs from all tests
|
||||
- When a test fails: Only shows the error message
|
||||
|
||||
## Desired Behavior
|
||||
- Default mode: Shows test results, and IF a test fails, shows all console logs from that failed test
|
||||
- Verbose mode: Shows all console logs from all tests (unchanged)
|
||||
- When a test fails: Shows all console logs from that test plus the error
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### 1. Update TapParser
|
||||
- Store console logs for each test temporarily
|
||||
- When a test fails, mark that its logs should be shown
|
||||
|
||||
### 2. Update TsTestLogger
|
||||
- Add a new method to handle failed test logs
|
||||
- Modify testConsoleOutput to buffer logs when not in verbose mode
|
||||
- When a test fails, flush the buffered logs for that test
|
||||
|
||||
### 3. Update test result handling
|
||||
- When a test fails, trigger display of all buffered logs for that test
|
||||
- Clear logs after each test completes successfully
|
||||
|
||||
## Code Changes Needed
|
||||
1. Add log buffering to TapParser
|
||||
2. Update TsTestLogger to handle failed test logs
|
||||
3. Modify test result processing to show logs on failure
|
||||
|
||||
## Files to Modify
|
||||
- `ts/tstest.classes.tap.parser.ts` - Add log buffering
|
||||
- `ts/tstest.logging.ts` - Add failed test log handling
|
||||
- `ts/tstest.classes.tap.testresult.ts` - May need to store logs
|
||||
23
test/test.failing-with-logs.ts
Normal file
23
test/test.failing-with-logs.ts
Normal file
@ -0,0 +1,23 @@
|
||||
import { expect, tap } from '@push.rocks/tapbundle';
|
||||
|
||||
tap.test('Test that will fail with console logs', async () => {
|
||||
console.log('Starting the test...');
|
||||
console.log('Doing some setup work');
|
||||
console.log('About to check assertion');
|
||||
|
||||
const value = 42;
|
||||
console.log(`The value is: ${value}`);
|
||||
|
||||
// This will fail
|
||||
expect(value).toEqual(100);
|
||||
|
||||
console.log('This log will not be reached');
|
||||
});
|
||||
|
||||
tap.test('Test that passes', async () => {
|
||||
console.log('This test passes');
|
||||
console.log('So these logs should not show in default mode');
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
tap.start();
|
||||
@ -3,6 +3,6 @@
|
||||
*/
|
||||
export const commitinfo = {
|
||||
name: '@git.zone/tstest',
|
||||
version: '1.3.1',
|
||||
version: '1.4.0',
|
||||
description: 'a test utility to run tests that match test/**/*.ts'
|
||||
}
|
||||
|
||||
@ -18,6 +18,8 @@ export class TapParser {
|
||||
|
||||
testStatusRegex = /(ok|not\sok)\s([0-9]+)\s-\s(.*)\s#\stime=(.*)ms$/;
|
||||
activeTapTestResult: TapTestResult;
|
||||
collectingErrorDetails: boolean = false;
|
||||
currentTestError: string[] = [];
|
||||
|
||||
pretaskRegex = /^::__PRETASK:(.*)$/;
|
||||
|
||||
@ -91,6 +93,9 @@ export class TapParser {
|
||||
this.logger.testResult(testSubject, true, testDuration);
|
||||
}
|
||||
} else {
|
||||
// Start collecting error details for failed test
|
||||
this.collectingErrorDetails = true;
|
||||
this.currentTestError = [];
|
||||
if (this.logger) {
|
||||
this.logger.testResult(testSubject, false, testDuration);
|
||||
}
|
||||
@ -101,13 +106,43 @@ export class TapParser {
|
||||
if (this.activeTapTestResult) {
|
||||
this.activeTapTestResult.addLogLine(logLine);
|
||||
}
|
||||
if (this.logger) {
|
||||
// This is console output from the test file, not TAP protocol
|
||||
this.logger.testConsoleOutput(logLine);
|
||||
|
||||
// Check if we're collecting error details
|
||||
if (this.collectingErrorDetails) {
|
||||
// Check if this line is an error detail (starts with Error: or has stack trace characteristics)
|
||||
if (logLine.trim().startsWith('Error:') || logLine.trim().match(/^\s*at\s/)) {
|
||||
this.currentTestError.push(logLine);
|
||||
} else if (this.currentTestError.length > 0) {
|
||||
// End of error details, show the error
|
||||
const errorMessage = this.currentTestError.join('\n');
|
||||
if (this.logger) {
|
||||
this.logger.testErrorDetails(errorMessage);
|
||||
}
|
||||
this.collectingErrorDetails = false;
|
||||
this.currentTestError = [];
|
||||
}
|
||||
}
|
||||
|
||||
// Don't output TAP error details as console output when we're collecting them
|
||||
if (!this.collectingErrorDetails || (!logLine.trim().startsWith('Error:') && !logLine.trim().match(/^\s*at\s/))) {
|
||||
if (this.logger) {
|
||||
// This is console output from the test file, not TAP protocol
|
||||
this.logger.testConsoleOutput(logLine);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (this.activeTapTestResult && this.activeTapTestResult.testSettled) {
|
||||
// Ensure any pending error is shown before settling the test
|
||||
if (this.collectingErrorDetails && this.currentTestError.length > 0) {
|
||||
const errorMessage = this.currentTestError.join('\n');
|
||||
if (this.logger) {
|
||||
this.logger.testErrorDetails(errorMessage);
|
||||
}
|
||||
this.collectingErrorDetails = false;
|
||||
this.currentTestError = [];
|
||||
}
|
||||
|
||||
this.testStore.push(this.activeTapTestResult);
|
||||
this._getNewTapTestResult();
|
||||
}
|
||||
|
||||
@ -40,6 +40,8 @@ export class TsTestLogger {
|
||||
private fileResults: TestFileResult[] = [];
|
||||
private currentFileResult: TestFileResult | null = null;
|
||||
private currentTestLogFile: string | null = null;
|
||||
private currentTestLogs: string[] = []; // Buffer for current test logs
|
||||
private currentTestFailed: boolean = false;
|
||||
|
||||
constructor(options: LogOptions = {}) {
|
||||
this.options = options;
|
||||
@ -145,6 +147,10 @@ export class TsTestLogger {
|
||||
tests: []
|
||||
};
|
||||
|
||||
// Reset test-specific state
|
||||
this.currentTestLogs = [];
|
||||
this.currentTestFailed = false;
|
||||
|
||||
// Only set up test log file if --logfile option is specified
|
||||
if (this.options.logFile) {
|
||||
const baseFilename = path.basename(filename, '.ts');
|
||||
@ -179,6 +185,7 @@ export class TsTestLogger {
|
||||
this.currentFileResult.passed++;
|
||||
} else {
|
||||
this.currentFileResult.failed++;
|
||||
this.currentTestFailed = true;
|
||||
}
|
||||
this.currentFileResult.duration += duration;
|
||||
}
|
||||
@ -188,6 +195,14 @@ export class TsTestLogger {
|
||||
return;
|
||||
}
|
||||
|
||||
// If test failed and we have buffered logs, show them now
|
||||
if (!passed && this.currentTestLogs.length > 0 && !this.options.verbose) {
|
||||
this.log(this.format(' 📋 Console output from failed test:', 'yellow'));
|
||||
this.currentTestLogs.forEach(logMessage => {
|
||||
this.log(this.format(` ${logMessage}`, 'dim'));
|
||||
});
|
||||
}
|
||||
|
||||
const icon = passed ? '✅' : '❌';
|
||||
const color = passed ? 'green' : 'red';
|
||||
|
||||
@ -199,6 +214,9 @@ export class TsTestLogger {
|
||||
this.log(this.format(` ${error}`, 'red'));
|
||||
}
|
||||
}
|
||||
|
||||
// Clear logs after each test
|
||||
this.currentTestLogs = [];
|
||||
}
|
||||
|
||||
testFileEnd(passed: number, failed: number, duration: number) {
|
||||
@ -242,9 +260,12 @@ export class TsTestLogger {
|
||||
testConsoleOutput(message: string) {
|
||||
if (this.options.json) return;
|
||||
|
||||
// Show console output from test files only in verbose mode
|
||||
// In verbose mode, show console output immediately
|
||||
if (this.options.verbose) {
|
||||
this.log(this.format(` ${message}`, 'dim'));
|
||||
} else {
|
||||
// In non-verbose mode, buffer the logs
|
||||
this.currentTestLogs.push(message);
|
||||
}
|
||||
|
||||
// Always log to test file if --logfile is specified
|
||||
@ -267,6 +288,26 @@ export class TsTestLogger {
|
||||
}
|
||||
}
|
||||
|
||||
// Test error details display
|
||||
testErrorDetails(errorMessage: string) {
|
||||
if (this.options.json) {
|
||||
this.logJson({ event: 'testError', error: errorMessage });
|
||||
return;
|
||||
}
|
||||
|
||||
if (!this.options.quiet) {
|
||||
this.log(this.format(' Error details:', 'red'));
|
||||
errorMessage.split('\n').forEach(line => {
|
||||
this.log(this.format(` ${line}`, 'red'));
|
||||
});
|
||||
}
|
||||
|
||||
// Always log to test file if --logfile is specified
|
||||
if (this.currentTestLogFile) {
|
||||
this.logToTestFile(` Error: ${errorMessage}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Final summary
|
||||
summary() {
|
||||
const totalDuration = Date.now() - this.startTime;
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user