feat(tapbundle_serverside): add network port discovery utilities and migrate file I/O to smartfs; refactor runtimes to use Node fs and SmartFs, update server APIs and bump dependencies

This commit is contained in:
2026-03-03 20:15:59 +00:00
parent 4d1896bdf9
commit f23c902658
24 changed files with 2562 additions and 2094 deletions

View File

@@ -11,31 +11,31 @@ pnpm install --save-dev @git.zone/tstest
## Issue Reporting and Security
For reporting bugs, issues, or security vulnerabilities, please visit https://community.foss.global/. This is the central community hub for all issue reporting. Developers who want to sign a contribution agreement and go through identification can also get a code.foss.global account to submit Pull Requests directly.
For reporting bugs, issues, or security vulnerabilities, please visit [community.foss.global/](https://community.foss.global/). This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull Requests directly.
## Overview
`@git.zone/tstest/tapbundle_serverside` provides server-side testing utilities exclusively for Node.js runtime. These tools enable shell command execution, environment variable management, HTTPS certificate generation, database testing, object storage testing, and test asset management - all functionality that only makes sense on the server-side.
`@git.zone/tstest/tapbundle_serverside` provides server-side testing utilities exclusively for Node.js runtime. These tools make it trivial to spin up test infrastructure — free ports, HTTPS certificates, ephemeral databases, S3 storage, shell commands, and environment variable management.
## Key Features
- 🔐 **Environment Variables** - On-demand environment variable loading with qenv
- 💻 **Shell Commands** - Execute bash commands during tests
- 🔒 **HTTPS Certificates** - Generate self-signed certificates for testing
- 🗄️ **MongoDB Testing** - Create ephemeral MongoDB instances
- 📦 **S3 Storage Testing** - Create local S3-compatible storage for tests
- 📁 **Test File Management** - Download and manage test assets
- 🌐 **Network Utilities** — Find free ports and port ranges for test servers
- 🔒 **HTTPS Certificates** — Generate self-signed certificates for testing
- 💻 **Shell Commands** — Execute bash commands during tests
- 🔐 **Environment Variables** — On-demand environment variable loading with qenv
- 🗄️ **MongoDB Testing** Create ephemeral MongoDB instances
- 📦 **S3 Storage Testing** — Create local S3-compatible storage
- 📁 **Test File Management** — Download and manage test assets
## Basic Usage
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
import { tap } from '@git.zone/tstest/tapbundle';
import { tap, expect } from '@git.zone/tstest/tapbundle';
tap.test('should use server-side tools', async () => {
// Execute shell commands on the server-side
const result = await tapNodeTools.runCommand('echo "hello"');
console.log(result);
tap.test('should start server on free port', async () => {
const port = await tapNodeTools.findFreePort();
// start your server on `port`
});
export default tap.start();
@@ -47,340 +47,224 @@ export default tap.start();
The main singleton instance providing all Node.js-specific utilities.
#### Environment Variables
---
##### `getQenv()`
### 🌐 Network Utilities
#### `findFreePort(options?)`
Find a single free port on the local machine.
```typescript
// Default: random free port in range 300060000
const port = await tapNodeTools.findFreePort();
// Custom range
const port = await tapNodeTools.findFreePort({
startPort: 8000,
endPort: 9000,
});
// With exclusions and sequential scan
const port = await tapNodeTools.findFreePort({
startPort: 3000,
endPort: 60000,
randomize: false, // default: true
exclude: [8080, 8443],
});
```
**Options:**
| Option | Type | Default | Description |
|---|---|---|---|
| `startPort` | `number` | `3000` | Start of port range (inclusive) |
| `endPort` | `number` | `60000` | End of port range (inclusive) |
| `randomize` | `boolean` | `true` | Pick a random port vs first available |
| `exclude` | `number[]` | `[]` | Ports to skip |
**Returns:** `Promise<number>` — Throws if no free port is found.
#### `findFreePorts(count, options?)`
Find multiple distinct free ports. Each found port is automatically excluded from subsequent searches, guaranteeing all returned ports are unique.
```typescript
const [httpPort, wsPort, adminPort] = await tapNodeTools.findFreePorts(3);
// With custom range
const ports = await tapNodeTools.findFreePorts(2, {
startPort: 10000,
endPort: 20000,
});
```
**Parameters:**
- `count` — Number of ports to find
- `options` — Same as `findFreePort()`
**Returns:** `Promise<number[]>` — Array of distinct free ports.
#### `findFreePortRange(count, options?)`
Find consecutive free ports (e.g., `[4000, 4001, 4002]`). Useful when you need a contiguous block.
```typescript
const ports = await tapNodeTools.findFreePortRange(3, {
startPort: 20000,
endPort: 30000,
});
// ports = [N, N+1, N+2] where all three are free
```
**Options:**
| Option | Type | Default | Description |
|---|---|---|---|
| `startPort` | `number` | `3000` | Start of search range |
| `endPort` | `number` | `60000` | End of search range |
| `exclude` | `number[]` | `[]` | Ports to skip |
**Returns:** `Promise<number[]>` — Array of consecutive free ports.
---
### 🔒 HTTPS Certificates
#### `createHttpsCert(commonName?, allowSelfSigned?)`
Generate a self-signed HTTPS certificate for testing secure connections.
```typescript
const { key, cert } = await tapNodeTools.createHttpsCert('localhost', true);
// Use with Node.js https module
const server = https.createServer({ key, cert }, handler);
server.listen(port);
```
**Parameters:**
- `commonName` (default: `'localhost'`) — Certificate common name
- `allowSelfSigned` (default: `true`) — Sets `NODE_TLS_REJECT_UNAUTHORIZED=0`
**Returns:** `Promise<{ key: string; cert: string }>` — PEM-encoded key and certificate.
---
### 💻 Shell Commands
#### `runCommand(command)`
Execute a bash command and return the result.
```typescript
const result = await tapNodeTools.runCommand('ls -la');
console.log(result.exitCode);
console.log(result.stdout);
```
---
### 🔐 Environment Variables
#### `getQenv()`
Get the qenv instance for managing environment variables from `.nogit/` directory.
```typescript
const qenv = await tapNodeTools.getQenv();
// qenv will load from .env files in .nogit/ directory
```
##### `getEnvVarOnDemand(envVarName)`
#### `getEnvVarOnDemand(envVarName)`
Request an environment variable. If not available, qenv will prompt for it and store it securely.
```typescript
tap.test('should get API key', async () => {
const apiKey = await tapNodeTools.getEnvVarOnDemand('GITHUB_API_KEY');
// If GITHUB_API_KEY is not set, qenv will prompt for it
// The value is stored in .nogit/.env for future use
});
const apiKey = await tapNodeTools.getEnvVarOnDemand('GITHUB_API_KEY');
// If not set, prompts interactively and stores in .nogit/.env
```
**Use Cases:**
- API keys for integration tests
- Database credentials
- Service endpoints
- Any sensitive configuration needed for testing
---
#### Shell Commands
### 🗄️ Database Testing
##### `runCommand(command)`
#### `createSmartmongo()`
Execute a bash command and return the result.
Create an ephemeral MongoDB instance. Automatically started and ready to use.
```typescript
tap.test('should execute shell commands', async () => {
const result = await tapNodeTools.runCommand('ls -la');
console.log(result.stdout);
});
const mongo = await tapNodeTools.createSmartmongo();
// ... run database tests ...
await mongo.stop();
```
**Use Cases:**
- Setup test environment
- Execute CLI tools
- File system operations
- Process management
Uses [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo).
#### HTTPS Certificates
---
##### `createHttpsCert(commonName?, allowSelfSigned?)`
### 📦 Storage Testing
Generate a self-signed HTTPS certificate for testing secure connections.
#### `createSmarts3()`
Create a local S3-compatible storage instance for testing.
```typescript
tap.test('should create HTTPS server', async () => {
const { key, cert } = await tapNodeTools.createHttpsCert('localhost', true);
// Use with Node.js https module
const server = https.createServer({ key, cert }, (req, res) => {
res.end('Hello Secure World');
});
server.listen(3000);
});
const s3 = await tapNodeTools.createSmarts3();
// ... run storage tests ...
await s3.stop();
```
**Parameters:**
- `commonName` (optional): Certificate common name, default: 'localhost'
- `allowSelfSigned` (optional): Allow self-signed certificates by setting `NODE_TLS_REJECT_UNAUTHORIZED=0`, default: true
Default config: port 3003, clean slate enabled. Uses [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3).
**Returns:**
- `key`: PEM-encoded private key
- `cert`: PEM-encoded certificate
---
**Use Cases:**
- Testing HTTPS servers
- Testing secure WebSocket connections
- Testing certificate validation logic
- Mocking secure external services
### 📁 Test File Provider
#### Database Testing
##### `createSmartmongo()`
Create an ephemeral MongoDB instance for testing. Automatically started and ready to use.
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
tap.test('should use MongoDB', async () => {
const mongoInstance = await tapNodeTools.createSmartmongo();
// Use the MongoDB instance
const db = await mongoInstance.getDatabase('testdb');
const collection = await db.getCollection('users');
await collection.insertOne({ name: 'Alice', age: 30 });
const user = await collection.findOne({ name: 'Alice' });
expect(user.age).toEqual(30);
// Cleanup (optional - instance will be cleaned up automatically)
await mongoInstance.stop();
});
export default tap.start();
```
**Features:**
- Ephemeral instance (starts fresh)
- Automatic cleanup
- Full MongoDB API via [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo)
**Use Cases:**
- Testing database operations
- Integration tests with MongoDB
- Testing data models
- Schema validation tests
#### Storage Testing
##### `createSmarts3()`
Create a local S3-compatible storage instance for testing object storage operations.
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
tap.test('should use S3 storage', async () => {
const s3Instance = await tapNodeTools.createSmarts3();
// Use the S3 instance (MinIO-compatible API)
const bucket = await s3Instance.createBucket('test-bucket');
await bucket.putObject('file.txt', Buffer.from('Hello S3'));
const file = await bucket.getObject('file.txt');
expect(file.toString()).toEqual('Hello S3');
// Cleanup
await s3Instance.stop();
});
export default tap.start();
```
**Configuration:**
- Port: 3003 (default)
- Clean slate: true (starts fresh each time)
- Full S3-compatible API via [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3)
**Use Cases:**
- Testing file uploads/downloads
- Testing object storage operations
- Testing backup/restore logic
- Mocking cloud storage
### TestFileProvider
Utility for downloading and managing test assets.
#### `getDockerAlpineImageAsLocalTarball()`
#### `testFileProvider.getDockerAlpineImageAsLocalTarball()`
Download the Alpine Linux Docker image as a tarball for testing.
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
tap.test('should provide docker image', async () => {
const tarballPath = await tapNodeTools.testFileProvider.getDockerAlpineImageAsLocalTarball();
// Use the tarball path
// Path: ./.nogit/testfiles/alpine.tar
expect(tarballPath).toMatch(/alpine\.tar$/);
});
export default tap.start();
```
**Features:**
- Downloads from https://code.foss.global/testassets/docker
- Caches in `.nogit/testfiles/` directory
- Returns local file path
**Use Cases:**
- Testing Docker operations
- Testing container deployment
- Testing image handling logic
### Path Utilities
The module exports useful path constants:
```typescript
import * as paths from '@git.zone/tstest/tapbundle_serverside/paths';
console.log(paths.cwd); // Current working directory
console.log(paths.testFilesDir); // ./.nogit/testfiles/
```
## Patterns and Best Practices
### Testing with External Services
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
import { tap, expect } from '@git.zone/tstest/tapbundle';
tap.describe('User Service Integration', () => {
let mongoInstance;
let db;
tap.beforeEach(async () => {
mongoInstance = await tapNodeTools.createSmartmongo();
db = await mongoInstance.getDatabase('testdb');
});
tap.test('should create user', async () => {
const users = await db.getCollection('users');
await users.insertOne({ name: 'Bob', email: 'bob@example.com' });
const user = await users.findOne({ name: 'Bob' });
expect(user.email).toEqual('bob@example.com');
});
tap.afterEach(async () => {
await mongoInstance.stop();
});
});
export default tap.start();
```
### Testing HTTPS Servers
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
import { tap, expect } from '@git.zone/tstest/tapbundle';
import * as https from 'https';
tap.test('should serve over HTTPS', async () => {
const { key, cert } = await tapNodeTools.createHttpsCert();
const server = https.createServer({ key, cert }, (req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Secure Response');
});
await new Promise((resolve) => {
server.listen(8443, () => resolve(undefined));
});
// Test the server
const response = await fetch('https://localhost:8443');
const text = await response.text();
expect(text).toEqual('Secure Response');
// Cleanup
server.close();
});
export default tap.start();
```
### Environment-Dependent Tests
```typescript
import { tapNodeTools } from '@git.zone/tstest/tapbundle_serverside';
import { tap, expect } from '@git.zone/tstest/tapbundle';
tap.test('should authenticate with GitHub', async () => {
const githubToken = await tapNodeTools.getEnvVarOnDemand('GITHUB_TOKEN');
// Use the token for API calls
const response = await fetch('https://api.github.com/user', {
headers: {
Authorization: `Bearer ${githubToken}`
}
});
expect(response.ok).toBeTruthy();
});
export default tap.start();
const tarballPath = await tapNodeTools.testFileProvider.getDockerAlpineImageAsLocalTarball();
// Path: ./.nogit/testfiles/alpine.tar
```
## Runtime Requirements
⚠️ **Server-Side Only (Node.js)**: All utilities in this module are designed exclusively for server-side testing in Node.js runtime. They provide functionality like shell command execution, file system operations, and process management that only make sense on the server.
**NOT available in:**
- Browser environments
- Deno runtime
- Bun runtime
**Important:** Import tapbundle_serverside only in tests that run exclusively on the server-side (`.node.ts` test files). For cross-runtime tests, these utilities will fail in non-Node environments.
## File Naming
Use Node.js-specific file naming when using these utilities:
⚠️ **Node.js only.** All utilities in this module require Node.js. Import only in `.node.ts` test files.
```
test/mytest.node.ts ✅ Node.js only
test/mytest.node+deno.ts ❌ Will fail in Deno
test/mytest.browser+node.ts ⚠️ Browser won't have access to these tools
test/mytest.node.ts ✅ Correct — Node.js only
test/mytest.ts ✅ Correct — defaults to Node.js
test/mytest.all.ts ❌ Will fail in Deno/Bun/Chromium
```
## Dependencies
This module uses the following packages:
- [@push.rocks/qenv](https://code.foss.global/push.rocks/qenv) - Environment variable management
- [@push.rocks/smartshell](https://code.foss.global/push.rocks/smartshell) - Shell command execution
- [@push.rocks/smartcrypto](https://code.foss.global/push.rocks/smartcrypto) - Certificate generation
- [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo) - MongoDB testing
- [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3) - S3 storage testing
- [@push.rocks/smartfile](https://code.foss.global/push.rocks/smartfile) - File operations
- [@push.rocks/smartrequest](https://code.foss.global/push.rocks/smartrequest) - HTTP requests
- [@push.rocks/smartnetwork](https://code.foss.global/push.rocks/smartnetwork) — Port discovery
- [@push.rocks/qenv](https://code.foss.global/push.rocks/qenv) Environment variable management
- [@push.rocks/smartshell](https://code.foss.global/push.rocks/smartshell) Shell command execution
- [@push.rocks/smartcrypto](https://code.foss.global/push.rocks/smartcrypto) Certificate generation
- [@push.rocks/smartmongo](https://code.foss.global/push.rocks/smartmongo) MongoDB testing
- [@push.rocks/smarts3](https://code.foss.global/push.rocks/smarts3) S3 storage testing
- [@push.rocks/smartfile](https://code.foss.global/push.rocks/smartfile) File operations
- [@push.rocks/smartrequest](https://code.foss.global/push.rocks/smartrequest) HTTP requests
## License and Legal Information
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](../license) file within this repository.
This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](../LICENSE) file.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.
Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.
### Company Information
Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany
Registered at District Court Bremen HRB 35230 HB, Germany
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
For any legal inquiries or further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.