14 Commits

Author SHA1 Message Date
8d918ec03b v5.1.1
Some checks failed
Default (tags) / security (push) Failing after 0s
Default (tags) / test (push) Failing after 0s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-03-26 08:02:04 +00:00
97e8298dcd fix(build): migrate smartconfig metadata and refresh build dependencies 2026-03-26 08:02:04 +00:00
c65c285296 v5.1.0
Some checks failed
Default (tags) / security (push) Successful in 38s
Default (tags) / test (push) Failing after 4m1s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-02-03 16:48:50 +00:00
b72174ca7b feat(localtsmdb): export ILocalTsmDbConnectionInfo and expand LocalTsmDb/TsmDB documentation and examples 2026-02-03 16:48:50 +00:00
1ff4d219af v5.0.0
Some checks failed
Default (tags) / security (push) Successful in 43s
Default (tags) / test (push) Failing after 4m2s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-02-03 16:42:49 +00:00
09f60de56f BREAKING CHANGE(localtsmdb): add Unix socket support and change LocalTsmDb API to return connection info instead of a MongoClient 2026-02-03 16:42:49 +00:00
e6a36ecb5f v4.3.0
Some checks failed
Default (tags) / security (push) Successful in 41s
Default (tags) / test (push) Failing after 4m0s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-02-03 08:14:58 +00:00
6a37a773ea feat(docs): add LocalTsmDb documentation and examples; update README code samples and imports; correct examples and variable names; update package author 2026-02-03 08:14:58 +00:00
1fff277698 v4.2.1
Some checks failed
Default (tags) / security (push) Successful in 44s
Default (tags) / test (push) Failing after 4m1s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-02-03 07:29:20 +00:00
0ad7f316c4 fix(package.json): replace main and typings with exports field pointing to ./dist_ts/index.js 2026-02-03 07:29:20 +00:00
0d450e7d4e v4.2.0
Some checks failed
Default (tags) / security (push) Successful in 43s
Default (tags) / test (push) Failing after 4m2s
Default (tags) / release (push) Has been skipped
Default (tags) / metadata (push) Has been skipped
2026-02-01 23:33:35 +00:00
fff77fbd8e feat(tsmdb): implement TsmDB Mongo-wire-compatible server, add storage/engine modules and reorganize exports 2026-02-01 23:33:35 +00:00
678bf15eb4 v4.1.1
Some checks failed
Default (tags) / security (push) Successful in 31s
Default (tags) / test (push) Failing after 12m4s
Default (tags) / release (push) Has been cancelled
Default (tags) / metadata (push) Has been cancelled
2026-02-01 16:16:45 +00:00
aa45e9579b fix(tsmdb): add comprehensive unit tests for tsmdb components: checksum, query planner, index engine, session, and WAL 2026-02-01 16:16:45 +00:00
50 changed files with 4013 additions and 3901 deletions

View File

@@ -5,7 +5,7 @@
"githost": "code.foss.global",
"gitscope": "push.rocks",
"gitrepo": "smartmongo",
"description": "A module for creating and managing a local MongoDB instance for testing purposes.",
"description": "A powerful MongoDB toolkit for testing and development with a real memory server, wire-protocol-compatible database server, and zero-config local database.",
"npmPackagename": "@push.rocks/smartmongo",
"license": "MIT",
"projectDomain": "push.rocks",

View File

@@ -1,7 +1,7 @@
{
"json.schemas": [
{
"fileMatch": ["/npmextra.json"],
"fileMatch": ["/.smartconfig.json"],
"schema": {
"type": "object",
"properties": {

View File

@@ -1,5 +1,68 @@
# Changelog
## 2026-03-26 - 5.1.1 - fix(build)
migrate smartconfig metadata and refresh build dependencies
- replace npmextra.json with .smartconfig.json and update packaged config file inclusion
- switch the build script to tsbuild tsfolders
- update core build and runtime dependencies and add explicit TypeScript type annotations to satisfy newer tooling
## 2026-02-03 - 5.1.0 - feat(localtsmdb)
export ILocalTsmDbConnectionInfo and expand LocalTsmDb/TsmDB documentation and examples
- Exported new type ILocalTsmDbConnectionInfo from ts_local (ts/index.ts)
- Added LocalTsmDb configuration example, methods table, and ConnectionInfo interface to README
- Documented Unix socket vs TCP connection modes and updated usage examples (TCP and socket examples)
- Expanded TsmDB docs: additional server properties, aggregation stages, regex examples, index operations, database ops, checksums, and wire protocol commands
- Updated architecture notes to include Unix socket support and new engine components (QueryEngine, UpdateEngine, AggregationEngine)
## 2026-02-03 - 5.0.0 - BREAKING CHANGE(localtsmdb)
add Unix socket support and change LocalTsmDb API to return connection info instead of a MongoClient
- LocalTsmDb.start() now returns ILocalTsmDbConnectionInfo { socketPath, connectionUri } instead of a connected MongoClient
- Removed internal MongoClient management: consumers must create/connect/close their own MongoClient using the returned connectionUri (close client before calling db.stop())
- Added ILocalTsmDbConnectionInfo type and getConnectionInfo() (replaces getClient())
- TsmdbServer: added socketPath option to listen on Unix sockets, cleans up stale socket files on start/stop, and encodes socket paths in getConnectionUri()
- LocalTsmDb can auto-generate socket paths in the OS temp dir; LocalTsmDb no longer depends on the mongodb package internally (lightweight Unix socket wrapper)
- Updated docs and tests to use MongoClient externally and to demonstrate socketPath/connectionUri workflow
- ts_local plugins no longer export net (net usage moved to server implementation)
## 2026-02-03 - 4.3.0 - feat(docs)
add LocalTsmDb documentation and examples; update README code samples and imports; correct examples and variable names; update package author
- Introduce LocalTsmDb: zero-config local database with automatic persistence, auto port discovery, and pre-connected client (added Quick Start, API, Features, and testing examples).
- Expand comparison table to include LocalTsmDb alongside SmartMongo and TsmDB.
- Update README examples: new LocalTsmDb usage, reorder options (LocalTsmDb, TsmDB, SmartMongo), rename test DB variable (db -> testDb), and adjust test snippets for Jest/Mocha and tap.
- Adjust code snippets and API notes: switch some example imports to use tsmdb, replace FileStorageAdapter references, change planner.createPlan to await planner.plan, and use wal.getEntriesAfter(...) without awaiting.
- Update package.json author from 'Lossless GmbH' to 'Task Venture Capital GmbH'.
## 2026-02-03 - 4.2.1 - fix(package.json)
replace main and typings with exports field pointing to ./dist_ts/index.js
- Added package.json exports field mapping "." to ./dist_ts/index.js to declare the package entrypoint.
- Removed main (dist_ts/index.js) and typings (dist_ts/index.d.ts) entries.
- Note: switching to exports improves Node resolution but removing the typings entry may affect TypeScript consumers expecting index.d.ts.
## 2026-02-01 - 4.2.0 - feat(tsmdb)
implement TsmDB Mongo-wire-compatible server, add storage/engine modules and reorganize exports
- Add full TsmDB implementation under ts/ts_tsmdb: wire protocol, server, command router, handlers, engines (Query, Update, Aggregation, Index, Transaction, Session), storage adapters (Memory, File), OpLog, WAL, utils and types.
- Remove legacy ts/tsmdb implementation and replace with new ts_tsmdb module exports.
- Introduce ts/ts_mongotools module and move SmartMongo class there; update top-level exports in ts/index.ts to export SmartMongo, tsmdb (from ts_tsmdb) and LocalTsmDb.
- Add LocalTsmDb convenience class (ts/ts_local) to start a file-backed TsmDB and return a connected MongoClient.
- Refactor plugin imports into per-module plugins files and add utilities (checksum, persistence, query planner, index engine).
## 2026-02-01 - 4.1.1 - fix(tsmdb)
add comprehensive unit tests for tsmdb components: checksum, query planner, index engine, session, and WAL
- Add new tests: test.tsmdb.checksum.ts — CRC32 and document checksum utilities (add/verify/remove)
- Add new tests: test.tsmdb.queryplanner.ts — QueryPlanner plans, index usage, selectivity, explain output, and edge cases
- Add new tests: test.tsmdb.indexengine.ts — Index creation, unique/sparse options, candidate selection, and constraints
- Add new tests: test.tsmdb.session.ts — Session lifecycle, touch/refresh/close, extractSessionId handling
- Add new tests: test.tsmdb.wal.ts — WAL initialization, LSN increments, logging/recovery for inserts/updates/deletes, binary and nested data handling
- Tests only — no production API changes; increases test coverage
- Recommend patch bump from 4.1.0 to 4.1.1
## 2026-02-01 - 4.1.0 - feat(readme)
expand README with storage integrity, WAL, query planner, session & transaction docs; update test script to enable verbose logging and increase timeout

View File

@@ -1,36 +1,37 @@
{
"name": "@push.rocks/smartmongo",
"version": "4.1.0",
"version": "5.1.1",
"private": false,
"description": "A module for creating and managing a local MongoDB instance for testing purposes.",
"main": "dist_ts/index.js",
"typings": "dist_ts/index.d.ts",
"description": "A powerful MongoDB toolkit for testing and development with a real memory server, wire-protocol-compatible database server, and zero-config local database.",
"exports": {
".": "./dist_ts/index.js"
},
"type": "module",
"author": "Lossless GmbH",
"author": "Task Venture Capital GmbH",
"license": "MIT",
"scripts": {
"test": "(tstest test/. --verbose --logfile --timeout 60)",
"build": "(tsbuild --web)",
"build": "(tsbuild tsfolders)",
"buildDocs": "tsdoc"
},
"devDependencies": {
"@git.zone/tsbuild": "^4.1.2",
"@git.zone/tsbundle": "^2.8.3",
"@git.zone/tsrun": "^2.0.1",
"@git.zone/tstest": "^3.1.8",
"@types/node": "^25.1.0",
"mongodb": "^7.0.0"
"@git.zone/tsbuild": "^4.4.0",
"@git.zone/tsbundle": "^2.10.0",
"@git.zone/tsrun": "^2.0.2",
"@git.zone/tstest": "^3.6.0",
"@types/node": "^25.5.0",
"mongodb": "^7.1.1"
},
"dependencies": {
"@push.rocks/mongodump": "^1.0.7",
"@push.rocks/smartdata": "^5.0.23",
"@push.rocks/smartfs": "^1.3.1",
"@push.rocks/smartpath": "^5.0.11",
"@push.rocks/smartpromise": "^4.0.3",
"@push.rocks/smartrx": "^3.0.0",
"bson": "^6.10.0",
"@push.rocks/mongodump": "^1.1.0",
"@push.rocks/smartdata": "^7.1.2",
"@push.rocks/smartfs": "^1.5.0",
"@push.rocks/smartpath": "^6.0.0",
"@push.rocks/smartpromise": "^4.2.3",
"@push.rocks/smartrx": "^3.0.10",
"bson": "^7.1.1",
"mingo": "^7.2.0",
"mongodb-memory-server": "^10.1.4"
"mongodb-memory-server": "^11.0.1"
},
"browserslist": [
"last 1 chrome versions"
@@ -44,7 +45,7 @@
"dist_ts_web/**/*",
"assets/**/*",
"cli.js",
"npmextra.json",
".smartconfig.json",
"readme.md"
],
"keywords": [

5251
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

431
readme.md
View File

@@ -1,6 +1,6 @@
# @push.rocks/smartmongo
A powerful MongoDB toolkit for testing and development — featuring both a real MongoDB memory server (**SmartMongo**) and an ultra-fast, lightweight wire-protocol-compatible in-memory database server (**TsmDB**). 🚀
A powerful MongoDB toolkit for testing and development — featuring a real MongoDB memory server (**SmartMongo**), an ultra-fast wire-protocol-compatible in-memory database server (**TsmDB**), and a zero-config local database (**LocalTsmDb**).
## Install
@@ -16,21 +16,88 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
## Overview
`@push.rocks/smartmongo` provides two powerful approaches for MongoDB in testing and development:
`@push.rocks/smartmongo` provides three powerful approaches for MongoDB in testing and development:
| Feature | SmartMongo | TsmDB |
|---------|------------|---------|
| **Type** | Real MongoDB (memory server) | Pure TypeScript wire protocol server |
| **Speed** | ~2-5s startup | Instant startup (~5ms) |
| **Compatibility** | 100% MongoDB | MongoDB driver compatible |
| **Dependencies** | Downloads MongoDB binary | Zero external dependencies |
| **Replication** | ✅ Full replica set support | Single node emulation |
| **Use Case** | Integration testing | Unit testing, CI/CD |
| **Persistence** | Dump to directory | Optional file/memory persistence |
| Feature | SmartMongo | TsmDB | LocalTsmDb |
|---------|------------|-------|------------|
| **Type** | Real MongoDB (memory server) | Wire protocol server | Zero-config local DB |
| **Speed** | ~2-5s startup | Instant (~5ms) | Instant (Unix socket) |
| **Compatibility** | 100% MongoDB | MongoDB driver compatible | MongoDB driver compatible |
| **Dependencies** | Downloads MongoDB binary | Zero external deps | Zero external deps (no MongoDB driver!) |
| **Connection** | TCP | TCP or Unix socket | Unix socket (default) |
| **Replication** | Full replica set | Single node | Single node |
| **Persistence** | Dump to directory | Memory or file | File-based (automatic) |
| **Use Case** | Integration testing | Unit testing, CI/CD | Quick prototyping, local dev |
## 🚀 Quick Start
## Quick Start
### Option 1: SmartMongo (Real MongoDB)
### Option 1: LocalTsmDb (Zero-Config Local Database)
The easiest way to get started — just point it at a folder and you have a persistent MongoDB-compatible database using Unix sockets. No port conflicts, no MongoDB driver dependency in LocalTsmDb!
```typescript
import { LocalTsmDb } from '@push.rocks/smartmongo';
import { MongoClient } from 'mongodb';
// Create a local database backed by files
const db = new LocalTsmDb({ folderPath: './my-data' });
// Start and get connection info (Unix socket path + connection URI)
const { connectionUri } = await db.start();
// Connect with your own MongoDB client
const client = new MongoClient(connectionUri, { directConnection: true });
await client.connect();
// Use exactly like MongoDB
const users = client.db('myapp').collection('users');
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
const user = await users.findOne({ name: 'Alice' });
console.log(user); // { _id: ObjectId(...), name: 'Alice', email: 'alice@example.com' }
// Data persists to disk automatically!
await client.close();
await db.stop();
// Later... data is still there
const db2 = new LocalTsmDb({ folderPath: './my-data' });
const { connectionUri: uri2 } = await db2.start();
const client2 = new MongoClient(uri2, { directConnection: true });
await client2.connect();
const savedUser = await client2.db('myapp').collection('users').findOne({ name: 'Alice' });
// savedUser exists!
```
### Option 2: TsmDB (Wire Protocol Server)
A lightweight, pure TypeScript MongoDB-compatible server — use the official `mongodb` driver directly!
```typescript
import { tsmdb } from '@push.rocks/smartmongo';
import { MongoClient } from 'mongodb';
// Start TsmDB server (TCP mode)
const server = new tsmdb.TsmdbServer({ port: 27017 });
await server.start();
// Connect with the official MongoDB driver
const client = new MongoClient('mongodb://127.0.0.1:27017');
await client.connect();
// Use exactly like real MongoDB
const db = client.db('myapp');
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
const user = await db.collection('users').findOne({ name: 'Alice' });
console.log(user); // { _id: ObjectId(...), name: 'Alice', age: 30 }
// Clean up
await client.close();
await server.stop();
```
### Option 3: SmartMongo (Real MongoDB)
Spin up a real MongoDB replica set in memory — perfect for integration tests that need full MongoDB compatibility.
@@ -51,35 +118,90 @@ console.log(descriptor.mongoDbUrl); // mongodb://127.0.0.1:xxxxx/...
await mongo.stop();
```
### Option 2: TsmDB (Wire Protocol Server)
---
A lightweight, pure TypeScript MongoDB-compatible server that speaks the wire protocol — use the official `mongodb` driver directly!
## LocalTsmDb API
The simplest option for local development and prototyping — lightweight, Unix socket-based, and automatic persistence.
### Configuration
```typescript
import { tsmdb } from '@push.rocks/smartmongo';
import { MongoClient } from 'mongodb';
import { LocalTsmDb } from '@push.rocks/smartmongo';
import type { ILocalTsmDbOptions, ILocalTsmDbConnectionInfo } from '@push.rocks/smartmongo';
// Start TsmDB server
const server = new tsmdb.TsmdbServer({ port: 27017 });
await server.start();
const options: ILocalTsmDbOptions = {
folderPath: './data', // Required: where to store data
socketPath: '/tmp/my.sock', // Optional: custom socket path (default: auto-generated)
};
// Connect with the official MongoDB driver!
const client = new MongoClient('mongodb://127.0.0.1:27017');
await client.connect();
// Use exactly like real MongoDB
const db = client.db('myapp');
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
const user = await db.collection('users').findOne({ name: 'Alice' });
console.log(user); // { _id: ObjectId(...), name: 'Alice', age: 30 }
// Clean up
await client.close();
await server.stop();
const db = new LocalTsmDb(options);
```
## 📖 SmartMongo API
### Methods
| Method | Returns | Description |
|--------|---------|-------------|
| `start()` | `Promise<ILocalTsmDbConnectionInfo>` | Starts the server and returns connection info |
| `stop()` | `Promise<void>` | Stops the server and cleans up the socket |
| `getConnectionInfo()` | `ILocalTsmDbConnectionInfo` | Returns current connection info |
| `getConnectionUri()` | `string` | Returns the MongoDB connection URI |
| `getServer()` | `TsmdbServer` | Returns the underlying TsmDB server instance |
| `running` | `boolean` | Property indicating if the server is running |
### Connection Info
The `start()` method returns an `ILocalTsmDbConnectionInfo` object:
```typescript
interface ILocalTsmDbConnectionInfo {
socketPath: string; // The Unix socket file path, e.g., /tmp/smartmongo-abc123.sock
connectionUri: string; // MongoDB URI, e.g., mongodb://%2Ftmp%2Fsmartmongo-abc123.sock
}
```
### Basic Usage
```typescript
import { LocalTsmDb } from '@push.rocks/smartmongo';
import { MongoClient } from 'mongodb';
const db = new LocalTsmDb({ folderPath: './data' });
// Start and get connection info
const { socketPath, connectionUri } = await db.start();
console.log(socketPath); // /tmp/smartmongo-abc123.sock (auto-generated)
console.log(connectionUri); // mongodb://%2Ftmp%2Fsmartmongo-abc123.sock
// Connect with your own MongoDB client
const client = new MongoClient(connectionUri, { directConnection: true });
await client.connect();
// Use the client
const users = client.db('mydb').collection('users');
await users.insertOne({ name: 'Alice' });
// Check status
console.log(db.running); // true
// Stop when done (close your client first!)
await client.close();
await db.stop();
```
### Features
- **Unix Sockets** — No port conflicts, faster IPC than TCP
- **Automatic Persistence** — Data saved to files, survives restarts
- **Lightweight** — No MongoDB driver dependency in LocalTsmDb itself
- **Zero Config** — Just specify a folder path and you're good to go
- **Connection URI** — Ready-to-use URI for your own MongoClient
---
## SmartMongo API
Full MongoDB replica set in memory using `mongodb-memory-server`.
### Creating an Instance
@@ -103,7 +225,7 @@ const descriptor = await mongo.getMongoDescriptor();
// }
```
### Stopping & Cleanup
### Stopping and Cleanup
```typescript
// Simple stop (data discarded)
@@ -116,27 +238,44 @@ await mongo.stopAndDumpToDir('./test-data');
await mongo.stopAndDumpToDir('./test-data', (doc) => `${doc.collection}-${doc._id}.bson`);
```
## 🔧 TsmDB API
---
## TsmDB API
Pure TypeScript MongoDB wire protocol server. No external dependencies.
### Server Configuration
```typescript
import { tsmdb } from '@push.rocks/smartmongo';
// TCP mode (default)
const server = new tsmdb.TsmdbServer({
port: 27017, // Default MongoDB port
host: '127.0.0.1', // Bind address
storage: 'memory', // 'memory' or 'file'
storagePath: './data', // For file-based storage
port: 27017, // Default MongoDB port
host: '127.0.0.1', // Bind address
storage: 'memory', // 'memory' or 'file'
storagePath: './data', // For file-based storage
});
// Unix socket mode (no port conflicts!)
const server = new tsmdb.TsmdbServer({
socketPath: '/tmp/my-tsmdb.sock',
storage: 'file',
storagePath: './data',
});
await server.start();
console.log(server.getConnectionUri()); // mongodb://127.0.0.1:27017
console.log(server.getConnectionUri());
// TCP: mongodb://127.0.0.1:27017
// Socket: mongodb://%2Ftmp%2Fmy-tsmdb.sock
// Server properties
console.log(server.running); // true
console.log(server.getUptime()); // seconds
console.log(server.getConnectionCount()); // active connections
console.log(server.port); // 27017 (TCP mode)
console.log(server.host); // '127.0.0.1' (TCP mode)
console.log(server.socketPath); // '/tmp/my-tsmdb.sock' (socket mode)
console.log(server.getUptime()); // seconds since start
console.log(server.getConnectionCount()); // active client connections
await server.stop();
```
@@ -145,7 +284,8 @@ await server.stop();
TsmDB supports the core MongoDB operations via the wire protocol:
#### 🔹 CRUD Operations
#### CRUD Operations
```typescript
// Insert
await collection.insertOne({ name: 'Bob' });
@@ -174,7 +314,8 @@ const result = await collection.findOneAndUpdate(
);
```
#### 🔹 Query Operators
#### Query Operators
```typescript
// Comparison
{ age: { $eq: 25 } }
@@ -197,9 +338,14 @@ const result = await collection.findOneAndUpdate(
{ tags: { $all: ['mongodb', 'database'] } }
{ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }
{ tags: { $size: 3 } }
// Regex
{ name: { $regex: /^Al/i } }
{ email: { $regex: '@example\\.com$' } }
```
#### 🔹 Update Operators
#### Update Operators
```typescript
{ $set: { name: 'New Name' } }
{ $unset: { tempField: '' } }
@@ -212,9 +358,12 @@ const result = await collection.findOneAndUpdate(
{ $addToSet: { tags: 'unique-tag' } }
{ $pop: { queue: 1 } } // Remove last
{ $pop: { queue: -1 } } // Remove first
{ $rename: { oldField: 'newField' } }
{ $currentDate: { lastModified: true } }
```
#### 🔹 Aggregation Pipeline
#### Aggregation Pipeline
```typescript
const results = await collection.aggregate([
{ $match: { status: 'active' } },
@@ -225,17 +374,27 @@ const results = await collection.aggregate([
]).toArray();
```
Supported stages: `$match`, `$project`, `$group`, `$sort`, `$limit`, `$skip`, `$unwind`, `$lookup`, `$addFields`, `$count`, `$facet`, and more.
**Supported stages:** `$match`, `$project`, `$group`, `$sort`, `$limit`, `$skip`, `$unwind`, `$lookup`, `$addFields`, `$count`, `$facet`, `$replaceRoot`, `$set`, `$unset`, and more.
**Supported group accumulators:** `$sum`, `$avg`, `$min`, `$max`, `$first`, `$last`, `$push`, `$addToSet`, `$count`.
#### Index Operations
#### 🔹 Index Operations
```typescript
// Create indexes
await collection.createIndex({ email: 1 }, { unique: true });
await collection.createIndex({ name: 1, age: -1 });
// List indexes
const indexes = await collection.listIndexes().toArray();
// Drop indexes
await collection.dropIndex('email_1');
await collection.dropIndexes(); // Drop all except _id
```
#### 🔹 Database Operations
#### Database Operations
```typescript
// List databases
const dbs = await client.db().admin().listDatabases();
@@ -249,9 +408,13 @@ await db.dropCollection('oldcollection');
// Drop database
await db.dropDatabase();
// Database stats
const stats = await db.stats();
```
#### 🔹 Count & Distinct
#### Count and Distinct
```typescript
// Count documents
const total = await collection.countDocuments({});
@@ -263,7 +426,8 @@ const departments = await collection.distinct('department');
const activeDepts = await collection.distinct('department', { status: 'active' });
```
#### 🔹 Bulk Operations
#### Bulk Operations
```typescript
const result = await collection.bulkWrite([
{ insertOne: { document: { name: 'Bulk1' } } },
@@ -292,22 +456,22 @@ const server = new tsmdb.TsmdbServer({
persistIntervalMs: 30000 // Save every 30 seconds
});
// File-based - persistent storage with optional checksums
import { FileStorageAdapter } from '@push.rocks/smartmongo/tsmdb';
const adapter = new FileStorageAdapter('./data/tsmdb', {
enableChecksums: true, // CRC32 checksums for data integrity
strictChecksums: false // Log warnings vs throw on mismatch
// File-based - persistent storage with checksums
const server = new tsmdb.TsmdbServer({
storage: 'file',
storagePath: './data/tsmdb'
});
```
## ⚡ Performance & Reliability Features
---
## Performance and Reliability Features
TsmDB includes enterprise-grade features for robustness:
### 🔍 Index-Accelerated Queries
### Index-Accelerated Queries
Indexes are automatically used to accelerate queries. Instead of scanning all documents, TsmDB uses:
Indexes are automatically used to accelerate queries:
- **Hash indexes** for equality queries (`$eq`, `$in`)
- **B-tree indexes** for range queries (`$gt`, `$gte`, `$lt`, `$lte`)
@@ -322,7 +486,7 @@ await collection.findOne({ email: 'alice@example.com' }); // Uses hash lookup
await collection.find({ age: { $gte: 18, $lt: 65 } }); // Uses B-tree range scan
```
### 📊 Query Planner
### Query Planner
TsmDB includes a query planner that analyzes queries and selects optimal execution strategies:
@@ -331,18 +495,18 @@ import { tsmdb } from '@push.rocks/smartmongo';
// For debugging, you can access the query planner
const planner = new tsmdb.QueryPlanner(indexEngine);
const plan = planner.createPlan(filter);
const plan = await planner.plan(filter);
console.log(plan);
// {
// type: 'IXSCAN', // or 'IXSCAN_RANGE', 'COLLSCAN'
// indexName: 'email_1',
// estimatedCost: 1,
// selectivity: 0.001
// selectivity: 0.01,
// indexCovering: true
// }
```
### 📝 Write-Ahead Logging (WAL)
### Write-Ahead Logging (WAL)
For durability, TsmDB supports write-ahead logging:
@@ -357,13 +521,13 @@ await wal.initialize();
// - Timestamp
// - Operation type (insert, update, delete, checkpoint)
// - Document data (BSON serialized)
// - CRC32 checksum
// - CRC32 checksum for integrity
// Recovery support
const entries = await wal.getEntriesAfter(lastCheckpointLsn);
const entries = wal.getEntriesAfter(lastCheckpointLsn);
```
### 🔐 Session Management
### Session Management
TsmDB tracks client sessions with automatic timeout and transaction linking:
@@ -388,37 +552,71 @@ try {
// - Session activity tracking
```
### Data Integrity Checksums
### Data Integrity Checksums
File-based storage supports CRC32 checksums to detect corruption:
```typescript
import { FileStorageAdapter } from '@push.rocks/smartmongo/tsmdb';
const adapter = new FileStorageAdapter('./data', {
enableChecksums: true,
strictChecksums: true // Throw error on corruption (vs warning)
});
import { tsmdb } from '@push.rocks/smartmongo';
// Checksums are used internally for WAL and data integrity
// Documents are checksummed on write, verified on read
// Checksums are automatically stripped before returning to client
const checksum = tsmdb.calculateDocumentChecksum(doc);
const isValid = tsmdb.verifyChecksum(docWithChecksum);
```
### 📋 Supported Wire Protocol Commands
### Supported Wire Protocol Commands
| Category | Commands |
|----------|----------|
| **Handshake** | `hello`, `isMaster` |
| **Handshake** | `hello`, `isMaster`, `ismaster` |
| **CRUD** | `find`, `insert`, `update`, `delete`, `findAndModify`, `getMore`, `killCursors` |
| **Aggregation** | `aggregate`, `count`, `distinct` |
| **Indexes** | `createIndexes`, `dropIndexes`, `listIndexes` |
| **Transactions** | `startTransaction`, `commitTransaction`, `abortTransaction` |
| **Sessions** | `startSession`, `endSessions` |
| **Admin** | `ping`, `listDatabases`, `listCollections`, `drop`, `dropDatabase`, `create`, `serverStatus`, `buildInfo`, `dbStats`, `collStats` |
| **Sessions** | `startSession`, `endSessions`, `refreshSessions` |
| **Admin** | `ping`, `listDatabases`, `listCollections`, `drop`, `dropDatabase`, `create`, `serverStatus`, `buildInfo`, `dbStats`, `collStats`, `connectionStatus`, `currentOp`, `collMod`, `renameCollection` |
TsmDB supports MongoDB wire protocol versions 0-21, compatible with MongoDB 3.6 through 7.0 drivers.
## 🧪 Testing Examples
---
## Testing Examples
### Jest/Mocha with LocalTsmDb
```typescript
import { LocalTsmDb } from '@push.rocks/smartmongo';
import { MongoClient, Db } from 'mongodb';
let db: LocalTsmDb;
let client: MongoClient;
beforeAll(async () => {
db = new LocalTsmDb({ folderPath: './test-data' });
const { connectionUri } = await db.start();
client = new MongoClient(connectionUri, { directConnection: true });
await client.connect();
});
afterAll(async () => {
await client.close();
await db.stop();
});
beforeEach(async () => {
// Clean slate for each test
await client.db('test').dropDatabase();
});
test('should insert and find user', async () => {
const users = client.db('test').collection('users');
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
const user = await users.findOne({ name: 'Alice' });
expect(user?.email).toBe('alice@example.com');
});
```
### Jest/Mocha with TsmDB
@@ -428,7 +626,7 @@ import { MongoClient, Db } from 'mongodb';
let server: tsmdb.TsmdbServer;
let client: MongoClient;
let db: Db;
let testDb: Db;
beforeAll(async () => {
server = new tsmdb.TsmdbServer({ port: 27117 });
@@ -436,7 +634,7 @@ beforeAll(async () => {
client = new MongoClient('mongodb://127.0.0.1:27117');
await client.connect();
db = client.db('test');
testDb = client.db('test');
});
afterAll(async () => {
@@ -445,12 +643,11 @@ afterAll(async () => {
});
beforeEach(async () => {
// Clean slate for each test
await db.dropDatabase();
await testDb.dropDatabase();
});
test('should insert and find user', async () => {
const users = db.collection('users');
const users = testDb.collection('users');
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
const user = await users.findOne({ name: 'Alice' });
@@ -462,22 +659,21 @@ test('should insert and find user', async () => {
```typescript
import { expect, tap } from '@git.zone/tstest/tapbundle';
import { tsmdb } from '@push.rocks/smartmongo';
import { LocalTsmDb } from '@push.rocks/smartmongo';
import { MongoClient } from 'mongodb';
let server: tsmdb.TsmdbServer;
let db: LocalTsmDb;
let client: MongoClient;
tap.test('setup', async () => {
server = new tsmdb.TsmdbServer({ port: 27117 });
await server.start();
client = new MongoClient('mongodb://127.0.0.1:27117');
db = new LocalTsmDb({ folderPath: './test-data' });
const { connectionUri } = await db.start();
client = new MongoClient(connectionUri, { directConnection: true });
await client.connect();
});
tap.test('should perform CRUD operations', async () => {
const db = client.db('test');
const col = db.collection('items');
const col = client.db('test').collection('items');
// Create
const result = await col.insertOne({ name: 'Widget', price: 9.99 });
@@ -500,13 +696,24 @@ tap.test('should perform CRUD operations', async () => {
tap.test('teardown', async () => {
await client.close();
await server.stop();
await db.stop();
});
export default tap.start();
```
## 🏗️ Architecture
---
## Architecture
### Module Structure
```
@push.rocks/smartmongo
├── SmartMongo -> Real MongoDB memory server (mongodb-memory-server wrapper)
├── tsmdb -> Wire protocol server with full engine stack
└── LocalTsmDb -> Lightweight Unix socket wrapper (no MongoDB driver dependency)
```
### TsmDB Wire Protocol Stack
@@ -515,17 +722,17 @@ export default tap.start();
│ Official MongoDB Driver │
│ (mongodb npm) │
└─────────────────────────┬───────────────────────────────────┘
│ TCP + OP_MSG/BSON
│ TCP/Unix Socket + OP_MSG/BSON
v
┌─────────────────────────────────────────────────────────────┐
│ TsmdbServer │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ WireProtocol ││CommandRouter ││ Handlers │ │
│ │ WireProtocol │->│CommandRouter │->│ Handlers │ │
│ │ (OP_MSG) │ │ │ │ (Find, Insert..) │ │
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
└─────────────────────────┬───────────────────────────────────┘
v
┌─────────────────────────────────────────────────────────────┐
│ Engines │
│ ┌─────────┐ ┌────────┐ ┌───────────┐ ┌───────┐ ┌───────┐ │
@@ -537,7 +744,7 @@ export default tap.start();
│ └──────────────────────┘ │
└─────────────────────────┬───────────────────────────────────┘
v
┌─────────────────────────────────────────────────────────────┐
│ Storage Layer │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────┐ │
@@ -554,28 +761,32 @@ export default tap.start();
| **WireProtocol** | Parses MongoDB OP_MSG binary protocol |
| **CommandRouter** | Routes commands to appropriate handlers |
| **QueryPlanner** | Analyzes queries and selects execution strategy |
| **QueryEngine** | Executes queries with filter matching via mingo |
| **UpdateEngine** | Processes update operators (`$set`, `$inc`, etc.) |
| **AggregationEngine** | Executes aggregation pipelines via mingo |
| **IndexEngine** | Manages B-tree and hash indexes |
| **SessionEngine** | Tracks client sessions and timeouts |
| **TransactionEngine** | Handles ACID transaction semantics |
| **WAL** | Write-ahead logging for durability |
| **TransactionEngine** | Handles ACID transaction semantics with snapshot isolation |
| **WAL** | Write-ahead logging for durability and crash recovery |
| **OpLog** | Operation log for change tracking and streaming |
---
## License and Legal Information
This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.
Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
### Company Information
Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany
Registered at District court Bremen HRB 35230 HB, Germany
For any legal inquiries or further information, please contact us via email at hello@task.vc.
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

232
test/test.tsmdb.checksum.ts Normal file
View File

@@ -0,0 +1,232 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import * as smartmongo from '../ts/index.js';
const {
calculateCRC32,
calculateCRC32Buffer,
calculateDocumentChecksum,
addChecksum,
verifyChecksum,
removeChecksum,
} = smartmongo.tsmdb;
// ============================================================================
// CRC32 String Tests
// ============================================================================
tap.test('checksum: calculateCRC32 should return consistent value for same input', async () => {
const result1 = calculateCRC32('hello world');
const result2 = calculateCRC32('hello world');
expect(result1).toEqual(result2);
});
tap.test('checksum: calculateCRC32 should return different values for different inputs', async () => {
const result1 = calculateCRC32('hello');
const result2 = calculateCRC32('world');
expect(result1).not.toEqual(result2);
});
tap.test('checksum: calculateCRC32 should return a 32-bit unsigned integer', async () => {
const result = calculateCRC32('test string');
expect(result).toBeGreaterThanOrEqual(0);
expect(result).toBeLessThanOrEqual(0xFFFFFFFF);
});
tap.test('checksum: calculateCRC32 should handle empty string', async () => {
const result = calculateCRC32('');
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
tap.test('checksum: calculateCRC32 should handle special characters', async () => {
const result = calculateCRC32('hello\nworld\t!"#$%&\'()');
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
tap.test('checksum: calculateCRC32 should handle unicode characters', async () => {
const result = calculateCRC32('hello 世界 🌍');
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
// ============================================================================
// CRC32 Buffer Tests
// ============================================================================
tap.test('checksum: calculateCRC32Buffer should return consistent value for same input', async () => {
const buffer = Buffer.from('hello world');
const result1 = calculateCRC32Buffer(buffer);
const result2 = calculateCRC32Buffer(buffer);
expect(result1).toEqual(result2);
});
tap.test('checksum: calculateCRC32Buffer should return different values for different inputs', async () => {
const buffer1 = Buffer.from('hello');
const buffer2 = Buffer.from('world');
const result1 = calculateCRC32Buffer(buffer1);
const result2 = calculateCRC32Buffer(buffer2);
expect(result1).not.toEqual(result2);
});
tap.test('checksum: calculateCRC32Buffer should handle empty buffer', async () => {
const result = calculateCRC32Buffer(Buffer.from(''));
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
tap.test('checksum: calculateCRC32Buffer should handle binary data', async () => {
const buffer = Buffer.from([0x00, 0xFF, 0x7F, 0x80, 0x01]);
const result = calculateCRC32Buffer(buffer);
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
// ============================================================================
// Document Checksum Tests
// ============================================================================
tap.test('checksum: calculateDocumentChecksum should return consistent value', async () => {
const doc = { name: 'John', age: 30 };
const result1 = calculateDocumentChecksum(doc);
const result2 = calculateDocumentChecksum(doc);
expect(result1).toEqual(result2);
});
tap.test('checksum: calculateDocumentChecksum should exclude _checksum field', async () => {
const doc1 = { name: 'John', age: 30 };
const doc2 = { name: 'John', age: 30, _checksum: 12345 };
const result1 = calculateDocumentChecksum(doc1);
const result2 = calculateDocumentChecksum(doc2);
expect(result1).toEqual(result2);
});
tap.test('checksum: calculateDocumentChecksum should handle empty document', async () => {
const result = calculateDocumentChecksum({});
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
tap.test('checksum: calculateDocumentChecksum should handle nested objects', async () => {
const doc = {
name: 'John',
address: {
street: '123 Main St',
city: 'Springfield',
zip: {
code: '12345',
plus4: '6789',
},
},
};
const result = calculateDocumentChecksum(doc);
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
tap.test('checksum: calculateDocumentChecksum should handle arrays', async () => {
const doc = {
name: 'John',
tags: ['developer', 'tester', 'admin'],
scores: [95, 87, 92],
};
const result = calculateDocumentChecksum(doc);
expect(typeof result).toEqual('number');
expect(result).toBeGreaterThanOrEqual(0);
});
// ============================================================================
// Add/Verify/Remove Checksum Tests
// ============================================================================
tap.test('checksum: addChecksum should add _checksum field to document', async () => {
const doc = { name: 'John', age: 30 };
const docWithChecksum = addChecksum(doc);
expect('_checksum' in docWithChecksum).toBeTrue();
expect(typeof docWithChecksum._checksum).toEqual('number');
expect(docWithChecksum.name).toEqual('John');
expect(docWithChecksum.age).toEqual(30);
});
tap.test('checksum: addChecksum should not modify the original document', async () => {
const doc = { name: 'John', age: 30 };
addChecksum(doc);
expect('_checksum' in doc).toBeFalse();
});
tap.test('checksum: verifyChecksum should return true for valid checksum', async () => {
const doc = { name: 'John', age: 30 };
const docWithChecksum = addChecksum(doc);
const isValid = verifyChecksum(docWithChecksum);
expect(isValid).toBeTrue();
});
tap.test('checksum: verifyChecksum should return false for tampered document', async () => {
const doc = { name: 'John', age: 30 };
const docWithChecksum = addChecksum(doc);
// Tamper with the document
docWithChecksum.age = 31;
const isValid = verifyChecksum(docWithChecksum);
expect(isValid).toBeFalse();
});
tap.test('checksum: verifyChecksum should return false for wrong checksum', async () => {
const doc = { name: 'John', age: 30, _checksum: 12345 };
const isValid = verifyChecksum(doc);
expect(isValid).toBeFalse();
});
tap.test('checksum: verifyChecksum should return true for document without checksum', async () => {
const doc = { name: 'John', age: 30 };
const isValid = verifyChecksum(doc);
expect(isValid).toBeTrue();
});
tap.test('checksum: removeChecksum should remove _checksum field', async () => {
const doc = { name: 'John', age: 30 };
const docWithChecksum = addChecksum(doc);
const docWithoutChecksum = removeChecksum(docWithChecksum);
expect('_checksum' in docWithoutChecksum).toBeFalse();
expect(docWithoutChecksum.name).toEqual('John');
expect(docWithoutChecksum.age).toEqual(30);
});
tap.test('checksum: removeChecksum should handle document without checksum', async () => {
const doc = { name: 'John', age: 30 };
const result = removeChecksum(doc);
expect('_checksum' in result).toBeFalse();
expect(result.name).toEqual('John');
expect(result.age).toEqual(30);
});
// ============================================================================
// Round-trip Tests
// ============================================================================
tap.test('checksum: full round-trip - add, verify, remove', async () => {
const original = { name: 'Test', value: 42, nested: { a: 1, b: 2 } };
// Add checksum
const withChecksum = addChecksum(original);
expect('_checksum' in withChecksum).toBeTrue();
// Verify checksum
expect(verifyChecksum(withChecksum)).toBeTrue();
// Remove checksum
const restored = removeChecksum(withChecksum);
expect('_checksum' in restored).toBeFalse();
// Original data should be intact
expect(restored.name).toEqual('Test');
expect(restored.value).toEqual(42);
expect(restored.nested.a).toEqual(1);
expect(restored.nested.b).toEqual(2);
});
export default tap.start();

View File

@@ -0,0 +1,417 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import * as smartmongo from '../ts/index.js';
const { IndexEngine, MemoryStorageAdapter, ObjectId } = smartmongo.tsmdb;
let storage: InstanceType<typeof MemoryStorageAdapter>;
let indexEngine: InstanceType<typeof IndexEngine>;
const TEST_DB = 'testdb';
const TEST_COLL = 'indextest';
// ============================================================================
// Setup
// ============================================================================
tap.test('indexengine: should create IndexEngine instance', async () => {
storage = new MemoryStorageAdapter();
await storage.initialize();
await storage.createCollection(TEST_DB, TEST_COLL);
indexEngine = new IndexEngine(TEST_DB, TEST_COLL, storage);
expect(indexEngine).toBeTruthy();
});
// ============================================================================
// Index Creation Tests
// ============================================================================
tap.test('indexengine: createIndex should create single-field index', async () => {
const indexName = await indexEngine.createIndex({ name: 1 });
expect(indexName).toEqual('name_1');
const exists = await indexEngine.indexExists('name_1');
expect(exists).toBeTrue();
});
tap.test('indexengine: createIndex should create compound index', async () => {
const indexName = await indexEngine.createIndex({ city: 1, state: -1 });
expect(indexName).toEqual('city_1_state_-1');
const exists = await indexEngine.indexExists('city_1_state_-1');
expect(exists).toBeTrue();
});
tap.test('indexengine: createIndex should use custom name if provided', async () => {
const indexName = await indexEngine.createIndex({ email: 1 }, { name: 'custom_email_index' });
expect(indexName).toEqual('custom_email_index');
const exists = await indexEngine.indexExists('custom_email_index');
expect(exists).toBeTrue();
});
tap.test('indexengine: createIndex should handle unique option', async () => {
const indexName = await indexEngine.createIndex({ uniqueField: 1 }, { unique: true });
expect(indexName).toEqual('uniqueField_1');
const indexes = await indexEngine.listIndexes();
const uniqueIndex = indexes.find(i => i.name === 'uniqueField_1');
expect(uniqueIndex!.unique).toBeTrue();
});
tap.test('indexengine: createIndex should handle sparse option', async () => {
const indexName = await indexEngine.createIndex({ sparseField: 1 }, { sparse: true });
expect(indexName).toEqual('sparseField_1');
const indexes = await indexEngine.listIndexes();
const sparseIndex = indexes.find(i => i.name === 'sparseField_1');
expect(sparseIndex!.sparse).toBeTrue();
});
tap.test('indexengine: createIndex should return existing index name if already exists', async () => {
const indexName1 = await indexEngine.createIndex({ existingField: 1 }, { name: 'existing_idx' });
const indexName2 = await indexEngine.createIndex({ existingField: 1 }, { name: 'existing_idx' });
expect(indexName1).toEqual(indexName2);
});
// ============================================================================
// Index Listing and Existence Tests
// ============================================================================
tap.test('indexengine: listIndexes should return all indexes', async () => {
const indexes = await indexEngine.listIndexes();
expect(indexes.length).toBeGreaterThanOrEqual(5); // _id_ + created indexes
expect(indexes.some(i => i.name === '_id_')).toBeTrue();
expect(indexes.some(i => i.name === 'name_1')).toBeTrue();
});
tap.test('indexengine: indexExists should return true for existing index', async () => {
const exists = await indexEngine.indexExists('name_1');
expect(exists).toBeTrue();
});
tap.test('indexengine: indexExists should return false for non-existent index', async () => {
const exists = await indexEngine.indexExists('nonexistent_index');
expect(exists).toBeFalse();
});
// ============================================================================
// Document Operations and Index Updates
// ============================================================================
tap.test('indexengine: should insert documents for index testing', async () => {
// Create a fresh index engine for document operations
await storage.dropCollection(TEST_DB, TEST_COLL);
await storage.createCollection(TEST_DB, TEST_COLL);
indexEngine = new IndexEngine(TEST_DB, TEST_COLL, storage);
// Create indexes first
await indexEngine.createIndex({ age: 1 });
await indexEngine.createIndex({ category: 1 });
// Insert test documents
const docs = [
{ _id: new ObjectId(), name: 'Alice', age: 25, category: 'A' },
{ _id: new ObjectId(), name: 'Bob', age: 30, category: 'B' },
{ _id: new ObjectId(), name: 'Charlie', age: 35, category: 'A' },
{ _id: new ObjectId(), name: 'Diana', age: 28, category: 'C' },
{ _id: new ObjectId(), name: 'Eve', age: 30, category: 'B' },
];
for (const doc of docs) {
const stored = await storage.insertOne(TEST_DB, TEST_COLL, doc);
await indexEngine.onInsert(stored);
}
});
tap.test('indexengine: onInsert should update indexes', async () => {
const newDoc = {
_id: new ObjectId(),
name: 'Frank',
age: 40,
category: 'D',
};
const stored = await storage.insertOne(TEST_DB, TEST_COLL, newDoc);
await indexEngine.onInsert(stored);
// Find by the indexed field
const candidates = await indexEngine.findCandidateIds({ age: 40 });
expect(candidates).toBeTruthy();
expect(candidates!.size).toEqual(1);
});
tap.test('indexengine: onUpdate should update indexes correctly', async () => {
// Get an existing document
const docs = await storage.findAll(TEST_DB, TEST_COLL);
const oldDoc = docs.find(d => d.name === 'Alice')!;
// Update the document
const newDoc = { ...oldDoc, age: 26 };
await storage.updateById(TEST_DB, TEST_COLL, oldDoc._id, newDoc);
await indexEngine.onUpdate(oldDoc, newDoc);
// Old value should not be in index
const oldCandidates = await indexEngine.findCandidateIds({ age: 25 });
expect(oldCandidates).toBeTruthy();
expect(oldCandidates!.has(oldDoc._id.toHexString())).toBeFalse();
// New value should be in index
const newCandidates = await indexEngine.findCandidateIds({ age: 26 });
expect(newCandidates).toBeTruthy();
expect(newCandidates!.has(oldDoc._id.toHexString())).toBeTrue();
});
tap.test('indexengine: onDelete should remove from indexes', async () => {
const docs = await storage.findAll(TEST_DB, TEST_COLL);
const docToDelete = docs.find(d => d.name === 'Frank')!;
await storage.deleteById(TEST_DB, TEST_COLL, docToDelete._id);
await indexEngine.onDelete(docToDelete);
const candidates = await indexEngine.findCandidateIds({ age: 40 });
expect(candidates).toBeTruthy();
expect(candidates!.has(docToDelete._id.toHexString())).toBeFalse();
});
// ============================================================================
// findCandidateIds Tests
// ============================================================================
tap.test('indexengine: findCandidateIds with equality filter', async () => {
const candidates = await indexEngine.findCandidateIds({ age: 30 });
expect(candidates).toBeTruthy();
expect(candidates!.size).toEqual(2); // Bob and Eve both have age 30
});
tap.test('indexengine: findCandidateIds with $in filter', async () => {
const candidates = await indexEngine.findCandidateIds({ age: { $in: [28, 30] } });
expect(candidates).toBeTruthy();
expect(candidates!.size).toEqual(3); // Diana (28), Bob (30), Eve (30)
});
tap.test('indexengine: findCandidateIds with no matching index', async () => {
const candidates = await indexEngine.findCandidateIds({ nonIndexedField: 'value' });
// Should return null when no index can be used
expect(candidates).toBeNull();
});
tap.test('indexengine: findCandidateIds with empty filter', async () => {
const candidates = await indexEngine.findCandidateIds({});
// Empty filter = no index can be used
expect(candidates).toBeNull();
});
// ============================================================================
// Range Query Tests (B-Tree)
// ============================================================================
tap.test('indexengine: findCandidateIds with $gt', async () => {
const candidates = await indexEngine.findCandidateIds({ age: { $gt: 30 } });
expect(candidates).toBeTruthy();
// Charlie (35) is > 30
expect(candidates!.size).toBeGreaterThanOrEqual(1);
});
tap.test('indexengine: findCandidateIds with $lt', async () => {
const candidates = await indexEngine.findCandidateIds({ age: { $lt: 28 } });
expect(candidates).toBeTruthy();
// Alice (26) is < 28
expect(candidates!.size).toBeGreaterThanOrEqual(1);
});
tap.test('indexengine: findCandidateIds with $gte', async () => {
const candidates = await indexEngine.findCandidateIds({ age: { $gte: 30 } });
expect(candidates).toBeTruthy();
// Bob (30), Eve (30), Charlie (35)
expect(candidates!.size).toBeGreaterThanOrEqual(3);
});
tap.test('indexengine: findCandidateIds with $lte', async () => {
const candidates = await indexEngine.findCandidateIds({ age: { $lte: 28 } });
expect(candidates).toBeTruthy();
// Alice (26), Diana (28)
expect(candidates!.size).toBeGreaterThanOrEqual(2);
});
tap.test('indexengine: findCandidateIds with range $gt and $lt', async () => {
const candidates = await indexEngine.findCandidateIds({ age: { $gt: 26, $lt: 35 } });
expect(candidates).toBeTruthy();
// Diana (28), Bob (30), Eve (30) are between 26 and 35 exclusive
expect(candidates!.size).toBeGreaterThanOrEqual(3);
});
// ============================================================================
// Index Selection Tests
// ============================================================================
tap.test('indexengine: selectIndex should return best index for equality', async () => {
const result = indexEngine.selectIndex({ age: 30 });
expect(result).toBeTruthy();
expect(result!.name).toEqual('age_1');
});
tap.test('indexengine: selectIndex should return best index for range query', async () => {
const result = indexEngine.selectIndex({ age: { $gt: 25 } });
expect(result).toBeTruthy();
expect(result!.name).toEqual('age_1');
});
tap.test('indexengine: selectIndex should return null for no matching filter', async () => {
const result = indexEngine.selectIndex({ nonIndexedField: 'value' });
expect(result).toBeNull();
});
tap.test('indexengine: selectIndex should return null for empty filter', async () => {
const result = indexEngine.selectIndex({});
expect(result).toBeNull();
});
tap.test('indexengine: selectIndex should prefer more specific indexes', async () => {
// Create a compound index
await indexEngine.createIndex({ age: 1, category: 1 }, { name: 'age_category_compound' });
// Query that matches compound index
const result = indexEngine.selectIndex({ age: 30, category: 'B' });
expect(result).toBeTruthy();
// Should prefer the compound index since it covers more fields
expect(result!.name).toEqual('age_category_compound');
});
// ============================================================================
// Drop Index Tests
// ============================================================================
tap.test('indexengine: dropIndex should remove the index', async () => {
await indexEngine.createIndex({ dropTest: 1 }, { name: 'drop_test_idx' });
expect(await indexEngine.indexExists('drop_test_idx')).toBeTrue();
await indexEngine.dropIndex('drop_test_idx');
expect(await indexEngine.indexExists('drop_test_idx')).toBeFalse();
});
tap.test('indexengine: dropIndex should throw for _id index', async () => {
let threw = false;
try {
await indexEngine.dropIndex('_id_');
} catch (e) {
threw = true;
}
expect(threw).toBeTrue();
});
tap.test('indexengine: dropIndex should throw for non-existent index', async () => {
let threw = false;
try {
await indexEngine.dropIndex('nonexistent_index');
} catch (e) {
threw = true;
}
expect(threw).toBeTrue();
});
tap.test('indexengine: dropAllIndexes should remove all indexes except _id', async () => {
// Create some indexes to drop
await indexEngine.createIndex({ toDrop1: 1 });
await indexEngine.createIndex({ toDrop2: 1 });
await indexEngine.dropAllIndexes();
const indexes = await indexEngine.listIndexes();
expect(indexes.length).toEqual(1);
expect(indexes[0].name).toEqual('_id_');
});
// ============================================================================
// Unique Index Constraint Tests
// ============================================================================
tap.test('indexengine: unique index should prevent duplicate inserts', async () => {
// Create fresh collection
await storage.dropCollection(TEST_DB, 'uniquetest');
await storage.createCollection(TEST_DB, 'uniquetest');
const uniqueIndexEngine = new IndexEngine(TEST_DB, 'uniquetest', storage);
await uniqueIndexEngine.createIndex({ email: 1 }, { unique: true });
// Insert first document
const doc1 = { _id: new ObjectId(), email: 'test@example.com', name: 'Test' };
const stored1 = await storage.insertOne(TEST_DB, 'uniquetest', doc1);
await uniqueIndexEngine.onInsert(stored1);
// Try to insert duplicate
const doc2 = { _id: new ObjectId(), email: 'test@example.com', name: 'Test2' };
const stored2 = await storage.insertOne(TEST_DB, 'uniquetest', doc2);
let threw = false;
try {
await uniqueIndexEngine.onInsert(stored2);
} catch (e: any) {
threw = true;
expect(e.message).toContain('duplicate key');
}
expect(threw).toBeTrue();
});
// ============================================================================
// Sparse Index Tests
// ============================================================================
tap.test('indexengine: sparse index should not include documents without the field', async () => {
// Create fresh collection
await storage.dropCollection(TEST_DB, 'sparsetest');
await storage.createCollection(TEST_DB, 'sparsetest');
const sparseIndexEngine = new IndexEngine(TEST_DB, 'sparsetest', storage);
await sparseIndexEngine.createIndex({ optionalField: 1 }, { sparse: true });
// Insert doc with the field
const doc1 = { _id: new ObjectId(), optionalField: 'hasValue', name: 'HasField' };
const stored1 = await storage.insertOne(TEST_DB, 'sparsetest', doc1);
await sparseIndexEngine.onInsert(stored1);
// Insert doc without the field
const doc2 = { _id: new ObjectId(), name: 'NoField' };
const stored2 = await storage.insertOne(TEST_DB, 'sparsetest', doc2);
await sparseIndexEngine.onInsert(stored2);
// Search for documents with the field
const candidates = await sparseIndexEngine.findCandidateIds({ optionalField: 'hasValue' });
expect(candidates).toBeTruthy();
expect(candidates!.size).toEqual(1);
expect(candidates!.has(stored1._id.toHexString())).toBeTrue();
});
// ============================================================================
// Cleanup
// ============================================================================
tap.test('indexengine: cleanup', async () => {
await storage.close();
expect(true).toBeTrue();
});
export default tap.start();

View File

@@ -0,0 +1,273 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import * as smartmongo from '../ts/index.js';
const { QueryPlanner, IndexEngine, MemoryStorageAdapter, ObjectId } = smartmongo.tsmdb;
let storage: InstanceType<typeof MemoryStorageAdapter>;
let indexEngine: InstanceType<typeof IndexEngine>;
let queryPlanner: InstanceType<typeof QueryPlanner>;
const TEST_DB = 'testdb';
const TEST_COLL = 'testcoll';
// ============================================================================
// Setup
// ============================================================================
tap.test('queryplanner: should create QueryPlanner instance', async () => {
storage = new MemoryStorageAdapter();
await storage.initialize();
await storage.createCollection(TEST_DB, TEST_COLL);
indexEngine = new IndexEngine(TEST_DB, TEST_COLL, storage);
queryPlanner = new QueryPlanner(indexEngine);
expect(queryPlanner).toBeTruthy();
});
tap.test('queryplanner: should insert test documents', async () => {
// Insert test documents
const docs = [
{ _id: new ObjectId(), name: 'Alice', age: 25, city: 'NYC', category: 'A' },
{ _id: new ObjectId(), name: 'Bob', age: 30, city: 'LA', category: 'B' },
{ _id: new ObjectId(), name: 'Charlie', age: 35, city: 'NYC', category: 'A' },
{ _id: new ObjectId(), name: 'Diana', age: 28, city: 'Chicago', category: 'C' },
{ _id: new ObjectId(), name: 'Eve', age: 32, city: 'LA', category: 'B' },
];
for (const doc of docs) {
await storage.insertOne(TEST_DB, TEST_COLL, doc);
}
});
// ============================================================================
// Basic Plan Tests
// ============================================================================
tap.test('queryplanner: empty filter should result in COLLSCAN', async () => {
const plan = await queryPlanner.plan({});
expect(plan.type).toEqual('COLLSCAN');
expect(plan.indexCovering).toBeFalse();
expect(plan.selectivity).toEqual(1.0);
expect(plan.explanation).toContain('No filter');
});
tap.test('queryplanner: null filter should result in COLLSCAN', async () => {
const plan = await queryPlanner.plan(null as any);
expect(plan.type).toEqual('COLLSCAN');
});
tap.test('queryplanner: filter with no matching index should result in COLLSCAN', async () => {
const plan = await queryPlanner.plan({ nonExistentField: 'value' });
expect(plan.type).toEqual('COLLSCAN');
expect(plan.explanation).toContain('No suitable index');
});
// ============================================================================
// Index Scan Tests (with indexes)
// ============================================================================
tap.test('queryplanner: should create test indexes', async () => {
await indexEngine.createIndex({ age: 1 }, { name: 'age_1' });
await indexEngine.createIndex({ name: 1 }, { name: 'name_1' });
await indexEngine.createIndex({ city: 1, category: 1 }, { name: 'city_category_1' });
const indexes = await indexEngine.listIndexes();
expect(indexes.length).toBeGreaterThanOrEqual(4); // _id_ + 3 created
});
tap.test('queryplanner: simple equality filter should use IXSCAN', async () => {
const plan = await queryPlanner.plan({ age: 30 });
expect(plan.type).toEqual('IXSCAN');
expect(plan.indexName).toEqual('age_1');
expect(plan.indexFieldsUsed).toContain('age');
expect(plan.usesRange).toBeFalse();
});
tap.test('queryplanner: $eq operator should use IXSCAN', async () => {
const plan = await queryPlanner.plan({ name: { $eq: 'Alice' } });
expect(plan.type).toEqual('IXSCAN');
expect(plan.indexName).toEqual('name_1');
expect(plan.indexFieldsUsed).toContain('name');
});
tap.test('queryplanner: range filter ($gt) should use IXSCAN_RANGE', async () => {
const plan = await queryPlanner.plan({ age: { $gt: 25 } });
expect(plan.type).toEqual('IXSCAN_RANGE');
expect(plan.indexName).toEqual('age_1');
expect(plan.usesRange).toBeTrue();
});
tap.test('queryplanner: range filter ($lt) should use IXSCAN_RANGE', async () => {
const plan = await queryPlanner.plan({ age: { $lt: 35 } });
expect(plan.type).toEqual('IXSCAN_RANGE');
expect(plan.usesRange).toBeTrue();
});
tap.test('queryplanner: range filter ($gte, $lte) should use IXSCAN_RANGE', async () => {
const plan = await queryPlanner.plan({ age: { $gte: 25, $lte: 35 } });
expect(plan.type).toEqual('IXSCAN_RANGE');
expect(plan.usesRange).toBeTrue();
});
tap.test('queryplanner: $in operator should use IXSCAN', async () => {
const plan = await queryPlanner.plan({ age: { $in: [25, 30, 35] } });
expect(plan.type).toEqual('IXSCAN');
expect(plan.indexName).toEqual('age_1');
});
// ============================================================================
// Compound Index Tests
// ============================================================================
tap.test('queryplanner: compound index - first field equality should use index', async () => {
const plan = await queryPlanner.plan({ city: 'NYC' });
expect(plan.type).toEqual('IXSCAN');
expect(plan.indexName).toEqual('city_category_1');
expect(plan.indexFieldsUsed).toContain('city');
});
tap.test('queryplanner: compound index - both fields should use full index', async () => {
const plan = await queryPlanner.plan({ city: 'NYC', category: 'A' });
expect(plan.type).toEqual('IXSCAN');
expect(plan.indexName).toEqual('city_category_1');
expect(plan.indexFieldsUsed).toContain('city');
expect(plan.indexFieldsUsed).toContain('category');
expect(plan.indexFieldsUsed.length).toEqual(2);
});
// ============================================================================
// Selectivity Tests
// ============================================================================
tap.test('queryplanner: equality query should have low selectivity', async () => {
const plan = await queryPlanner.plan({ age: 30 });
expect(plan.selectivity).toBeLessThan(0.1);
});
tap.test('queryplanner: range query should have moderate selectivity', async () => {
const plan = await queryPlanner.plan({ age: { $gt: 25 } });
expect(plan.selectivity).toBeGreaterThan(0);
expect(plan.selectivity).toBeLessThan(1);
});
tap.test('queryplanner: $in query selectivity depends on array size', async () => {
const smallInPlan = await queryPlanner.plan({ age: { $in: [25] } });
const largeInPlan = await queryPlanner.plan({ age: { $in: [25, 26, 27, 28, 29, 30] } });
// Larger $in should have higher selectivity (less selective = more documents)
expect(largeInPlan.selectivity).toBeGreaterThanOrEqual(smallInPlan.selectivity);
});
// ============================================================================
// Index Covering Tests
// ============================================================================
tap.test('queryplanner: query covering all filter fields should be index covering', async () => {
const plan = await queryPlanner.plan({ age: 30 });
// All filter fields are covered by the index
expect(plan.indexCovering).toBeTrue();
});
tap.test('queryplanner: query with residual filter should not be index covering', async () => {
const plan = await queryPlanner.plan({ city: 'NYC', name: 'Alice' });
// 'name' is not in the compound index city_category, so it's residual
expect(plan.indexCovering).toBeFalse();
expect(plan.residualFilter).toBeTruthy();
});
// ============================================================================
// Explain Tests
// ============================================================================
tap.test('queryplanner: explain should return detailed plan info', async () => {
const explanation = await queryPlanner.explain({ age: 30 });
expect(explanation.queryPlanner).toBeTruthy();
expect(explanation.queryPlanner.plannerVersion).toEqual(1);
expect(explanation.queryPlanner.winningPlan).toBeTruthy();
expect(explanation.queryPlanner.rejectedPlans).toBeArray();
});
tap.test('queryplanner: explain should include winning and rejected plans', async () => {
const explanation = await queryPlanner.explain({ age: 30 });
expect(explanation.queryPlanner.winningPlan.type).toBeTruthy();
expect(explanation.queryPlanner.rejectedPlans.length).toBeGreaterThan(0);
});
tap.test('queryplanner: explain winning plan should be the best plan', async () => {
const explanation = await queryPlanner.explain({ age: 30 });
// Winning plan should use an index, not collection scan (if index exists)
expect(explanation.queryPlanner.winningPlan.type).toEqual('IXSCAN');
// There should be a COLLSCAN in rejected plans
const hasCOLLSCAN = explanation.queryPlanner.rejectedPlans.some(p => p.type === 'COLLSCAN');
expect(hasCOLLSCAN).toBeTrue();
});
// ============================================================================
// $and Operator Tests
// ============================================================================
tap.test('queryplanner: $and conditions should be analyzed', async () => {
const plan = await queryPlanner.plan({
$and: [
{ age: { $gte: 25 } },
{ age: { $lte: 35 } },
],
});
expect(plan.type).toEqual('IXSCAN_RANGE');
expect(plan.indexName).toEqual('age_1');
});
// ============================================================================
// Edge Cases
// ============================================================================
tap.test('queryplanner: should handle complex nested operators', async () => {
const plan = await queryPlanner.plan({
age: { $gte: 20, $lte: 40 },
city: 'NYC',
});
expect(plan).toBeTruthy();
expect(plan.type).not.toBeUndefined();
});
tap.test('queryplanner: should handle $exists operator', async () => {
await indexEngine.createIndex({ email: 1 }, { name: 'email_1', sparse: true });
const plan = await queryPlanner.plan({ email: { $exists: true } });
// $exists can use sparse indexes
expect(plan).toBeTruthy();
});
// ============================================================================
// Cleanup
// ============================================================================
tap.test('queryplanner: cleanup', async () => {
await storage.close();
expect(true).toBeTrue();
});
export default tap.start();

361
test/test.tsmdb.session.ts Normal file
View File

@@ -0,0 +1,361 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import * as smartmongo from '../ts/index.js';
const { SessionEngine } = smartmongo.tsmdb;
let sessionEngine: InstanceType<typeof SessionEngine>;
// ============================================================================
// Setup
// ============================================================================
tap.test('session: should create SessionEngine instance', async () => {
sessionEngine = new SessionEngine({
sessionTimeoutMs: 1000, // 1 second for testing
cleanupIntervalMs: 10000, // 10 seconds to avoid cleanup during tests
});
expect(sessionEngine).toBeTruthy();
});
// ============================================================================
// Session Lifecycle Tests
// ============================================================================
tap.test('session: startSession should create session with auto-generated ID', async () => {
const session = sessionEngine.startSession();
expect(session).toBeTruthy();
expect(session.id).toBeTruthy();
expect(session.id.length).toBeGreaterThanOrEqual(32); // UUID hex string (32 or 36 with hyphens)
expect(session.createdAt).toBeGreaterThan(0);
expect(session.lastActivityAt).toBeGreaterThan(0);
expect(session.inTransaction).toBeFalse();
});
tap.test('session: startSession should create session with specified ID', async () => {
const customId = 'custom-session-id-12345';
const session = sessionEngine.startSession(customId);
expect(session.id).toEqual(customId);
});
tap.test('session: startSession should create session with metadata', async () => {
const metadata = { client: 'test-client', version: '1.0' };
const session = sessionEngine.startSession(undefined, metadata);
expect(session.metadata).toBeTruthy();
expect(session.metadata!.client).toEqual('test-client');
expect(session.metadata!.version).toEqual('1.0');
});
tap.test('session: getSession should return session by ID', async () => {
const created = sessionEngine.startSession('get-session-test');
const retrieved = sessionEngine.getSession('get-session-test');
expect(retrieved).toBeTruthy();
expect(retrieved!.id).toEqual('get-session-test');
expect(retrieved!.id).toEqual(created.id);
});
tap.test('session: getSession should return undefined for non-existent session', async () => {
const session = sessionEngine.getSession('non-existent-session-id');
expect(session).toBeUndefined();
});
tap.test('session: touchSession should update lastActivityAt', async () => {
const session = sessionEngine.startSession('touch-test-session');
const originalLastActivity = session.lastActivityAt;
// Wait a bit to ensure time difference
await new Promise(resolve => setTimeout(resolve, 10));
const touched = sessionEngine.touchSession('touch-test-session');
expect(touched).toBeTrue();
const updated = sessionEngine.getSession('touch-test-session');
expect(updated!.lastActivityAt).toBeGreaterThanOrEqual(originalLastActivity);
});
tap.test('session: touchSession should return false for non-existent session', async () => {
const touched = sessionEngine.touchSession('non-existent-touch-session');
expect(touched).toBeFalse();
});
tap.test('session: endSession should remove the session', async () => {
sessionEngine.startSession('end-session-test');
expect(sessionEngine.getSession('end-session-test')).toBeTruthy();
const ended = await sessionEngine.endSession('end-session-test');
expect(ended).toBeTrue();
expect(sessionEngine.getSession('end-session-test')).toBeUndefined();
});
tap.test('session: endSession should return false for non-existent session', async () => {
const ended = await sessionEngine.endSession('non-existent-end-session');
expect(ended).toBeFalse();
});
// ============================================================================
// Session Expiry Tests
// ============================================================================
tap.test('session: isSessionExpired should return false for fresh session', async () => {
const session = sessionEngine.startSession('fresh-session');
const isExpired = sessionEngine.isSessionExpired(session);
expect(isExpired).toBeFalse();
});
tap.test('session: isSessionExpired should return true for old session', async () => {
// Create a session with old lastActivityAt
const session = sessionEngine.startSession('old-session');
// Manually set lastActivityAt to old value (sessionTimeoutMs is 1000ms)
(session as any).lastActivityAt = Date.now() - 2000;
const isExpired = sessionEngine.isSessionExpired(session);
expect(isExpired).toBeTrue();
});
tap.test('session: getSession should return undefined for expired session', async () => {
const session = sessionEngine.startSession('expiring-session');
// Manually expire the session
(session as any).lastActivityAt = Date.now() - 2000;
const retrieved = sessionEngine.getSession('expiring-session');
expect(retrieved).toBeUndefined();
});
// ============================================================================
// Transaction Integration Tests
// ============================================================================
tap.test('session: startTransaction should mark session as in transaction', async () => {
sessionEngine.startSession('txn-session-1');
const started = sessionEngine.startTransaction('txn-session-1', 'txn-id-1', 1);
expect(started).toBeTrue();
const session = sessionEngine.getSession('txn-session-1');
expect(session!.inTransaction).toBeTrue();
expect(session!.txnId).toEqual('txn-id-1');
expect(session!.txnNumber).toEqual(1);
});
tap.test('session: startTransaction should return false for non-existent session', async () => {
const started = sessionEngine.startTransaction('non-existent-txn-session', 'txn-id');
expect(started).toBeFalse();
});
tap.test('session: endTransaction should clear transaction state', async () => {
sessionEngine.startSession('txn-session-2');
sessionEngine.startTransaction('txn-session-2', 'txn-id-2');
const ended = sessionEngine.endTransaction('txn-session-2');
expect(ended).toBeTrue();
const session = sessionEngine.getSession('txn-session-2');
expect(session!.inTransaction).toBeFalse();
expect(session!.txnId).toBeUndefined();
expect(session!.txnNumber).toBeUndefined();
});
tap.test('session: endTransaction should return false for non-existent session', async () => {
const ended = sessionEngine.endTransaction('non-existent-end-txn-session');
expect(ended).toBeFalse();
});
tap.test('session: getTransactionId should return transaction ID', async () => {
sessionEngine.startSession('txn-id-session');
sessionEngine.startTransaction('txn-id-session', 'my-txn-id');
const txnId = sessionEngine.getTransactionId('txn-id-session');
expect(txnId).toEqual('my-txn-id');
});
tap.test('session: getTransactionId should return undefined for session without transaction', async () => {
sessionEngine.startSession('no-txn-session');
const txnId = sessionEngine.getTransactionId('no-txn-session');
expect(txnId).toBeUndefined();
});
tap.test('session: getTransactionId should return undefined for non-existent session', async () => {
const txnId = sessionEngine.getTransactionId('non-existent-txn-id-session');
expect(txnId).toBeUndefined();
});
tap.test('session: isInTransaction should return correct state', async () => {
sessionEngine.startSession('in-txn-check-session');
expect(sessionEngine.isInTransaction('in-txn-check-session')).toBeFalse();
sessionEngine.startTransaction('in-txn-check-session', 'txn-check');
expect(sessionEngine.isInTransaction('in-txn-check-session')).toBeTrue();
sessionEngine.endTransaction('in-txn-check-session');
expect(sessionEngine.isInTransaction('in-txn-check-session')).toBeFalse();
});
tap.test('session: isInTransaction should return false for non-existent session', async () => {
expect(sessionEngine.isInTransaction('non-existent-in-txn-session')).toBeFalse();
});
// ============================================================================
// Session Listing Tests
// ============================================================================
tap.test('session: listSessions should return all active sessions', async () => {
// Close and recreate to have a clean slate
sessionEngine.close();
sessionEngine = new SessionEngine({
sessionTimeoutMs: 10000,
cleanupIntervalMs: 60000,
});
sessionEngine.startSession('list-session-1');
sessionEngine.startSession('list-session-2');
sessionEngine.startSession('list-session-3');
const sessions = sessionEngine.listSessions();
expect(sessions.length).toEqual(3);
});
tap.test('session: listSessions should not include expired sessions', async () => {
const session = sessionEngine.startSession('expired-list-session');
// Expire the session
(session as any).lastActivityAt = Date.now() - 20000;
const sessions = sessionEngine.listSessions();
const found = sessions.find(s => s.id === 'expired-list-session');
expect(found).toBeUndefined();
});
tap.test('session: getSessionCount should return correct count', async () => {
const count = sessionEngine.getSessionCount();
expect(count).toBeGreaterThanOrEqual(3); // We created 3 sessions above
});
tap.test('session: getSessionsWithTransactions should filter correctly', async () => {
// Clean slate
sessionEngine.close();
sessionEngine = new SessionEngine({
sessionTimeoutMs: 10000,
cleanupIntervalMs: 60000,
});
sessionEngine.startSession('no-txn-1');
sessionEngine.startSession('no-txn-2');
sessionEngine.startSession('with-txn-1');
sessionEngine.startSession('with-txn-2');
sessionEngine.startTransaction('with-txn-1', 'txn-a');
sessionEngine.startTransaction('with-txn-2', 'txn-b');
const txnSessions = sessionEngine.getSessionsWithTransactions();
expect(txnSessions.length).toEqual(2);
expect(txnSessions.every(s => s.inTransaction)).toBeTrue();
});
// ============================================================================
// getOrCreateSession Tests
// ============================================================================
tap.test('session: getOrCreateSession should create if missing', async () => {
const session = sessionEngine.getOrCreateSession('get-or-create-new');
expect(session).toBeTruthy();
expect(session.id).toEqual('get-or-create-new');
});
tap.test('session: getOrCreateSession should return existing session', async () => {
const created = sessionEngine.startSession('get-or-create-existing');
const retrieved = sessionEngine.getOrCreateSession('get-or-create-existing');
expect(retrieved.id).toEqual(created.id);
expect(retrieved.createdAt).toEqual(created.createdAt);
});
tap.test('session: getOrCreateSession should touch existing session', async () => {
const session = sessionEngine.startSession('get-or-create-touch');
const originalLastActivity = session.lastActivityAt;
await new Promise(resolve => setTimeout(resolve, 10));
sessionEngine.getOrCreateSession('get-or-create-touch');
const updated = sessionEngine.getSession('get-or-create-touch');
expect(updated!.lastActivityAt).toBeGreaterThanOrEqual(originalLastActivity);
});
// ============================================================================
// extractSessionId Static Method Tests
// ============================================================================
tap.test('session: extractSessionId should handle UUID object', async () => {
const { ObjectId } = smartmongo.tsmdb;
const uuid = new smartmongo.tsmdb.plugins.bson.UUID();
const lsid = { id: uuid };
const extracted = SessionEngine.extractSessionId(lsid);
expect(extracted).toEqual(uuid.toHexString());
});
tap.test('session: extractSessionId should handle string ID', async () => {
const lsid = { id: 'string-session-id' };
const extracted = SessionEngine.extractSessionId(lsid);
expect(extracted).toEqual('string-session-id');
});
tap.test('session: extractSessionId should handle binary format', async () => {
const binaryData = Buffer.from('test-binary-uuid', 'utf8').toString('base64');
const lsid = { id: { $binary: { base64: binaryData } } };
const extracted = SessionEngine.extractSessionId(lsid);
expect(extracted).toBeTruthy();
expect(typeof extracted).toEqual('string');
});
tap.test('session: extractSessionId should return undefined for null/undefined', async () => {
expect(SessionEngine.extractSessionId(null)).toBeUndefined();
expect(SessionEngine.extractSessionId(undefined)).toBeUndefined();
});
tap.test('session: extractSessionId should return undefined for empty object', async () => {
expect(SessionEngine.extractSessionId({})).toBeUndefined();
});
// ============================================================================
// refreshSession Tests
// ============================================================================
tap.test('session: refreshSession should update lastActivityAt', async () => {
const session = sessionEngine.startSession('refresh-session-test');
const originalLastActivity = session.lastActivityAt;
await new Promise(resolve => setTimeout(resolve, 10));
const refreshed = sessionEngine.refreshSession('refresh-session-test');
expect(refreshed).toBeTrue();
const updated = sessionEngine.getSession('refresh-session-test');
expect(updated!.lastActivityAt).toBeGreaterThanOrEqual(originalLastActivity);
});
tap.test('session: refreshSession should return false for non-existent session', async () => {
const refreshed = sessionEngine.refreshSession('non-existent-refresh-session');
expect(refreshed).toBeFalse();
});
// ============================================================================
// Cleanup
// ============================================================================
tap.test('session: close should clear all sessions', async () => {
sessionEngine.startSession('close-test-session');
expect(sessionEngine.getSessionCount()).toBeGreaterThan(0);
sessionEngine.close();
expect(sessionEngine.getSessionCount()).toEqual(0);
});
export default tap.start();

411
test/test.tsmdb.wal.ts Normal file
View File

@@ -0,0 +1,411 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import * as smartmongo from '../ts/index.js';
import * as path from 'path';
import * as fs from 'fs/promises';
const { WAL, ObjectId } = smartmongo.tsmdb;
let wal: InstanceType<typeof WAL>;
const TEST_WAL_PATH = '/tmp/tsmdb-test-wal/test.wal';
// Helper to clean up test files
async function cleanupTestFiles() {
try {
await fs.rm('/tmp/tsmdb-test-wal', { recursive: true, force: true });
} catch {
// Ignore if doesn't exist
}
}
// ============================================================================
// Setup
// ============================================================================
tap.test('wal: cleanup before tests', async () => {
await cleanupTestFiles();
});
tap.test('wal: should create WAL instance', async () => {
wal = new WAL(TEST_WAL_PATH, { checkpointInterval: 100 });
expect(wal).toBeTruthy();
});
tap.test('wal: should initialize WAL', async () => {
const result = await wal.initialize();
expect(result).toBeTruthy();
expect(result.recoveredEntries).toBeArray();
expect(result.recoveredEntries.length).toEqual(0);
});
// ============================================================================
// LSN Tests
// ============================================================================
tap.test('wal: getCurrentLsn should return 0 initially', async () => {
const lsn = wal.getCurrentLsn();
expect(lsn).toEqual(0);
});
tap.test('wal: LSN should increment after logging', async () => {
const doc = { _id: new ObjectId(), name: 'Test' };
const lsn = await wal.logInsert('testdb', 'testcoll', doc as any);
expect(lsn).toEqual(1);
expect(wal.getCurrentLsn()).toEqual(1);
});
// ============================================================================
// Insert Logging Tests
// ============================================================================
tap.test('wal: logInsert should create entry with correct structure', async () => {
const doc = { _id: new ObjectId(), name: 'InsertTest', value: 42 };
const lsn = await wal.logInsert('testdb', 'insertcoll', doc as any);
expect(lsn).toBeGreaterThan(0);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry).toBeTruthy();
expect(entry!.operation).toEqual('insert');
expect(entry!.dbName).toEqual('testdb');
expect(entry!.collName).toEqual('insertcoll');
expect(entry!.documentId).toEqual(doc._id.toHexString());
expect(entry!.data).toBeTruthy();
expect(entry!.timestamp).toBeGreaterThan(0);
expect(entry!.checksum).toBeGreaterThan(0);
});
tap.test('wal: logInsert with transaction ID', async () => {
const doc = { _id: new ObjectId(), name: 'TxnInsertTest' };
const lsn = await wal.logInsert('testdb', 'insertcoll', doc as any, 'txn-123');
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry!.txnId).toEqual('txn-123');
});
// ============================================================================
// Update Logging Tests
// ============================================================================
tap.test('wal: logUpdate should store old and new document', async () => {
const oldDoc = { _id: new ObjectId(), name: 'OldName', value: 1 };
const newDoc = { ...oldDoc, name: 'NewName', value: 2 };
const lsn = await wal.logUpdate('testdb', 'updatecoll', oldDoc as any, newDoc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry).toBeTruthy();
expect(entry!.operation).toEqual('update');
expect(entry!.data).toBeTruthy();
expect(entry!.previousData).toBeTruthy();
expect(entry!.documentId).toEqual(oldDoc._id.toHexString());
});
// ============================================================================
// Delete Logging Tests
// ============================================================================
tap.test('wal: logDelete should record deleted document', async () => {
const doc = { _id: new ObjectId(), name: 'ToDelete' };
const lsn = await wal.logDelete('testdb', 'deletecoll', doc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry).toBeTruthy();
expect(entry!.operation).toEqual('delete');
expect(entry!.previousData).toBeTruthy();
expect(entry!.data).toBeUndefined();
});
// ============================================================================
// Transaction Logging Tests
// ============================================================================
tap.test('wal: logBeginTransaction should create begin entry', async () => {
const lsn = await wal.logBeginTransaction('txn-begin-test');
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry).toBeTruthy();
expect(entry!.operation).toEqual('begin');
expect(entry!.txnId).toEqual('txn-begin-test');
});
tap.test('wal: logCommitTransaction should create commit entry', async () => {
const lsn = await wal.logCommitTransaction('txn-commit-test');
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry).toBeTruthy();
expect(entry!.operation).toEqual('commit');
expect(entry!.txnId).toEqual('txn-commit-test');
});
tap.test('wal: logAbortTransaction should create abort entry', async () => {
const lsn = await wal.logAbortTransaction('txn-abort-test');
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry).toBeTruthy();
expect(entry!.operation).toEqual('abort');
expect(entry!.txnId).toEqual('txn-abort-test');
});
// ============================================================================
// getTransactionEntries Tests
// ============================================================================
tap.test('wal: getTransactionEntries should return entries for transaction', async () => {
// Log a complete transaction
const txnId = 'txn-entries-test';
await wal.logBeginTransaction(txnId);
const doc1 = { _id: new ObjectId(), name: 'TxnDoc1' };
await wal.logInsert('testdb', 'txncoll', doc1 as any, txnId);
const doc2 = { _id: new ObjectId(), name: 'TxnDoc2' };
await wal.logInsert('testdb', 'txncoll', doc2 as any, txnId);
await wal.logCommitTransaction(txnId);
const entries = wal.getTransactionEntries(txnId);
expect(entries.length).toEqual(4); // begin + 2 inserts + commit
expect(entries.every(e => e.txnId === txnId)).toBeTrue();
});
tap.test('wal: getTransactionEntries should return empty for unknown transaction', async () => {
const entries = wal.getTransactionEntries('unknown-txn-id');
expect(entries.length).toEqual(0);
});
// ============================================================================
// getEntriesAfter Tests
// ============================================================================
tap.test('wal: getEntriesAfter should filter by LSN', async () => {
const currentLsn = wal.getCurrentLsn();
// Add more entries
const doc = { _id: new ObjectId(), name: 'AfterTest' };
await wal.logInsert('testdb', 'aftercoll', doc as any);
const entries = wal.getEntriesAfter(currentLsn);
expect(entries.length).toEqual(1);
expect(entries[0].lsn).toBeGreaterThan(currentLsn);
});
tap.test('wal: getEntriesAfter with LSN 0 should return all entries', async () => {
const entries = wal.getEntriesAfter(0);
expect(entries.length).toBeGreaterThan(0);
expect(entries.length).toEqual(wal.getCurrentLsn());
});
// ============================================================================
// Checkpoint Tests
// ============================================================================
tap.test('wal: checkpoint should create checkpoint entry', async () => {
const lsn = await wal.checkpoint();
expect(lsn).toBeGreaterThan(0);
// After checkpoint, getEntriesAfter(checkpoint) should be limited
const entries = wal.getEntriesAfter(0);
expect(entries.some(e => e.operation === 'checkpoint')).toBeTrue();
});
// ============================================================================
// Document Recovery Tests
// ============================================================================
tap.test('wal: recoverDocument should deserialize document from entry', async () => {
const originalDoc = { _id: new ObjectId(), name: 'RecoverTest', nested: { a: 1, b: 2 } };
const lsn = await wal.logInsert('testdb', 'recovercoll', originalDoc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const recovered = wal.recoverDocument(entry!);
expect(recovered).toBeTruthy();
expect(recovered!.name).toEqual('RecoverTest');
expect(recovered!.nested.a).toEqual(1);
expect(recovered!.nested.b).toEqual(2);
});
tap.test('wal: recoverDocument should return null for entry without data', async () => {
const lsn = await wal.logBeginTransaction('recover-no-data');
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const recovered = wal.recoverDocument(entry!);
expect(recovered).toBeNull();
});
tap.test('wal: recoverPreviousDocument should deserialize previous state', async () => {
const oldDoc = { _id: new ObjectId(), name: 'Old', value: 100 };
const newDoc = { ...oldDoc, name: 'New', value: 200 };
const lsn = await wal.logUpdate('testdb', 'recovercoll', oldDoc as any, newDoc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const previous = wal.recoverPreviousDocument(entry!);
expect(previous).toBeTruthy();
expect(previous!.name).toEqual('Old');
expect(previous!.value).toEqual(100);
});
tap.test('wal: recoverPreviousDocument should return null for insert entry', async () => {
const doc = { _id: new ObjectId(), name: 'NoPrevious' };
const lsn = await wal.logInsert('testdb', 'recovercoll', doc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const previous = wal.recoverPreviousDocument(entry!);
expect(previous).toBeNull();
});
// ============================================================================
// WAL Persistence and Recovery Tests
// ============================================================================
tap.test('wal: should persist and recover entries', async () => {
// Close current WAL
await wal.close();
// Create new WAL instance and initialize (should recover)
const wal2 = new WAL(TEST_WAL_PATH, { checkpointInterval: 100 });
const result = await wal2.initialize();
// Should have recovered entries
expect(result.recoveredEntries).toBeArray();
// After checkpoint, there might not be many recoverable entries
// but getCurrentLsn should be preserved or reset
await wal2.close();
});
// ============================================================================
// Entry Checksum Tests
// ============================================================================
tap.test('wal: entries should have valid checksums', async () => {
wal = new WAL(TEST_WAL_PATH + '.checksum', { checkpointInterval: 100 });
await wal.initialize();
const doc = { _id: new ObjectId(), name: 'ChecksumTest' };
const lsn = await wal.logInsert('testdb', 'checksumcoll', doc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
expect(entry!.checksum).toBeGreaterThan(0);
expect(typeof entry!.checksum).toEqual('number');
});
// ============================================================================
// Edge Cases
// ============================================================================
tap.test('wal: should handle special characters in document', async () => {
const doc = {
_id: new ObjectId(),
name: 'Test\nWith\tSpecial\r\nChars',
emoji: '🎉',
unicode: '日本語',
};
const lsn = await wal.logInsert('testdb', 'specialcoll', doc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const recovered = wal.recoverDocument(entry!);
expect(recovered!.name).toEqual('Test\nWith\tSpecial\r\nChars');
expect(recovered!.emoji).toEqual('🎉');
expect(recovered!.unicode).toEqual('日本語');
});
tap.test('wal: should handle binary data in documents', async () => {
const doc = {
_id: new ObjectId(),
binaryField: Buffer.from([0x00, 0xFF, 0x7F, 0x80]),
};
const lsn = await wal.logInsert('testdb', 'binarycoll', doc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const recovered = wal.recoverDocument(entry!);
expect(recovered).toBeTruthy();
});
tap.test('wal: should handle nested documents', async () => {
const doc = {
_id: new ObjectId(),
level1: {
level2: {
level3: {
value: 'deep',
},
},
},
};
const lsn = await wal.logInsert('testdb', 'nestedcoll', doc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const recovered = wal.recoverDocument(entry!);
expect(recovered!.level1.level2.level3.value).toEqual('deep');
});
tap.test('wal: should handle arrays in documents', async () => {
const doc = {
_id: new ObjectId(),
tags: ['a', 'b', 'c'],
numbers: [1, 2, 3],
mixed: [1, 'two', { three: 3 }],
};
const lsn = await wal.logInsert('testdb', 'arraycoll', doc as any);
const entries = wal.getEntriesAfter(lsn - 1);
const entry = entries.find(e => e.lsn === lsn);
const recovered = wal.recoverDocument(entry!);
expect(recovered!.tags).toEqual(['a', 'b', 'c']);
expect(recovered!.numbers).toEqual([1, 2, 3]);
expect(recovered!.mixed[2].three).toEqual(3);
});
// ============================================================================
// Cleanup
// ============================================================================
tap.test('wal: cleanup', async () => {
await wal.close();
await cleanupTestFiles();
expect(true).toBeTrue();
});
export default tap.start();

View File

@@ -3,6 +3,6 @@
*/
export const commitinfo = {
name: '@push.rocks/smartmongo',
version: '4.1.0',
description: 'A module for creating and managing a local MongoDB instance for testing purposes.'
version: '5.1.1',
description: 'A powerful MongoDB toolkit for testing and development with a real memory server, wire-protocol-compatible database server, and zero-config local database.'
}

View File

@@ -1,74 +1,14 @@
import { commitinfo } from './00_commitinfo_data.js';
import * as plugins from './smartmongo.plugins.js';
// Export SmartMongo from ts_mongotools
export { SmartMongo } from './ts_mongotools/index.js';
// Export TsmDB module
export * as tsmdb from './tsmdb/index.js';
export * as tsmdb from './ts_tsmdb/index.js';
export class SmartMongo {
// STATIC
public static async createAndStart(replCountArg: number = 1) {
const smartMongoInstance = new SmartMongo();
await smartMongoInstance.start(replCountArg);
return smartMongoInstance;
}
// Export LocalTsmDb from ts_local
export { LocalTsmDb } from './ts_local/index.js';
export type { ILocalTsmDbOptions, ILocalTsmDbConnectionInfo } from './ts_local/index.js';
// INSTANCE
private _readyDeferred = plugins.smartpromise.defer();
public readyPromise = this._readyDeferred.promise;
public mongoReplicaSet: plugins.mongoPlugin.MongoMemoryReplSet;
constructor() {}
public async start(countArg: number = 1) {
this.mongoReplicaSet = await plugins.mongoPlugin.MongoMemoryReplSet.create({
replSet: { count: countArg },
instanceOpts: [
{
storageEngine: 'wiredTiger',
},
],
});
this._readyDeferred.resolve();
console.log(`mongoReplicaSet with ${countArg} replicas started.`);
console.log(`@pushrocks/smartmongo version ${commitinfo.version}`);
}
/**
* returns a mongo descriptor for modules like
* @pushrocks/smartfile.
*/
public async getMongoDescriptor(): Promise<plugins.smartdata.IMongoDescriptor> {
await this.readyPromise;
return {
mongoDbName: `smartmongo_testdatabase`,
mongoDbUrl: this.mongoReplicaSet.getUri(),
};
}
/**
* stops the smartmongo instance
* and cleans up after itself
*/
public async stop() {
await this.mongoReplicaSet.stop();
await this.mongoReplicaSet.cleanup();
}
/**
* like stop() but allows you to actually store
* the database on disk
*/
public async stopAndDumpToDir(
dirArg: string,
nameFunctionArg?: (doc: any) => string,
emptyDirArg = true,
) {
const mongodumpInstance = new plugins.mongodump.MongoDump();
const mongodumpTarget = await mongodumpInstance.addMongoTargetByMongoDescriptor(
await this.getMongoDescriptor(),
);
await mongodumpTarget.dumpAllCollectionsToDir(dirArg, nameFunctionArg, emptyDirArg);
await mongodumpInstance.stop();
await this.stop();
}
}
// Export commitinfo
export { commitinfo };

View File

@@ -0,0 +1,143 @@
import * as plugins from './plugins.js';
import * as crypto from 'crypto';
import * as path from 'path';
import * as os from 'os';
import { TsmdbServer } from '../ts_tsmdb/index.js';
/**
* Connection information returned by LocalTsmDb.start()
*/
export interface ILocalTsmDbConnectionInfo {
/** The Unix socket file path */
socketPath: string;
/** MongoDB connection URI ready for MongoClient */
connectionUri: string;
}
export interface ILocalTsmDbOptions {
/** Required: where to store data */
folderPath: string;
/** Optional: custom socket path (default: auto-generated in /tmp) */
socketPath?: string;
}
/**
* LocalTsmDb - Lightweight local MongoDB-compatible database using Unix sockets
*
* This class wraps TsmdbServer and provides a simple interface for
* starting a local file-based MongoDB-compatible server. Returns connection
* info that you can use with your own MongoDB driver instance.
*
* @example
* ```typescript
* import { LocalTsmDb } from '@push.rocks/smartmongo';
* import { MongoClient } from 'mongodb';
*
* const db = new LocalTsmDb({ folderPath: './data' });
* const { connectionUri } = await db.start();
*
* // Connect with your own MongoDB client
* const client = new MongoClient(connectionUri, { directConnection: true });
* await client.connect();
*
* // Use the MongoDB client
* const collection = client.db('mydb').collection('users');
* await collection.insertOne({ name: 'Alice' });
*
* // When done
* await client.close();
* await db.stop();
* ```
*/
export class LocalTsmDb {
private options: ILocalTsmDbOptions;
private server: TsmdbServer | null = null;
private generatedSocketPath: string | null = null;
constructor(options: ILocalTsmDbOptions) {
this.options = options;
}
/**
* Generate a unique socket path in /tmp
*/
private generateSocketPath(): string {
const randomId = crypto.randomBytes(8).toString('hex');
return path.join(os.tmpdir(), `smartmongo-${randomId}.sock`);
}
/**
* Start the local TsmDB server and return connection info
*/
async start(): Promise<ILocalTsmDbConnectionInfo> {
if (this.server) {
throw new Error('LocalTsmDb is already running');
}
// Use provided socket path or generate one
this.generatedSocketPath = this.options.socketPath ?? this.generateSocketPath();
this.server = new TsmdbServer({
socketPath: this.generatedSocketPath,
storage: 'file',
storagePath: this.options.folderPath,
});
await this.server.start();
return {
socketPath: this.generatedSocketPath,
connectionUri: this.server.getConnectionUri(),
};
}
/**
* Get connection info (throws if not started)
*/
getConnectionInfo(): ILocalTsmDbConnectionInfo {
if (!this.server || !this.generatedSocketPath) {
throw new Error('LocalTsmDb is not running. Call start() first.');
}
return {
socketPath: this.generatedSocketPath,
connectionUri: this.server.getConnectionUri(),
};
}
/**
* Get the underlying TsmdbServer instance (throws if not started)
*/
getServer(): TsmdbServer {
if (!this.server) {
throw new Error('LocalTsmDb is not running. Call start() first.');
}
return this.server;
}
/**
* Get the connection URI
*/
getConnectionUri(): string {
if (!this.server) {
throw new Error('LocalTsmDb is not running. Call start() first.');
}
return this.server.getConnectionUri();
}
/**
* Check if the server is running
*/
get running(): boolean {
return this.server !== null && this.server.running;
}
/**
* Stop the local TsmDB server
*/
async stop(): Promise<void> {
if (this.server) {
await this.server.stop();
this.server = null;
this.generatedSocketPath = null;
}
}
}

2
ts/ts_local/index.ts Normal file
View File

@@ -0,0 +1,2 @@
export { LocalTsmDb } from './classes.localtsmdb.js';
export type { ILocalTsmDbOptions, ILocalTsmDbConnectionInfo } from './classes.localtsmdb.js';

3
ts/ts_local/plugins.ts Normal file
View File

@@ -0,0 +1,3 @@
import * as smartpromise from '@push.rocks/smartpromise';
export { smartpromise };

View File

@@ -0,0 +1,71 @@
import { commitinfo } from '../00_commitinfo_data.js';
import * as plugins from './plugins.js';
export class SmartMongo {
// STATIC
public static async createAndStart(replCountArg: number = 1) {
const smartMongoInstance = new SmartMongo();
await smartMongoInstance.start(replCountArg);
return smartMongoInstance;
}
// INSTANCE
private _readyDeferred = plugins.smartpromise.defer();
public readyPromise = this._readyDeferred.promise;
public mongoReplicaSet!: plugins.mongoPlugin.MongoMemoryReplSet;
constructor() {}
public async start(countArg: number = 1) {
this.mongoReplicaSet = await plugins.mongoPlugin.MongoMemoryReplSet.create({
replSet: { count: countArg },
instanceOpts: [
{
storageEngine: 'wiredTiger',
},
],
});
this._readyDeferred.resolve();
console.log(`mongoReplicaSet with ${countArg} replicas started.`);
console.log(`@pushrocks/smartmongo version ${commitinfo.version}`);
}
/**
* returns a mongo descriptor for modules like
* @pushrocks/smartfile.
*/
public async getMongoDescriptor(): Promise<plugins.smartdata.IMongoDescriptor> {
await this.readyPromise;
return {
mongoDbName: `smartmongo_testdatabase`,
mongoDbUrl: this.mongoReplicaSet.getUri(),
};
}
/**
* stops the smartmongo instance
* and cleans up after itself
*/
public async stop() {
await this.mongoReplicaSet.stop();
await this.mongoReplicaSet.cleanup();
}
/**
* like stop() but allows you to actually store
* the database on disk
*/
public async stopAndDumpToDir(
dirArg: string,
nameFunctionArg?: (doc: any) => string,
emptyDirArg = true,
) {
const mongodumpInstance = new plugins.mongodump.MongoDump();
const mongodumpTarget = await mongodumpInstance.addMongoTargetByMongoDescriptor(
await this.getMongoDescriptor(),
);
await mongodumpTarget.dumpAllCollectionsToDir(dirArg, nameFunctionArg, emptyDirArg);
await mongodumpInstance.stop();
await this.stop();
}
}

View File

@@ -0,0 +1,2 @@
export * from './plugins.js';
export { SmartMongo } from './classes.smartmongo.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { Document, IStoredDocument, IAggregateOptions } from '../types/interfaces.js';
// Import mingo Aggregator
@@ -47,7 +47,7 @@ export class AggregationEngine {
const cursor = aggregator.stream(documents);
for (const doc of cursor) {
yield doc;
yield doc as Document;
}
}

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
// Simple B-Tree implementation for range queries

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { Document, IStoredDocument, ISortSpecification, ISortDirection } from '../types/interfaces.js';
// Import mingo Query class

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { Document, IStoredDocument } from '../types/interfaces.js';
import { IndexEngine } from './IndexEngine.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { TransactionEngine } from './TransactionEngine.js';
/**

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
import type { Document, IStoredDocument, ITransactionOptions } from '../types/interfaces.js';
import { TsmdbTransactionError, TsmdbWriteConflictError } from '../errors/TsmdbErrors.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { Document, IStoredDocument } from '../types/interfaces.js';
import { QueryEngine } from './QueryEngine.js';

View File

@@ -2,7 +2,7 @@
// Use the official MongoDB driver to connect to TsmdbServer
// Re-export plugins for external use
import * as plugins from './tsmdb.plugins.js';
import * as plugins from './plugins.js';
export { plugins };
// Export BSON types for convenience

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
import type { IParsedCommand } from './WireProtocol.js';
import type { TsmdbServer } from './TsmdbServer.js';

View File

@@ -1,5 +1,6 @@
import * as net from 'net';
import * as plugins from '../tsmdb.plugins.js';
import * as fs from 'fs/promises';
import * as plugins from '../plugins.js';
import { WireProtocol, OP_QUERY } from './WireProtocol.js';
import { CommandRouter } from './CommandRouter.js';
import { MemoryStorageAdapter } from '../storage/MemoryStorageAdapter.js';
@@ -10,10 +11,12 @@ import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
* Server configuration options
*/
export interface ITsmdbServerOptions {
/** Port to listen on (default: 27017) */
/** Port to listen on (default: 27017) - ignored if socketPath is set */
port?: number;
/** Host to bind to (default: 127.0.0.1) */
/** Host to bind to (default: 127.0.0.1) - ignored if socketPath is set */
host?: string;
/** Unix socket path - if set, server listens on socket instead of TCP */
socketPath?: string;
/** Storage type: 'memory' or 'file' (default: 'memory') */
storage?: 'memory' | 'file';
/** Path for file storage (required if storage is 'file') */
@@ -54,7 +57,7 @@ interface IConnectionState {
* ```
*/
export class TsmdbServer {
private options: Required<ITsmdbServerOptions>;
private options: Required<Omit<ITsmdbServerOptions, 'socketPath'>> & { socketPath: string };
private server: net.Server | null = null;
private storage: IStorageAdapter;
private commandRouter: CommandRouter;
@@ -62,11 +65,14 @@ export class TsmdbServer {
private connectionIdCounter = 0;
private isRunning = false;
private startTime: Date = new Date();
private useSocket: boolean;
constructor(options: ITsmdbServerOptions = {}) {
this.useSocket = !!options.socketPath;
this.options = {
port: options.port ?? 27017,
host: options.host ?? '127.0.0.1',
socketPath: options.socketPath ?? '',
storage: options.storage ?? 'memory',
storagePath: options.storagePath ?? './data',
persistPath: options.persistPath ?? '',
@@ -119,6 +125,18 @@ export class TsmdbServer {
// Initialize storage
await this.storage.initialize();
// Clean up stale socket file if using Unix socket
if (this.useSocket && this.options.socketPath) {
try {
await fs.unlink(this.options.socketPath);
} catch (err: any) {
// Ignore ENOENT (file doesn't exist)
if (err.code !== 'ENOENT') {
throw err;
}
}
}
return new Promise((resolve, reject) => {
this.server = net.createServer((socket) => {
this.handleConnection(socket);
@@ -132,11 +150,21 @@ export class TsmdbServer {
}
});
this.server.listen(this.options.port, this.options.host, () => {
this.isRunning = true;
this.startTime = new Date();
resolve();
});
if (this.useSocket && this.options.socketPath) {
// Listen on Unix socket
this.server.listen(this.options.socketPath, () => {
this.isRunning = true;
this.startTime = new Date();
resolve();
});
} else {
// Listen on TCP
this.server.listen(this.options.port, this.options.host, () => {
this.isRunning = true;
this.startTime = new Date();
resolve();
});
}
});
}
@@ -161,9 +189,22 @@ export class TsmdbServer {
await this.storage.close();
return new Promise((resolve) => {
this.server!.close(() => {
this.server!.close(async () => {
this.isRunning = false;
this.server = null;
// Clean up socket file if using Unix socket
if (this.useSocket && this.options.socketPath) {
try {
await fs.unlink(this.options.socketPath);
} catch (err: any) {
// Ignore ENOENT (file doesn't exist)
if (err.code !== 'ENOENT') {
console.error('Failed to remove socket file:', err);
}
}
}
resolve();
});
});
@@ -275,9 +316,21 @@ export class TsmdbServer {
* Get the connection URI for this server
*/
getConnectionUri(): string {
if (this.useSocket && this.options.socketPath) {
// URL-encode the socket path (replace / with %2F)
const encodedPath = encodeURIComponent(this.options.socketPath);
return `mongodb://${encodedPath}`;
}
return `mongodb://${this.options.host}:${this.options.port}`;
}
/**
* Get the socket path (if using Unix socket mode)
*/
get socketPath(): string | undefined {
return this.useSocket ? this.options.socketPath : undefined;
}
/**
* Check if the server is running
*/

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
/**
* MongoDB Wire Protocol Implementation

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
import { SessionEngine } from '../../engine/SessionEngine.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext, ICursorState } from '../CommandRouter.js';
import { AggregationEngine } from '../../engine/AggregationEngine.js';
@@ -297,7 +297,7 @@ export class AggregateHandler implements ICommandHandler {
const existingDocs = await storage.findAll(targetDb, targetColl);
const onFields = Array.isArray(on) ? on : [on];
let matchingDoc = null;
let matchingDoc: plugins.bson.Document | null = null;
for (const existing of existingDocs) {
let matches = true;
for (const field of onFields) {

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
import type { IStoredDocument } from '../../types/interfaces.js';
import { QueryEngine } from '../../engine/QueryEngine.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext, ICursorState } from '../CommandRouter.js';
import type { IStoredDocument } from '../../types/interfaces.js';
import { QueryEngine } from '../../engine/QueryEngine.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
/**

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
import { IndexEngine } from '../../engine/IndexEngine.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
import type { IStoredDocument } from '../../types/interfaces.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../../tsmdb.plugins.js';
import * as plugins from '../../plugins.js';
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
import type { IStoredDocument } from '../../types/interfaces.js';
import { QueryEngine } from '../../engine/QueryEngine.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { IStorageAdapter } from './IStorageAdapter.js';
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
import { calculateDocumentChecksum, verifyChecksum } from '../utils/checksum.js';

View File

@@ -1,4 +1,4 @@
import type * as plugins from '../tsmdb.plugins.js';
import type * as plugins from '../plugins.js';
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
/**

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { IStorageAdapter } from './IStorageAdapter.js';
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { IStorageAdapter } from './IStorageAdapter.js';
import type { IOpLogEntry, Document, IResumeToken, ChangeStreamOperationType } from '../types/interfaces.js';

View File

@@ -1,4 +1,4 @@
import * as plugins from '../tsmdb.plugins.js';
import * as plugins from '../plugins.js';
import type { Document, IStoredDocument } from '../types/interfaces.js';
/**

View File

@@ -1,4 +1,4 @@
import type * as plugins from '../tsmdb.plugins.js';
import type * as plugins from '../plugins.js';
// ============================================================================
// Document Types