Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c65c285296 | |||
| b72174ca7b | |||
| 1ff4d219af | |||
| 09f60de56f | |||
| e6a36ecb5f | |||
| 6a37a773ea | |||
| 1fff277698 | |||
| 0ad7f316c4 | |||
| 0d450e7d4e | |||
| fff77fbd8e | |||
| 678bf15eb4 | |||
| aa45e9579b | |||
| e3dc19aa7c | |||
| 316af45b5e |
65
changelog.md
65
changelog.md
@@ -1,5 +1,70 @@
|
||||
# Changelog
|
||||
|
||||
## 2026-02-03 - 5.1.0 - feat(localtsmdb)
|
||||
export ILocalTsmDbConnectionInfo and expand LocalTsmDb/TsmDB documentation and examples
|
||||
|
||||
- Exported new type ILocalTsmDbConnectionInfo from ts_local (ts/index.ts)
|
||||
- Added LocalTsmDb configuration example, methods table, and ConnectionInfo interface to README
|
||||
- Documented Unix socket vs TCP connection modes and updated usage examples (TCP and socket examples)
|
||||
- Expanded TsmDB docs: additional server properties, aggregation stages, regex examples, index operations, database ops, checksums, and wire protocol commands
|
||||
- Updated architecture notes to include Unix socket support and new engine components (QueryEngine, UpdateEngine, AggregationEngine)
|
||||
|
||||
## 2026-02-03 - 5.0.0 - BREAKING CHANGE(localtsmdb)
|
||||
add Unix socket support and change LocalTsmDb API to return connection info instead of a MongoClient
|
||||
|
||||
- LocalTsmDb.start() now returns ILocalTsmDbConnectionInfo { socketPath, connectionUri } instead of a connected MongoClient
|
||||
- Removed internal MongoClient management: consumers must create/connect/close their own MongoClient using the returned connectionUri (close client before calling db.stop())
|
||||
- Added ILocalTsmDbConnectionInfo type and getConnectionInfo() (replaces getClient())
|
||||
- TsmdbServer: added socketPath option to listen on Unix sockets, cleans up stale socket files on start/stop, and encodes socket paths in getConnectionUri()
|
||||
- LocalTsmDb can auto-generate socket paths in the OS temp dir; LocalTsmDb no longer depends on the mongodb package internally (lightweight Unix socket wrapper)
|
||||
- Updated docs and tests to use MongoClient externally and to demonstrate socketPath/connectionUri workflow
|
||||
- ts_local plugins no longer export net (net usage moved to server implementation)
|
||||
|
||||
## 2026-02-03 - 4.3.0 - feat(docs)
|
||||
add LocalTsmDb documentation and examples; update README code samples and imports; correct examples and variable names; update package author
|
||||
|
||||
- Introduce LocalTsmDb: zero-config local database with automatic persistence, auto port discovery, and pre-connected client (added Quick Start, API, Features, and testing examples).
|
||||
- Expand comparison table to include LocalTsmDb alongside SmartMongo and TsmDB.
|
||||
- Update README examples: new LocalTsmDb usage, reorder options (LocalTsmDb, TsmDB, SmartMongo), rename test DB variable (db -> testDb), and adjust test snippets for Jest/Mocha and tap.
|
||||
- Adjust code snippets and API notes: switch some example imports to use tsmdb, replace FileStorageAdapter references, change planner.createPlan to await planner.plan, and use wal.getEntriesAfter(...) without awaiting.
|
||||
- Update package.json author from 'Lossless GmbH' to 'Task Venture Capital GmbH'.
|
||||
|
||||
## 2026-02-03 - 4.2.1 - fix(package.json)
|
||||
replace main and typings with exports field pointing to ./dist_ts/index.js
|
||||
|
||||
- Added package.json exports field mapping "." to ./dist_ts/index.js to declare the package entrypoint.
|
||||
- Removed main (dist_ts/index.js) and typings (dist_ts/index.d.ts) entries.
|
||||
- Note: switching to exports improves Node resolution but removing the typings entry may affect TypeScript consumers expecting index.d.ts.
|
||||
|
||||
## 2026-02-01 - 4.2.0 - feat(tsmdb)
|
||||
implement TsmDB Mongo-wire-compatible server, add storage/engine modules and reorganize exports
|
||||
|
||||
- Add full TsmDB implementation under ts/ts_tsmdb: wire protocol, server, command router, handlers, engines (Query, Update, Aggregation, Index, Transaction, Session), storage adapters (Memory, File), OpLog, WAL, utils and types.
|
||||
- Remove legacy ts/tsmdb implementation and replace with new ts_tsmdb module exports.
|
||||
- Introduce ts/ts_mongotools module and move SmartMongo class there; update top-level exports in ts/index.ts to export SmartMongo, tsmdb (from ts_tsmdb) and LocalTsmDb.
|
||||
- Add LocalTsmDb convenience class (ts/ts_local) to start a file-backed TsmDB and return a connected MongoClient.
|
||||
- Refactor plugin imports into per-module plugins files and add utilities (checksum, persistence, query planner, index engine).
|
||||
|
||||
## 2026-02-01 - 4.1.1 - fix(tsmdb)
|
||||
add comprehensive unit tests for tsmdb components: checksum, query planner, index engine, session, and WAL
|
||||
|
||||
- Add new tests: test.tsmdb.checksum.ts — CRC32 and document checksum utilities (add/verify/remove)
|
||||
- Add new tests: test.tsmdb.queryplanner.ts — QueryPlanner plans, index usage, selectivity, explain output, and edge cases
|
||||
- Add new tests: test.tsmdb.indexengine.ts — Index creation, unique/sparse options, candidate selection, and constraints
|
||||
- Add new tests: test.tsmdb.session.ts — Session lifecycle, touch/refresh/close, extractSessionId handling
|
||||
- Add new tests: test.tsmdb.wal.ts — WAL initialization, LSN increments, logging/recovery for inserts/updates/deletes, binary and nested data handling
|
||||
- Tests only — no production API changes; increases test coverage
|
||||
- Recommend patch bump from 4.1.0 to 4.1.1
|
||||
|
||||
## 2026-02-01 - 4.1.0 - feat(readme)
|
||||
expand README with storage integrity, WAL, query planner, session & transaction docs; update test script to enable verbose logging and increase timeout
|
||||
|
||||
- Updated npm test script to run tstest with --verbose, --logfile and --timeout 60 to improve test output and avoid timeouts.
|
||||
- Extensive README additions: file storage adapter examples with checksum options, write-ahead logging (WAL) usage and recovery, query planner examples, index and query execution details, session and transaction examples and features.
|
||||
- Wire protocol / features table updated to include Transactions and Sessions and added admin commands (dbStats, collStats).
|
||||
- Architecture diagram and component list updated to include QueryPlanner, SessionEngine, TransactionEngine and WAL; storage layer annotated with checksums and WAL.
|
||||
- Minor example import tweak: MongoClient import now includes Db type in test examples.
|
||||
|
||||
## 2026-02-01 - 4.0.0 - BREAKING CHANGE(storage,engine,server)
|
||||
add session & transaction management, index/query planner, WAL and checksum support; integrate index-accelerated queries and update storage API (findByIds) to enable index optimizations
|
||||
|
||||
|
||||
11
package.json
11
package.json
@@ -1,15 +1,16 @@
|
||||
{
|
||||
"name": "@push.rocks/smartmongo",
|
||||
"version": "4.0.0",
|
||||
"version": "5.1.0",
|
||||
"private": false,
|
||||
"description": "A module for creating and managing a local MongoDB instance for testing purposes.",
|
||||
"main": "dist_ts/index.js",
|
||||
"typings": "dist_ts/index.d.ts",
|
||||
"exports": {
|
||||
".": "./dist_ts/index.js"
|
||||
},
|
||||
"type": "module",
|
||||
"author": "Lossless GmbH",
|
||||
"author": "Task Venture Capital GmbH",
|
||||
"license": "MIT",
|
||||
"scripts": {
|
||||
"test": "(tstest test/)",
|
||||
"test": "(tstest test/. --verbose --logfile --timeout 60)",
|
||||
"build": "(tsbuild --web)",
|
||||
"buildDocs": "tsdoc"
|
||||
},
|
||||
|
||||
474
readme.md
474
readme.md
@@ -1,6 +1,6 @@
|
||||
# @push.rocks/smartmongo
|
||||
|
||||
A powerful MongoDB toolkit for testing and development — featuring both a real MongoDB memory server (**SmartMongo**) and an ultra-fast, lightweight wire-protocol-compatible in-memory database server (**TsmDB**). 🚀
|
||||
A powerful MongoDB toolkit for testing and development — featuring a real MongoDB memory server (**SmartMongo**), an ultra-fast wire-protocol-compatible in-memory database server (**TsmDB**), and a zero-config local database (**LocalTsmDb**). 🚀
|
||||
|
||||
## Install
|
||||
|
||||
@@ -16,21 +16,88 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
|
||||
|
||||
## Overview
|
||||
|
||||
`@push.rocks/smartmongo` provides two powerful approaches for MongoDB in testing and development:
|
||||
`@push.rocks/smartmongo` provides three powerful approaches for MongoDB in testing and development:
|
||||
|
||||
| Feature | SmartMongo | TsmDB |
|
||||
|---------|------------|---------|
|
||||
| **Type** | Real MongoDB (memory server) | Pure TypeScript wire protocol server |
|
||||
| **Speed** | ~2-5s startup | ⚡ Instant startup (~5ms) |
|
||||
| **Compatibility** | 100% MongoDB | MongoDB driver compatible |
|
||||
| **Dependencies** | Downloads MongoDB binary | Zero external dependencies |
|
||||
| **Replication** | ✅ Full replica set support | Single node emulation |
|
||||
| **Use Case** | Integration testing | Unit testing, CI/CD |
|
||||
| **Persistence** | Dump to directory | Optional file/memory persistence |
|
||||
| Feature | SmartMongo | TsmDB | LocalTsmDb |
|
||||
|---------|------------|-------|------------|
|
||||
| **Type** | Real MongoDB (memory server) | Wire protocol server | Zero-config local DB |
|
||||
| **Speed** | ~2-5s startup | ⚡ Instant (~5ms) | ⚡ Instant (Unix socket) |
|
||||
| **Compatibility** | 100% MongoDB | MongoDB driver compatible | MongoDB driver compatible |
|
||||
| **Dependencies** | Downloads MongoDB binary | Zero external deps | Zero external deps (no MongoDB driver!) |
|
||||
| **Connection** | TCP | TCP or Unix socket | Unix socket (default) |
|
||||
| **Replication** | ✅ Full replica set | Single node | Single node |
|
||||
| **Persistence** | Dump to directory | Memory or file | File-based (automatic) |
|
||||
| **Use Case** | Integration testing | Unit testing, CI/CD | Quick prototyping, local dev |
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Option 1: SmartMongo (Real MongoDB)
|
||||
### Option 1: LocalTsmDb (Zero-Config Local Database) ⭐
|
||||
|
||||
The easiest way to get started — just point it at a folder and you have a persistent MongoDB-compatible database using Unix sockets. No port conflicts, no MongoDB driver dependency in LocalTsmDb!
|
||||
|
||||
```typescript
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
|
||||
// Create a local database backed by files
|
||||
const db = new LocalTsmDb({ folderPath: './my-data' });
|
||||
|
||||
// Start and get connection info (Unix socket path + connection URI)
|
||||
const { connectionUri } = await db.start();
|
||||
|
||||
// Connect with your own MongoDB client
|
||||
const client = new MongoClient(connectionUri, { directConnection: true });
|
||||
await client.connect();
|
||||
|
||||
// Use exactly like MongoDB
|
||||
const users = client.db('myapp').collection('users');
|
||||
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
|
||||
|
||||
const user = await users.findOne({ name: 'Alice' });
|
||||
console.log(user); // { _id: ObjectId(...), name: 'Alice', email: 'alice@example.com' }
|
||||
|
||||
// Data persists to disk automatically!
|
||||
await client.close();
|
||||
await db.stop();
|
||||
|
||||
// Later... data is still there
|
||||
const db2 = new LocalTsmDb({ folderPath: './my-data' });
|
||||
const { connectionUri: uri2 } = await db2.start();
|
||||
const client2 = new MongoClient(uri2, { directConnection: true });
|
||||
await client2.connect();
|
||||
const savedUser = await client2.db('myapp').collection('users').findOne({ name: 'Alice' });
|
||||
// savedUser exists!
|
||||
```
|
||||
|
||||
### Option 2: TsmDB (Wire Protocol Server)
|
||||
|
||||
A lightweight, pure TypeScript MongoDB-compatible server — use the official `mongodb` driver directly!
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
|
||||
// Start TsmDB server (TCP mode)
|
||||
const server = new tsmdb.TsmdbServer({ port: 27017 });
|
||||
await server.start();
|
||||
|
||||
// Connect with the official MongoDB driver
|
||||
const client = new MongoClient('mongodb://127.0.0.1:27017');
|
||||
await client.connect();
|
||||
|
||||
// Use exactly like real MongoDB
|
||||
const db = client.db('myapp');
|
||||
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
|
||||
|
||||
const user = await db.collection('users').findOne({ name: 'Alice' });
|
||||
console.log(user); // { _id: ObjectId(...), name: 'Alice', age: 30 }
|
||||
|
||||
// Clean up
|
||||
await client.close();
|
||||
await server.stop();
|
||||
```
|
||||
|
||||
### Option 3: SmartMongo (Real MongoDB)
|
||||
|
||||
Spin up a real MongoDB replica set in memory — perfect for integration tests that need full MongoDB compatibility.
|
||||
|
||||
@@ -51,36 +118,91 @@ console.log(descriptor.mongoDbUrl); // mongodb://127.0.0.1:xxxxx/...
|
||||
await mongo.stop();
|
||||
```
|
||||
|
||||
### Option 2: TsmDB (Wire Protocol Server)
|
||||
---
|
||||
|
||||
A lightweight, pure TypeScript MongoDB-compatible server that speaks the wire protocol — use the official `mongodb` driver directly!
|
||||
## 📖 LocalTsmDb API
|
||||
|
||||
The simplest option for local development and prototyping — lightweight, Unix socket-based, and automatic persistence.
|
||||
|
||||
### Configuration
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
import type { ILocalTsmDbOptions, ILocalTsmDbConnectionInfo } from '@push.rocks/smartmongo';
|
||||
|
||||
// Start TsmDB server
|
||||
const server = new tsmdb.TsmdbServer({ port: 27017 });
|
||||
await server.start();
|
||||
const options: ILocalTsmDbOptions = {
|
||||
folderPath: './data', // Required: where to store data
|
||||
socketPath: '/tmp/my.sock', // Optional: custom socket path (default: auto-generated)
|
||||
};
|
||||
|
||||
// Connect with the official MongoDB driver!
|
||||
const client = new MongoClient('mongodb://127.0.0.1:27017');
|
||||
await client.connect();
|
||||
|
||||
// Use exactly like real MongoDB
|
||||
const db = client.db('myapp');
|
||||
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
|
||||
|
||||
const user = await db.collection('users').findOne({ name: 'Alice' });
|
||||
console.log(user); // { _id: ObjectId(...), name: 'Alice', age: 30 }
|
||||
|
||||
// Clean up
|
||||
await client.close();
|
||||
await server.stop();
|
||||
const db = new LocalTsmDb(options);
|
||||
```
|
||||
|
||||
### Methods
|
||||
|
||||
| Method | Returns | Description |
|
||||
|--------|---------|-------------|
|
||||
| `start()` | `Promise<ILocalTsmDbConnectionInfo>` | Starts the server and returns connection info |
|
||||
| `stop()` | `Promise<void>` | Stops the server and cleans up the socket |
|
||||
| `getConnectionInfo()` | `ILocalTsmDbConnectionInfo` | Returns current connection info |
|
||||
| `getConnectionUri()` | `string` | Returns the MongoDB connection URI |
|
||||
| `getServer()` | `TsmdbServer` | Returns the underlying TsmDB server instance |
|
||||
| `running` | `boolean` | Property indicating if the server is running |
|
||||
|
||||
### Connection Info
|
||||
|
||||
The `start()` method returns an `ILocalTsmDbConnectionInfo` object:
|
||||
|
||||
```typescript
|
||||
interface ILocalTsmDbConnectionInfo {
|
||||
socketPath: string; // The Unix socket file path, e.g., /tmp/smartmongo-abc123.sock
|
||||
connectionUri: string; // MongoDB URI, e.g., mongodb://%2Ftmp%2Fsmartmongo-abc123.sock
|
||||
}
|
||||
```
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```typescript
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
|
||||
const db = new LocalTsmDb({ folderPath: './data' });
|
||||
|
||||
// Start and get connection info
|
||||
const { socketPath, connectionUri } = await db.start();
|
||||
console.log(socketPath); // /tmp/smartmongo-abc123.sock (auto-generated)
|
||||
console.log(connectionUri); // mongodb://%2Ftmp%2Fsmartmongo-abc123.sock
|
||||
|
||||
// Connect with your own MongoDB client
|
||||
const client = new MongoClient(connectionUri, { directConnection: true });
|
||||
await client.connect();
|
||||
|
||||
// Use the client
|
||||
const users = client.db('mydb').collection('users');
|
||||
await users.insertOne({ name: 'Alice' });
|
||||
|
||||
// Check status
|
||||
console.log(db.running); // true
|
||||
|
||||
// Stop when done (close your client first!)
|
||||
await client.close();
|
||||
await db.stop();
|
||||
```
|
||||
|
||||
### Features
|
||||
|
||||
- 🔌 **Unix Sockets** — No port conflicts, faster IPC than TCP
|
||||
- 💾 **Automatic Persistence** — Data saved to files, survives restarts
|
||||
- 🪶 **Lightweight** — No MongoDB driver dependency in LocalTsmDb itself
|
||||
- 🎯 **Zero Config** — Just specify a folder path and you're good to go
|
||||
- 🔗 **Connection URI** — Ready-to-use URI for your own MongoClient
|
||||
|
||||
---
|
||||
|
||||
## 📖 SmartMongo API
|
||||
|
||||
Full MongoDB replica set in memory using `mongodb-memory-server`.
|
||||
|
||||
### Creating an Instance
|
||||
|
||||
```typescript
|
||||
@@ -116,27 +238,44 @@ await mongo.stopAndDumpToDir('./test-data');
|
||||
await mongo.stopAndDumpToDir('./test-data', (doc) => `${doc.collection}-${doc._id}.bson`);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 TsmDB API
|
||||
|
||||
Pure TypeScript MongoDB wire protocol server. No external dependencies.
|
||||
|
||||
### Server Configuration
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
|
||||
// TCP mode (default)
|
||||
const server = new tsmdb.TsmdbServer({
|
||||
port: 27017, // Default MongoDB port
|
||||
host: '127.0.0.1', // Bind address
|
||||
storage: 'memory', // 'memory' or 'file'
|
||||
storagePath: './data', // For file-based storage
|
||||
port: 27017, // Default MongoDB port
|
||||
host: '127.0.0.1', // Bind address
|
||||
storage: 'memory', // 'memory' or 'file'
|
||||
storagePath: './data', // For file-based storage
|
||||
});
|
||||
|
||||
// Unix socket mode (no port conflicts!)
|
||||
const server = new tsmdb.TsmdbServer({
|
||||
socketPath: '/tmp/my-tsmdb.sock',
|
||||
storage: 'file',
|
||||
storagePath: './data',
|
||||
});
|
||||
|
||||
await server.start();
|
||||
console.log(server.getConnectionUri()); // mongodb://127.0.0.1:27017
|
||||
console.log(server.getConnectionUri());
|
||||
// TCP: mongodb://127.0.0.1:27017
|
||||
// Socket: mongodb://%2Ftmp%2Fmy-tsmdb.sock
|
||||
|
||||
// Server properties
|
||||
console.log(server.running); // true
|
||||
console.log(server.getUptime()); // seconds
|
||||
console.log(server.getConnectionCount()); // active connections
|
||||
console.log(server.running); // true
|
||||
console.log(server.port); // 27017 (TCP mode)
|
||||
console.log(server.host); // '127.0.0.1' (TCP mode)
|
||||
console.log(server.socketPath); // '/tmp/my-tsmdb.sock' (socket mode)
|
||||
console.log(server.getUptime()); // seconds since start
|
||||
console.log(server.getConnectionCount()); // active client connections
|
||||
|
||||
await server.stop();
|
||||
```
|
||||
@@ -146,6 +285,7 @@ await server.stop();
|
||||
TsmDB supports the core MongoDB operations via the wire protocol:
|
||||
|
||||
#### 🔹 CRUD Operations
|
||||
|
||||
```typescript
|
||||
// Insert
|
||||
await collection.insertOne({ name: 'Bob' });
|
||||
@@ -175,6 +315,7 @@ const result = await collection.findOneAndUpdate(
|
||||
```
|
||||
|
||||
#### 🔹 Query Operators
|
||||
|
||||
```typescript
|
||||
// Comparison
|
||||
{ age: { $eq: 25 } }
|
||||
@@ -197,9 +338,14 @@ const result = await collection.findOneAndUpdate(
|
||||
{ tags: { $all: ['mongodb', 'database'] } }
|
||||
{ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }
|
||||
{ tags: { $size: 3 } }
|
||||
|
||||
// Regex
|
||||
{ name: { $regex: /^Al/i } }
|
||||
{ email: { $regex: '@example\\.com$' } }
|
||||
```
|
||||
|
||||
#### 🔹 Update Operators
|
||||
|
||||
```typescript
|
||||
{ $set: { name: 'New Name' } }
|
||||
{ $unset: { tempField: '' } }
|
||||
@@ -212,9 +358,12 @@ const result = await collection.findOneAndUpdate(
|
||||
{ $addToSet: { tags: 'unique-tag' } }
|
||||
{ $pop: { queue: 1 } } // Remove last
|
||||
{ $pop: { queue: -1 } } // Remove first
|
||||
{ $rename: { oldField: 'newField' } }
|
||||
{ $currentDate: { lastModified: true } }
|
||||
```
|
||||
|
||||
#### 🔹 Aggregation Pipeline
|
||||
|
||||
```typescript
|
||||
const results = await collection.aggregate([
|
||||
{ $match: { status: 'active' } },
|
||||
@@ -225,17 +374,28 @@ const results = await collection.aggregate([
|
||||
]).toArray();
|
||||
```
|
||||
|
||||
Supported stages: `$match`, `$project`, `$group`, `$sort`, `$limit`, `$skip`, `$unwind`, `$lookup`, `$addFields`, `$count`, `$facet`, and more.
|
||||
**Supported stages:** `$match`, `$project`, `$group`, `$sort`, `$limit`, `$skip`, `$unwind`, `$lookup`, `$addFields`, `$count`, `$facet`, `$replaceRoot`, `$set`, `$unset`, and more.
|
||||
|
||||
**Supported group accumulators:** `$sum`, `$avg`, `$min`, `$max`, `$first`, `$last`, `$push`, `$addToSet`, `$count`.
|
||||
|
||||
#### 🔹 Index Operations
|
||||
|
||||
```typescript
|
||||
// Create indexes
|
||||
await collection.createIndex({ email: 1 }, { unique: true });
|
||||
await collection.createIndex({ name: 1, age: -1 });
|
||||
await collection.createIndex({ location: '2dsphere' }); // Geospatial
|
||||
|
||||
// List indexes
|
||||
const indexes = await collection.listIndexes().toArray();
|
||||
|
||||
// Drop indexes
|
||||
await collection.dropIndex('email_1');
|
||||
await collection.dropIndexes(); // Drop all except _id
|
||||
```
|
||||
|
||||
#### 🔹 Database Operations
|
||||
|
||||
```typescript
|
||||
// List databases
|
||||
const dbs = await client.db().admin().listDatabases();
|
||||
@@ -249,9 +409,13 @@ await db.dropCollection('oldcollection');
|
||||
|
||||
// Drop database
|
||||
await db.dropDatabase();
|
||||
|
||||
// Database stats
|
||||
const stats = await db.stats();
|
||||
```
|
||||
|
||||
#### 🔹 Count & Distinct
|
||||
|
||||
```typescript
|
||||
// Count documents
|
||||
const total = await collection.countDocuments({});
|
||||
@@ -264,6 +428,7 @@ const activeDepts = await collection.distinct('department', { status: 'active' }
|
||||
```
|
||||
|
||||
#### 🔹 Bulk Operations
|
||||
|
||||
```typescript
|
||||
const result = await collection.bulkWrite([
|
||||
{ insertOne: { document: { name: 'Bulk1' } } },
|
||||
@@ -279,7 +444,7 @@ console.log(result.deletedCount); // 1
|
||||
|
||||
### Storage Adapters
|
||||
|
||||
TsmDB supports pluggable storage:
|
||||
TsmDB supports pluggable storage with data integrity features:
|
||||
|
||||
```typescript
|
||||
// In-memory (default) - fast, data lost on stop
|
||||
@@ -292,36 +457,177 @@ const server = new tsmdb.TsmdbServer({
|
||||
persistIntervalMs: 30000 // Save every 30 seconds
|
||||
});
|
||||
|
||||
// File-based - persistent storage
|
||||
// File-based - persistent storage with checksums
|
||||
const server = new tsmdb.TsmdbServer({
|
||||
storage: 'file',
|
||||
storagePath: './data/tsmdb'
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Performance & Reliability Features
|
||||
|
||||
TsmDB includes enterprise-grade features for robustness:
|
||||
|
||||
### 🔍 Index-Accelerated Queries
|
||||
|
||||
Indexes are automatically used to accelerate queries:
|
||||
|
||||
- **Hash indexes** for equality queries (`$eq`, `$in`)
|
||||
- **B-tree indexes** for range queries (`$gt`, `$gte`, `$lt`, `$lte`)
|
||||
|
||||
```typescript
|
||||
// Create an index
|
||||
await collection.createIndex({ email: 1 });
|
||||
await collection.createIndex({ age: 1 });
|
||||
|
||||
// These queries will use the index (fast!)
|
||||
await collection.findOne({ email: 'alice@example.com' }); // Uses hash lookup
|
||||
await collection.find({ age: { $gte: 18, $lt: 65 } }); // Uses B-tree range scan
|
||||
```
|
||||
|
||||
### 📊 Query Planner
|
||||
|
||||
TsmDB includes a query planner that analyzes queries and selects optimal execution strategies:
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
|
||||
// For debugging, you can access the query planner
|
||||
const planner = new tsmdb.QueryPlanner(indexEngine);
|
||||
const plan = await planner.plan(filter);
|
||||
|
||||
console.log(plan);
|
||||
// {
|
||||
// type: 'IXSCAN', // or 'IXSCAN_RANGE', 'COLLSCAN'
|
||||
// indexName: 'email_1',
|
||||
// selectivity: 0.01,
|
||||
// indexCovering: true
|
||||
// }
|
||||
```
|
||||
|
||||
### 📝 Write-Ahead Logging (WAL)
|
||||
|
||||
For durability, TsmDB supports write-ahead logging:
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
|
||||
const wal = new tsmdb.WAL('./data/wal.log');
|
||||
await wal.initialize();
|
||||
|
||||
// WAL entries include:
|
||||
// - LSN (Log Sequence Number)
|
||||
// - Timestamp
|
||||
// - Operation type (insert, update, delete, checkpoint)
|
||||
// - Document data (BSON serialized)
|
||||
// - CRC32 checksum for integrity
|
||||
|
||||
// Recovery support
|
||||
const entries = wal.getEntriesAfter(lastCheckpointLsn);
|
||||
```
|
||||
|
||||
### 🔐 Session Management
|
||||
|
||||
TsmDB tracks client sessions with automatic timeout and transaction linking:
|
||||
|
||||
```typescript
|
||||
// Sessions are automatically managed when using the MongoDB driver
|
||||
const session = client.startSession();
|
||||
|
||||
try {
|
||||
session.startTransaction();
|
||||
await collection.insertOne({ name: 'Alice' }, { session });
|
||||
await collection.updateOne({ name: 'Bob' }, { $inc: { balance: 100 } }, { session });
|
||||
await session.commitTransaction();
|
||||
} catch (error) {
|
||||
await session.abortTransaction();
|
||||
} finally {
|
||||
session.endSession();
|
||||
}
|
||||
|
||||
// Session features:
|
||||
// - Automatic session timeout (30 minutes default)
|
||||
// - Transaction auto-abort on session expiry
|
||||
// - Session activity tracking
|
||||
```
|
||||
|
||||
### ✅ Data Integrity Checksums
|
||||
|
||||
File-based storage supports CRC32 checksums to detect corruption:
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
|
||||
// Checksums are used internally for WAL and data integrity
|
||||
// Documents are checksummed on write, verified on read
|
||||
const checksum = tsmdb.calculateDocumentChecksum(doc);
|
||||
const isValid = tsmdb.verifyChecksum(docWithChecksum);
|
||||
```
|
||||
|
||||
### 📋 Supported Wire Protocol Commands
|
||||
|
||||
| Category | Commands |
|
||||
|----------|----------|
|
||||
| **Handshake** | `hello`, `isMaster` |
|
||||
| **Handshake** | `hello`, `isMaster`, `ismaster` |
|
||||
| **CRUD** | `find`, `insert`, `update`, `delete`, `findAndModify`, `getMore`, `killCursors` |
|
||||
| **Aggregation** | `aggregate`, `count`, `distinct` |
|
||||
| **Indexes** | `createIndexes`, `dropIndexes`, `listIndexes` |
|
||||
| **Admin** | `ping`, `listDatabases`, `listCollections`, `drop`, `dropDatabase`, `create`, `serverStatus`, `buildInfo` |
|
||||
| **Transactions** | `startTransaction`, `commitTransaction`, `abortTransaction` |
|
||||
| **Sessions** | `startSession`, `endSessions`, `refreshSessions` |
|
||||
| **Admin** | `ping`, `listDatabases`, `listCollections`, `drop`, `dropDatabase`, `create`, `serverStatus`, `buildInfo`, `dbStats`, `collStats`, `connectionStatus` |
|
||||
|
||||
TsmDB supports MongoDB wire protocol versions 0-21, compatible with MongoDB 3.6 through 7.0 drivers.
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing Examples
|
||||
|
||||
### Jest/Mocha with LocalTsmDb
|
||||
|
||||
```typescript
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient, Db } from 'mongodb';
|
||||
|
||||
let db: LocalTsmDb;
|
||||
let client: MongoClient;
|
||||
|
||||
beforeAll(async () => {
|
||||
db = new LocalTsmDb({ folderPath: './test-data' });
|
||||
const { connectionUri } = await db.start();
|
||||
client = new MongoClient(connectionUri, { directConnection: true });
|
||||
await client.connect();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await client.close();
|
||||
await db.stop();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
// Clean slate for each test
|
||||
await client.db('test').dropDatabase();
|
||||
});
|
||||
|
||||
test('should insert and find user', async () => {
|
||||
const users = client.db('test').collection('users');
|
||||
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
|
||||
|
||||
const user = await users.findOne({ name: 'Alice' });
|
||||
expect(user?.email).toBe('alice@example.com');
|
||||
});
|
||||
```
|
||||
|
||||
### Jest/Mocha with TsmDB
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
import { MongoClient, Db } from 'mongodb';
|
||||
|
||||
let server: tsmdb.TsmdbServer;
|
||||
let client: MongoClient;
|
||||
let db: Db;
|
||||
let testDb: Db;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = new tsmdb.TsmdbServer({ port: 27117 });
|
||||
@@ -329,7 +635,7 @@ beforeAll(async () => {
|
||||
|
||||
client = new MongoClient('mongodb://127.0.0.1:27117');
|
||||
await client.connect();
|
||||
db = client.db('test');
|
||||
testDb = client.db('test');
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
@@ -338,12 +644,11 @@ afterAll(async () => {
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
// Clean slate for each test
|
||||
await db.dropDatabase();
|
||||
await testDb.dropDatabase();
|
||||
});
|
||||
|
||||
test('should insert and find user', async () => {
|
||||
const users = db.collection('users');
|
||||
const users = testDb.collection('users');
|
||||
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
|
||||
|
||||
const user = await users.findOne({ name: 'Alice' });
|
||||
@@ -355,22 +660,21 @@ test('should insert and find user', async () => {
|
||||
|
||||
```typescript
|
||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
|
||||
let server: tsmdb.TsmdbServer;
|
||||
let db: LocalTsmDb;
|
||||
let client: MongoClient;
|
||||
|
||||
tap.test('setup', async () => {
|
||||
server = new tsmdb.TsmdbServer({ port: 27117 });
|
||||
await server.start();
|
||||
client = new MongoClient('mongodb://127.0.0.1:27117');
|
||||
db = new LocalTsmDb({ folderPath: './test-data' });
|
||||
const { connectionUri } = await db.start();
|
||||
client = new MongoClient(connectionUri, { directConnection: true });
|
||||
await client.connect();
|
||||
});
|
||||
|
||||
tap.test('should perform CRUD operations', async () => {
|
||||
const db = client.db('test');
|
||||
const col = db.collection('items');
|
||||
const col = client.db('test').collection('items');
|
||||
|
||||
// Create
|
||||
const result = await col.insertOne({ name: 'Widget', price: 9.99 });
|
||||
@@ -393,14 +697,25 @@ tap.test('should perform CRUD operations', async () => {
|
||||
|
||||
tap.test('teardown', async () => {
|
||||
await client.close();
|
||||
await server.stop();
|
||||
await db.stop();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
### Module Structure
|
||||
|
||||
```
|
||||
@push.rocks/smartmongo
|
||||
├── SmartMongo → Real MongoDB memory server (mongodb-memory-server wrapper)
|
||||
├── tsmdb → Wire protocol server with full engine stack
|
||||
└── LocalTsmDb → Lightweight Unix socket wrapper (no MongoDB driver dependency)
|
||||
```
|
||||
|
||||
### TsmDB Wire Protocol Stack
|
||||
|
||||
```
|
||||
@@ -408,7 +723,7 @@ export default tap.start();
|
||||
│ Official MongoDB Driver │
|
||||
│ (mongodb npm) │
|
||||
└─────────────────────────┬───────────────────────────────────┘
|
||||
│ TCP + OP_MSG/BSON
|
||||
│ TCP/Unix Socket + OP_MSG/BSON
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ TsmdbServer │
|
||||
@@ -421,21 +736,42 @@ export default tap.start();
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Engines │
|
||||
│ ┌───────────┐ ┌────────────┐ ┌────────────┐ ┌───────────┐ │
|
||||
│ │ Query │ │ Update │ │Aggregation │ │ Index │ │
|
||||
│ │ Engine │ │ Engine │ │ Engine │ │ Engine │ │
|
||||
│ └───────────┘ └────────────┘ └────────────┘ └───────────┘ │
|
||||
│ ┌─────────┐ ┌────────┐ ┌───────────┐ ┌───────┐ ┌───────┐ │
|
||||
│ │ Query │ │ Update │ │Aggregation│ │ Index │ │Session│ │
|
||||
│ │ Planner │ │ Engine │ │ Engine │ │Engine │ │Engine │ │
|
||||
│ └─────────┘ └────────┘ └───────────┘ └───────┘ └───────┘ │
|
||||
│ ┌──────────────────────┐ │
|
||||
│ │ Transaction Engine │ │
|
||||
│ └──────────────────────┘ │
|
||||
└─────────────────────────┬───────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Storage Adapters │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ │
|
||||
│ │ MemoryStorage │ │ FileStorage │ │
|
||||
│ └──────────────────┘ └──────────────────┘ │
|
||||
│ Storage Layer │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────┐ │
|
||||
│ │ MemoryStorage │ │ FileStorage │ │ WAL │ │
|
||||
│ │ │ │ (+ Checksums) │ │ │ │
|
||||
│ └──────────────────┘ └──────────────────┘ └──────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
| Component | Description |
|
||||
|-----------|-------------|
|
||||
| **WireProtocol** | Parses MongoDB OP_MSG binary protocol |
|
||||
| **CommandRouter** | Routes commands to appropriate handlers |
|
||||
| **QueryPlanner** | Analyzes queries and selects execution strategy |
|
||||
| **QueryEngine** | Executes queries with filter matching |
|
||||
| **UpdateEngine** | Processes update operators (`$set`, `$inc`, etc.) |
|
||||
| **AggregationEngine** | Executes aggregation pipelines |
|
||||
| **IndexEngine** | Manages B-tree and hash indexes |
|
||||
| **SessionEngine** | Tracks client sessions and timeouts |
|
||||
| **TransactionEngine** | Handles ACID transaction semantics |
|
||||
| **WAL** | Write-ahead logging for durability |
|
||||
|
||||
---
|
||||
|
||||
## License and Legal Information
|
||||
|
||||
This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.
|
||||
|
||||
232
test/test.tsmdb.checksum.ts
Normal file
232
test/test.tsmdb.checksum.ts
Normal file
@@ -0,0 +1,232 @@
|
||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||
import * as smartmongo from '../ts/index.js';
|
||||
|
||||
const {
|
||||
calculateCRC32,
|
||||
calculateCRC32Buffer,
|
||||
calculateDocumentChecksum,
|
||||
addChecksum,
|
||||
verifyChecksum,
|
||||
removeChecksum,
|
||||
} = smartmongo.tsmdb;
|
||||
|
||||
// ============================================================================
|
||||
// CRC32 String Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('checksum: calculateCRC32 should return consistent value for same input', async () => {
|
||||
const result1 = calculateCRC32('hello world');
|
||||
const result2 = calculateCRC32('hello world');
|
||||
expect(result1).toEqual(result2);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32 should return different values for different inputs', async () => {
|
||||
const result1 = calculateCRC32('hello');
|
||||
const result2 = calculateCRC32('world');
|
||||
expect(result1).not.toEqual(result2);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32 should return a 32-bit unsigned integer', async () => {
|
||||
const result = calculateCRC32('test string');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
expect(result).toBeLessThanOrEqual(0xFFFFFFFF);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32 should handle empty string', async () => {
|
||||
const result = calculateCRC32('');
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32 should handle special characters', async () => {
|
||||
const result = calculateCRC32('hello\nworld\t!"#$%&\'()');
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32 should handle unicode characters', async () => {
|
||||
const result = calculateCRC32('hello 世界 🌍');
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// CRC32 Buffer Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('checksum: calculateCRC32Buffer should return consistent value for same input', async () => {
|
||||
const buffer = Buffer.from('hello world');
|
||||
const result1 = calculateCRC32Buffer(buffer);
|
||||
const result2 = calculateCRC32Buffer(buffer);
|
||||
expect(result1).toEqual(result2);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32Buffer should return different values for different inputs', async () => {
|
||||
const buffer1 = Buffer.from('hello');
|
||||
const buffer2 = Buffer.from('world');
|
||||
const result1 = calculateCRC32Buffer(buffer1);
|
||||
const result2 = calculateCRC32Buffer(buffer2);
|
||||
expect(result1).not.toEqual(result2);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32Buffer should handle empty buffer', async () => {
|
||||
const result = calculateCRC32Buffer(Buffer.from(''));
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateCRC32Buffer should handle binary data', async () => {
|
||||
const buffer = Buffer.from([0x00, 0xFF, 0x7F, 0x80, 0x01]);
|
||||
const result = calculateCRC32Buffer(buffer);
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Document Checksum Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('checksum: calculateDocumentChecksum should return consistent value', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
const result1 = calculateDocumentChecksum(doc);
|
||||
const result2 = calculateDocumentChecksum(doc);
|
||||
expect(result1).toEqual(result2);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateDocumentChecksum should exclude _checksum field', async () => {
|
||||
const doc1 = { name: 'John', age: 30 };
|
||||
const doc2 = { name: 'John', age: 30, _checksum: 12345 };
|
||||
const result1 = calculateDocumentChecksum(doc1);
|
||||
const result2 = calculateDocumentChecksum(doc2);
|
||||
expect(result1).toEqual(result2);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateDocumentChecksum should handle empty document', async () => {
|
||||
const result = calculateDocumentChecksum({});
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateDocumentChecksum should handle nested objects', async () => {
|
||||
const doc = {
|
||||
name: 'John',
|
||||
address: {
|
||||
street: '123 Main St',
|
||||
city: 'Springfield',
|
||||
zip: {
|
||||
code: '12345',
|
||||
plus4: '6789',
|
||||
},
|
||||
},
|
||||
};
|
||||
const result = calculateDocumentChecksum(doc);
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
tap.test('checksum: calculateDocumentChecksum should handle arrays', async () => {
|
||||
const doc = {
|
||||
name: 'John',
|
||||
tags: ['developer', 'tester', 'admin'],
|
||||
scores: [95, 87, 92],
|
||||
};
|
||||
const result = calculateDocumentChecksum(doc);
|
||||
expect(typeof result).toEqual('number');
|
||||
expect(result).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Add/Verify/Remove Checksum Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('checksum: addChecksum should add _checksum field to document', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
const docWithChecksum = addChecksum(doc);
|
||||
|
||||
expect('_checksum' in docWithChecksum).toBeTrue();
|
||||
expect(typeof docWithChecksum._checksum).toEqual('number');
|
||||
expect(docWithChecksum.name).toEqual('John');
|
||||
expect(docWithChecksum.age).toEqual(30);
|
||||
});
|
||||
|
||||
tap.test('checksum: addChecksum should not modify the original document', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
addChecksum(doc);
|
||||
expect('_checksum' in doc).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('checksum: verifyChecksum should return true for valid checksum', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
const docWithChecksum = addChecksum(doc);
|
||||
const isValid = verifyChecksum(docWithChecksum);
|
||||
expect(isValid).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('checksum: verifyChecksum should return false for tampered document', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
const docWithChecksum = addChecksum(doc);
|
||||
|
||||
// Tamper with the document
|
||||
docWithChecksum.age = 31;
|
||||
|
||||
const isValid = verifyChecksum(docWithChecksum);
|
||||
expect(isValid).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('checksum: verifyChecksum should return false for wrong checksum', async () => {
|
||||
const doc = { name: 'John', age: 30, _checksum: 12345 };
|
||||
const isValid = verifyChecksum(doc);
|
||||
expect(isValid).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('checksum: verifyChecksum should return true for document without checksum', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
const isValid = verifyChecksum(doc);
|
||||
expect(isValid).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('checksum: removeChecksum should remove _checksum field', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
const docWithChecksum = addChecksum(doc);
|
||||
const docWithoutChecksum = removeChecksum(docWithChecksum);
|
||||
|
||||
expect('_checksum' in docWithoutChecksum).toBeFalse();
|
||||
expect(docWithoutChecksum.name).toEqual('John');
|
||||
expect(docWithoutChecksum.age).toEqual(30);
|
||||
});
|
||||
|
||||
tap.test('checksum: removeChecksum should handle document without checksum', async () => {
|
||||
const doc = { name: 'John', age: 30 };
|
||||
const result = removeChecksum(doc);
|
||||
|
||||
expect('_checksum' in result).toBeFalse();
|
||||
expect(result.name).toEqual('John');
|
||||
expect(result.age).toEqual(30);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Round-trip Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('checksum: full round-trip - add, verify, remove', async () => {
|
||||
const original = { name: 'Test', value: 42, nested: { a: 1, b: 2 } };
|
||||
|
||||
// Add checksum
|
||||
const withChecksum = addChecksum(original);
|
||||
expect('_checksum' in withChecksum).toBeTrue();
|
||||
|
||||
// Verify checksum
|
||||
expect(verifyChecksum(withChecksum)).toBeTrue();
|
||||
|
||||
// Remove checksum
|
||||
const restored = removeChecksum(withChecksum);
|
||||
expect('_checksum' in restored).toBeFalse();
|
||||
|
||||
// Original data should be intact
|
||||
expect(restored.name).toEqual('Test');
|
||||
expect(restored.value).toEqual(42);
|
||||
expect(restored.nested.a).toEqual(1);
|
||||
expect(restored.nested.b).toEqual(2);
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
417
test/test.tsmdb.indexengine.ts
Normal file
417
test/test.tsmdb.indexengine.ts
Normal file
@@ -0,0 +1,417 @@
|
||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||
import * as smartmongo from '../ts/index.js';
|
||||
|
||||
const { IndexEngine, MemoryStorageAdapter, ObjectId } = smartmongo.tsmdb;
|
||||
|
||||
let storage: InstanceType<typeof MemoryStorageAdapter>;
|
||||
let indexEngine: InstanceType<typeof IndexEngine>;
|
||||
|
||||
const TEST_DB = 'testdb';
|
||||
const TEST_COLL = 'indextest';
|
||||
|
||||
// ============================================================================
|
||||
// Setup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: should create IndexEngine instance', async () => {
|
||||
storage = new MemoryStorageAdapter();
|
||||
await storage.initialize();
|
||||
await storage.createCollection(TEST_DB, TEST_COLL);
|
||||
|
||||
indexEngine = new IndexEngine(TEST_DB, TEST_COLL, storage);
|
||||
expect(indexEngine).toBeTruthy();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Index Creation Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: createIndex should create single-field index', async () => {
|
||||
const indexName = await indexEngine.createIndex({ name: 1 });
|
||||
|
||||
expect(indexName).toEqual('name_1');
|
||||
|
||||
const exists = await indexEngine.indexExists('name_1');
|
||||
expect(exists).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: createIndex should create compound index', async () => {
|
||||
const indexName = await indexEngine.createIndex({ city: 1, state: -1 });
|
||||
|
||||
expect(indexName).toEqual('city_1_state_-1');
|
||||
|
||||
const exists = await indexEngine.indexExists('city_1_state_-1');
|
||||
expect(exists).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: createIndex should use custom name if provided', async () => {
|
||||
const indexName = await indexEngine.createIndex({ email: 1 }, { name: 'custom_email_index' });
|
||||
|
||||
expect(indexName).toEqual('custom_email_index');
|
||||
|
||||
const exists = await indexEngine.indexExists('custom_email_index');
|
||||
expect(exists).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: createIndex should handle unique option', async () => {
|
||||
const indexName = await indexEngine.createIndex({ uniqueField: 1 }, { unique: true });
|
||||
|
||||
expect(indexName).toEqual('uniqueField_1');
|
||||
|
||||
const indexes = await indexEngine.listIndexes();
|
||||
const uniqueIndex = indexes.find(i => i.name === 'uniqueField_1');
|
||||
expect(uniqueIndex!.unique).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: createIndex should handle sparse option', async () => {
|
||||
const indexName = await indexEngine.createIndex({ sparseField: 1 }, { sparse: true });
|
||||
|
||||
expect(indexName).toEqual('sparseField_1');
|
||||
|
||||
const indexes = await indexEngine.listIndexes();
|
||||
const sparseIndex = indexes.find(i => i.name === 'sparseField_1');
|
||||
expect(sparseIndex!.sparse).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: createIndex should return existing index name if already exists', async () => {
|
||||
const indexName1 = await indexEngine.createIndex({ existingField: 1 }, { name: 'existing_idx' });
|
||||
const indexName2 = await indexEngine.createIndex({ existingField: 1 }, { name: 'existing_idx' });
|
||||
|
||||
expect(indexName1).toEqual(indexName2);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Index Listing and Existence Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: listIndexes should return all indexes', async () => {
|
||||
const indexes = await indexEngine.listIndexes();
|
||||
|
||||
expect(indexes.length).toBeGreaterThanOrEqual(5); // _id_ + created indexes
|
||||
expect(indexes.some(i => i.name === '_id_')).toBeTrue();
|
||||
expect(indexes.some(i => i.name === 'name_1')).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: indexExists should return true for existing index', async () => {
|
||||
const exists = await indexEngine.indexExists('name_1');
|
||||
expect(exists).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: indexExists should return false for non-existent index', async () => {
|
||||
const exists = await indexEngine.indexExists('nonexistent_index');
|
||||
expect(exists).toBeFalse();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Document Operations and Index Updates
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: should insert documents for index testing', async () => {
|
||||
// Create a fresh index engine for document operations
|
||||
await storage.dropCollection(TEST_DB, TEST_COLL);
|
||||
await storage.createCollection(TEST_DB, TEST_COLL);
|
||||
|
||||
indexEngine = new IndexEngine(TEST_DB, TEST_COLL, storage);
|
||||
|
||||
// Create indexes first
|
||||
await indexEngine.createIndex({ age: 1 });
|
||||
await indexEngine.createIndex({ category: 1 });
|
||||
|
||||
// Insert test documents
|
||||
const docs = [
|
||||
{ _id: new ObjectId(), name: 'Alice', age: 25, category: 'A' },
|
||||
{ _id: new ObjectId(), name: 'Bob', age: 30, category: 'B' },
|
||||
{ _id: new ObjectId(), name: 'Charlie', age: 35, category: 'A' },
|
||||
{ _id: new ObjectId(), name: 'Diana', age: 28, category: 'C' },
|
||||
{ _id: new ObjectId(), name: 'Eve', age: 30, category: 'B' },
|
||||
];
|
||||
|
||||
for (const doc of docs) {
|
||||
const stored = await storage.insertOne(TEST_DB, TEST_COLL, doc);
|
||||
await indexEngine.onInsert(stored);
|
||||
}
|
||||
});
|
||||
|
||||
tap.test('indexengine: onInsert should update indexes', async () => {
|
||||
const newDoc = {
|
||||
_id: new ObjectId(),
|
||||
name: 'Frank',
|
||||
age: 40,
|
||||
category: 'D',
|
||||
};
|
||||
|
||||
const stored = await storage.insertOne(TEST_DB, TEST_COLL, newDoc);
|
||||
await indexEngine.onInsert(stored);
|
||||
|
||||
// Find by the indexed field
|
||||
const candidates = await indexEngine.findCandidateIds({ age: 40 });
|
||||
expect(candidates).toBeTruthy();
|
||||
expect(candidates!.size).toEqual(1);
|
||||
});
|
||||
|
||||
tap.test('indexengine: onUpdate should update indexes correctly', async () => {
|
||||
// Get an existing document
|
||||
const docs = await storage.findAll(TEST_DB, TEST_COLL);
|
||||
const oldDoc = docs.find(d => d.name === 'Alice')!;
|
||||
|
||||
// Update the document
|
||||
const newDoc = { ...oldDoc, age: 26 };
|
||||
await storage.updateById(TEST_DB, TEST_COLL, oldDoc._id, newDoc);
|
||||
await indexEngine.onUpdate(oldDoc, newDoc);
|
||||
|
||||
// Old value should not be in index
|
||||
const oldCandidates = await indexEngine.findCandidateIds({ age: 25 });
|
||||
expect(oldCandidates).toBeTruthy();
|
||||
expect(oldCandidates!.has(oldDoc._id.toHexString())).toBeFalse();
|
||||
|
||||
// New value should be in index
|
||||
const newCandidates = await indexEngine.findCandidateIds({ age: 26 });
|
||||
expect(newCandidates).toBeTruthy();
|
||||
expect(newCandidates!.has(oldDoc._id.toHexString())).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: onDelete should remove from indexes', async () => {
|
||||
const docs = await storage.findAll(TEST_DB, TEST_COLL);
|
||||
const docToDelete = docs.find(d => d.name === 'Frank')!;
|
||||
|
||||
await storage.deleteById(TEST_DB, TEST_COLL, docToDelete._id);
|
||||
await indexEngine.onDelete(docToDelete);
|
||||
|
||||
const candidates = await indexEngine.findCandidateIds({ age: 40 });
|
||||
expect(candidates).toBeTruthy();
|
||||
expect(candidates!.has(docToDelete._id.toHexString())).toBeFalse();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// findCandidateIds Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: findCandidateIds with equality filter', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ age: 30 });
|
||||
|
||||
expect(candidates).toBeTruthy();
|
||||
expect(candidates!.size).toEqual(2); // Bob and Eve both have age 30
|
||||
});
|
||||
|
||||
tap.test('indexengine: findCandidateIds with $in filter', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ age: { $in: [28, 30] } });
|
||||
|
||||
expect(candidates).toBeTruthy();
|
||||
expect(candidates!.size).toEqual(3); // Diana (28), Bob (30), Eve (30)
|
||||
});
|
||||
|
||||
tap.test('indexengine: findCandidateIds with no matching index', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ nonIndexedField: 'value' });
|
||||
|
||||
// Should return null when no index can be used
|
||||
expect(candidates).toBeNull();
|
||||
});
|
||||
|
||||
tap.test('indexengine: findCandidateIds with empty filter', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({});
|
||||
|
||||
// Empty filter = no index can be used
|
||||
expect(candidates).toBeNull();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Range Query Tests (B-Tree)
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: findCandidateIds with $gt', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ age: { $gt: 30 } });
|
||||
|
||||
expect(candidates).toBeTruthy();
|
||||
// Charlie (35) is > 30
|
||||
expect(candidates!.size).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
tap.test('indexengine: findCandidateIds with $lt', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ age: { $lt: 28 } });
|
||||
|
||||
expect(candidates).toBeTruthy();
|
||||
// Alice (26) is < 28
|
||||
expect(candidates!.size).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
tap.test('indexengine: findCandidateIds with $gte', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ age: { $gte: 30 } });
|
||||
|
||||
expect(candidates).toBeTruthy();
|
||||
// Bob (30), Eve (30), Charlie (35)
|
||||
expect(candidates!.size).toBeGreaterThanOrEqual(3);
|
||||
});
|
||||
|
||||
tap.test('indexengine: findCandidateIds with $lte', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ age: { $lte: 28 } });
|
||||
|
||||
expect(candidates).toBeTruthy();
|
||||
// Alice (26), Diana (28)
|
||||
expect(candidates!.size).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
tap.test('indexengine: findCandidateIds with range $gt and $lt', async () => {
|
||||
const candidates = await indexEngine.findCandidateIds({ age: { $gt: 26, $lt: 35 } });
|
||||
|
||||
expect(candidates).toBeTruthy();
|
||||
// Diana (28), Bob (30), Eve (30) are between 26 and 35 exclusive
|
||||
expect(candidates!.size).toBeGreaterThanOrEqual(3);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Index Selection Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: selectIndex should return best index for equality', async () => {
|
||||
const result = indexEngine.selectIndex({ age: 30 });
|
||||
|
||||
expect(result).toBeTruthy();
|
||||
expect(result!.name).toEqual('age_1');
|
||||
});
|
||||
|
||||
tap.test('indexengine: selectIndex should return best index for range query', async () => {
|
||||
const result = indexEngine.selectIndex({ age: { $gt: 25 } });
|
||||
|
||||
expect(result).toBeTruthy();
|
||||
expect(result!.name).toEqual('age_1');
|
||||
});
|
||||
|
||||
tap.test('indexengine: selectIndex should return null for no matching filter', async () => {
|
||||
const result = indexEngine.selectIndex({ nonIndexedField: 'value' });
|
||||
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
tap.test('indexengine: selectIndex should return null for empty filter', async () => {
|
||||
const result = indexEngine.selectIndex({});
|
||||
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
tap.test('indexengine: selectIndex should prefer more specific indexes', async () => {
|
||||
// Create a compound index
|
||||
await indexEngine.createIndex({ age: 1, category: 1 }, { name: 'age_category_compound' });
|
||||
|
||||
// Query that matches compound index
|
||||
const result = indexEngine.selectIndex({ age: 30, category: 'B' });
|
||||
|
||||
expect(result).toBeTruthy();
|
||||
// Should prefer the compound index since it covers more fields
|
||||
expect(result!.name).toEqual('age_category_compound');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Drop Index Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: dropIndex should remove the index', async () => {
|
||||
await indexEngine.createIndex({ dropTest: 1 }, { name: 'drop_test_idx' });
|
||||
expect(await indexEngine.indexExists('drop_test_idx')).toBeTrue();
|
||||
|
||||
await indexEngine.dropIndex('drop_test_idx');
|
||||
expect(await indexEngine.indexExists('drop_test_idx')).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('indexengine: dropIndex should throw for _id index', async () => {
|
||||
let threw = false;
|
||||
try {
|
||||
await indexEngine.dropIndex('_id_');
|
||||
} catch (e) {
|
||||
threw = true;
|
||||
}
|
||||
expect(threw).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: dropIndex should throw for non-existent index', async () => {
|
||||
let threw = false;
|
||||
try {
|
||||
await indexEngine.dropIndex('nonexistent_index');
|
||||
} catch (e) {
|
||||
threw = true;
|
||||
}
|
||||
expect(threw).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('indexengine: dropAllIndexes should remove all indexes except _id', async () => {
|
||||
// Create some indexes to drop
|
||||
await indexEngine.createIndex({ toDrop1: 1 });
|
||||
await indexEngine.createIndex({ toDrop2: 1 });
|
||||
|
||||
await indexEngine.dropAllIndexes();
|
||||
|
||||
const indexes = await indexEngine.listIndexes();
|
||||
expect(indexes.length).toEqual(1);
|
||||
expect(indexes[0].name).toEqual('_id_');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Unique Index Constraint Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: unique index should prevent duplicate inserts', async () => {
|
||||
// Create fresh collection
|
||||
await storage.dropCollection(TEST_DB, 'uniquetest');
|
||||
await storage.createCollection(TEST_DB, 'uniquetest');
|
||||
|
||||
const uniqueIndexEngine = new IndexEngine(TEST_DB, 'uniquetest', storage);
|
||||
await uniqueIndexEngine.createIndex({ email: 1 }, { unique: true });
|
||||
|
||||
// Insert first document
|
||||
const doc1 = { _id: new ObjectId(), email: 'test@example.com', name: 'Test' };
|
||||
const stored1 = await storage.insertOne(TEST_DB, 'uniquetest', doc1);
|
||||
await uniqueIndexEngine.onInsert(stored1);
|
||||
|
||||
// Try to insert duplicate
|
||||
const doc2 = { _id: new ObjectId(), email: 'test@example.com', name: 'Test2' };
|
||||
const stored2 = await storage.insertOne(TEST_DB, 'uniquetest', doc2);
|
||||
|
||||
let threw = false;
|
||||
try {
|
||||
await uniqueIndexEngine.onInsert(stored2);
|
||||
} catch (e: any) {
|
||||
threw = true;
|
||||
expect(e.message).toContain('duplicate key');
|
||||
}
|
||||
|
||||
expect(threw).toBeTrue();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Sparse Index Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: sparse index should not include documents without the field', async () => {
|
||||
// Create fresh collection
|
||||
await storage.dropCollection(TEST_DB, 'sparsetest');
|
||||
await storage.createCollection(TEST_DB, 'sparsetest');
|
||||
|
||||
const sparseIndexEngine = new IndexEngine(TEST_DB, 'sparsetest', storage);
|
||||
await sparseIndexEngine.createIndex({ optionalField: 1 }, { sparse: true });
|
||||
|
||||
// Insert doc with the field
|
||||
const doc1 = { _id: new ObjectId(), optionalField: 'hasValue', name: 'HasField' };
|
||||
const stored1 = await storage.insertOne(TEST_DB, 'sparsetest', doc1);
|
||||
await sparseIndexEngine.onInsert(stored1);
|
||||
|
||||
// Insert doc without the field
|
||||
const doc2 = { _id: new ObjectId(), name: 'NoField' };
|
||||
const stored2 = await storage.insertOne(TEST_DB, 'sparsetest', doc2);
|
||||
await sparseIndexEngine.onInsert(stored2);
|
||||
|
||||
// Search for documents with the field
|
||||
const candidates = await sparseIndexEngine.findCandidateIds({ optionalField: 'hasValue' });
|
||||
expect(candidates).toBeTruthy();
|
||||
expect(candidates!.size).toEqual(1);
|
||||
expect(candidates!.has(stored1._id.toHexString())).toBeTrue();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Cleanup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('indexengine: cleanup', async () => {
|
||||
await storage.close();
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
273
test/test.tsmdb.queryplanner.ts
Normal file
273
test/test.tsmdb.queryplanner.ts
Normal file
@@ -0,0 +1,273 @@
|
||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||
import * as smartmongo from '../ts/index.js';
|
||||
|
||||
const { QueryPlanner, IndexEngine, MemoryStorageAdapter, ObjectId } = smartmongo.tsmdb;
|
||||
|
||||
let storage: InstanceType<typeof MemoryStorageAdapter>;
|
||||
let indexEngine: InstanceType<typeof IndexEngine>;
|
||||
let queryPlanner: InstanceType<typeof QueryPlanner>;
|
||||
|
||||
const TEST_DB = 'testdb';
|
||||
const TEST_COLL = 'testcoll';
|
||||
|
||||
// ============================================================================
|
||||
// Setup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: should create QueryPlanner instance', async () => {
|
||||
storage = new MemoryStorageAdapter();
|
||||
await storage.initialize();
|
||||
await storage.createCollection(TEST_DB, TEST_COLL);
|
||||
|
||||
indexEngine = new IndexEngine(TEST_DB, TEST_COLL, storage);
|
||||
queryPlanner = new QueryPlanner(indexEngine);
|
||||
|
||||
expect(queryPlanner).toBeTruthy();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: should insert test documents', async () => {
|
||||
// Insert test documents
|
||||
const docs = [
|
||||
{ _id: new ObjectId(), name: 'Alice', age: 25, city: 'NYC', category: 'A' },
|
||||
{ _id: new ObjectId(), name: 'Bob', age: 30, city: 'LA', category: 'B' },
|
||||
{ _id: new ObjectId(), name: 'Charlie', age: 35, city: 'NYC', category: 'A' },
|
||||
{ _id: new ObjectId(), name: 'Diana', age: 28, city: 'Chicago', category: 'C' },
|
||||
{ _id: new ObjectId(), name: 'Eve', age: 32, city: 'LA', category: 'B' },
|
||||
];
|
||||
|
||||
for (const doc of docs) {
|
||||
await storage.insertOne(TEST_DB, TEST_COLL, doc);
|
||||
}
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Basic Plan Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: empty filter should result in COLLSCAN', async () => {
|
||||
const plan = await queryPlanner.plan({});
|
||||
|
||||
expect(plan.type).toEqual('COLLSCAN');
|
||||
expect(plan.indexCovering).toBeFalse();
|
||||
expect(plan.selectivity).toEqual(1.0);
|
||||
expect(plan.explanation).toContain('No filter');
|
||||
});
|
||||
|
||||
tap.test('queryplanner: null filter should result in COLLSCAN', async () => {
|
||||
const plan = await queryPlanner.plan(null as any);
|
||||
|
||||
expect(plan.type).toEqual('COLLSCAN');
|
||||
});
|
||||
|
||||
tap.test('queryplanner: filter with no matching index should result in COLLSCAN', async () => {
|
||||
const plan = await queryPlanner.plan({ nonExistentField: 'value' });
|
||||
|
||||
expect(plan.type).toEqual('COLLSCAN');
|
||||
expect(plan.explanation).toContain('No suitable index');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Index Scan Tests (with indexes)
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: should create test indexes', async () => {
|
||||
await indexEngine.createIndex({ age: 1 }, { name: 'age_1' });
|
||||
await indexEngine.createIndex({ name: 1 }, { name: 'name_1' });
|
||||
await indexEngine.createIndex({ city: 1, category: 1 }, { name: 'city_category_1' });
|
||||
|
||||
const indexes = await indexEngine.listIndexes();
|
||||
expect(indexes.length).toBeGreaterThanOrEqual(4); // _id_ + 3 created
|
||||
});
|
||||
|
||||
tap.test('queryplanner: simple equality filter should use IXSCAN', async () => {
|
||||
const plan = await queryPlanner.plan({ age: 30 });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN');
|
||||
expect(plan.indexName).toEqual('age_1');
|
||||
expect(plan.indexFieldsUsed).toContain('age');
|
||||
expect(plan.usesRange).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: $eq operator should use IXSCAN', async () => {
|
||||
const plan = await queryPlanner.plan({ name: { $eq: 'Alice' } });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN');
|
||||
expect(plan.indexName).toEqual('name_1');
|
||||
expect(plan.indexFieldsUsed).toContain('name');
|
||||
});
|
||||
|
||||
tap.test('queryplanner: range filter ($gt) should use IXSCAN_RANGE', async () => {
|
||||
const plan = await queryPlanner.plan({ age: { $gt: 25 } });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN_RANGE');
|
||||
expect(plan.indexName).toEqual('age_1');
|
||||
expect(plan.usesRange).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: range filter ($lt) should use IXSCAN_RANGE', async () => {
|
||||
const plan = await queryPlanner.plan({ age: { $lt: 35 } });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN_RANGE');
|
||||
expect(plan.usesRange).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: range filter ($gte, $lte) should use IXSCAN_RANGE', async () => {
|
||||
const plan = await queryPlanner.plan({ age: { $gte: 25, $lte: 35 } });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN_RANGE');
|
||||
expect(plan.usesRange).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: $in operator should use IXSCAN', async () => {
|
||||
const plan = await queryPlanner.plan({ age: { $in: [25, 30, 35] } });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN');
|
||||
expect(plan.indexName).toEqual('age_1');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Compound Index Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: compound index - first field equality should use index', async () => {
|
||||
const plan = await queryPlanner.plan({ city: 'NYC' });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN');
|
||||
expect(plan.indexName).toEqual('city_category_1');
|
||||
expect(plan.indexFieldsUsed).toContain('city');
|
||||
});
|
||||
|
||||
tap.test('queryplanner: compound index - both fields should use full index', async () => {
|
||||
const plan = await queryPlanner.plan({ city: 'NYC', category: 'A' });
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN');
|
||||
expect(plan.indexName).toEqual('city_category_1');
|
||||
expect(plan.indexFieldsUsed).toContain('city');
|
||||
expect(plan.indexFieldsUsed).toContain('category');
|
||||
expect(plan.indexFieldsUsed.length).toEqual(2);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Selectivity Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: equality query should have low selectivity', async () => {
|
||||
const plan = await queryPlanner.plan({ age: 30 });
|
||||
|
||||
expect(plan.selectivity).toBeLessThan(0.1);
|
||||
});
|
||||
|
||||
tap.test('queryplanner: range query should have moderate selectivity', async () => {
|
||||
const plan = await queryPlanner.plan({ age: { $gt: 25 } });
|
||||
|
||||
expect(plan.selectivity).toBeGreaterThan(0);
|
||||
expect(plan.selectivity).toBeLessThan(1);
|
||||
});
|
||||
|
||||
tap.test('queryplanner: $in query selectivity depends on array size', async () => {
|
||||
const smallInPlan = await queryPlanner.plan({ age: { $in: [25] } });
|
||||
const largeInPlan = await queryPlanner.plan({ age: { $in: [25, 26, 27, 28, 29, 30] } });
|
||||
|
||||
// Larger $in should have higher selectivity (less selective = more documents)
|
||||
expect(largeInPlan.selectivity).toBeGreaterThanOrEqual(smallInPlan.selectivity);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Index Covering Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: query covering all filter fields should be index covering', async () => {
|
||||
const plan = await queryPlanner.plan({ age: 30 });
|
||||
|
||||
// All filter fields are covered by the index
|
||||
expect(plan.indexCovering).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: query with residual filter should not be index covering', async () => {
|
||||
const plan = await queryPlanner.plan({ city: 'NYC', name: 'Alice' });
|
||||
|
||||
// 'name' is not in the compound index city_category, so it's residual
|
||||
expect(plan.indexCovering).toBeFalse();
|
||||
expect(plan.residualFilter).toBeTruthy();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Explain Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: explain should return detailed plan info', async () => {
|
||||
const explanation = await queryPlanner.explain({ age: 30 });
|
||||
|
||||
expect(explanation.queryPlanner).toBeTruthy();
|
||||
expect(explanation.queryPlanner.plannerVersion).toEqual(1);
|
||||
expect(explanation.queryPlanner.winningPlan).toBeTruthy();
|
||||
expect(explanation.queryPlanner.rejectedPlans).toBeArray();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: explain should include winning and rejected plans', async () => {
|
||||
const explanation = await queryPlanner.explain({ age: 30 });
|
||||
|
||||
expect(explanation.queryPlanner.winningPlan.type).toBeTruthy();
|
||||
expect(explanation.queryPlanner.rejectedPlans.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
tap.test('queryplanner: explain winning plan should be the best plan', async () => {
|
||||
const explanation = await queryPlanner.explain({ age: 30 });
|
||||
|
||||
// Winning plan should use an index, not collection scan (if index exists)
|
||||
expect(explanation.queryPlanner.winningPlan.type).toEqual('IXSCAN');
|
||||
|
||||
// There should be a COLLSCAN in rejected plans
|
||||
const hasCOLLSCAN = explanation.queryPlanner.rejectedPlans.some(p => p.type === 'COLLSCAN');
|
||||
expect(hasCOLLSCAN).toBeTrue();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// $and Operator Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: $and conditions should be analyzed', async () => {
|
||||
const plan = await queryPlanner.plan({
|
||||
$and: [
|
||||
{ age: { $gte: 25 } },
|
||||
{ age: { $lte: 35 } },
|
||||
],
|
||||
});
|
||||
|
||||
expect(plan.type).toEqual('IXSCAN_RANGE');
|
||||
expect(plan.indexName).toEqual('age_1');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Edge Cases
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: should handle complex nested operators', async () => {
|
||||
const plan = await queryPlanner.plan({
|
||||
age: { $gte: 20, $lte: 40 },
|
||||
city: 'NYC',
|
||||
});
|
||||
|
||||
expect(plan).toBeTruthy();
|
||||
expect(plan.type).not.toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('queryplanner: should handle $exists operator', async () => {
|
||||
await indexEngine.createIndex({ email: 1 }, { name: 'email_1', sparse: true });
|
||||
|
||||
const plan = await queryPlanner.plan({ email: { $exists: true } });
|
||||
|
||||
// $exists can use sparse indexes
|
||||
expect(plan).toBeTruthy();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Cleanup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('queryplanner: cleanup', async () => {
|
||||
await storage.close();
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
361
test/test.tsmdb.session.ts
Normal file
361
test/test.tsmdb.session.ts
Normal file
@@ -0,0 +1,361 @@
|
||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||
import * as smartmongo from '../ts/index.js';
|
||||
|
||||
const { SessionEngine } = smartmongo.tsmdb;
|
||||
|
||||
let sessionEngine: InstanceType<typeof SessionEngine>;
|
||||
|
||||
// ============================================================================
|
||||
// Setup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: should create SessionEngine instance', async () => {
|
||||
sessionEngine = new SessionEngine({
|
||||
sessionTimeoutMs: 1000, // 1 second for testing
|
||||
cleanupIntervalMs: 10000, // 10 seconds to avoid cleanup during tests
|
||||
});
|
||||
expect(sessionEngine).toBeTruthy();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Session Lifecycle Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: startSession should create session with auto-generated ID', async () => {
|
||||
const session = sessionEngine.startSession();
|
||||
|
||||
expect(session).toBeTruthy();
|
||||
expect(session.id).toBeTruthy();
|
||||
expect(session.id.length).toBeGreaterThanOrEqual(32); // UUID hex string (32 or 36 with hyphens)
|
||||
expect(session.createdAt).toBeGreaterThan(0);
|
||||
expect(session.lastActivityAt).toBeGreaterThan(0);
|
||||
expect(session.inTransaction).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('session: startSession should create session with specified ID', async () => {
|
||||
const customId = 'custom-session-id-12345';
|
||||
const session = sessionEngine.startSession(customId);
|
||||
|
||||
expect(session.id).toEqual(customId);
|
||||
});
|
||||
|
||||
tap.test('session: startSession should create session with metadata', async () => {
|
||||
const metadata = { client: 'test-client', version: '1.0' };
|
||||
const session = sessionEngine.startSession(undefined, metadata);
|
||||
|
||||
expect(session.metadata).toBeTruthy();
|
||||
expect(session.metadata!.client).toEqual('test-client');
|
||||
expect(session.metadata!.version).toEqual('1.0');
|
||||
});
|
||||
|
||||
tap.test('session: getSession should return session by ID', async () => {
|
||||
const created = sessionEngine.startSession('get-session-test');
|
||||
const retrieved = sessionEngine.getSession('get-session-test');
|
||||
|
||||
expect(retrieved).toBeTruthy();
|
||||
expect(retrieved!.id).toEqual('get-session-test');
|
||||
expect(retrieved!.id).toEqual(created.id);
|
||||
});
|
||||
|
||||
tap.test('session: getSession should return undefined for non-existent session', async () => {
|
||||
const session = sessionEngine.getSession('non-existent-session-id');
|
||||
expect(session).toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('session: touchSession should update lastActivityAt', async () => {
|
||||
const session = sessionEngine.startSession('touch-test-session');
|
||||
const originalLastActivity = session.lastActivityAt;
|
||||
|
||||
// Wait a bit to ensure time difference
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
|
||||
const touched = sessionEngine.touchSession('touch-test-session');
|
||||
expect(touched).toBeTrue();
|
||||
|
||||
const updated = sessionEngine.getSession('touch-test-session');
|
||||
expect(updated!.lastActivityAt).toBeGreaterThanOrEqual(originalLastActivity);
|
||||
});
|
||||
|
||||
tap.test('session: touchSession should return false for non-existent session', async () => {
|
||||
const touched = sessionEngine.touchSession('non-existent-touch-session');
|
||||
expect(touched).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('session: endSession should remove the session', async () => {
|
||||
sessionEngine.startSession('end-session-test');
|
||||
expect(sessionEngine.getSession('end-session-test')).toBeTruthy();
|
||||
|
||||
const ended = await sessionEngine.endSession('end-session-test');
|
||||
expect(ended).toBeTrue();
|
||||
|
||||
expect(sessionEngine.getSession('end-session-test')).toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('session: endSession should return false for non-existent session', async () => {
|
||||
const ended = await sessionEngine.endSession('non-existent-end-session');
|
||||
expect(ended).toBeFalse();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Session Expiry Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: isSessionExpired should return false for fresh session', async () => {
|
||||
const session = sessionEngine.startSession('fresh-session');
|
||||
const isExpired = sessionEngine.isSessionExpired(session);
|
||||
expect(isExpired).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('session: isSessionExpired should return true for old session', async () => {
|
||||
// Create a session with old lastActivityAt
|
||||
const session = sessionEngine.startSession('old-session');
|
||||
// Manually set lastActivityAt to old value (sessionTimeoutMs is 1000ms)
|
||||
(session as any).lastActivityAt = Date.now() - 2000;
|
||||
|
||||
const isExpired = sessionEngine.isSessionExpired(session);
|
||||
expect(isExpired).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('session: getSession should return undefined for expired session', async () => {
|
||||
const session = sessionEngine.startSession('expiring-session');
|
||||
// Manually expire the session
|
||||
(session as any).lastActivityAt = Date.now() - 2000;
|
||||
|
||||
const retrieved = sessionEngine.getSession('expiring-session');
|
||||
expect(retrieved).toBeUndefined();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Transaction Integration Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: startTransaction should mark session as in transaction', async () => {
|
||||
sessionEngine.startSession('txn-session-1');
|
||||
const started = sessionEngine.startTransaction('txn-session-1', 'txn-id-1', 1);
|
||||
|
||||
expect(started).toBeTrue();
|
||||
|
||||
const session = sessionEngine.getSession('txn-session-1');
|
||||
expect(session!.inTransaction).toBeTrue();
|
||||
expect(session!.txnId).toEqual('txn-id-1');
|
||||
expect(session!.txnNumber).toEqual(1);
|
||||
});
|
||||
|
||||
tap.test('session: startTransaction should return false for non-existent session', async () => {
|
||||
const started = sessionEngine.startTransaction('non-existent-txn-session', 'txn-id');
|
||||
expect(started).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('session: endTransaction should clear transaction state', async () => {
|
||||
sessionEngine.startSession('txn-session-2');
|
||||
sessionEngine.startTransaction('txn-session-2', 'txn-id-2');
|
||||
|
||||
const ended = sessionEngine.endTransaction('txn-session-2');
|
||||
expect(ended).toBeTrue();
|
||||
|
||||
const session = sessionEngine.getSession('txn-session-2');
|
||||
expect(session!.inTransaction).toBeFalse();
|
||||
expect(session!.txnId).toBeUndefined();
|
||||
expect(session!.txnNumber).toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('session: endTransaction should return false for non-existent session', async () => {
|
||||
const ended = sessionEngine.endTransaction('non-existent-end-txn-session');
|
||||
expect(ended).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('session: getTransactionId should return transaction ID', async () => {
|
||||
sessionEngine.startSession('txn-id-session');
|
||||
sessionEngine.startTransaction('txn-id-session', 'my-txn-id');
|
||||
|
||||
const txnId = sessionEngine.getTransactionId('txn-id-session');
|
||||
expect(txnId).toEqual('my-txn-id');
|
||||
});
|
||||
|
||||
tap.test('session: getTransactionId should return undefined for session without transaction', async () => {
|
||||
sessionEngine.startSession('no-txn-session');
|
||||
const txnId = sessionEngine.getTransactionId('no-txn-session');
|
||||
expect(txnId).toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('session: getTransactionId should return undefined for non-existent session', async () => {
|
||||
const txnId = sessionEngine.getTransactionId('non-existent-txn-id-session');
|
||||
expect(txnId).toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('session: isInTransaction should return correct state', async () => {
|
||||
sessionEngine.startSession('in-txn-check-session');
|
||||
|
||||
expect(sessionEngine.isInTransaction('in-txn-check-session')).toBeFalse();
|
||||
|
||||
sessionEngine.startTransaction('in-txn-check-session', 'txn-check');
|
||||
expect(sessionEngine.isInTransaction('in-txn-check-session')).toBeTrue();
|
||||
|
||||
sessionEngine.endTransaction('in-txn-check-session');
|
||||
expect(sessionEngine.isInTransaction('in-txn-check-session')).toBeFalse();
|
||||
});
|
||||
|
||||
tap.test('session: isInTransaction should return false for non-existent session', async () => {
|
||||
expect(sessionEngine.isInTransaction('non-existent-in-txn-session')).toBeFalse();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Session Listing Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: listSessions should return all active sessions', async () => {
|
||||
// Close and recreate to have a clean slate
|
||||
sessionEngine.close();
|
||||
sessionEngine = new SessionEngine({
|
||||
sessionTimeoutMs: 10000,
|
||||
cleanupIntervalMs: 60000,
|
||||
});
|
||||
|
||||
sessionEngine.startSession('list-session-1');
|
||||
sessionEngine.startSession('list-session-2');
|
||||
sessionEngine.startSession('list-session-3');
|
||||
|
||||
const sessions = sessionEngine.listSessions();
|
||||
expect(sessions.length).toEqual(3);
|
||||
});
|
||||
|
||||
tap.test('session: listSessions should not include expired sessions', async () => {
|
||||
const session = sessionEngine.startSession('expired-list-session');
|
||||
// Expire the session
|
||||
(session as any).lastActivityAt = Date.now() - 20000;
|
||||
|
||||
const sessions = sessionEngine.listSessions();
|
||||
const found = sessions.find(s => s.id === 'expired-list-session');
|
||||
expect(found).toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('session: getSessionCount should return correct count', async () => {
|
||||
const count = sessionEngine.getSessionCount();
|
||||
expect(count).toBeGreaterThanOrEqual(3); // We created 3 sessions above
|
||||
});
|
||||
|
||||
tap.test('session: getSessionsWithTransactions should filter correctly', async () => {
|
||||
// Clean slate
|
||||
sessionEngine.close();
|
||||
sessionEngine = new SessionEngine({
|
||||
sessionTimeoutMs: 10000,
|
||||
cleanupIntervalMs: 60000,
|
||||
});
|
||||
|
||||
sessionEngine.startSession('no-txn-1');
|
||||
sessionEngine.startSession('no-txn-2');
|
||||
sessionEngine.startSession('with-txn-1');
|
||||
sessionEngine.startSession('with-txn-2');
|
||||
|
||||
sessionEngine.startTransaction('with-txn-1', 'txn-a');
|
||||
sessionEngine.startTransaction('with-txn-2', 'txn-b');
|
||||
|
||||
const txnSessions = sessionEngine.getSessionsWithTransactions();
|
||||
expect(txnSessions.length).toEqual(2);
|
||||
expect(txnSessions.every(s => s.inTransaction)).toBeTrue();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// getOrCreateSession Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: getOrCreateSession should create if missing', async () => {
|
||||
const session = sessionEngine.getOrCreateSession('get-or-create-new');
|
||||
expect(session).toBeTruthy();
|
||||
expect(session.id).toEqual('get-or-create-new');
|
||||
});
|
||||
|
||||
tap.test('session: getOrCreateSession should return existing session', async () => {
|
||||
const created = sessionEngine.startSession('get-or-create-existing');
|
||||
const retrieved = sessionEngine.getOrCreateSession('get-or-create-existing');
|
||||
|
||||
expect(retrieved.id).toEqual(created.id);
|
||||
expect(retrieved.createdAt).toEqual(created.createdAt);
|
||||
});
|
||||
|
||||
tap.test('session: getOrCreateSession should touch existing session', async () => {
|
||||
const session = sessionEngine.startSession('get-or-create-touch');
|
||||
const originalLastActivity = session.lastActivityAt;
|
||||
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
|
||||
sessionEngine.getOrCreateSession('get-or-create-touch');
|
||||
const updated = sessionEngine.getSession('get-or-create-touch');
|
||||
|
||||
expect(updated!.lastActivityAt).toBeGreaterThanOrEqual(originalLastActivity);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// extractSessionId Static Method Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: extractSessionId should handle UUID object', async () => {
|
||||
const { ObjectId } = smartmongo.tsmdb;
|
||||
const uuid = new smartmongo.tsmdb.plugins.bson.UUID();
|
||||
const lsid = { id: uuid };
|
||||
|
||||
const extracted = SessionEngine.extractSessionId(lsid);
|
||||
expect(extracted).toEqual(uuid.toHexString());
|
||||
});
|
||||
|
||||
tap.test('session: extractSessionId should handle string ID', async () => {
|
||||
const lsid = { id: 'string-session-id' };
|
||||
|
||||
const extracted = SessionEngine.extractSessionId(lsid);
|
||||
expect(extracted).toEqual('string-session-id');
|
||||
});
|
||||
|
||||
tap.test('session: extractSessionId should handle binary format', async () => {
|
||||
const binaryData = Buffer.from('test-binary-uuid', 'utf8').toString('base64');
|
||||
const lsid = { id: { $binary: { base64: binaryData } } };
|
||||
|
||||
const extracted = SessionEngine.extractSessionId(lsid);
|
||||
expect(extracted).toBeTruthy();
|
||||
expect(typeof extracted).toEqual('string');
|
||||
});
|
||||
|
||||
tap.test('session: extractSessionId should return undefined for null/undefined', async () => {
|
||||
expect(SessionEngine.extractSessionId(null)).toBeUndefined();
|
||||
expect(SessionEngine.extractSessionId(undefined)).toBeUndefined();
|
||||
});
|
||||
|
||||
tap.test('session: extractSessionId should return undefined for empty object', async () => {
|
||||
expect(SessionEngine.extractSessionId({})).toBeUndefined();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// refreshSession Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: refreshSession should update lastActivityAt', async () => {
|
||||
const session = sessionEngine.startSession('refresh-session-test');
|
||||
const originalLastActivity = session.lastActivityAt;
|
||||
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
|
||||
const refreshed = sessionEngine.refreshSession('refresh-session-test');
|
||||
expect(refreshed).toBeTrue();
|
||||
|
||||
const updated = sessionEngine.getSession('refresh-session-test');
|
||||
expect(updated!.lastActivityAt).toBeGreaterThanOrEqual(originalLastActivity);
|
||||
});
|
||||
|
||||
tap.test('session: refreshSession should return false for non-existent session', async () => {
|
||||
const refreshed = sessionEngine.refreshSession('non-existent-refresh-session');
|
||||
expect(refreshed).toBeFalse();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Cleanup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('session: close should clear all sessions', async () => {
|
||||
sessionEngine.startSession('close-test-session');
|
||||
expect(sessionEngine.getSessionCount()).toBeGreaterThan(0);
|
||||
|
||||
sessionEngine.close();
|
||||
|
||||
expect(sessionEngine.getSessionCount()).toEqual(0);
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
411
test/test.tsmdb.wal.ts
Normal file
411
test/test.tsmdb.wal.ts
Normal file
@@ -0,0 +1,411 @@
|
||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||
import * as smartmongo from '../ts/index.js';
|
||||
import * as path from 'path';
|
||||
import * as fs from 'fs/promises';
|
||||
|
||||
const { WAL, ObjectId } = smartmongo.tsmdb;
|
||||
|
||||
let wal: InstanceType<typeof WAL>;
|
||||
const TEST_WAL_PATH = '/tmp/tsmdb-test-wal/test.wal';
|
||||
|
||||
// Helper to clean up test files
|
||||
async function cleanupTestFiles() {
|
||||
try {
|
||||
await fs.rm('/tmp/tsmdb-test-wal', { recursive: true, force: true });
|
||||
} catch {
|
||||
// Ignore if doesn't exist
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Setup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: cleanup before tests', async () => {
|
||||
await cleanupTestFiles();
|
||||
});
|
||||
|
||||
tap.test('wal: should create WAL instance', async () => {
|
||||
wal = new WAL(TEST_WAL_PATH, { checkpointInterval: 100 });
|
||||
expect(wal).toBeTruthy();
|
||||
});
|
||||
|
||||
tap.test('wal: should initialize WAL', async () => {
|
||||
const result = await wal.initialize();
|
||||
expect(result).toBeTruthy();
|
||||
expect(result.recoveredEntries).toBeArray();
|
||||
expect(result.recoveredEntries.length).toEqual(0);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// LSN Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: getCurrentLsn should return 0 initially', async () => {
|
||||
const lsn = wal.getCurrentLsn();
|
||||
expect(lsn).toEqual(0);
|
||||
});
|
||||
|
||||
tap.test('wal: LSN should increment after logging', async () => {
|
||||
const doc = { _id: new ObjectId(), name: 'Test' };
|
||||
const lsn = await wal.logInsert('testdb', 'testcoll', doc as any);
|
||||
|
||||
expect(lsn).toEqual(1);
|
||||
expect(wal.getCurrentLsn()).toEqual(1);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Insert Logging Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: logInsert should create entry with correct structure', async () => {
|
||||
const doc = { _id: new ObjectId(), name: 'InsertTest', value: 42 };
|
||||
const lsn = await wal.logInsert('testdb', 'insertcoll', doc as any);
|
||||
|
||||
expect(lsn).toBeGreaterThan(0);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry).toBeTruthy();
|
||||
expect(entry!.operation).toEqual('insert');
|
||||
expect(entry!.dbName).toEqual('testdb');
|
||||
expect(entry!.collName).toEqual('insertcoll');
|
||||
expect(entry!.documentId).toEqual(doc._id.toHexString());
|
||||
expect(entry!.data).toBeTruthy();
|
||||
expect(entry!.timestamp).toBeGreaterThan(0);
|
||||
expect(entry!.checksum).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
tap.test('wal: logInsert with transaction ID', async () => {
|
||||
const doc = { _id: new ObjectId(), name: 'TxnInsertTest' };
|
||||
const lsn = await wal.logInsert('testdb', 'insertcoll', doc as any, 'txn-123');
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry!.txnId).toEqual('txn-123');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Update Logging Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: logUpdate should store old and new document', async () => {
|
||||
const oldDoc = { _id: new ObjectId(), name: 'OldName', value: 1 };
|
||||
const newDoc = { ...oldDoc, name: 'NewName', value: 2 };
|
||||
|
||||
const lsn = await wal.logUpdate('testdb', 'updatecoll', oldDoc as any, newDoc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry).toBeTruthy();
|
||||
expect(entry!.operation).toEqual('update');
|
||||
expect(entry!.data).toBeTruthy();
|
||||
expect(entry!.previousData).toBeTruthy();
|
||||
expect(entry!.documentId).toEqual(oldDoc._id.toHexString());
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Delete Logging Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: logDelete should record deleted document', async () => {
|
||||
const doc = { _id: new ObjectId(), name: 'ToDelete' };
|
||||
|
||||
const lsn = await wal.logDelete('testdb', 'deletecoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry).toBeTruthy();
|
||||
expect(entry!.operation).toEqual('delete');
|
||||
expect(entry!.previousData).toBeTruthy();
|
||||
expect(entry!.data).toBeUndefined();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Transaction Logging Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: logBeginTransaction should create begin entry', async () => {
|
||||
const lsn = await wal.logBeginTransaction('txn-begin-test');
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry).toBeTruthy();
|
||||
expect(entry!.operation).toEqual('begin');
|
||||
expect(entry!.txnId).toEqual('txn-begin-test');
|
||||
});
|
||||
|
||||
tap.test('wal: logCommitTransaction should create commit entry', async () => {
|
||||
const lsn = await wal.logCommitTransaction('txn-commit-test');
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry).toBeTruthy();
|
||||
expect(entry!.operation).toEqual('commit');
|
||||
expect(entry!.txnId).toEqual('txn-commit-test');
|
||||
});
|
||||
|
||||
tap.test('wal: logAbortTransaction should create abort entry', async () => {
|
||||
const lsn = await wal.logAbortTransaction('txn-abort-test');
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry).toBeTruthy();
|
||||
expect(entry!.operation).toEqual('abort');
|
||||
expect(entry!.txnId).toEqual('txn-abort-test');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// getTransactionEntries Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: getTransactionEntries should return entries for transaction', async () => {
|
||||
// Log a complete transaction
|
||||
const txnId = 'txn-entries-test';
|
||||
await wal.logBeginTransaction(txnId);
|
||||
|
||||
const doc1 = { _id: new ObjectId(), name: 'TxnDoc1' };
|
||||
await wal.logInsert('testdb', 'txncoll', doc1 as any, txnId);
|
||||
|
||||
const doc2 = { _id: new ObjectId(), name: 'TxnDoc2' };
|
||||
await wal.logInsert('testdb', 'txncoll', doc2 as any, txnId);
|
||||
|
||||
await wal.logCommitTransaction(txnId);
|
||||
|
||||
const entries = wal.getTransactionEntries(txnId);
|
||||
|
||||
expect(entries.length).toEqual(4); // begin + 2 inserts + commit
|
||||
expect(entries.every(e => e.txnId === txnId)).toBeTrue();
|
||||
});
|
||||
|
||||
tap.test('wal: getTransactionEntries should return empty for unknown transaction', async () => {
|
||||
const entries = wal.getTransactionEntries('unknown-txn-id');
|
||||
expect(entries.length).toEqual(0);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// getEntriesAfter Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: getEntriesAfter should filter by LSN', async () => {
|
||||
const currentLsn = wal.getCurrentLsn();
|
||||
|
||||
// Add more entries
|
||||
const doc = { _id: new ObjectId(), name: 'AfterTest' };
|
||||
await wal.logInsert('testdb', 'aftercoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(currentLsn);
|
||||
expect(entries.length).toEqual(1);
|
||||
expect(entries[0].lsn).toBeGreaterThan(currentLsn);
|
||||
});
|
||||
|
||||
tap.test('wal: getEntriesAfter with LSN 0 should return all entries', async () => {
|
||||
const entries = wal.getEntriesAfter(0);
|
||||
expect(entries.length).toBeGreaterThan(0);
|
||||
expect(entries.length).toEqual(wal.getCurrentLsn());
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Checkpoint Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: checkpoint should create checkpoint entry', async () => {
|
||||
const lsn = await wal.checkpoint();
|
||||
|
||||
expect(lsn).toBeGreaterThan(0);
|
||||
|
||||
// After checkpoint, getEntriesAfter(checkpoint) should be limited
|
||||
const entries = wal.getEntriesAfter(0);
|
||||
expect(entries.some(e => e.operation === 'checkpoint')).toBeTrue();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Document Recovery Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: recoverDocument should deserialize document from entry', async () => {
|
||||
const originalDoc = { _id: new ObjectId(), name: 'RecoverTest', nested: { a: 1, b: 2 } };
|
||||
const lsn = await wal.logInsert('testdb', 'recovercoll', originalDoc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const recovered = wal.recoverDocument(entry!);
|
||||
|
||||
expect(recovered).toBeTruthy();
|
||||
expect(recovered!.name).toEqual('RecoverTest');
|
||||
expect(recovered!.nested.a).toEqual(1);
|
||||
expect(recovered!.nested.b).toEqual(2);
|
||||
});
|
||||
|
||||
tap.test('wal: recoverDocument should return null for entry without data', async () => {
|
||||
const lsn = await wal.logBeginTransaction('recover-no-data');
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const recovered = wal.recoverDocument(entry!);
|
||||
expect(recovered).toBeNull();
|
||||
});
|
||||
|
||||
tap.test('wal: recoverPreviousDocument should deserialize previous state', async () => {
|
||||
const oldDoc = { _id: new ObjectId(), name: 'Old', value: 100 };
|
||||
const newDoc = { ...oldDoc, name: 'New', value: 200 };
|
||||
|
||||
const lsn = await wal.logUpdate('testdb', 'recovercoll', oldDoc as any, newDoc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const previous = wal.recoverPreviousDocument(entry!);
|
||||
|
||||
expect(previous).toBeTruthy();
|
||||
expect(previous!.name).toEqual('Old');
|
||||
expect(previous!.value).toEqual(100);
|
||||
});
|
||||
|
||||
tap.test('wal: recoverPreviousDocument should return null for insert entry', async () => {
|
||||
const doc = { _id: new ObjectId(), name: 'NoPrevious' };
|
||||
const lsn = await wal.logInsert('testdb', 'recovercoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const previous = wal.recoverPreviousDocument(entry!);
|
||||
expect(previous).toBeNull();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// WAL Persistence and Recovery Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: should persist and recover entries', async () => {
|
||||
// Close current WAL
|
||||
await wal.close();
|
||||
|
||||
// Create new WAL instance and initialize (should recover)
|
||||
const wal2 = new WAL(TEST_WAL_PATH, { checkpointInterval: 100 });
|
||||
const result = await wal2.initialize();
|
||||
|
||||
// Should have recovered entries
|
||||
expect(result.recoveredEntries).toBeArray();
|
||||
// After checkpoint, there might not be many recoverable entries
|
||||
// but getCurrentLsn should be preserved or reset
|
||||
|
||||
await wal2.close();
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Entry Checksum Tests
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: entries should have valid checksums', async () => {
|
||||
wal = new WAL(TEST_WAL_PATH + '.checksum', { checkpointInterval: 100 });
|
||||
await wal.initialize();
|
||||
|
||||
const doc = { _id: new ObjectId(), name: 'ChecksumTest' };
|
||||
const lsn = await wal.logInsert('testdb', 'checksumcoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
expect(entry!.checksum).toBeGreaterThan(0);
|
||||
expect(typeof entry!.checksum).toEqual('number');
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Edge Cases
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: should handle special characters in document', async () => {
|
||||
const doc = {
|
||||
_id: new ObjectId(),
|
||||
name: 'Test\nWith\tSpecial\r\nChars',
|
||||
emoji: '🎉',
|
||||
unicode: '日本語',
|
||||
};
|
||||
|
||||
const lsn = await wal.logInsert('testdb', 'specialcoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const recovered = wal.recoverDocument(entry!);
|
||||
expect(recovered!.name).toEqual('Test\nWith\tSpecial\r\nChars');
|
||||
expect(recovered!.emoji).toEqual('🎉');
|
||||
expect(recovered!.unicode).toEqual('日本語');
|
||||
});
|
||||
|
||||
tap.test('wal: should handle binary data in documents', async () => {
|
||||
const doc = {
|
||||
_id: new ObjectId(),
|
||||
binaryField: Buffer.from([0x00, 0xFF, 0x7F, 0x80]),
|
||||
};
|
||||
|
||||
const lsn = await wal.logInsert('testdb', 'binarycoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const recovered = wal.recoverDocument(entry!);
|
||||
expect(recovered).toBeTruthy();
|
||||
});
|
||||
|
||||
tap.test('wal: should handle nested documents', async () => {
|
||||
const doc = {
|
||||
_id: new ObjectId(),
|
||||
level1: {
|
||||
level2: {
|
||||
level3: {
|
||||
value: 'deep',
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
const lsn = await wal.logInsert('testdb', 'nestedcoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const recovered = wal.recoverDocument(entry!);
|
||||
expect(recovered!.level1.level2.level3.value).toEqual('deep');
|
||||
});
|
||||
|
||||
tap.test('wal: should handle arrays in documents', async () => {
|
||||
const doc = {
|
||||
_id: new ObjectId(),
|
||||
tags: ['a', 'b', 'c'],
|
||||
numbers: [1, 2, 3],
|
||||
mixed: [1, 'two', { three: 3 }],
|
||||
};
|
||||
|
||||
const lsn = await wal.logInsert('testdb', 'arraycoll', doc as any);
|
||||
|
||||
const entries = wal.getEntriesAfter(lsn - 1);
|
||||
const entry = entries.find(e => e.lsn === lsn);
|
||||
|
||||
const recovered = wal.recoverDocument(entry!);
|
||||
expect(recovered!.tags).toEqual(['a', 'b', 'c']);
|
||||
expect(recovered!.numbers).toEqual([1, 2, 3]);
|
||||
expect(recovered!.mixed[2].three).toEqual(3);
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// Cleanup
|
||||
// ============================================================================
|
||||
|
||||
tap.test('wal: cleanup', async () => {
|
||||
await wal.close();
|
||||
await cleanupTestFiles();
|
||||
expect(true).toBeTrue();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
@@ -3,6 +3,6 @@
|
||||
*/
|
||||
export const commitinfo = {
|
||||
name: '@push.rocks/smartmongo',
|
||||
version: '4.0.0',
|
||||
version: '5.1.0',
|
||||
description: 'A module for creating and managing a local MongoDB instance for testing purposes.'
|
||||
}
|
||||
|
||||
78
ts/index.ts
78
ts/index.ts
@@ -1,74 +1,14 @@
|
||||
import { commitinfo } from './00_commitinfo_data.js';
|
||||
import * as plugins from './smartmongo.plugins.js';
|
||||
|
||||
// Export SmartMongo from ts_mongotools
|
||||
export { SmartMongo } from './ts_mongotools/index.js';
|
||||
|
||||
// Export TsmDB module
|
||||
export * as tsmdb from './tsmdb/index.js';
|
||||
export * as tsmdb from './ts_tsmdb/index.js';
|
||||
|
||||
export class SmartMongo {
|
||||
// STATIC
|
||||
public static async createAndStart(replCountArg: number = 1) {
|
||||
const smartMongoInstance = new SmartMongo();
|
||||
await smartMongoInstance.start(replCountArg);
|
||||
return smartMongoInstance;
|
||||
}
|
||||
// Export LocalTsmDb from ts_local
|
||||
export { LocalTsmDb } from './ts_local/index.js';
|
||||
export type { ILocalTsmDbOptions, ILocalTsmDbConnectionInfo } from './ts_local/index.js';
|
||||
|
||||
// INSTANCE
|
||||
private _readyDeferred = plugins.smartpromise.defer();
|
||||
public readyPromise = this._readyDeferred.promise;
|
||||
public mongoReplicaSet: plugins.mongoPlugin.MongoMemoryReplSet;
|
||||
|
||||
constructor() {}
|
||||
|
||||
public async start(countArg: number = 1) {
|
||||
this.mongoReplicaSet = await plugins.mongoPlugin.MongoMemoryReplSet.create({
|
||||
replSet: { count: countArg },
|
||||
instanceOpts: [
|
||||
{
|
||||
storageEngine: 'wiredTiger',
|
||||
},
|
||||
],
|
||||
});
|
||||
this._readyDeferred.resolve();
|
||||
console.log(`mongoReplicaSet with ${countArg} replicas started.`);
|
||||
console.log(`@pushrocks/smartmongo version ${commitinfo.version}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* returns a mongo descriptor for modules like
|
||||
* @pushrocks/smartfile.
|
||||
*/
|
||||
public async getMongoDescriptor(): Promise<plugins.smartdata.IMongoDescriptor> {
|
||||
await this.readyPromise;
|
||||
return {
|
||||
mongoDbName: `smartmongo_testdatabase`,
|
||||
mongoDbUrl: this.mongoReplicaSet.getUri(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* stops the smartmongo instance
|
||||
* and cleans up after itself
|
||||
*/
|
||||
public async stop() {
|
||||
await this.mongoReplicaSet.stop();
|
||||
await this.mongoReplicaSet.cleanup();
|
||||
}
|
||||
|
||||
/**
|
||||
* like stop() but allows you to actually store
|
||||
* the database on disk
|
||||
*/
|
||||
public async stopAndDumpToDir(
|
||||
dirArg: string,
|
||||
nameFunctionArg?: (doc: any) => string,
|
||||
emptyDirArg = true,
|
||||
) {
|
||||
const mongodumpInstance = new plugins.mongodump.MongoDump();
|
||||
const mongodumpTarget = await mongodumpInstance.addMongoTargetByMongoDescriptor(
|
||||
await this.getMongoDescriptor(),
|
||||
);
|
||||
await mongodumpTarget.dumpAllCollectionsToDir(dirArg, nameFunctionArg, emptyDirArg);
|
||||
await mongodumpInstance.stop();
|
||||
await this.stop();
|
||||
}
|
||||
}
|
||||
// Export commitinfo
|
||||
export { commitinfo };
|
||||
|
||||
143
ts/ts_local/classes.localtsmdb.ts
Normal file
143
ts/ts_local/classes.localtsmdb.ts
Normal file
@@ -0,0 +1,143 @@
|
||||
import * as plugins from './plugins.js';
|
||||
import * as crypto from 'crypto';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { TsmdbServer } from '../ts_tsmdb/index.js';
|
||||
|
||||
/**
|
||||
* Connection information returned by LocalTsmDb.start()
|
||||
*/
|
||||
export interface ILocalTsmDbConnectionInfo {
|
||||
/** The Unix socket file path */
|
||||
socketPath: string;
|
||||
/** MongoDB connection URI ready for MongoClient */
|
||||
connectionUri: string;
|
||||
}
|
||||
|
||||
export interface ILocalTsmDbOptions {
|
||||
/** Required: where to store data */
|
||||
folderPath: string;
|
||||
/** Optional: custom socket path (default: auto-generated in /tmp) */
|
||||
socketPath?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* LocalTsmDb - Lightweight local MongoDB-compatible database using Unix sockets
|
||||
*
|
||||
* This class wraps TsmdbServer and provides a simple interface for
|
||||
* starting a local file-based MongoDB-compatible server. Returns connection
|
||||
* info that you can use with your own MongoDB driver instance.
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
* import { MongoClient } from 'mongodb';
|
||||
*
|
||||
* const db = new LocalTsmDb({ folderPath: './data' });
|
||||
* const { connectionUri } = await db.start();
|
||||
*
|
||||
* // Connect with your own MongoDB client
|
||||
* const client = new MongoClient(connectionUri, { directConnection: true });
|
||||
* await client.connect();
|
||||
*
|
||||
* // Use the MongoDB client
|
||||
* const collection = client.db('mydb').collection('users');
|
||||
* await collection.insertOne({ name: 'Alice' });
|
||||
*
|
||||
* // When done
|
||||
* await client.close();
|
||||
* await db.stop();
|
||||
* ```
|
||||
*/
|
||||
export class LocalTsmDb {
|
||||
private options: ILocalTsmDbOptions;
|
||||
private server: TsmdbServer | null = null;
|
||||
private generatedSocketPath: string | null = null;
|
||||
|
||||
constructor(options: ILocalTsmDbOptions) {
|
||||
this.options = options;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a unique socket path in /tmp
|
||||
*/
|
||||
private generateSocketPath(): string {
|
||||
const randomId = crypto.randomBytes(8).toString('hex');
|
||||
return path.join(os.tmpdir(), `smartmongo-${randomId}.sock`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the local TsmDB server and return connection info
|
||||
*/
|
||||
async start(): Promise<ILocalTsmDbConnectionInfo> {
|
||||
if (this.server) {
|
||||
throw new Error('LocalTsmDb is already running');
|
||||
}
|
||||
|
||||
// Use provided socket path or generate one
|
||||
this.generatedSocketPath = this.options.socketPath ?? this.generateSocketPath();
|
||||
|
||||
this.server = new TsmdbServer({
|
||||
socketPath: this.generatedSocketPath,
|
||||
storage: 'file',
|
||||
storagePath: this.options.folderPath,
|
||||
});
|
||||
await this.server.start();
|
||||
|
||||
return {
|
||||
socketPath: this.generatedSocketPath,
|
||||
connectionUri: this.server.getConnectionUri(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get connection info (throws if not started)
|
||||
*/
|
||||
getConnectionInfo(): ILocalTsmDbConnectionInfo {
|
||||
if (!this.server || !this.generatedSocketPath) {
|
||||
throw new Error('LocalTsmDb is not running. Call start() first.');
|
||||
}
|
||||
return {
|
||||
socketPath: this.generatedSocketPath,
|
||||
connectionUri: this.server.getConnectionUri(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the underlying TsmdbServer instance (throws if not started)
|
||||
*/
|
||||
getServer(): TsmdbServer {
|
||||
if (!this.server) {
|
||||
throw new Error('LocalTsmDb is not running. Call start() first.');
|
||||
}
|
||||
return this.server;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the connection URI
|
||||
*/
|
||||
getConnectionUri(): string {
|
||||
if (!this.server) {
|
||||
throw new Error('LocalTsmDb is not running. Call start() first.');
|
||||
}
|
||||
return this.server.getConnectionUri();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the server is running
|
||||
*/
|
||||
get running(): boolean {
|
||||
return this.server !== null && this.server.running;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop the local TsmDB server
|
||||
*/
|
||||
async stop(): Promise<void> {
|
||||
if (this.server) {
|
||||
await this.server.stop();
|
||||
this.server = null;
|
||||
this.generatedSocketPath = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
2
ts/ts_local/index.ts
Normal file
2
ts/ts_local/index.ts
Normal file
@@ -0,0 +1,2 @@
|
||||
export { LocalTsmDb } from './classes.localtsmdb.js';
|
||||
export type { ILocalTsmDbOptions, ILocalTsmDbConnectionInfo } from './classes.localtsmdb.js';
|
||||
3
ts/ts_local/plugins.ts
Normal file
3
ts/ts_local/plugins.ts
Normal file
@@ -0,0 +1,3 @@
|
||||
import * as smartpromise from '@push.rocks/smartpromise';
|
||||
|
||||
export { smartpromise };
|
||||
71
ts/ts_mongotools/classes.smartmongo.ts
Normal file
71
ts/ts_mongotools/classes.smartmongo.ts
Normal file
@@ -0,0 +1,71 @@
|
||||
import { commitinfo } from '../00_commitinfo_data.js';
|
||||
import * as plugins from './plugins.js';
|
||||
|
||||
export class SmartMongo {
|
||||
// STATIC
|
||||
public static async createAndStart(replCountArg: number = 1) {
|
||||
const smartMongoInstance = new SmartMongo();
|
||||
await smartMongoInstance.start(replCountArg);
|
||||
return smartMongoInstance;
|
||||
}
|
||||
|
||||
// INSTANCE
|
||||
private _readyDeferred = plugins.smartpromise.defer();
|
||||
public readyPromise = this._readyDeferred.promise;
|
||||
public mongoReplicaSet: plugins.mongoPlugin.MongoMemoryReplSet;
|
||||
|
||||
constructor() {}
|
||||
|
||||
public async start(countArg: number = 1) {
|
||||
this.mongoReplicaSet = await plugins.mongoPlugin.MongoMemoryReplSet.create({
|
||||
replSet: { count: countArg },
|
||||
instanceOpts: [
|
||||
{
|
||||
storageEngine: 'wiredTiger',
|
||||
},
|
||||
],
|
||||
});
|
||||
this._readyDeferred.resolve();
|
||||
console.log(`mongoReplicaSet with ${countArg} replicas started.`);
|
||||
console.log(`@pushrocks/smartmongo version ${commitinfo.version}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* returns a mongo descriptor for modules like
|
||||
* @pushrocks/smartfile.
|
||||
*/
|
||||
public async getMongoDescriptor(): Promise<plugins.smartdata.IMongoDescriptor> {
|
||||
await this.readyPromise;
|
||||
return {
|
||||
mongoDbName: `smartmongo_testdatabase`,
|
||||
mongoDbUrl: this.mongoReplicaSet.getUri(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* stops the smartmongo instance
|
||||
* and cleans up after itself
|
||||
*/
|
||||
public async stop() {
|
||||
await this.mongoReplicaSet.stop();
|
||||
await this.mongoReplicaSet.cleanup();
|
||||
}
|
||||
|
||||
/**
|
||||
* like stop() but allows you to actually store
|
||||
* the database on disk
|
||||
*/
|
||||
public async stopAndDumpToDir(
|
||||
dirArg: string,
|
||||
nameFunctionArg?: (doc: any) => string,
|
||||
emptyDirArg = true,
|
||||
) {
|
||||
const mongodumpInstance = new plugins.mongodump.MongoDump();
|
||||
const mongodumpTarget = await mongodumpInstance.addMongoTargetByMongoDescriptor(
|
||||
await this.getMongoDescriptor(),
|
||||
);
|
||||
await mongodumpTarget.dumpAllCollectionsToDir(dirArg, nameFunctionArg, emptyDirArg);
|
||||
await mongodumpInstance.stop();
|
||||
await this.stop();
|
||||
}
|
||||
}
|
||||
2
ts/ts_mongotools/index.ts
Normal file
2
ts/ts_mongotools/index.ts
Normal file
@@ -0,0 +1,2 @@
|
||||
export * from './plugins.js';
|
||||
export { SmartMongo } from './classes.smartmongo.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument, IAggregateOptions } from '../types/interfaces.js';
|
||||
|
||||
// Import mingo Aggregator
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
|
||||
|
||||
// Simple B-Tree implementation for range queries
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument, ISortSpecification, ISortDirection } from '../types/interfaces.js';
|
||||
|
||||
// Import mingo Query class
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument } from '../types/interfaces.js';
|
||||
import { IndexEngine } from './IndexEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { TransactionEngine } from './TransactionEngine.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
|
||||
import type { Document, IStoredDocument, ITransactionOptions } from '../types/interfaces.js';
|
||||
import { TsmdbTransactionError, TsmdbWriteConflictError } from '../errors/TsmdbErrors.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument } from '../types/interfaces.js';
|
||||
import { QueryEngine } from './QueryEngine.js';
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
// Use the official MongoDB driver to connect to TsmdbServer
|
||||
|
||||
// Re-export plugins for external use
|
||||
import * as plugins from './tsmdb.plugins.js';
|
||||
import * as plugins from './plugins.js';
|
||||
export { plugins };
|
||||
|
||||
// Export BSON types for convenience
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
|
||||
import type { IParsedCommand } from './WireProtocol.js';
|
||||
import type { TsmdbServer } from './TsmdbServer.js';
|
||||
@@ -1,5 +1,6 @@
|
||||
import * as net from 'net';
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as fs from 'fs/promises';
|
||||
import * as plugins from '../plugins.js';
|
||||
import { WireProtocol, OP_QUERY } from './WireProtocol.js';
|
||||
import { CommandRouter } from './CommandRouter.js';
|
||||
import { MemoryStorageAdapter } from '../storage/MemoryStorageAdapter.js';
|
||||
@@ -10,10 +11,12 @@ import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
|
||||
* Server configuration options
|
||||
*/
|
||||
export interface ITsmdbServerOptions {
|
||||
/** Port to listen on (default: 27017) */
|
||||
/** Port to listen on (default: 27017) - ignored if socketPath is set */
|
||||
port?: number;
|
||||
/** Host to bind to (default: 127.0.0.1) */
|
||||
/** Host to bind to (default: 127.0.0.1) - ignored if socketPath is set */
|
||||
host?: string;
|
||||
/** Unix socket path - if set, server listens on socket instead of TCP */
|
||||
socketPath?: string;
|
||||
/** Storage type: 'memory' or 'file' (default: 'memory') */
|
||||
storage?: 'memory' | 'file';
|
||||
/** Path for file storage (required if storage is 'file') */
|
||||
@@ -54,7 +57,7 @@ interface IConnectionState {
|
||||
* ```
|
||||
*/
|
||||
export class TsmdbServer {
|
||||
private options: Required<ITsmdbServerOptions>;
|
||||
private options: Required<Omit<ITsmdbServerOptions, 'socketPath'>> & { socketPath: string };
|
||||
private server: net.Server | null = null;
|
||||
private storage: IStorageAdapter;
|
||||
private commandRouter: CommandRouter;
|
||||
@@ -62,11 +65,14 @@ export class TsmdbServer {
|
||||
private connectionIdCounter = 0;
|
||||
private isRunning = false;
|
||||
private startTime: Date = new Date();
|
||||
private useSocket: boolean;
|
||||
|
||||
constructor(options: ITsmdbServerOptions = {}) {
|
||||
this.useSocket = !!options.socketPath;
|
||||
this.options = {
|
||||
port: options.port ?? 27017,
|
||||
host: options.host ?? '127.0.0.1',
|
||||
socketPath: options.socketPath ?? '',
|
||||
storage: options.storage ?? 'memory',
|
||||
storagePath: options.storagePath ?? './data',
|
||||
persistPath: options.persistPath ?? '',
|
||||
@@ -119,6 +125,18 @@ export class TsmdbServer {
|
||||
// Initialize storage
|
||||
await this.storage.initialize();
|
||||
|
||||
// Clean up stale socket file if using Unix socket
|
||||
if (this.useSocket && this.options.socketPath) {
|
||||
try {
|
||||
await fs.unlink(this.options.socketPath);
|
||||
} catch (err: any) {
|
||||
// Ignore ENOENT (file doesn't exist)
|
||||
if (err.code !== 'ENOENT') {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
this.server = net.createServer((socket) => {
|
||||
this.handleConnection(socket);
|
||||
@@ -132,11 +150,21 @@ export class TsmdbServer {
|
||||
}
|
||||
});
|
||||
|
||||
this.server.listen(this.options.port, this.options.host, () => {
|
||||
this.isRunning = true;
|
||||
this.startTime = new Date();
|
||||
resolve();
|
||||
});
|
||||
if (this.useSocket && this.options.socketPath) {
|
||||
// Listen on Unix socket
|
||||
this.server.listen(this.options.socketPath, () => {
|
||||
this.isRunning = true;
|
||||
this.startTime = new Date();
|
||||
resolve();
|
||||
});
|
||||
} else {
|
||||
// Listen on TCP
|
||||
this.server.listen(this.options.port, this.options.host, () => {
|
||||
this.isRunning = true;
|
||||
this.startTime = new Date();
|
||||
resolve();
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -161,9 +189,22 @@ export class TsmdbServer {
|
||||
await this.storage.close();
|
||||
|
||||
return new Promise((resolve) => {
|
||||
this.server!.close(() => {
|
||||
this.server!.close(async () => {
|
||||
this.isRunning = false;
|
||||
this.server = null;
|
||||
|
||||
// Clean up socket file if using Unix socket
|
||||
if (this.useSocket && this.options.socketPath) {
|
||||
try {
|
||||
await fs.unlink(this.options.socketPath);
|
||||
} catch (err: any) {
|
||||
// Ignore ENOENT (file doesn't exist)
|
||||
if (err.code !== 'ENOENT') {
|
||||
console.error('Failed to remove socket file:', err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
@@ -275,9 +316,21 @@ export class TsmdbServer {
|
||||
* Get the connection URI for this server
|
||||
*/
|
||||
getConnectionUri(): string {
|
||||
if (this.useSocket && this.options.socketPath) {
|
||||
// URL-encode the socket path (replace / with %2F)
|
||||
const encodedPath = encodeURIComponent(this.options.socketPath);
|
||||
return `mongodb://${encodedPath}`;
|
||||
}
|
||||
return `mongodb://${this.options.host}:${this.options.port}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the socket path (if using Unix socket mode)
|
||||
*/
|
||||
get socketPath(): string | undefined {
|
||||
return this.useSocket ? this.options.socketPath : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the server is running
|
||||
*/
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
|
||||
/**
|
||||
* MongoDB Wire Protocol Implementation
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import { SessionEngine } from '../../engine/SessionEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext, ICursorState } from '../CommandRouter.js';
|
||||
import { AggregationEngine } from '../../engine/AggregationEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
import { QueryEngine } from '../../engine/QueryEngine.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext, ICursorState } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
import { QueryEngine } from '../../engine/QueryEngine.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import { IndexEngine } from '../../engine/IndexEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
import { QueryEngine } from '../../engine/QueryEngine.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from './IStorageAdapter.js';
|
||||
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
|
||||
import { calculateDocumentChecksum, verifyChecksum } from '../utils/checksum.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import type * as plugins from '../tsmdb.plugins.js';
|
||||
import type * as plugins from '../plugins.js';
|
||||
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from './IStorageAdapter.js';
|
||||
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from './IStorageAdapter.js';
|
||||
import type { IOpLogEntry, Document, IResumeToken, ChangeStreamOperationType } from '../types/interfaces.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument } from '../types/interfaces.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import type * as plugins from '../tsmdb.plugins.js';
|
||||
import type * as plugins from '../plugins.js';
|
||||
|
||||
// ============================================================================
|
||||
// Document Types
|
||||
Reference in New Issue
Block a user