Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e6a36ecb5f | |||
| 6a37a773ea | |||
| 1fff277698 | |||
| 0ad7f316c4 | |||
| 0d450e7d4e | |||
| fff77fbd8e |
25
changelog.md
25
changelog.md
@@ -1,5 +1,30 @@
|
||||
# Changelog
|
||||
|
||||
## 2026-02-03 - 4.3.0 - feat(docs)
|
||||
add LocalTsmDb documentation and examples; update README code samples and imports; correct examples and variable names; update package author
|
||||
|
||||
- Introduce LocalTsmDb: zero-config local database with automatic persistence, auto port discovery, and pre-connected client (added Quick Start, API, Features, and testing examples).
|
||||
- Expand comparison table to include LocalTsmDb alongside SmartMongo and TsmDB.
|
||||
- Update README examples: new LocalTsmDb usage, reorder options (LocalTsmDb, TsmDB, SmartMongo), rename test DB variable (db -> testDb), and adjust test snippets for Jest/Mocha and tap.
|
||||
- Adjust code snippets and API notes: switch some example imports to use tsmdb, replace FileStorageAdapter references, change planner.createPlan to await planner.plan, and use wal.getEntriesAfter(...) without awaiting.
|
||||
- Update package.json author from 'Lossless GmbH' to 'Task Venture Capital GmbH'.
|
||||
|
||||
## 2026-02-03 - 4.2.1 - fix(package.json)
|
||||
replace main and typings with exports field pointing to ./dist_ts/index.js
|
||||
|
||||
- Added package.json exports field mapping "." to ./dist_ts/index.js to declare the package entrypoint.
|
||||
- Removed main (dist_ts/index.js) and typings (dist_ts/index.d.ts) entries.
|
||||
- Note: switching to exports improves Node resolution but removing the typings entry may affect TypeScript consumers expecting index.d.ts.
|
||||
|
||||
## 2026-02-01 - 4.2.0 - feat(tsmdb)
|
||||
implement TsmDB Mongo-wire-compatible server, add storage/engine modules and reorganize exports
|
||||
|
||||
- Add full TsmDB implementation under ts/ts_tsmdb: wire protocol, server, command router, handlers, engines (Query, Update, Aggregation, Index, Transaction, Session), storage adapters (Memory, File), OpLog, WAL, utils and types.
|
||||
- Remove legacy ts/tsmdb implementation and replace with new ts_tsmdb module exports.
|
||||
- Introduce ts/ts_mongotools module and move SmartMongo class there; update top-level exports in ts/index.ts to export SmartMongo, tsmdb (from ts_tsmdb) and LocalTsmDb.
|
||||
- Add LocalTsmDb convenience class (ts/ts_local) to start a file-backed TsmDB and return a connected MongoClient.
|
||||
- Refactor plugin imports into per-module plugins files and add utilities (checksum, persistence, query planner, index engine).
|
||||
|
||||
## 2026-02-01 - 4.1.1 - fix(tsmdb)
|
||||
add comprehensive unit tests for tsmdb components: checksum, query planner, index engine, session, and WAL
|
||||
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
{
|
||||
"name": "@push.rocks/smartmongo",
|
||||
"version": "4.1.1",
|
||||
"version": "4.3.0",
|
||||
"private": false,
|
||||
"description": "A module for creating and managing a local MongoDB instance for testing purposes.",
|
||||
"main": "dist_ts/index.js",
|
||||
"typings": "dist_ts/index.d.ts",
|
||||
"exports": {
|
||||
".": "./dist_ts/index.js"
|
||||
},
|
||||
"type": "module",
|
||||
"author": "Lossless GmbH",
|
||||
"author": "Task Venture Capital GmbH",
|
||||
"license": "MIT",
|
||||
"scripts": {
|
||||
"test": "(tstest test/. --verbose --logfile --timeout 60)",
|
||||
|
||||
222
readme.md
222
readme.md
@@ -1,6 +1,6 @@
|
||||
# @push.rocks/smartmongo
|
||||
|
||||
A powerful MongoDB toolkit for testing and development — featuring both a real MongoDB memory server (**SmartMongo**) and an ultra-fast, lightweight wire-protocol-compatible in-memory database server (**TsmDB**). 🚀
|
||||
A powerful MongoDB toolkit for testing and development — featuring a real MongoDB memory server (**SmartMongo**), an ultra-fast wire-protocol-compatible in-memory database server (**TsmDB**), and a zero-config local database (**LocalTsmDb**). 🚀
|
||||
|
||||
## Install
|
||||
|
||||
@@ -16,21 +16,79 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
|
||||
|
||||
## Overview
|
||||
|
||||
`@push.rocks/smartmongo` provides two powerful approaches for MongoDB in testing and development:
|
||||
`@push.rocks/smartmongo` provides three powerful approaches for MongoDB in testing and development:
|
||||
|
||||
| Feature | SmartMongo | TsmDB |
|
||||
|---------|------------|---------|
|
||||
| **Type** | Real MongoDB (memory server) | Pure TypeScript wire protocol server |
|
||||
| **Speed** | ~2-5s startup | ⚡ Instant startup (~5ms) |
|
||||
| **Compatibility** | 100% MongoDB | MongoDB driver compatible |
|
||||
| **Dependencies** | Downloads MongoDB binary | Zero external dependencies |
|
||||
| **Replication** | ✅ Full replica set support | Single node emulation |
|
||||
| **Use Case** | Integration testing | Unit testing, CI/CD |
|
||||
| **Persistence** | Dump to directory | Optional file/memory persistence |
|
||||
| Feature | SmartMongo | TsmDB | LocalTsmDb |
|
||||
|---------|------------|-------|------------|
|
||||
| **Type** | Real MongoDB (memory server) | Wire protocol server | Zero-config local DB |
|
||||
| **Speed** | ~2-5s startup | ⚡ Instant (~5ms) | ⚡ Instant + auto-connect |
|
||||
| **Compatibility** | 100% MongoDB | MongoDB driver compatible | MongoDB driver compatible |
|
||||
| **Dependencies** | Downloads MongoDB binary | Zero external deps | Zero external deps |
|
||||
| **Replication** | ✅ Full replica set | Single node | Single node |
|
||||
| **Persistence** | Dump to directory | Memory or file | File-based (automatic) |
|
||||
| **Use Case** | Integration testing | Unit testing, CI/CD | Quick prototyping, local dev |
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Option 1: SmartMongo (Real MongoDB)
|
||||
### Option 1: LocalTsmDb (Zero-Config Local Database) ⭐ NEW
|
||||
|
||||
The easiest way to get started — just point it at a folder and you have a persistent MongoDB-compatible database with automatic port discovery!
|
||||
|
||||
```typescript
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
|
||||
// Create a local database backed by files
|
||||
const db = new LocalTsmDb({ folderPath: './my-data' });
|
||||
|
||||
// Start and get a connected MongoDB client
|
||||
const client = await db.start();
|
||||
|
||||
// Use exactly like MongoDB
|
||||
const users = client.db('myapp').collection('users');
|
||||
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
|
||||
|
||||
const user = await users.findOne({ name: 'Alice' });
|
||||
console.log(user); // { _id: ObjectId(...), name: 'Alice', email: 'alice@example.com' }
|
||||
|
||||
// Data persists to disk automatically!
|
||||
await db.stop();
|
||||
|
||||
// Later... data is still there
|
||||
const db2 = new LocalTsmDb({ folderPath: './my-data' });
|
||||
const client2 = await db2.start();
|
||||
const savedUser = await client2.db('myapp').collection('users').findOne({ name: 'Alice' });
|
||||
// savedUser exists!
|
||||
```
|
||||
|
||||
### Option 2: TsmDB (Wire Protocol Server)
|
||||
|
||||
A lightweight, pure TypeScript MongoDB-compatible server — use the official `mongodb` driver directly!
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
|
||||
// Start TsmDB server
|
||||
const server = new tsmdb.TsmdbServer({ port: 27017 });
|
||||
await server.start();
|
||||
|
||||
// Connect with the official MongoDB driver
|
||||
const client = new MongoClient('mongodb://127.0.0.1:27017');
|
||||
await client.connect();
|
||||
|
||||
// Use exactly like real MongoDB
|
||||
const db = client.db('myapp');
|
||||
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
|
||||
|
||||
const user = await db.collection('users').findOne({ name: 'Alice' });
|
||||
console.log(user); // { _id: ObjectId(...), name: 'Alice', age: 30 }
|
||||
|
||||
// Clean up
|
||||
await client.close();
|
||||
await server.stop();
|
||||
```
|
||||
|
||||
### Option 3: SmartMongo (Real MongoDB)
|
||||
|
||||
Spin up a real MongoDB replica set in memory — perfect for integration tests that need full MongoDB compatibility.
|
||||
|
||||
@@ -51,34 +109,42 @@ console.log(descriptor.mongoDbUrl); // mongodb://127.0.0.1:xxxxx/...
|
||||
await mongo.stop();
|
||||
```
|
||||
|
||||
### Option 2: TsmDB (Wire Protocol Server)
|
||||
## 📖 LocalTsmDb API
|
||||
|
||||
A lightweight, pure TypeScript MongoDB-compatible server that speaks the wire protocol — use the official `mongodb` driver directly!
|
||||
The simplest option for local development and prototyping — zero config, auto port discovery, and automatic persistence.
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```typescript
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
|
||||
// Start TsmDB server
|
||||
const server = new tsmdb.TsmdbServer({ port: 27017 });
|
||||
await server.start();
|
||||
const db = new LocalTsmDb({
|
||||
folderPath: './data', // Required: where to store data
|
||||
port: 27017, // Optional: defaults to auto-discovery
|
||||
host: '127.0.0.1', // Optional: bind address
|
||||
});
|
||||
|
||||
// Connect with the official MongoDB driver!
|
||||
const client = new MongoClient('mongodb://127.0.0.1:27017');
|
||||
await client.connect();
|
||||
// Start and get connected client
|
||||
const client = await db.start();
|
||||
|
||||
// Use exactly like real MongoDB
|
||||
const db = client.db('myapp');
|
||||
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
|
||||
// Access the underlying server if needed
|
||||
const server = db.getServer();
|
||||
const uri = db.getConnectionUri();
|
||||
|
||||
const user = await db.collection('users').findOne({ name: 'Alice' });
|
||||
console.log(user); // { _id: ObjectId(...), name: 'Alice', age: 30 }
|
||||
// Check status
|
||||
console.log(db.running); // true
|
||||
|
||||
// Clean up
|
||||
await client.close();
|
||||
await server.stop();
|
||||
// Stop when done
|
||||
await db.stop();
|
||||
```
|
||||
|
||||
### Features
|
||||
|
||||
- 🔍 **Auto Port Discovery** — Automatically finds an available port if 27017 is in use
|
||||
- 💾 **Automatic Persistence** — Data saved to files, survives restarts
|
||||
- 🔌 **Pre-connected Client** — `start()` returns a ready-to-use MongoDB client
|
||||
- 🎯 **Zero Config** — Just specify a folder path and you're good to go
|
||||
|
||||
## 📖 SmartMongo API
|
||||
|
||||
### Creating an Instance
|
||||
@@ -293,11 +359,11 @@ const server = new tsmdb.TsmdbServer({
|
||||
});
|
||||
|
||||
// File-based - persistent storage with optional checksums
|
||||
import { FileStorageAdapter } from '@push.rocks/smartmongo/tsmdb';
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
|
||||
const adapter = new FileStorageAdapter('./data/tsmdb', {
|
||||
enableChecksums: true, // CRC32 checksums for data integrity
|
||||
strictChecksums: false // Log warnings vs throw on mismatch
|
||||
const server = new tsmdb.TsmdbServer({
|
||||
storage: 'file',
|
||||
storagePath: './data/tsmdb'
|
||||
});
|
||||
```
|
||||
|
||||
@@ -331,14 +397,14 @@ import { tsmdb } from '@push.rocks/smartmongo';
|
||||
|
||||
// For debugging, you can access the query planner
|
||||
const planner = new tsmdb.QueryPlanner(indexEngine);
|
||||
const plan = planner.createPlan(filter);
|
||||
const plan = await planner.plan(filter);
|
||||
|
||||
console.log(plan);
|
||||
// {
|
||||
// type: 'IXSCAN', // or 'IXSCAN_RANGE', 'COLLSCAN'
|
||||
// indexName: 'email_1',
|
||||
// estimatedCost: 1,
|
||||
// selectivity: 0.001
|
||||
// selectivity: 0.01,
|
||||
// indexCovering: true
|
||||
// }
|
||||
```
|
||||
|
||||
@@ -357,10 +423,10 @@ await wal.initialize();
|
||||
// - Timestamp
|
||||
// - Operation type (insert, update, delete, checkpoint)
|
||||
// - Document data (BSON serialized)
|
||||
// - CRC32 checksum
|
||||
// - CRC32 checksum for integrity
|
||||
|
||||
// Recovery support
|
||||
const entries = await wal.getEntriesAfter(lastCheckpointLsn);
|
||||
const entries = wal.getEntriesAfter(lastCheckpointLsn);
|
||||
```
|
||||
|
||||
### 🔐 Session Management
|
||||
@@ -393,15 +459,10 @@ try {
|
||||
File-based storage supports CRC32 checksums to detect corruption:
|
||||
|
||||
```typescript
|
||||
import { FileStorageAdapter } from '@push.rocks/smartmongo/tsmdb';
|
||||
|
||||
const adapter = new FileStorageAdapter('./data', {
|
||||
enableChecksums: true,
|
||||
strictChecksums: true // Throw error on corruption (vs warning)
|
||||
});
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
|
||||
// Checksums are used internally for WAL and data integrity
|
||||
// Documents are checksummed on write, verified on read
|
||||
// Checksums are automatically stripped before returning to client
|
||||
```
|
||||
|
||||
### 📋 Supported Wire Protocol Commands
|
||||
@@ -420,6 +481,38 @@ TsmDB supports MongoDB wire protocol versions 0-21, compatible with MongoDB 3.6
|
||||
|
||||
## 🧪 Testing Examples
|
||||
|
||||
### Jest/Mocha with LocalTsmDb
|
||||
|
||||
```typescript
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient, Db } from 'mongodb';
|
||||
|
||||
let db: LocalTsmDb;
|
||||
let client: MongoClient;
|
||||
|
||||
beforeAll(async () => {
|
||||
db = new LocalTsmDb({ folderPath: './test-data' });
|
||||
client = await db.start();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await db.stop();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
// Clean slate for each test
|
||||
await client.db('test').dropDatabase();
|
||||
});
|
||||
|
||||
test('should insert and find user', async () => {
|
||||
const users = client.db('test').collection('users');
|
||||
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
|
||||
|
||||
const user = await users.findOne({ name: 'Alice' });
|
||||
expect(user?.email).toBe('alice@example.com');
|
||||
});
|
||||
```
|
||||
|
||||
### Jest/Mocha with TsmDB
|
||||
|
||||
```typescript
|
||||
@@ -428,7 +521,7 @@ import { MongoClient, Db } from 'mongodb';
|
||||
|
||||
let server: tsmdb.TsmdbServer;
|
||||
let client: MongoClient;
|
||||
let db: Db;
|
||||
let testDb: Db;
|
||||
|
||||
beforeAll(async () => {
|
||||
server = new tsmdb.TsmdbServer({ port: 27117 });
|
||||
@@ -436,7 +529,7 @@ beforeAll(async () => {
|
||||
|
||||
client = new MongoClient('mongodb://127.0.0.1:27117');
|
||||
await client.connect();
|
||||
db = client.db('test');
|
||||
testDb = client.db('test');
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
@@ -445,12 +538,11 @@ afterAll(async () => {
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
// Clean slate for each test
|
||||
await db.dropDatabase();
|
||||
await testDb.dropDatabase();
|
||||
});
|
||||
|
||||
test('should insert and find user', async () => {
|
||||
const users = db.collection('users');
|
||||
const users = testDb.collection('users');
|
||||
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
|
||||
|
||||
const user = await users.findOne({ name: 'Alice' });
|
||||
@@ -462,22 +554,18 @@ test('should insert and find user', async () => {
|
||||
|
||||
```typescript
|
||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||
import { tsmdb } from '@push.rocks/smartmongo';
|
||||
import { MongoClient } from 'mongodb';
|
||||
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
|
||||
let server: tsmdb.TsmdbServer;
|
||||
let client: MongoClient;
|
||||
let db: LocalTsmDb;
|
||||
|
||||
tap.test('setup', async () => {
|
||||
server = new tsmdb.TsmdbServer({ port: 27117 });
|
||||
await server.start();
|
||||
client = new MongoClient('mongodb://127.0.0.1:27117');
|
||||
await client.connect();
|
||||
db = new LocalTsmDb({ folderPath: './test-data' });
|
||||
await db.start();
|
||||
});
|
||||
|
||||
tap.test('should perform CRUD operations', async () => {
|
||||
const db = client.db('test');
|
||||
const col = db.collection('items');
|
||||
const client = db.getClient();
|
||||
const col = client.db('test').collection('items');
|
||||
|
||||
// Create
|
||||
const result = await col.insertOne({ name: 'Widget', price: 9.99 });
|
||||
@@ -499,8 +587,7 @@ tap.test('should perform CRUD operations', async () => {
|
||||
});
|
||||
|
||||
tap.test('teardown', async () => {
|
||||
await client.close();
|
||||
await server.stop();
|
||||
await db.stop();
|
||||
});
|
||||
|
||||
export default tap.start();
|
||||
@@ -508,6 +595,15 @@ export default tap.start();
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
### Module Structure
|
||||
|
||||
```
|
||||
@push.rocks/smartmongo
|
||||
├── SmartMongo → Real MongoDB memory server (mongodb-memory-server wrapper)
|
||||
├── tsmdb → Wire protocol server with full engine stack
|
||||
└── LocalTsmDb → Zero-config local database (convenience wrapper)
|
||||
```
|
||||
|
||||
### TsmDB Wire Protocol Stack
|
||||
|
||||
```
|
||||
|
||||
@@ -3,6 +3,6 @@
|
||||
*/
|
||||
export const commitinfo = {
|
||||
name: '@push.rocks/smartmongo',
|
||||
version: '4.1.1',
|
||||
version: '4.3.0',
|
||||
description: 'A module for creating and managing a local MongoDB instance for testing purposes.'
|
||||
}
|
||||
|
||||
78
ts/index.ts
78
ts/index.ts
@@ -1,74 +1,14 @@
|
||||
import { commitinfo } from './00_commitinfo_data.js';
|
||||
import * as plugins from './smartmongo.plugins.js';
|
||||
|
||||
// Export SmartMongo from ts_mongotools
|
||||
export { SmartMongo } from './ts_mongotools/index.js';
|
||||
|
||||
// Export TsmDB module
|
||||
export * as tsmdb from './tsmdb/index.js';
|
||||
export * as tsmdb from './ts_tsmdb/index.js';
|
||||
|
||||
export class SmartMongo {
|
||||
// STATIC
|
||||
public static async createAndStart(replCountArg: number = 1) {
|
||||
const smartMongoInstance = new SmartMongo();
|
||||
await smartMongoInstance.start(replCountArg);
|
||||
return smartMongoInstance;
|
||||
}
|
||||
// Export LocalTsmDb from ts_local
|
||||
export { LocalTsmDb } from './ts_local/index.js';
|
||||
export type { ILocalTsmDbOptions } from './ts_local/index.js';
|
||||
|
||||
// INSTANCE
|
||||
private _readyDeferred = plugins.smartpromise.defer();
|
||||
public readyPromise = this._readyDeferred.promise;
|
||||
public mongoReplicaSet: plugins.mongoPlugin.MongoMemoryReplSet;
|
||||
|
||||
constructor() {}
|
||||
|
||||
public async start(countArg: number = 1) {
|
||||
this.mongoReplicaSet = await plugins.mongoPlugin.MongoMemoryReplSet.create({
|
||||
replSet: { count: countArg },
|
||||
instanceOpts: [
|
||||
{
|
||||
storageEngine: 'wiredTiger',
|
||||
},
|
||||
],
|
||||
});
|
||||
this._readyDeferred.resolve();
|
||||
console.log(`mongoReplicaSet with ${countArg} replicas started.`);
|
||||
console.log(`@pushrocks/smartmongo version ${commitinfo.version}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* returns a mongo descriptor for modules like
|
||||
* @pushrocks/smartfile.
|
||||
*/
|
||||
public async getMongoDescriptor(): Promise<plugins.smartdata.IMongoDescriptor> {
|
||||
await this.readyPromise;
|
||||
return {
|
||||
mongoDbName: `smartmongo_testdatabase`,
|
||||
mongoDbUrl: this.mongoReplicaSet.getUri(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* stops the smartmongo instance
|
||||
* and cleans up after itself
|
||||
*/
|
||||
public async stop() {
|
||||
await this.mongoReplicaSet.stop();
|
||||
await this.mongoReplicaSet.cleanup();
|
||||
}
|
||||
|
||||
/**
|
||||
* like stop() but allows you to actually store
|
||||
* the database on disk
|
||||
*/
|
||||
public async stopAndDumpToDir(
|
||||
dirArg: string,
|
||||
nameFunctionArg?: (doc: any) => string,
|
||||
emptyDirArg = true,
|
||||
) {
|
||||
const mongodumpInstance = new plugins.mongodump.MongoDump();
|
||||
const mongodumpTarget = await mongodumpInstance.addMongoTargetByMongoDescriptor(
|
||||
await this.getMongoDescriptor(),
|
||||
);
|
||||
await mongodumpTarget.dumpAllCollectionsToDir(dirArg, nameFunctionArg, emptyDirArg);
|
||||
await mongodumpInstance.stop();
|
||||
await this.stop();
|
||||
}
|
||||
}
|
||||
// Export commitinfo
|
||||
export { commitinfo };
|
||||
|
||||
138
ts/ts_local/classes.localtsmdb.ts
Normal file
138
ts/ts_local/classes.localtsmdb.ts
Normal file
@@ -0,0 +1,138 @@
|
||||
import * as plugins from './plugins.js';
|
||||
import { TsmdbServer } from '../ts_tsmdb/index.js';
|
||||
import type { MongoClient } from 'mongodb';
|
||||
|
||||
export interface ILocalTsmDbOptions {
|
||||
folderPath: string;
|
||||
port?: number;
|
||||
host?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* LocalTsmDb - Convenience class for local MongoDB-compatible database
|
||||
*
|
||||
* This class wraps TsmdbServer and provides a simple interface for
|
||||
* starting a local file-based MongoDB-compatible server and connecting to it.
|
||||
*
|
||||
* @example
|
||||
* ```typescript
|
||||
* import { LocalTsmDb } from '@push.rocks/smartmongo';
|
||||
*
|
||||
* const db = new LocalTsmDb({ folderPath: './data' });
|
||||
* const client = await db.start();
|
||||
*
|
||||
* // Use the MongoDB client
|
||||
* const collection = client.db('mydb').collection('users');
|
||||
* await collection.insertOne({ name: 'Alice' });
|
||||
*
|
||||
* // When done
|
||||
* await db.stop();
|
||||
* ```
|
||||
*/
|
||||
export class LocalTsmDb {
|
||||
private options: ILocalTsmDbOptions;
|
||||
private server: TsmdbServer | null = null;
|
||||
private client: MongoClient | null = null;
|
||||
|
||||
constructor(options: ILocalTsmDbOptions) {
|
||||
this.options = options;
|
||||
}
|
||||
|
||||
/**
|
||||
* Find an available port starting from the given port
|
||||
*/
|
||||
private async findAvailablePort(startPort = 27017): Promise<number> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const server = plugins.net.createServer();
|
||||
server.listen(startPort, '127.0.0.1', () => {
|
||||
const addr = server.address();
|
||||
const port = typeof addr === 'object' && addr ? addr.port : startPort;
|
||||
server.close(() => resolve(port));
|
||||
});
|
||||
server.on('error', () => {
|
||||
this.findAvailablePort(startPort + 1).then(resolve).catch(reject);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the local TsmDB server and return a connected MongoDB client
|
||||
*/
|
||||
async start(): Promise<MongoClient> {
|
||||
if (this.server && this.client) {
|
||||
throw new Error('LocalTsmDb is already running');
|
||||
}
|
||||
|
||||
const port = this.options.port ?? await this.findAvailablePort();
|
||||
const host = this.options.host ?? '127.0.0.1';
|
||||
|
||||
this.server = new TsmdbServer({
|
||||
port,
|
||||
host,
|
||||
storage: 'file',
|
||||
storagePath: this.options.folderPath,
|
||||
});
|
||||
await this.server.start();
|
||||
|
||||
// Dynamically import mongodb to avoid requiring it as a hard dependency
|
||||
const mongodb = await import('mongodb');
|
||||
this.client = new mongodb.MongoClient(this.server.getConnectionUri(), {
|
||||
directConnection: true,
|
||||
serverSelectionTimeoutMS: 5000,
|
||||
});
|
||||
await this.client.connect();
|
||||
|
||||
return this.client;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the MongoDB client (throws if not started)
|
||||
*/
|
||||
getClient(): MongoClient {
|
||||
if (!this.client) {
|
||||
throw new Error('LocalTsmDb is not running. Call start() first.');
|
||||
}
|
||||
return this.client;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the underlying TsmdbServer instance (throws if not started)
|
||||
*/
|
||||
getServer(): TsmdbServer {
|
||||
if (!this.server) {
|
||||
throw new Error('LocalTsmDb is not running. Call start() first.');
|
||||
}
|
||||
return this.server;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the connection URI
|
||||
*/
|
||||
getConnectionUri(): string {
|
||||
if (!this.server) {
|
||||
throw new Error('LocalTsmDb is not running. Call start() first.');
|
||||
}
|
||||
return this.server.getConnectionUri();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if the server is running
|
||||
*/
|
||||
get running(): boolean {
|
||||
return this.server !== null && this.server.running;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop the local TsmDB server and close the client connection
|
||||
*/
|
||||
async stop(): Promise<void> {
|
||||
if (this.client) {
|
||||
await this.client.close();
|
||||
this.client = null;
|
||||
}
|
||||
if (this.server) {
|
||||
await this.server.stop();
|
||||
this.server = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
2
ts/ts_local/index.ts
Normal file
2
ts/ts_local/index.ts
Normal file
@@ -0,0 +1,2 @@
|
||||
export { LocalTsmDb } from './classes.localtsmdb.js';
|
||||
export type { ILocalTsmDbOptions } from './classes.localtsmdb.js';
|
||||
4
ts/ts_local/plugins.ts
Normal file
4
ts/ts_local/plugins.ts
Normal file
@@ -0,0 +1,4 @@
|
||||
import * as smartpromise from '@push.rocks/smartpromise';
|
||||
import * as net from 'net';
|
||||
|
||||
export { smartpromise, net };
|
||||
71
ts/ts_mongotools/classes.smartmongo.ts
Normal file
71
ts/ts_mongotools/classes.smartmongo.ts
Normal file
@@ -0,0 +1,71 @@
|
||||
import { commitinfo } from '../00_commitinfo_data.js';
|
||||
import * as plugins from './plugins.js';
|
||||
|
||||
export class SmartMongo {
|
||||
// STATIC
|
||||
public static async createAndStart(replCountArg: number = 1) {
|
||||
const smartMongoInstance = new SmartMongo();
|
||||
await smartMongoInstance.start(replCountArg);
|
||||
return smartMongoInstance;
|
||||
}
|
||||
|
||||
// INSTANCE
|
||||
private _readyDeferred = plugins.smartpromise.defer();
|
||||
public readyPromise = this._readyDeferred.promise;
|
||||
public mongoReplicaSet: plugins.mongoPlugin.MongoMemoryReplSet;
|
||||
|
||||
constructor() {}
|
||||
|
||||
public async start(countArg: number = 1) {
|
||||
this.mongoReplicaSet = await plugins.mongoPlugin.MongoMemoryReplSet.create({
|
||||
replSet: { count: countArg },
|
||||
instanceOpts: [
|
||||
{
|
||||
storageEngine: 'wiredTiger',
|
||||
},
|
||||
],
|
||||
});
|
||||
this._readyDeferred.resolve();
|
||||
console.log(`mongoReplicaSet with ${countArg} replicas started.`);
|
||||
console.log(`@pushrocks/smartmongo version ${commitinfo.version}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* returns a mongo descriptor for modules like
|
||||
* @pushrocks/smartfile.
|
||||
*/
|
||||
public async getMongoDescriptor(): Promise<plugins.smartdata.IMongoDescriptor> {
|
||||
await this.readyPromise;
|
||||
return {
|
||||
mongoDbName: `smartmongo_testdatabase`,
|
||||
mongoDbUrl: this.mongoReplicaSet.getUri(),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* stops the smartmongo instance
|
||||
* and cleans up after itself
|
||||
*/
|
||||
public async stop() {
|
||||
await this.mongoReplicaSet.stop();
|
||||
await this.mongoReplicaSet.cleanup();
|
||||
}
|
||||
|
||||
/**
|
||||
* like stop() but allows you to actually store
|
||||
* the database on disk
|
||||
*/
|
||||
public async stopAndDumpToDir(
|
||||
dirArg: string,
|
||||
nameFunctionArg?: (doc: any) => string,
|
||||
emptyDirArg = true,
|
||||
) {
|
||||
const mongodumpInstance = new plugins.mongodump.MongoDump();
|
||||
const mongodumpTarget = await mongodumpInstance.addMongoTargetByMongoDescriptor(
|
||||
await this.getMongoDescriptor(),
|
||||
);
|
||||
await mongodumpTarget.dumpAllCollectionsToDir(dirArg, nameFunctionArg, emptyDirArg);
|
||||
await mongodumpInstance.stop();
|
||||
await this.stop();
|
||||
}
|
||||
}
|
||||
2
ts/ts_mongotools/index.ts
Normal file
2
ts/ts_mongotools/index.ts
Normal file
@@ -0,0 +1,2 @@
|
||||
export * from './plugins.js';
|
||||
export { SmartMongo } from './classes.smartmongo.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument, IAggregateOptions } from '../types/interfaces.js';
|
||||
|
||||
// Import mingo Aggregator
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
|
||||
|
||||
// Simple B-Tree implementation for range queries
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument, ISortSpecification, ISortDirection } from '../types/interfaces.js';
|
||||
|
||||
// Import mingo Query class
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument } from '../types/interfaces.js';
|
||||
import { IndexEngine } from './IndexEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { TransactionEngine } from './TransactionEngine.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
|
||||
import type { Document, IStoredDocument, ITransactionOptions } from '../types/interfaces.js';
|
||||
import { TsmdbTransactionError, TsmdbWriteConflictError } from '../errors/TsmdbErrors.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument } from '../types/interfaces.js';
|
||||
import { QueryEngine } from './QueryEngine.js';
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
// Use the official MongoDB driver to connect to TsmdbServer
|
||||
|
||||
// Re-export plugins for external use
|
||||
import * as plugins from './tsmdb.plugins.js';
|
||||
import * as plugins from './plugins.js';
|
||||
export { plugins };
|
||||
|
||||
// Export BSON types for convenience
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from '../storage/IStorageAdapter.js';
|
||||
import type { IParsedCommand } from './WireProtocol.js';
|
||||
import type { TsmdbServer } from './TsmdbServer.js';
|
||||
@@ -1,5 +1,5 @@
|
||||
import * as net from 'net';
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import { WireProtocol, OP_QUERY } from './WireProtocol.js';
|
||||
import { CommandRouter } from './CommandRouter.js';
|
||||
import { MemoryStorageAdapter } from '../storage/MemoryStorageAdapter.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
|
||||
/**
|
||||
* MongoDB Wire Protocol Implementation
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import { SessionEngine } from '../../engine/SessionEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext, ICursorState } from '../CommandRouter.js';
|
||||
import { AggregationEngine } from '../../engine/AggregationEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
import { QueryEngine } from '../../engine/QueryEngine.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext, ICursorState } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
import { QueryEngine } from '../../engine/QueryEngine.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import { IndexEngine } from '../../engine/IndexEngine.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../../tsmdb.plugins.js';
|
||||
import * as plugins from '../../plugins.js';
|
||||
import type { ICommandHandler, IHandlerContext } from '../CommandRouter.js';
|
||||
import type { IStoredDocument } from '../../types/interfaces.js';
|
||||
import { QueryEngine } from '../../engine/QueryEngine.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from './IStorageAdapter.js';
|
||||
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
|
||||
import { calculateDocumentChecksum, verifyChecksum } from '../utils/checksum.js';
|
||||
@@ -1,4 +1,4 @@
|
||||
import type * as plugins from '../tsmdb.plugins.js';
|
||||
import type * as plugins from '../plugins.js';
|
||||
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from './IStorageAdapter.js';
|
||||
import type { IStoredDocument, IOpLogEntry, Document } from '../types/interfaces.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { IStorageAdapter } from './IStorageAdapter.js';
|
||||
import type { IOpLogEntry, Document, IResumeToken, ChangeStreamOperationType } from '../types/interfaces.js';
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import * as plugins from '../tsmdb.plugins.js';
|
||||
import * as plugins from '../plugins.js';
|
||||
import type { Document, IStoredDocument } from '../types/interfaces.js';
|
||||
|
||||
/**
|
||||
@@ -1,4 +1,4 @@
|
||||
import type * as plugins from '../tsmdb.plugins.js';
|
||||
import type * as plugins from '../plugins.js';
|
||||
|
||||
// ============================================================================
|
||||
// Document Types
|
||||
Reference in New Issue
Block a user