|
|
|
@@ -1,6 +1,6 @@
|
|
|
|
# @push.rocks/smartmongo
|
|
|
|
# @push.rocks/smartmongo
|
|
|
|
|
|
|
|
|
|
|
|
A powerful MongoDB toolkit for testing and development — featuring a real MongoDB memory server (**SmartMongo**), an ultra-fast wire-protocol-compatible in-memory database server (**TsmDB**), and a zero-config local database (**LocalTsmDb**). 🚀
|
|
|
|
A powerful MongoDB toolkit for testing and development — featuring a real MongoDB memory server (**SmartMongo**), an ultra-fast wire-protocol-compatible in-memory database server (**TsmDB**), and a zero-config local database (**LocalTsmDb**).
|
|
|
|
|
|
|
|
|
|
|
|
## Install
|
|
|
|
## Install
|
|
|
|
|
|
|
|
|
|
|
|
@@ -21,16 +21,17 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
|
|
|
|
| Feature | SmartMongo | TsmDB | LocalTsmDb |
|
|
|
|
| Feature | SmartMongo | TsmDB | LocalTsmDb |
|
|
|
|
|---------|------------|-------|------------|
|
|
|
|
|---------|------------|-------|------------|
|
|
|
|
| **Type** | Real MongoDB (memory server) | Wire protocol server | Zero-config local DB |
|
|
|
|
| **Type** | Real MongoDB (memory server) | Wire protocol server | Zero-config local DB |
|
|
|
|
| **Speed** | ~2-5s startup | ⚡ Instant (~5ms) | ⚡ Instant (Unix socket) |
|
|
|
|
| **Speed** | ~2-5s startup | Instant (~5ms) | Instant (Unix socket) |
|
|
|
|
| **Compatibility** | 100% MongoDB | MongoDB driver compatible | MongoDB driver compatible |
|
|
|
|
| **Compatibility** | 100% MongoDB | MongoDB driver compatible | MongoDB driver compatible |
|
|
|
|
| **Dependencies** | Downloads MongoDB binary | Zero external deps | Zero external deps (no MongoDB driver!) |
|
|
|
|
| **Dependencies** | Downloads MongoDB binary | Zero external deps | Zero external deps (no MongoDB driver!) |
|
|
|
|
| **Replication** | ✅ Full replica set | Single node | Single node |
|
|
|
|
| **Connection** | TCP | TCP or Unix socket | Unix socket (default) |
|
|
|
|
|
|
|
|
| **Replication** | Full replica set | Single node | Single node |
|
|
|
|
| **Persistence** | Dump to directory | Memory or file | File-based (automatic) |
|
|
|
|
| **Persistence** | Dump to directory | Memory or file | File-based (automatic) |
|
|
|
|
| **Use Case** | Integration testing | Unit testing, CI/CD | Quick prototyping, local dev |
|
|
|
|
| **Use Case** | Integration testing | Unit testing, CI/CD | Quick prototyping, local dev |
|
|
|
|
|
|
|
|
|
|
|
|
## 🚀 Quick Start
|
|
|
|
## Quick Start
|
|
|
|
|
|
|
|
|
|
|
|
### Option 1: LocalTsmDb (Zero-Config Local Database) ⭐ NEW
|
|
|
|
### Option 1: LocalTsmDb (Zero-Config Local Database)
|
|
|
|
|
|
|
|
|
|
|
|
The easiest way to get started — just point it at a folder and you have a persistent MongoDB-compatible database using Unix sockets. No port conflicts, no MongoDB driver dependency in LocalTsmDb!
|
|
|
|
The easiest way to get started — just point it at a folder and you have a persistent MongoDB-compatible database using Unix sockets. No port conflicts, no MongoDB driver dependency in LocalTsmDb!
|
|
|
|
|
|
|
|
|
|
|
|
@@ -76,7 +77,7 @@ A lightweight, pure TypeScript MongoDB-compatible server — use the official `m
|
|
|
|
import { tsmdb } from '@push.rocks/smartmongo';
|
|
|
|
import { tsmdb } from '@push.rocks/smartmongo';
|
|
|
|
import { MongoClient } from 'mongodb';
|
|
|
|
import { MongoClient } from 'mongodb';
|
|
|
|
|
|
|
|
|
|
|
|
// Start TsmDB server
|
|
|
|
// Start TsmDB server (TCP mode)
|
|
|
|
const server = new tsmdb.TsmdbServer({ port: 27017 });
|
|
|
|
const server = new tsmdb.TsmdbServer({ port: 27017 });
|
|
|
|
await server.start();
|
|
|
|
await server.start();
|
|
|
|
|
|
|
|
|
|
|
|
@@ -117,20 +118,55 @@ console.log(descriptor.mongoDbUrl); // mongodb://127.0.0.1:xxxxx/...
|
|
|
|
await mongo.stop();
|
|
|
|
await mongo.stop();
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## 📖 LocalTsmDb API
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## LocalTsmDb API
|
|
|
|
|
|
|
|
|
|
|
|
The simplest option for local development and prototyping — lightweight, Unix socket-based, and automatic persistence.
|
|
|
|
The simplest option for local development and prototyping — lightweight, Unix socket-based, and automatic persistence.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Configuration
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
|
|
|
|
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
|
|
|
|
|
|
|
import type { ILocalTsmDbOptions, ILocalTsmDbConnectionInfo } from '@push.rocks/smartmongo';
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
const options: ILocalTsmDbOptions = {
|
|
|
|
|
|
|
|
folderPath: './data', // Required: where to store data
|
|
|
|
|
|
|
|
socketPath: '/tmp/my.sock', // Optional: custom socket path (default: auto-generated)
|
|
|
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
const db = new LocalTsmDb(options);
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Methods
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Method | Returns | Description |
|
|
|
|
|
|
|
|
|--------|---------|-------------|
|
|
|
|
|
|
|
|
| `start()` | `Promise<ILocalTsmDbConnectionInfo>` | Starts the server and returns connection info |
|
|
|
|
|
|
|
|
| `stop()` | `Promise<void>` | Stops the server and cleans up the socket |
|
|
|
|
|
|
|
|
| `getConnectionInfo()` | `ILocalTsmDbConnectionInfo` | Returns current connection info |
|
|
|
|
|
|
|
|
| `getConnectionUri()` | `string` | Returns the MongoDB connection URI |
|
|
|
|
|
|
|
|
| `getServer()` | `TsmdbServer` | Returns the underlying TsmDB server instance |
|
|
|
|
|
|
|
|
| `running` | `boolean` | Property indicating if the server is running |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Connection Info
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The `start()` method returns an `ILocalTsmDbConnectionInfo` object:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
|
|
|
|
interface ILocalTsmDbConnectionInfo {
|
|
|
|
|
|
|
|
socketPath: string; // The Unix socket file path, e.g., /tmp/smartmongo-abc123.sock
|
|
|
|
|
|
|
|
connectionUri: string; // MongoDB URI, e.g., mongodb://%2Ftmp%2Fsmartmongo-abc123.sock
|
|
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### Basic Usage
|
|
|
|
### Basic Usage
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
|
|
|
import { LocalTsmDb } from '@push.rocks/smartmongo';
|
|
|
|
import { MongoClient } from 'mongodb';
|
|
|
|
import { MongoClient } from 'mongodb';
|
|
|
|
|
|
|
|
|
|
|
|
const db = new LocalTsmDb({
|
|
|
|
const db = new LocalTsmDb({ folderPath: './data' });
|
|
|
|
folderPath: './data', // Required: where to store data
|
|
|
|
|
|
|
|
socketPath: '/tmp/my.sock', // Optional: custom socket path (default: auto-generated)
|
|
|
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// Start and get connection info
|
|
|
|
// Start and get connection info
|
|
|
|
const { socketPath, connectionUri } = await db.start();
|
|
|
|
const { socketPath, connectionUri } = await db.start();
|
|
|
|
@@ -145,10 +181,6 @@ await client.connect();
|
|
|
|
const users = client.db('mydb').collection('users');
|
|
|
|
const users = client.db('mydb').collection('users');
|
|
|
|
await users.insertOne({ name: 'Alice' });
|
|
|
|
await users.insertOne({ name: 'Alice' });
|
|
|
|
|
|
|
|
|
|
|
|
// Access the underlying server if needed
|
|
|
|
|
|
|
|
const server = db.getServer();
|
|
|
|
|
|
|
|
const uri = db.getConnectionUri();
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// Check status
|
|
|
|
// Check status
|
|
|
|
console.log(db.running); // true
|
|
|
|
console.log(db.running); // true
|
|
|
|
|
|
|
|
|
|
|
|
@@ -159,13 +191,17 @@ await db.stop();
|
|
|
|
|
|
|
|
|
|
|
|
### Features
|
|
|
|
### Features
|
|
|
|
|
|
|
|
|
|
|
|
- 🔌 **Unix Sockets** — No port conflicts, faster IPC than TCP
|
|
|
|
- **Unix Sockets** — No port conflicts, faster IPC than TCP
|
|
|
|
- 💾 **Automatic Persistence** — Data saved to files, survives restarts
|
|
|
|
- **Automatic Persistence** — Data saved to files, survives restarts
|
|
|
|
- 🪶 **Lightweight** — No MongoDB driver dependency in LocalTsmDb itself
|
|
|
|
- **Lightweight** — No MongoDB driver dependency in LocalTsmDb itself
|
|
|
|
- 🎯 **Zero Config** — Just specify a folder path and you're good to go
|
|
|
|
- **Zero Config** — Just specify a folder path and you're good to go
|
|
|
|
- 🔗 **Connection URI** — Ready-to-use URI for your own MongoClient
|
|
|
|
- **Connection URI** — Ready-to-use URI for your own MongoClient
|
|
|
|
|
|
|
|
|
|
|
|
## 📖 SmartMongo API
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## SmartMongo API
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Full MongoDB replica set in memory using `mongodb-memory-server`.
|
|
|
|
|
|
|
|
|
|
|
|
### Creating an Instance
|
|
|
|
### Creating an Instance
|
|
|
|
|
|
|
|
|
|
|
|
@@ -189,7 +225,7 @@ const descriptor = await mongo.getMongoDescriptor();
|
|
|
|
// }
|
|
|
|
// }
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### Stopping & Cleanup
|
|
|
|
### Stopping and Cleanup
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
// Simple stop (data discarded)
|
|
|
|
// Simple stop (data discarded)
|
|
|
|
@@ -202,13 +238,18 @@ await mongo.stopAndDumpToDir('./test-data');
|
|
|
|
await mongo.stopAndDumpToDir('./test-data', (doc) => `${doc.collection}-${doc._id}.bson`);
|
|
|
|
await mongo.stopAndDumpToDir('./test-data', (doc) => `${doc.collection}-${doc._id}.bson`);
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## 🔧 TsmDB API
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## TsmDB API
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Pure TypeScript MongoDB wire protocol server. No external dependencies.
|
|
|
|
|
|
|
|
|
|
|
|
### Server Configuration
|
|
|
|
### Server Configuration
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
import { tsmdb } from '@push.rocks/smartmongo';
|
|
|
|
import { tsmdb } from '@push.rocks/smartmongo';
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// TCP mode (default)
|
|
|
|
const server = new tsmdb.TsmdbServer({
|
|
|
|
const server = new tsmdb.TsmdbServer({
|
|
|
|
port: 27017, // Default MongoDB port
|
|
|
|
port: 27017, // Default MongoDB port
|
|
|
|
host: '127.0.0.1', // Bind address
|
|
|
|
host: '127.0.0.1', // Bind address
|
|
|
|
@@ -216,13 +257,25 @@ const server = new tsmdb.TsmdbServer({
|
|
|
|
storagePath: './data', // For file-based storage
|
|
|
|
storagePath: './data', // For file-based storage
|
|
|
|
});
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// Unix socket mode (no port conflicts!)
|
|
|
|
|
|
|
|
const server = new tsmdb.TsmdbServer({
|
|
|
|
|
|
|
|
socketPath: '/tmp/my-tsmdb.sock',
|
|
|
|
|
|
|
|
storage: 'file',
|
|
|
|
|
|
|
|
storagePath: './data',
|
|
|
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
|
|
await server.start();
|
|
|
|
await server.start();
|
|
|
|
console.log(server.getConnectionUri()); // mongodb://127.0.0.1:27017
|
|
|
|
console.log(server.getConnectionUri());
|
|
|
|
|
|
|
|
// TCP: mongodb://127.0.0.1:27017
|
|
|
|
|
|
|
|
// Socket: mongodb://%2Ftmp%2Fmy-tsmdb.sock
|
|
|
|
|
|
|
|
|
|
|
|
// Server properties
|
|
|
|
// Server properties
|
|
|
|
console.log(server.running); // true
|
|
|
|
console.log(server.running); // true
|
|
|
|
console.log(server.getUptime()); // seconds
|
|
|
|
console.log(server.port); // 27017 (TCP mode)
|
|
|
|
console.log(server.getConnectionCount()); // active connections
|
|
|
|
console.log(server.host); // '127.0.0.1' (TCP mode)
|
|
|
|
|
|
|
|
console.log(server.socketPath); // '/tmp/my-tsmdb.sock' (socket mode)
|
|
|
|
|
|
|
|
console.log(server.getUptime()); // seconds since start
|
|
|
|
|
|
|
|
console.log(server.getConnectionCount()); // active client connections
|
|
|
|
|
|
|
|
|
|
|
|
await server.stop();
|
|
|
|
await server.stop();
|
|
|
|
```
|
|
|
|
```
|
|
|
|
@@ -231,7 +284,8 @@ await server.stop();
|
|
|
|
|
|
|
|
|
|
|
|
TsmDB supports the core MongoDB operations via the wire protocol:
|
|
|
|
TsmDB supports the core MongoDB operations via the wire protocol:
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 CRUD Operations
|
|
|
|
#### CRUD Operations
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
// Insert
|
|
|
|
// Insert
|
|
|
|
await collection.insertOne({ name: 'Bob' });
|
|
|
|
await collection.insertOne({ name: 'Bob' });
|
|
|
|
@@ -260,7 +314,8 @@ const result = await collection.findOneAndUpdate(
|
|
|
|
);
|
|
|
|
);
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 Query Operators
|
|
|
|
#### Query Operators
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
// Comparison
|
|
|
|
// Comparison
|
|
|
|
{ age: { $eq: 25 } }
|
|
|
|
{ age: { $eq: 25 } }
|
|
|
|
@@ -283,9 +338,14 @@ const result = await collection.findOneAndUpdate(
|
|
|
|
{ tags: { $all: ['mongodb', 'database'] } }
|
|
|
|
{ tags: { $all: ['mongodb', 'database'] } }
|
|
|
|
{ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }
|
|
|
|
{ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }
|
|
|
|
{ tags: { $size: 3 } }
|
|
|
|
{ tags: { $size: 3 } }
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// Regex
|
|
|
|
|
|
|
|
{ name: { $regex: /^Al/i } }
|
|
|
|
|
|
|
|
{ email: { $regex: '@example\\.com$' } }
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 Update Operators
|
|
|
|
#### Update Operators
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
{ $set: { name: 'New Name' } }
|
|
|
|
{ $set: { name: 'New Name' } }
|
|
|
|
{ $unset: { tempField: '' } }
|
|
|
|
{ $unset: { tempField: '' } }
|
|
|
|
@@ -298,9 +358,12 @@ const result = await collection.findOneAndUpdate(
|
|
|
|
{ $addToSet: { tags: 'unique-tag' } }
|
|
|
|
{ $addToSet: { tags: 'unique-tag' } }
|
|
|
|
{ $pop: { queue: 1 } } // Remove last
|
|
|
|
{ $pop: { queue: 1 } } // Remove last
|
|
|
|
{ $pop: { queue: -1 } } // Remove first
|
|
|
|
{ $pop: { queue: -1 } } // Remove first
|
|
|
|
|
|
|
|
{ $rename: { oldField: 'newField' } }
|
|
|
|
|
|
|
|
{ $currentDate: { lastModified: true } }
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 Aggregation Pipeline
|
|
|
|
#### Aggregation Pipeline
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
const results = await collection.aggregate([
|
|
|
|
const results = await collection.aggregate([
|
|
|
|
{ $match: { status: 'active' } },
|
|
|
|
{ $match: { status: 'active' } },
|
|
|
|
@@ -311,17 +374,27 @@ const results = await collection.aggregate([
|
|
|
|
]).toArray();
|
|
|
|
]).toArray();
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Supported stages: `$match`, `$project`, `$group`, `$sort`, `$limit`, `$skip`, `$unwind`, `$lookup`, `$addFields`, `$count`, `$facet`, and more.
|
|
|
|
**Supported stages:** `$match`, `$project`, `$group`, `$sort`, `$limit`, `$skip`, `$unwind`, `$lookup`, `$addFields`, `$count`, `$facet`, `$replaceRoot`, `$set`, `$unset`, and more.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Supported group accumulators:** `$sum`, `$avg`, `$min`, `$max`, `$first`, `$last`, `$push`, `$addToSet`, `$count`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### Index Operations
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 Index Operations
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
|
|
|
|
// Create indexes
|
|
|
|
await collection.createIndex({ email: 1 }, { unique: true });
|
|
|
|
await collection.createIndex({ email: 1 }, { unique: true });
|
|
|
|
await collection.createIndex({ name: 1, age: -1 });
|
|
|
|
await collection.createIndex({ name: 1, age: -1 });
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// List indexes
|
|
|
|
const indexes = await collection.listIndexes().toArray();
|
|
|
|
const indexes = await collection.listIndexes().toArray();
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// Drop indexes
|
|
|
|
await collection.dropIndex('email_1');
|
|
|
|
await collection.dropIndex('email_1');
|
|
|
|
|
|
|
|
await collection.dropIndexes(); // Drop all except _id
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 Database Operations
|
|
|
|
#### Database Operations
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
// List databases
|
|
|
|
// List databases
|
|
|
|
const dbs = await client.db().admin().listDatabases();
|
|
|
|
const dbs = await client.db().admin().listDatabases();
|
|
|
|
@@ -335,9 +408,13 @@ await db.dropCollection('oldcollection');
|
|
|
|
|
|
|
|
|
|
|
|
// Drop database
|
|
|
|
// Drop database
|
|
|
|
await db.dropDatabase();
|
|
|
|
await db.dropDatabase();
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
// Database stats
|
|
|
|
|
|
|
|
const stats = await db.stats();
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 Count & Distinct
|
|
|
|
#### Count and Distinct
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
// Count documents
|
|
|
|
// Count documents
|
|
|
|
const total = await collection.countDocuments({});
|
|
|
|
const total = await collection.countDocuments({});
|
|
|
|
@@ -349,7 +426,8 @@ const departments = await collection.distinct('department');
|
|
|
|
const activeDepts = await collection.distinct('department', { status: 'active' });
|
|
|
|
const activeDepts = await collection.distinct('department', { status: 'active' });
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### 🔹 Bulk Operations
|
|
|
|
#### Bulk Operations
|
|
|
|
|
|
|
|
|
|
|
|
```typescript
|
|
|
|
```typescript
|
|
|
|
const result = await collection.bulkWrite([
|
|
|
|
const result = await collection.bulkWrite([
|
|
|
|
{ insertOne: { document: { name: 'Bulk1' } } },
|
|
|
|
{ insertOne: { document: { name: 'Bulk1' } } },
|
|
|
|
@@ -378,22 +456,22 @@ const server = new tsmdb.TsmdbServer({
|
|
|
|
persistIntervalMs: 30000 // Save every 30 seconds
|
|
|
|
persistIntervalMs: 30000 // Save every 30 seconds
|
|
|
|
});
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
|
|
// File-based - persistent storage with optional checksums
|
|
|
|
// File-based - persistent storage with checksums
|
|
|
|
import { tsmdb } from '@push.rocks/smartmongo';
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
const server = new tsmdb.TsmdbServer({
|
|
|
|
const server = new tsmdb.TsmdbServer({
|
|
|
|
storage: 'file',
|
|
|
|
storage: 'file',
|
|
|
|
storagePath: './data/tsmdb'
|
|
|
|
storagePath: './data/tsmdb'
|
|
|
|
});
|
|
|
|
});
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## ⚡ Performance & Reliability Features
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Performance and Reliability Features
|
|
|
|
|
|
|
|
|
|
|
|
TsmDB includes enterprise-grade features for robustness:
|
|
|
|
TsmDB includes enterprise-grade features for robustness:
|
|
|
|
|
|
|
|
|
|
|
|
### 🔍 Index-Accelerated Queries
|
|
|
|
### Index-Accelerated Queries
|
|
|
|
|
|
|
|
|
|
|
|
Indexes are automatically used to accelerate queries. Instead of scanning all documents, TsmDB uses:
|
|
|
|
Indexes are automatically used to accelerate queries:
|
|
|
|
|
|
|
|
|
|
|
|
- **Hash indexes** for equality queries (`$eq`, `$in`)
|
|
|
|
- **Hash indexes** for equality queries (`$eq`, `$in`)
|
|
|
|
- **B-tree indexes** for range queries (`$gt`, `$gte`, `$lt`, `$lte`)
|
|
|
|
- **B-tree indexes** for range queries (`$gt`, `$gte`, `$lt`, `$lte`)
|
|
|
|
@@ -408,7 +486,7 @@ await collection.findOne({ email: 'alice@example.com' }); // Uses hash lookup
|
|
|
|
await collection.find({ age: { $gte: 18, $lt: 65 } }); // Uses B-tree range scan
|
|
|
|
await collection.find({ age: { $gte: 18, $lt: 65 } }); // Uses B-tree range scan
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### 📊 Query Planner
|
|
|
|
### Query Planner
|
|
|
|
|
|
|
|
|
|
|
|
TsmDB includes a query planner that analyzes queries and selects optimal execution strategies:
|
|
|
|
TsmDB includes a query planner that analyzes queries and selects optimal execution strategies:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -428,7 +506,7 @@ console.log(plan);
|
|
|
|
// }
|
|
|
|
// }
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### 📝 Write-Ahead Logging (WAL)
|
|
|
|
### Write-Ahead Logging (WAL)
|
|
|
|
|
|
|
|
|
|
|
|
For durability, TsmDB supports write-ahead logging:
|
|
|
|
For durability, TsmDB supports write-ahead logging:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -449,7 +527,7 @@ await wal.initialize();
|
|
|
|
const entries = wal.getEntriesAfter(lastCheckpointLsn);
|
|
|
|
const entries = wal.getEntriesAfter(lastCheckpointLsn);
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### 🔐 Session Management
|
|
|
|
### Session Management
|
|
|
|
|
|
|
|
|
|
|
|
TsmDB tracks client sessions with automatic timeout and transaction linking:
|
|
|
|
TsmDB tracks client sessions with automatic timeout and transaction linking:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -474,7 +552,7 @@ try {
|
|
|
|
// - Session activity tracking
|
|
|
|
// - Session activity tracking
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### ✅ Data Integrity Checksums
|
|
|
|
### Data Integrity Checksums
|
|
|
|
|
|
|
|
|
|
|
|
File-based storage supports CRC32 checksums to detect corruption:
|
|
|
|
File-based storage supports CRC32 checksums to detect corruption:
|
|
|
|
|
|
|
|
|
|
|
|
@@ -483,23 +561,27 @@ import { tsmdb } from '@push.rocks/smartmongo';
|
|
|
|
|
|
|
|
|
|
|
|
// Checksums are used internally for WAL and data integrity
|
|
|
|
// Checksums are used internally for WAL and data integrity
|
|
|
|
// Documents are checksummed on write, verified on read
|
|
|
|
// Documents are checksummed on write, verified on read
|
|
|
|
|
|
|
|
const checksum = tsmdb.calculateDocumentChecksum(doc);
|
|
|
|
|
|
|
|
const isValid = tsmdb.verifyChecksum(docWithChecksum);
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### 📋 Supported Wire Protocol Commands
|
|
|
|
### Supported Wire Protocol Commands
|
|
|
|
|
|
|
|
|
|
|
|
| Category | Commands |
|
|
|
|
| Category | Commands |
|
|
|
|
|----------|----------|
|
|
|
|
|----------|----------|
|
|
|
|
| **Handshake** | `hello`, `isMaster` |
|
|
|
|
| **Handshake** | `hello`, `isMaster`, `ismaster` |
|
|
|
|
| **CRUD** | `find`, `insert`, `update`, `delete`, `findAndModify`, `getMore`, `killCursors` |
|
|
|
|
| **CRUD** | `find`, `insert`, `update`, `delete`, `findAndModify`, `getMore`, `killCursors` |
|
|
|
|
| **Aggregation** | `aggregate`, `count`, `distinct` |
|
|
|
|
| **Aggregation** | `aggregate`, `count`, `distinct` |
|
|
|
|
| **Indexes** | `createIndexes`, `dropIndexes`, `listIndexes` |
|
|
|
|
| **Indexes** | `createIndexes`, `dropIndexes`, `listIndexes` |
|
|
|
|
| **Transactions** | `startTransaction`, `commitTransaction`, `abortTransaction` |
|
|
|
|
| **Transactions** | `startTransaction`, `commitTransaction`, `abortTransaction` |
|
|
|
|
| **Sessions** | `startSession`, `endSessions` |
|
|
|
|
| **Sessions** | `startSession`, `endSessions`, `refreshSessions` |
|
|
|
|
| **Admin** | `ping`, `listDatabases`, `listCollections`, `drop`, `dropDatabase`, `create`, `serverStatus`, `buildInfo`, `dbStats`, `collStats` |
|
|
|
|
| **Admin** | `ping`, `listDatabases`, `listCollections`, `drop`, `dropDatabase`, `create`, `serverStatus`, `buildInfo`, `dbStats`, `collStats`, `connectionStatus`, `currentOp`, `collMod`, `renameCollection` |
|
|
|
|
|
|
|
|
|
|
|
|
TsmDB supports MongoDB wire protocol versions 0-21, compatible with MongoDB 3.6 through 7.0 drivers.
|
|
|
|
TsmDB supports MongoDB wire protocol versions 0-21, compatible with MongoDB 3.6 through 7.0 drivers.
|
|
|
|
|
|
|
|
|
|
|
|
## 🧪 Testing Examples
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Testing Examples
|
|
|
|
|
|
|
|
|
|
|
|
### Jest/Mocha with LocalTsmDb
|
|
|
|
### Jest/Mocha with LocalTsmDb
|
|
|
|
|
|
|
|
|
|
|
|
@@ -620,15 +702,17 @@ tap.test('teardown', async () => {
|
|
|
|
export default tap.start();
|
|
|
|
export default tap.start();
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## 🏗️ Architecture
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Architecture
|
|
|
|
|
|
|
|
|
|
|
|
### Module Structure
|
|
|
|
### Module Structure
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
@push.rocks/smartmongo
|
|
|
|
@push.rocks/smartmongo
|
|
|
|
├── SmartMongo → Real MongoDB memory server (mongodb-memory-server wrapper)
|
|
|
|
├── SmartMongo -> Real MongoDB memory server (mongodb-memory-server wrapper)
|
|
|
|
├── tsmdb → Wire protocol server with full engine stack
|
|
|
|
├── tsmdb -> Wire protocol server with full engine stack
|
|
|
|
└── LocalTsmDb → Lightweight Unix socket wrapper (no MongoDB driver dependency)
|
|
|
|
└── LocalTsmDb -> Lightweight Unix socket wrapper (no MongoDB driver dependency)
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### TsmDB Wire Protocol Stack
|
|
|
|
### TsmDB Wire Protocol Stack
|
|
|
|
@@ -638,17 +722,17 @@ export default tap.start();
|
|
|
|
│ Official MongoDB Driver │
|
|
|
|
│ Official MongoDB Driver │
|
|
|
|
│ (mongodb npm) │
|
|
|
|
│ (mongodb npm) │
|
|
|
|
└─────────────────────────┬───────────────────────────────────┘
|
|
|
|
└─────────────────────────┬───────────────────────────────────┘
|
|
|
|
│ TCP + OP_MSG/BSON
|
|
|
|
│ TCP/Unix Socket + OP_MSG/BSON
|
|
|
|
▼
|
|
|
|
v
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
|
|
│ TsmdbServer │
|
|
|
|
│ TsmdbServer │
|
|
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
|
|
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
|
|
|
|
│ │ WireProtocol │→ │CommandRouter │→ │ Handlers │ │
|
|
|
|
│ │ WireProtocol │->│CommandRouter │->│ Handlers │ │
|
|
|
|
│ │ (OP_MSG) │ │ │ │ (Find, Insert..) │ │
|
|
|
|
│ │ (OP_MSG) │ │ │ │ (Find, Insert..) │ │
|
|
|
|
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
|
|
|
|
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
|
|
|
|
└─────────────────────────┬───────────────────────────────────┘
|
|
|
|
└─────────────────────────┬───────────────────────────────────┘
|
|
|
|
│
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
v
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
|
|
│ Engines │
|
|
|
|
│ Engines │
|
|
|
|
│ ┌─────────┐ ┌────────┐ ┌───────────┐ ┌───────┐ ┌───────┐ │
|
|
|
|
│ ┌─────────┐ ┌────────┐ ┌───────────┐ ┌───────┐ ┌───────┐ │
|
|
|
|
@@ -660,7 +744,7 @@ export default tap.start();
|
|
|
|
│ └──────────────────────┘ │
|
|
|
|
│ └──────────────────────┘ │
|
|
|
|
└─────────────────────────┬───────────────────────────────────┘
|
|
|
|
└─────────────────────────┬───────────────────────────────────┘
|
|
|
|
│
|
|
|
|
│
|
|
|
|
▼
|
|
|
|
v
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
|
|
│ Storage Layer │
|
|
|
|
│ Storage Layer │
|
|
|
|
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────┐ │
|
|
|
|
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────┐ │
|
|
|
|
@@ -677,28 +761,32 @@ export default tap.start();
|
|
|
|
| **WireProtocol** | Parses MongoDB OP_MSG binary protocol |
|
|
|
|
| **WireProtocol** | Parses MongoDB OP_MSG binary protocol |
|
|
|
|
| **CommandRouter** | Routes commands to appropriate handlers |
|
|
|
|
| **CommandRouter** | Routes commands to appropriate handlers |
|
|
|
|
| **QueryPlanner** | Analyzes queries and selects execution strategy |
|
|
|
|
| **QueryPlanner** | Analyzes queries and selects execution strategy |
|
|
|
|
|
|
|
|
| **QueryEngine** | Executes queries with filter matching via mingo |
|
|
|
|
|
|
|
|
| **UpdateEngine** | Processes update operators (`$set`, `$inc`, etc.) |
|
|
|
|
|
|
|
|
| **AggregationEngine** | Executes aggregation pipelines via mingo |
|
|
|
|
| **IndexEngine** | Manages B-tree and hash indexes |
|
|
|
|
| **IndexEngine** | Manages B-tree and hash indexes |
|
|
|
|
| **SessionEngine** | Tracks client sessions and timeouts |
|
|
|
|
| **SessionEngine** | Tracks client sessions and timeouts |
|
|
|
|
| **TransactionEngine** | Handles ACID transaction semantics |
|
|
|
|
| **TransactionEngine** | Handles ACID transaction semantics with snapshot isolation |
|
|
|
|
| **WAL** | Write-ahead logging for durability |
|
|
|
|
| **WAL** | Write-ahead logging for durability and crash recovery |
|
|
|
|
|
|
|
|
| **OpLog** | Operation log for change tracking and streaming |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## License and Legal Information
|
|
|
|
## License and Legal Information
|
|
|
|
|
|
|
|
|
|
|
|
This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.
|
|
|
|
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
|
|
|
|
|
|
|
|
|
|
|
|
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
|
|
|
|
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
|
|
|
|
|
|
|
|
|
|
|
|
### Trademarks
|
|
|
|
### Trademarks
|
|
|
|
|
|
|
|
|
|
|
|
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.
|
|
|
|
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
|
|
|
|
|
|
|
|
|
|
|
|
Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Company Information
|
|
|
|
### Company Information
|
|
|
|
|
|
|
|
|
|
|
|
Task Venture Capital GmbH
|
|
|
|
Task Venture Capital GmbH
|
|
|
|
Registered at District Court Bremen HRB 35230 HB, Germany
|
|
|
|
Registered at District court Bremen HRB 35230 HB, Germany
|
|
|
|
|
|
|
|
|
|
|
|
For any legal inquiries or further information, please contact us via email at hello@task.vc.
|
|
|
|
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
|
|
|
|
|
|
|
|
|
|
|
|
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
|
|
|
|
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
|
|
|
|
|