A MongoDB-compatible embedded database server powered by Rust 🦀⚡ — use the official mongodb driver and it just works. No binary downloads, instant startup, zero config. Features a built-in operation log with point-in-time revert and a web-based debug dashboard.

Install

pnpm add @push.rocks/smartdb
# or
npm install @push.rocks/smartdb

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

What It Does

@push.rocks/smartdb is a real database server that speaks the wire protocol used by MongoDB drivers. The core engine is written in Rust for high performance, with a thin TypeScript orchestration layer. Connect with the standard mongodb Node.js driver — no mocks, no stubs, no external binaries required.

Why SmartDB?

	SmartDB	External DB Server
Startup time	~30ms	~2-5s
Binary download	Bundled (~7MB)	~200MB+
Install	`pnpm add`	System package / Docker
Persistence	Memory or file-based	Full disk engine
Debug UI	Built-in 🖥️	External tooling
Point-in-time revert	Built-in ⏪	Requires oplog tailing
Perfect for	Unit tests, CI/CD, prototyping, local dev, embedded	Production at scale

Three Ways to Use It

🎯 LocalSmartDb — Zero-config convenience. Give it a folder path, get a persistent database over a Unix socket. Done.
🏗️ SmartdbServer — Full control. Configure port, host, storage backend, Unix sockets. Great for test fixtures or custom setups.
🖥️ SmartdbDebugServer — Launch a web dashboard to visually browse collections, inspect the operation log, and revert to any point in time.

Architecture: TypeScript + Rust 🦀

SmartDB uses a sidecar binary pattern — TypeScript handles lifecycle, Rust handles all database operations:

┌──────────────────────────────────────────────────────────────┐
│                   Your Application                           │
│                  (TypeScript / Node.js)                      │
│  ┌──────────────────┐      ┌───────────────────────────┐     │
│  │  SmartdbServer   │─────▶│  RustDbBridge (IPC)       │     │
│  │  or LocalSmartDb │      │  @push.rocks/smartrust    │     │
│  └──────────────────┘      └───────────┬───────────────┘     │
└────────────────────────────────────────┼─────────────────────┘
                                         │ spawn + JSON IPC
                                         ▼
┌──────────────────────────────────────────────────────────────┐
│                    rustdb binary                             │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────┐       │
│  │ Wire Protocol│→ │Command Router│→ │   Handlers    │       │
│  │  (OP_MSG)    │  │  (40+ cmds)  │  │ Find,Insert.. │       │
│  └──────────────┘  └──────────────┘  └───────┬───────┘       │
│                                              │               │
│  ┌─────────┐ ┌────────┐ ┌───────────┐ ┌──────┴──────┐        │
│  │  Query  │ │ Update │ │Aggregation│ │   Index     │        │
│  │ Matcher │ │ Engine │ │  Engine   │ │   Engine    │        │
│  └─────────┘ └────────┘ └───────────┘ └─────────────┘        │
│                                                              │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────┐    │
│  │  MemoryStorage   │  │   FileStorage    │  │  OpLog   │    │
│  └──────────────────┘  └──────────────────┘  └──────────┘    │
└──────────────────────────────────────────────────────────────┘
              ▲
              │ TCP / Unix Socket (wire protocol)
              │
┌─────────────┴────────────────────────────────────────────────┐
│              MongoClient (mongodb npm driver)                │
│              Connects directly to Rust binary                │
└──────────────────────────────────────────────────────────────┘

The TypeScript layer handles lifecycle only (start/stop/configure via IPC). All database operations flow directly from the MongoClient to the Rust binary over TCP or Unix sockets — zero per-query IPC overhead.

Quick Start

Option 1: LocalSmartDb (Zero Config) 🎯

The fastest way to get a persistent local database:

import { LocalSmartDb } from '@push.rocks/smartdb';
import { MongoClient } from 'mongodb';

// Point it at a folder — that's it
const db = new LocalSmartDb({ folderPath: './my-data' });
const { connectionUri } = await db.start();

// Connect with the standard driver
const client = new MongoClient(connectionUri, { directConnection: true });
await client.connect();

// Use it like any wire-protocol-compatible database
const users = client.db('myapp').collection('users');
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
const user = await users.findOne({ name: 'Alice' });
console.log(user); // { _id: ObjectId(...), name: 'Alice', email: 'alice@example.com' }

// Data persists to disk automatically — survives restarts!
await client.close();
await db.stop();

Option 2: SmartdbServer (Full Control) 🏗️

import { SmartdbServer } from '@push.rocks/smartdb';
import { MongoClient } from 'mongodb';

// TCP mode
const server = new SmartdbServer({ port: 27017 });
await server.start();

const client = new MongoClient('mongodb://127.0.0.1:27017');
await client.connect();

const db = client.db('myapp');
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
const user = await db.collection('users').findOne({ name: 'Alice' });

await client.close();
await server.stop();

Option 3: Debug Server (Visual Dashboard) 🖥️

Launch a web-based dashboard to inspect your database in real time:

debugserver and debugui are optional subpath exports. Install their peer dependencies only when you use them:

pnpm add @api.global/typedserver @design.estate/dees-element

import { SmartdbServer } from '@push.rocks/smartdb';
import { SmartdbDebugServer } from '@push.rocks/smartdb/debugserver';

const server = new SmartdbServer({ storage: 'memory' });
await server.start();

const debugServer = new SmartdbDebugServer(server, { port: 4000 });
await debugServer.start();
// Open http://localhost:4000 in your browser 🚀

The debug dashboard gives you:

📊 Dashboard — server status, uptime, database/collection counts, operation breakdown
📁 Collection Browser — browse databases, collections, and documents interactively
📝 OpLog Timeline — every insert, update, and delete with expandable field-level diffs
⏪ Point-in-Time Revert — select any oplog sequence, preview what will be undone, and execute

📝 Operation Log & Point-in-Time Revert

Every write operation (insert, update, delete) is automatically recorded in an in-memory operation log (OpLog) with full before/after document snapshots. The OpLog lives in RAM and resets on restart — it covers the current session only, and retention is bounded (default: 10,000 entries or 64 MiB, whichever is hit first; configurable via the oplog server option). Once a limit is exceeded, the oldest entries are evicted. This enables:

Change tracking — see exactly what changed, when, and in which collection
Field-level diffs — compare previous and new document states
Point-in-time revert — undo operations back to any retained sequence number
Dry-run preview — see what would be reverted before executing

Programmatic OpLog API

import { SmartdbServer } from '@push.rocks/smartdb';

const server = new SmartdbServer({ port: 27017 });
await server.start();

// ... perform some CRUD operations via MongoClient ...

// Get oplog entries
const oplog = await server.getOpLog({ limit: 50 });
console.log(oplog.entries);
// [{ seq: 1, op: 'insert', db: 'myapp', collection: 'users', document: {...}, previousDocument: null }, ...]

// Get aggregate stats
const stats = await server.getOpLogStats();
console.log(stats);
// { currentSeq: 42, totalEntries: 42, oldestSeq: 1, approxBytes: 18342, entriesByOp: { insert: 20, update: 15, delete: 7 } }

// Preview a revert (dry run)
const preview = await server.revertToSeq(30, true);
console.log(`Would undo ${preview.reverted} operations`);

// Execute the revert — undoes all operations after seq 30
const result = await server.revertToSeq(30, false);
console.log(`Reverted ${result.reverted} operations`);

// Reverts can only reach back as far as retained oplog history —
// revertToSeq returns an error if the target sequence is older than the
// oldest retained entry (evicted by the retention limits).

// Browse collections programmatically
const collections = await server.getCollections();
const docs = await server.getDocuments('myapp', 'users', 50, 0);

OpLog Entry Structure

Each entry contains:

Field	Type	Description
`seq`	`number`	Monotonically increasing sequence number
`timestampMs`	`number`	Unix timestamp in milliseconds
`op`	`'insert' \| 'update' \| 'delete'`	Operation type
`db`	`string`	Database name
`collection`	`string`	Collection name
`documentId`	`string`	Document `_id` as hex string
`document`	`object \| null`	New document state (null for deletes)
`previousDocument`	`object \| null`	Previous document state (null for inserts)

✋ No-Op Write Detection

The engine detects document rewrites that change nothing and skips them entirely — no storage write, no WAL append, no index update, and no oplog entry. A skipped rewrite still counts as matched (matchedCount: 1) but reports modifiedCount: 0.

Two classes are skipped:

Identical — the post-image equals the stored document byte for byte.
Volatile-only — the post-image differs only in top-level volatile metadata fields (currently _updatedAt, the field ODM layers such as @push.rocks/smartdata restamp on every save). The stored document is kept as-is, including its existing _updatedAt, so the timestamp means "last real change" rather than "last save call".

This makes periodic reconcile loops that re-save unchanged documents cost nothing at the engine level. A caller that needs a document to actually change must change a non-volatile field.

Counters are exposed through serverStatus:

const status = await db.command({ serverStatus: 1 });
console.log(status.writes);
// {
//   updatesWritten: 42,        // rewrites that reached storage, index, and oplog
//   noopSkipped: {
//     identical: 1337,         // post-image equal to the stored document
//     volatileOnly: 271,       // only volatile metadata differed
//   },
// }

API Reference

SmartdbServer

The core server class. Manages the Rust database engine and exposes connection details.

Constructor Options (`ISmartdbServerOptions`)

import { SmartdbServer } from '@push.rocks/smartdb';

// TCP mode (default)
const server = new SmartdbServer({
  port: 27017,              // Default: 27017
  host: '127.0.0.1',        // Default: 127.0.0.1
  storage: 'memory',        // 'memory' or 'file' (default: 'memory')
  storagePath: './data',     // Required when storage is 'file'
});

// Unix socket mode — no port conflicts!
const server = new SmartdbServer({
  socketPath: '/tmp/smartdb.sock',
  storage: 'file',
  storagePath: './data',
});

// Memory storage with periodic persistence
const server = new SmartdbServer({
  storage: 'memory',
  persistPath: './data/snapshot.json',
  persistIntervalMs: 30000, // Save every 30s
});

// Bounded in-memory oplog retention
const server = new SmartdbServer({
  port: 27017,
  oplog: {
    maxEntries: 50000,            // Default: 10000
    maxBytes: 128 * 1024 * 1024,  // Default: 64 MiB
  },
});

// TLS transport for TCP mode
const tlsServer = new SmartdbServer({
  port: 27017,
  tls: {
    enabled: true,
    certPath: './certs/server.pem',
    keyPath: './certs/server.key',
    // caPath: './certs/client-ca.pem',
    // requireClientCert: true, // Enables mTLS client certificate checks
  },
});

// SCRAM-SHA-256 authentication
const secureServer = new SmartdbServer({
  port: 27017,
  auth: {
    enabled: true,
    usersPath: './data/smartdb-users.json', // Optional: persists derived SCRAM credentials
    users: [
      {
        username: 'root',
        password: 'change-me',
        database: 'admin',
        roles: ['root'],
      },
    ],
  },
});

When auth.enabled is true, protected commands require successful SCRAM-SHA-256 authentication through the official MongoDB driver:

const client = new MongoClient('mongodb://root:change-me@127.0.0.1:27017/admin?authSource=admin', {
  directConnection: true,
});
await client.connect();

TLS is available for TCP listeners. getConnectionUri() includes ?tls=true when TLS is enabled; pass the trusted CA to the MongoDB driver with tlsCAFile, ca, or secureContext.

Authentication verifies SCRAM credentials, denies unauthenticated commands, and enforces command-level built-in roles for supported operations. connectionStatus reports the authenticated users and roles for the current socket.

Supported built-in role names are root, read, readWrite, dbAdmin, userAdmin, clusterMonitor, plus readAnyDatabase, readWriteAnyDatabase, dbAdminAnyDatabase, and userAdminAnyDatabase. When usersPath is set, SmartDB persists SCRAM credential material atomically and does not store plaintext passwords.

Persisted users also carry a random principal identity and a monotonic generation. SmartDB reloads and resolves that identity for every authenticated command, so password or role changes take effect immediately and stale sockets are rejected. Deleting and recreating the same username creates a different principal; an old connection cannot inherit the replacement user's authority. Cross-process user updates are serialized through the persisted users-file lock.

Single-node transactions are supported through official MongoDB driver sessions. Writes with startTransaction and autocommit: false are buffered per logical session, reads inside the transaction see the buffered overlay, commitTransaction applies the write set with conflict checks, and abortTransaction discards it. Live logical sessions remain resumable across socket disconnects. Bounded background cleanup aborts expired transactions, removes expired sessions, and releases publication leases; explicit endSessions and killSessions do the same for their active transactions.

Durable Database Publication Holds

File-backed SmartDB can keep a replaced or deleted database closed after the local mutation is durable and until a downstream coordinator confirms its own fsync. Check the explicit health contract before using this flow:

const health = await server.getHealth();
if (
  health.publicationHoldVersion !== 1 ||
  !health.publicationHoldSupported ||
  health.publicationHoldRequiresExternalDrain
) {
  throw new Error('SmartDB publication holds are unavailable');
}

Set holdPublication: true on an exact fenced importDatabase() or deleteDatabaseTenant() operation. The result contains a held resourceFence with a one-time publicationCapability. Treat that capability as a secret: do not log it or persist it outside protected control-plane state. After downstream state is durable, submit the exact receipt to commitDatabasePublication():

import type { ISmartDbHeldPublicationReceipt } from '@push.rocks/smartdb';

const held = await server.importDatabase({
  databaseName: 'tenant_a',
  source: snapshot,
  username: 'tenant_a_user',
  fence: {
    version: 1,
    scopeId: 'corestore-node-1.tenant-a',
    token: 42,
    mutationId: 'restore-42',
    payloadSha256: controlPlanePayloadSha256,
  },
  holdPublication: true,
});

const receipt = held.resourceFence as ISmartDbHeldPublicationReceipt;
await persistAndFsyncDownstreamState(receipt);
const released = await server.commitDatabasePublication({
  databaseName: 'tenant_a',
  resourceFence: receipt,
});

The held barrier survives restart and blocks wire commands, transactions, startup recovery, compaction, index restoration, and close-time hint writes for that database while unrelated databases remain available. Commit is exact and idempotent. A successful commit removes the raw capability from durable state and retains only protected verification material. Provider/root identity, durable fenced-mode markers, and bounded startup validation make copied, missing, corrupt, or legacy-active publication state fail closed. A higher fencing token compacts resolved older receipts, so long-lived coordinators do not exhaust receipt capacity.

Basic user management commands are available for authenticated users with root or userAdmin privileges:

await client.db('admin').command({
  createUser: 'reader',
  pwd: 'readpass',
  roles: [{ role: 'read', db: 'myapp' }],
});

await client.db('admin').command({ usersInfo: 'reader' });

Methods & Properties

Method / Property	Type	Description
`start()`	`Promise<void>`	Start the server (spawns Rust binary)
`stop()`	`Promise<void>`	Stop the server and clean up
`getConnectionUri()`	`string`	Get the `mongodb://` connection URI
`running`	`boolean`	Whether the server is currently running
`port`	`number`	Configured port (TCP mode)
`host`	`string`	Configured host (TCP mode)
`socketPath`	`string \| undefined`	Socket path (socket mode)
`getMetrics()`	`Promise<ISmartDbMetrics>`	Server metrics (db/collection counts, sessions, transactions, auth, uptime)
`getOpLog(params?)`	`Promise<IOpLogResult>`	Query oplog entries with optional filters
`getOpLogStats()`	`Promise<IOpLogStats>`	Aggregate oplog statistics
`revertToSeq(seq, dryRun?)`	`Promise<IRevertResult>`	Revert to a specific oplog sequence (must be within retained oplog history)
`getCollections(db?)`	`Promise<ICollectionInfo[]>`	List all collections with counts
`getDocuments(db, coll, limit?, skip?)`	`Promise<IDocumentsResult>`	Browse documents with pagination
`getHealth()`	`Promise<ISmartDbHealth>`	Read readiness and explicit publication-hold capability fields
`importDatabase(params)`	`Promise<ISmartDbImportDatabaseResult>`	Durably replace one database, optionally leaving publication held
`deleteDatabaseTenant(params)`	`Promise<ISmartDbDeleteDatabaseTenantResult>`	Durably delete an exact tenant database/user, optionally leaving publication held
`commitDatabasePublication(params)`	`Promise<TSmartDbCommitDatabasePublicationResult>`	Idempotently release an exact held publication receipt

LocalSmartDb

Zero-config wrapper around SmartdbServer. Uses Unix sockets and file-based persistence.

Constructor Options (`ILocalSmartDbOptions`)

import { LocalSmartDb } from '@push.rocks/smartdb';

const db = new LocalSmartDb({
  folderPath: './data',                  // Required: data storage directory
  socketPath: '/tmp/custom.sock',        // Optional: custom socket (default: auto-generated)
});

Methods & Properties

Method / Property	Type	Description
`start()`	`Promise<ILocalSmartDbConnectionInfo>`	Start and return connection info
`stop()`	`Promise<void>`	Stop the server
`getConnectionInfo()`	`ILocalSmartDbConnectionInfo`	Get current connection info
`getConnectionUri()`	`string`	Get the connection URI
`getServer()`	`SmartdbServer`	Access the underlying server
`running`	`boolean`	Whether the server is running

SmartdbDebugServer

Web-based debug dashboard served via @api.global/typedserver. Import from the debugserver subpath:

import { SmartdbDebugServer } from '@push.rocks/smartdb/debugserver';

const debugServer = new SmartdbDebugServer(server, { port: 4000 });
await debugServer.start();
// Dashboard at http://localhost:4000

await debugServer.stop();

The UI is bundled as base64-encoded content (via @git.zone/tsbundle) and served from memory — no static file directory needed.

SmartdbDebugUi (Web Component)

For embedding the debug UI directly into your own web application, import the <smartdb-debugui> web component:

import { SmartdbDebugUi } from '@push.rocks/smartdb/debugui';

// In your HTML/lit template:
// <smartdb-debugui .server=${mySmartdbServer}></smartdb-debugui>
//
// Or in HTTP mode (when served by SmartdbDebugServer):
// <smartdb-debugui apiBaseUrl=""></smartdb-debugui>

Supported Operations

SmartDB supports the core operations through the wire protocol. Use the standard mongodb driver — these all work:

CRUD

// Insert
await collection.insertOne({ name: 'Bob' });
await collection.insertMany([{ a: 1 }, { a: 2 }]);

// Find
const doc = await collection.findOne({ name: 'Bob' });
const docs = await collection.find({ age: { $gte: 18 } }).toArray();

// Update
await collection.updateOne({ name: 'Bob' }, { $set: { age: 25 } });
await collection.updateMany({ active: false }, { $set: { archived: true } });

// Delete
await collection.deleteOne({ name: 'Bob' });
await collection.deleteMany({ archived: true });

// Replace
await collection.replaceOne({ _id: id }, { name: 'New Bob', age: 30 });

// Find and Modify
await collection.findOneAndUpdate({ name: 'Bob' }, { $inc: { visits: 1 } }, { returnDocument: 'after' });
await collection.findOneAndDelete({ expired: true });
await collection.findOneAndReplace({ _id: id }, { name: 'Replaced' }, { returnDocument: 'after' });

Query Operators

// Comparison
{ age: { $eq: 25 } }     { age: { $ne: 25 } }
{ age: { $gt: 18 } }     { age: { $lt: 65 } }
{ age: { $gte: 18 } }    { age: { $lte: 65 } }
{ status: { $in: ['active', 'pending'] } }
{ status: { $nin: ['deleted'] } }

// Logical
{ $and: [{ age: { $gte: 18 } }, { active: true }] }
{ $or: [{ status: 'active' }, { admin: true }] }
{ $not: { status: 'deleted' } }

// Element
{ email: { $exists: true } }
{ type: { $type: 'string' } }

// Array
{ tags: { $all: ['mongodb', 'database'] } }
{ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }
{ tags: { $size: 3 } }

// Regex
{ name: { $regex: /^Al/i } }

Update Operators

{ $set: { name: 'New Name' } }
{ $unset: { tempField: '' } }
{ $inc: { count: 1 } }
{ $mul: { price: 1.1 } }
{ $min: { low: 50 } }        { $max: { high: 100 } }
{ $push: { tags: 'new' } }   { $pull: { tags: 'old' } }
{ $addToSet: { tags: 'unique' } }
{ $pop: { queue: 1 } }       // Remove last
{ $pop: { queue: -1 } }      // Remove first
{ $rename: { old: 'new' } }
{ $currentDate: { lastModified: true } }

Aggregation Pipeline

const results = await collection.aggregate([
  { $match: { status: 'active' } },
  { $group: { _id: '$category', total: { $sum: '$amount' } } },
  { $sort: { total: -1 } },
  { $limit: 10 },
  { $project: { category: '$_id', total: 1, _id: 0 } },
]).toArray();

Supported stages: $match, $project, $group, $sort, $limit, $skip, $unwind, $lookup, $addFields, $count, $facet, $replaceRoot, $set, $unionWith, $out, $merge

Group accumulators: $sum, $avg, $min, $max, $first, $last, $push, $addToSet, $count

Indexes

await collection.createIndex({ email: 1 }, { unique: true });
await collection.createIndex({ name: 1, age: -1 });    // compound
await collection.createIndex({ field: 1 }, { sparse: true });
const indexes = await collection.listIndexes().toArray();
await collection.dropIndex('email_1');
await collection.dropIndexes();  // drop all except _id

🛡️ Unique indexes are enforced at the engine level. Duplicate values are rejected with a DuplicateKey error (code 11000) before the document is written to disk — on insertOne, updateOne, findAndModify, and upserts. Index definitions are persisted to indexes.json and automatically restored on restart.

Database & Admin

await db.listCollections().toArray();
await db.createCollection('new');
await db.dropCollection('old');
await db.dropDatabase();
await db.stats();

const admin = client.db().admin();
await admin.listDatabases();
await admin.ping();
await admin.serverStatus();

Bulk Operations

const result = await collection.bulkWrite([
  { insertOne: { document: { name: 'Bulk1' } } },
  { updateOne: { filter: { name: 'X' }, update: { $set: { bulk: true } } } },
  { deleteOne: { filter: { name: 'Expired' } } },
]);

Count & Distinct

const count = await collection.countDocuments({ status: 'active' });
const estimated = await collection.estimatedDocumentCount();
const names = await collection.distinct('name');

Wire Protocol Commands

Category	Commands
Handshake	`hello`, `isMaster`, `ismaster`
CRUD	`find`, `insert`, `update`, `delete`, `findAndModify`, `getMore`, `killCursors`
Aggregation	`aggregate`, `count`, `distinct`
Indexes	`createIndexes`, `dropIndexes`, `listIndexes`
Sessions	`startSession`, `endSessions`
Transactions	`startTransaction`, `commitTransaction`, `abortTransaction` through driver sessions
Admin	`ping`, `listDatabases`, `listCollections`, `drop`, `dropDatabase`, `create`, `serverStatus`, `buildInfo`, `dbStats`, `collStats`, `connectionStatus`, `currentOp`, `renameCollection`

Compatible with wire protocol versions 0–21 (driver versions 3.6 through 7.0).

Rust Crate Architecture 🦀

The Rust engine is organized as a Cargo workspace with 9 focused crates:

Crate	Purpose
`rustdb`	Binary entry point: TCP/Unix listener, management IPC, CLI
`rustdb-config`	Server configuration types (serde, camelCase JSON)
`rustdb-wire`	Wire protocol parser/encoder (OP_MSG, OP_QUERY, OP_REPLY)
`rustdb-query`	Query matcher, update engine, aggregation, sort, projection
`rustdb-storage`	Storage backends (memory, file), OpLog with point-in-time replay
`rustdb-index`	B-tree/hash indexes, query planner (IXSCAN/COLLSCAN)
`rustdb-txn`	Transaction + session management with snapshot isolation
`rustdb-auth`	SCRAM-SHA-256 credential handling, user metadata persistence, RBAC checks
`rustdb-commands`	40+ command handlers wiring everything together

Cross-compiled for linux_amd64 and linux_arm64 via @git.zone/tsrust.

Storage Engine Reliability 🔒

The Bitcask-style file storage engine includes several reliability features:

Write-ahead log (WAL) — every write is logged before being applied, with crash recovery on restart; once committed entries grow past 16 MiB the WAL is checkpointed and truncated at runtime, so wal.rdb does not grow unbounded
CRC32 checksums — every record is integrity-checked on read
Automatic compaction — dead records are reclaimed when they exceed 50% of file size, runs on startup and after every write
Hint file staleness detection — the hint file records the data file size at write time; if data.rdb changed since (e.g. crash after a delete), the engine falls back to a full scan to ensure tombstones are not lost
Torn-tail repair — startup scans data.rdb to the last valid record, truncates invalid trailing bytes, and preserves all verified records after interrupted writes
Stale socket cleanup — orphaned /tmp/smartdb-*.sock files from crashed instances are automatically cleaned up on startup

Data Integrity CLI 🔍

The Rust binary includes an offline integrity checker:

# Check all collections in a data directory
./dist_rust/rustdb_linux_amd64 --validate-data /path/to/data

# Output:
# === SmartDB Data Integrity Report ===
#
# Database: mydb
#   Collection: users
#     Header:       OK
#     Records:      1,234 (1,200 live, 34 tombstones)
#     Data size:    2.1 MB
#     Duplicates:   0
#     CRC errors:   0
#     Hint file:    OK

Checks file headers, record CRC32 checksums, duplicate _id entries, and hint file consistency. Exit code 1 if any errors are found.

Testing Example

import { expect, tap } from '@git.zone/tstest/tapbundle';
import { SmartdbServer } from '@push.rocks/smartdb';
import { MongoClient } from 'mongodb';

let server: SmartdbServer;
let client: MongoClient;

tap.test('setup', async () => {
  server = new SmartdbServer({ port: 27117 });
  await server.start();
  client = new MongoClient('mongodb://127.0.0.1:27117', { directConnection: true });
  await client.connect();
});

tap.test('should insert and find', async () => {
  const col = client.db('test').collection('items');
  await col.insertOne({ name: 'Widget', price: 9.99 });
  const item = await col.findOne({ name: 'Widget' });
  expect(item?.price).toEqual(9.99);
});

tap.test('should track changes in oplog', async () => {
  const oplog = await server.getOpLog();
  expect(oplog.entries.length).toBeGreaterThan(0);
  expect(oplog.entries[0].op).toEqual('insert');
});

tap.test('teardown', async () => {
  await client.close();
  await server.stop();
});

export default tap.start();

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

readme.md Unescape Escape

@push.rocks/smartdb

Install

Issue Reporting and Security

What It Does

Why SmartDB?

Three Ways to Use It

Architecture: TypeScript + Rust 🦀

Quick Start

Option 1: LocalSmartDb (Zero Config) 🎯

Option 2: SmartdbServer (Full Control) 🏗️

Option 3: Debug Server (Visual Dashboard) 🖥️

📝 Operation Log & Point-in-Time Revert

Programmatic OpLog API

OpLog Entry Structure

✋ No-Op Write Detection

API Reference

SmartdbServer

Constructor Options (ISmartdbServerOptions)

Durable Database Publication Holds

Methods & Properties

LocalSmartDb

Constructor Options (ILocalSmartDbOptions)

Methods & Properties

SmartdbDebugServer

SmartdbDebugUi (Web Component)

Supported Operations

CRUD

Query Operators

Update Operators

Aggregation Pipeline

Indexes

Database & Admin

Bulk Operations

Count & Distinct

Wire Protocol Commands

Rust Crate Architecture 🦀

Storage Engine Reliability 🔒

Data Integrity CLI 🔍

Testing Example

License and Legal Information

Trademarks

Company Information

readme.md

Constructor Options (`ISmartdbServerOptions`)

Constructor Options (`ILocalSmartDbOptions`)