@push.rocks/smartdb

A MongoDB-wire-compatible embedded database server powered by Rust 🦀⚡. It supports the documented command surface through the official mongodb driver without an external MongoDB server. No binary downloads, instant startup, zero config. Features a built-in operation log with point-in-time revert and a web-based debug dashboard.

Install

pnpm add @push.rocks/smartdb
# or
npm install @push.rocks/smartdb

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

---

What It Does

@push.rocks/smartdb is a real database server that speaks the wire protocol used by MongoDB drivers. The core engine is written in Rust for high performance, with a thin TypeScript orchestration layer. Connect with the standard mongodb Node.js driver — no mocks, no stubs, no external binaries required.

Why SmartDB?

	SmartDB	External DB Server
Startup time	~30ms	~2-5s
Binary download	Bundled (~7MB)	~200MB+
Install	pnpm add	System package / Docker
Persistence	Memory or file-based	Full disk engine
Debug UI	Built-in 🖥️	External tooling
Point-in-time revert	Built-in ⏪	Requires oplog tailing
Perfect for	Unit tests, CI/CD, prototyping, local dev, embedded	Production at scale

Three Ways to Use It

• 🎯 LocalSmartDb — Zero-config convenience. Give it a folder path, get a persistent database over a Unix socket. Done.
• 🏗️ SmartdbServer — Full control. Configure port, host, storage backend, Unix sockets. Great for test fixtures or custom setups.
• 🖥️ SmartdbDebugServer — Launch a web dashboard to visually browse collections, inspect the operation log, and revert to any point in time.

Architecture: TypeScript + Rust 🦀

SmartDB uses a sidecar binary pattern — TypeScript handles lifecycle, Rust handles all database operations:

┌──────────────────────────────────────────────────────────────┐
│                   Your Application                           │
│                  (TypeScript / Node.js)                      │
│  ┌──────────────────┐      ┌───────────────────────────┐     │
│  │  SmartdbServer   │─────▶│  RustDbBridge (IPC)       │     │
│  │  or LocalSmartDb │      │  @push.rocks/smartrust    │     │
│  └──────────────────┘      └───────────┬───────────────┘     │
└────────────────────────────────────────┼─────────────────────┘
                                         │ spawn + JSON IPC
                                         ▼
┌──────────────────────────────────────────────────────────────┐
│                    rustdb binary                             │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────┐       │
│  │ Wire Protocol│→ │Command Router│→ │   Handlers    │       │
│  │  (OP_MSG)    │  │  (40+ cmds)  │  │ Find,Insert.. │       │
│  └──────────────┘  └──────────────┘  └───────┬───────┘       │
│                                              │               │
│  ┌─────────┐ ┌────────┐ ┌───────────┐ ┌──────┴──────┐        │
│  │  Query  │ │ Update │ │Aggregation│ │   Index     │        │
│  │ Matcher │ │ Engine │ │  Engine   │ │   Engine    │        │
│  └─────────┘ └────────┘ └───────────┘ └─────────────┘        │
│                                                              │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────┐    │
│  │  MemoryStorage   │  │   FileStorage    │  │  OpLog   │    │
│  └──────────────────┘  └──────────────────┘  └──────────┘    │
└──────────────────────────────────────────────────────────────┘
              ▲
              │ TCP / Unix Socket (wire protocol)
              │
┌─────────────┴────────────────────────────────────────────────┐
│              MongoClient (mongodb npm driver)                │
│              Connects directly to Rust binary                │
└──────────────────────────────────────────────────────────────┘

The TypeScript layer handles lifecycle only (start/stop/configure via IPC). All database operations flow directly from the MongoClient to the Rust binary over TCP or Unix sockets — zero per-query IPC overhead.

---

Quick Start

Option 1: LocalSmartDb (Zero Config) 🎯

The fastest way to get a persistent local database:

import { LocalSmartDb } from '@push.rocks/smartdb';
import { MongoClient } from 'mongodb';

// Point it at a folder — that's it
const db = new LocalSmartDb({ folderPath: './my-data' });
const { connectionUri } = await db.start();

// Connect with the standard driver
const client = new MongoClient(connectionUri, { directConnection: true });
await client.connect();

// Use it like any wire-protocol-compatible database
const users = client.db('myapp').collection('users');
await users.insertOne({ name: 'Alice', email: 'alice@example.com' });
const user = await users.findOne({ name: 'Alice' });
console.log(user); // { _id: ObjectId(...), name: 'Alice', email: 'alice@example.com' }

// Data persists to disk automatically — survives restarts!
await client.close();
await db.stop();

Option 2: SmartdbServer (Full Control) 🏗️

import { SmartdbServer } from '@push.rocks/smartdb';
import { MongoClient } from 'mongodb';

// TCP mode
const server = new SmartdbServer({ port: 27017 });
await server.start();

const client = new MongoClient('mongodb://127.0.0.1:27017');
await client.connect();

const db = client.db('myapp');
await db.collection('users').insertOne({ name: 'Alice', age: 30 });
const user = await db.collection('users').findOne({ name: 'Alice' });

await client.close();
await server.stop();

Option 3: Debug Server (Visual Dashboard) 🖥️

Launch a web-based dashboard to inspect your database in real time:

debugserver and debugui are optional subpath exports. Install their debug dependencies only when you use them:

pnpm add '@api.global/typedserver@^8' '@design.estate/dees-element@^2'

import { SmartdbServer } from '@push.rocks/smartdb';
import { SmartdbDebugServer } from '@push.rocks/smartdb/debugserver';

const server = new SmartdbServer({ storage: 'memory' });
await server.start();

const debugServer = new SmartdbDebugServer(server, { port: 4000 });
await debugServer.start();
// Open http://localhost:4000 in your browser 🚀

The debug dashboard gives you:

• 📊 Dashboard — server status, uptime, database/collection counts, operation breakdown
• 📁 Collection Browser — browse databases, collections, and documents interactively
• 📝 OpLog Timeline — every insert, update, and delete with expandable field-level diffs
• ⏪ Point-in-Time Revert — select any oplog sequence, preview what will be undone, and execute

---

📝 Operation Log & Point-in-Time Revert

Every write operation (insert, update, delete) is automatically recorded in an in-memory operation log (OpLog) with full before/after document snapshots. The OpLog lives in RAM and resets on restart — it covers the current session only, and retention is bounded (default: 10,000 entries or 64 MiB, whichever is hit first; configurable via the oplog server option). Once a limit is exceeded, the oldest entries are evicted. This enables:

• Change tracking — see exactly what changed, when, and in which collection
• Field-level diffs — compare previous and new document states
• Point-in-time revert — undo operations back to any retained sequence number
• Dry-run preview — see what would be reverted before executing

Programmatic OpLog API

import { SmartdbServer } from '@push.rocks/smartdb';

const server = new SmartdbServer({ port: 27017 });
await server.start();

// ... perform some CRUD operations via MongoClient ...

// Get oplog entries
const oplog = await server.getOpLog({ limit: 50 });
console.log(oplog.entries);
// [{ seq: 1, op: 'insert', db: 'myapp', collection: 'users', document: {...}, previousDocument: null }, ...]

// Get aggregate stats
const stats = await server.getOpLogStats();
console.log(stats);
// { currentSeq: 42, totalEntries: 42, oldestSeq: 1, approxBytes: 18342, entriesByOp: { insert: 20, update: 15, delete: 7 } }

// Preview a revert (dry run)
const preview = await server.revertToSeq(30, true);
console.log(`Would undo ${preview.reverted} operations`);

// Execute the revert — undoes all operations after seq 30
const result = await server.revertToSeq(30, false);
console.log(`Reverted ${result.reverted} operations`);

// Reverts can only reach back as far as retained oplog history —
// revertToSeq returns an error if the target sequence is older than the
// oldest retained entry (evicted by the retention limits).

// Browse collections programmatically
const collections = await server.getCollections();
const docs = await server.getDocuments('myapp', 'users', 50, 0);

OpLog Entry Structure

Each entry contains:

Field	Type	Description
seq	number	Monotonically increasing sequence number
timestampMs	number	Unix timestamp in milliseconds
op	'insert' | 'update' | 'delete'	Operation type
db	string	Database name
collection	string	Collection name
documentId	string	Document _id as hex string
document	object | null	New document state (null for deletes)
previousDocument	object | null	Previous document state (null for inserts)

---

✋ No-Op Write Detection

The engine detects document rewrites that change nothing and skips them entirely — no storage write, no WAL append, no index update, and no oplog entry. A skipped rewrite still counts as matched (matchedCount: 1) but reports modifiedCount: 0.

Two classes are skipped:

• Identical — the post-image equals the stored document byte for byte.
• Volatile-only — the post-image differs only in top-level volatile metadata fields (currently _updatedAt, the field ODM layers such as @push.rocks/smartdata restamp on every save). The stored document is kept as-is, including its existing _updatedAt, so the timestamp means "last real change" rather than "last save call".

This makes periodic reconcile loops that re-save unchanged documents cost nothing at the engine level. A caller that needs a document to actually change must change a non-volatile field.

Counters are exposed through serverStatus:

const status = await db.command({ serverStatus: 1 });
console.log(status.writes);
// {
//   updatesWritten: 42,        // rewrites that reached storage, index, and oplog
//   noopSkipped: {
//     identical: 1337,         // post-image equal to the stored document
//     volatileOnly: 271,       // only volatile metadata differed
//   },
// }

---

API Reference

SmartdbServer

The core server class. Manages the Rust database engine and exposes connection details.

Constructor Options (ISmartdbServerOptions)

binaryPath?: string selects one exact engine executable for this server. When supplied, Smartrust validates a regular executable file and never searches SMARTDB_RUST_BINARY, npm packages, development directories or PATH, even if the selected path is unusable. Missing, empty, malformed, non-file and non-executable paths reject startup with RustBinaryLocatorError, code ERR_RUST_BINARY_EXPLICIT_PATH_INVALID. Validation never changes the target's permissions. Executable launch or readiness failures also fail startup without selecting another engine. Omitting the option preserves normal discovery.

Use an absolute path to the caller-installed, verified artifact. Smartrust anchors relative paths when the underlying bridge is constructed, without normalizing away filesystem symlink traversal. A bare name is a relative file path, not a PATH lookup. Options are instance-local; no environment variables are changed. Installer verification and protected installation paths remain the caller's responsibility.

import { SmartdbServer } from '@push.rocks/smartdb';

const server = new SmartdbServer({
  binaryPath: '/opt/spark/engines/smartdb/rustdb_linux_amd64',
  socketPath: '/run/spark/private/smartdb.sock',
  storage: 'file',
  storagePath: '/var/lib/spark/smartdb',
});
await server.start({ timeoutMs: 30_000 });
try {
  const connectionUri = server.getConnectionUri();
  // Pass connectionUri to SmartData for application persistence.
} finally {
  await server.stop(); // Resolves after the owned engine has terminated.
}

The socket parent directory must already exist with the caller's desired access permissions. RustDbBridge also accepts { binaryPath } for direct lifecycle use.

Current-format-only file startup

ISmartdbServerOptions.fileStorageStartup and ILocalSmartDbOptions.fileStorageStartup accept 'migrate' (the default) or 'current-format-only'. Invalid values throw TypeError at construction, before filesystem or process side effects. SmartdbServer requires storage: 'file' when selecting 'current-format-only'. The exported option type is TSmartdbFileStorageStartup.

import { SmartdbServer } from '@push.rocks/smartdb';

const server = new SmartdbServer({
  binaryPath: '/opt/pallet/engines/smartdb/rustdb_linux_amd64',
  socketPath: '/run/pallet/private/identity.sock',
  storage: 'file',
  storagePath: '/var/lib/pallet/private-identity',
  fileStorageStartup: 'current-format-only',
});
await server.start({ timeoutMs: 30_000 });
try {
  const connectionUri = server.getConnectionUri();
  // Connect SmartData, prepare its schema, then run application migrations.
} finally {
  await server.stop(); // Confirms native engine exit before handing off the root.
}

This mode uses SmartDB's existing format detector to accept a missing, empty, or current file root and reject legacy collection JSON, mixed current/legacy layouts, and orphaned migration staging. Ambiguous database or collection symlinks are also rejected. The format check does not read or log document contents, write files, change permissions, rename, delete, or initialize storage. The entire pre-engine phase is read-only, including when used through LocalSmartDb; strict startup skips its global stale-socket scan and skips the legacy auth-permission migration. Existing auth metadata must already meet the native security requirements, including mode 0600 on Linux.

After format admission, the native engine acquires its existing exclusive file-root lease before storage recovery/initialization, auth loading, or index preparation. A second engine using the same root fails and is terminated without disturbing the owner. Only after start() succeeds should consumers initialize SmartData models or run application migrations. No additional ownership store or consumer-side format detector is needed.

The format check is not a lease, full offline integrity inspection, or protection against concurrent edits by an installer or legacy migration process. Callers must control the storage path and exclude those edits during startup. Native startup continues to validate and recover current-format storage under its lease. SmartdbServer.start() cancellation and its deadline cover format inspection through readiness; failed-start cleanup and retryable stop() retain their normal guarantees. Omitting the policy retains automatic legacy storage and auth-permission migration. A root migrated by that default path retains its old JSON files and therefore remains a mixed layout until those files are removed through an explicit, separately owned offline operation.

import { SmartdbServer } from '@push.rocks/smartdb';

// TCP mode (default)
const server = new SmartdbServer({
  port: 0,                  // Default: 27017; 0 requests an OS-assigned port
  host: '127.0.0.1',        // Default: 127.0.0.1
  storage: 'memory',        // 'memory' or 'file' (default: 'memory')
  storagePath: './data',     // Required when storage is 'file'
});
const startupController = new AbortController();
await server.start({
  signal: startupController.signal,
  timeoutMs: 30_000,
});
console.log(server.port); // Actual bound port while running
console.log(server.getConnectionUri()); // mongodb://127.0.0.1:<actual-port>

// Unix socket mode — no port conflicts!
const server = new SmartdbServer({
  socketPath: '/tmp/smartdb.sock',
  storage: 'file',
  storagePath: './data',
});

// Memory storage with periodic persistence
const server = new SmartdbServer({
  storage: 'memory',
  persistPath: './data/snapshot.json',
  persistIntervalMs: 30000, // Save every 30s
});

// Bounded in-memory oplog retention
const server = new SmartdbServer({
  port: 27017,
  oplog: {
    maxEntries: 50000,            // Default: 10000
    maxBytes: 128 * 1024 * 1024,  // Default: 64 MiB
  },
});

// TLS transport for TCP mode
const tlsServer = new SmartdbServer({
  port: 27017,
  tls: {
    enabled: true,
    certPath: './certs/server.pem',
    keyPath: './certs/server.key',
    // caPath: './certs/client-ca.pem',
    // requireClientCert: true, // Enables mTLS client certificate checks
  },
});

// SCRAM-SHA-256 authentication
const secureServer = new SmartdbServer({
  port: 27017,
  auth: {
    enabled: true,
    usersPath: './data/smartdb-users.json', // Optional: persists derived SCRAM credentials
    users: [
      {
        username: 'root',
        password: 'change-me',
        database: 'admin',
        roles: ['root'],
      },
    ],
  },
});

When auth.enabled is true, protected commands require successful SCRAM-SHA-256 authentication through the official MongoDB driver:

const client = new MongoClient('mongodb://root:change-me@127.0.0.1:27017/admin?authSource=admin', {
  directConnection: true,
});
await client.connect();

TLS is available for TCP listeners. getConnectionUri() includes ?tls=true when TLS is enabled; pass the trusted CA to the MongoDB driver with tlsCAFile, ca, or secureContext.

Authentication verifies SCRAM credentials, denies unauthenticated commands, and enforces command-level built-in roles for supported operations. connectionStatus reports the authenticated users and roles for the current socket.

Supported built-in role names are root, read, readWrite, dbAdmin, userAdmin, clusterMonitor, plus readAnyDatabase, readWriteAnyDatabase, dbAdminAnyDatabase, and userAdminAnyDatabase. When usersPath is set, SmartDB persists SCRAM credential material atomically and does not store plaintext passwords. Auth metadata version 3 contains protected decoy material, an immutable persisted SCRAM profile, allocation read-grant principals, and bounded anti-replay state. auth.scramIterations must match the profile already established by that usersPath on every later startup and in every attached process; SmartDB fails closed on a mismatch rather than attempting to rederive credentials without plaintext passwords. SmartDB 4.0.0 startup persistently rewrites exact version 2 owner metadata to version 3 without changing existing owner SCRAM material, principal identities, generations, or roles. This is a one-way major-version compatibility boundary: after migration, SmartDB 3.x must not be restarted against that usersPath. Unsupported versions and malformed auth or grant metadata fail closed.

On Linux, startup performs one narrow compatibility migration before the Rust engine opens usersPath: an effective-user-owned, single-link regular file with the exact legacy mode 0644 is changed to 0600. Current 0600 files are left unchanged. Symlinks, hard links, unexpected modes or ownership, unsafe parent directories, cross-device targets, and identity changes remain fail-closed errors.

Legacy v0 JSON collections are converted into hidden sibling staging directories. SmartDB fsyncs every generated file and the staging directory before atomically publishing a complete v1 collection, while the v0 files remain unchanged. Cancellation removes unpublished staging and preserves the v0 input. Startup fails closed if it finds staging left by an unclean process exit or an incomplete published target, so neither state is mistaken for a complete migration.

Persisted users also carry a random principal identity and a monotonic generation. SmartDB reloads and resolves that identity for every authenticated command, so password or role changes take effect immediately and stale sockets are rejected. Deleting and recreating the same username creates a different principal; an old connection cannot inherit the replacement user's authority. Cross-process user updates are serialized through the persisted users-file lock.

Single-node transactions are supported through official MongoDB driver sessions. find, count, distinct, insert, update, findAndModify, and delete use the transaction snapshot and buffered write set; commitTransaction applies that write set with conflict checks, and abortTransaction discards it. The snapshot is database-wide: the first namespace a transaction materializes pins a publication sequence on the namespace clock, and any later namespace that was published after that sequence fails closed with SnapshotUnavailable (code 246) and the TransientTransactionError label so the driver restarts the transaction on fresh state; write-write conflicts surface at commit as WriteConflict (code 112) with the same label. Each session remembers the outcome of its last transaction number: a repeated commitTransaction replays ok, abortTransaction after a commit reports TransactionCommitted (code 256), commitTransaction after an abort reports NoSuchTransaction (code 251), and starting a transaction with an older number reports TransactionTooOld (code 225). After applicable authentication, authorization, and allocation-policy checks, collection, database, and index DDL, user-management mutations, and aggregate pipelines ending in $out or $merge reject transaction envelopes with OperationNotSupportedInTransaction (code 263) before session or transaction creation, maintenance gates, or mutation. Live logical sessions remain resumable across socket disconnects. Bounded background cleanup aborts expired transactions, removes expired sessions, and releases publication leases; explicit endSessions and killSessions do the same for their active transactions.

Durable Database Publication Holds

On Linux, file-backed SmartDB can keep a replaced or deleted database closed after the local mutation is durable and until a downstream coordinator confirms its own fsync. Check the explicit health contract before using this flow:

const health = await server.getHealth();
if (
  health.publicationHoldVersion !== 1 ||
  !health.publicationHoldSupported ||
  health.publicationHoldRequiresExternalDrain
) {
  throw new Error('SmartDB publication holds are unavailable');
}

Set holdPublication: true on an exact fenced importDatabase() or deleteDatabaseTenant() operation. The result contains a held resourceFence with a one-time publicationCapability. Treat that capability as a secret: do not log it or persist it outside protected control-plane state. After downstream state is durable, submit the exact receipt to commitDatabasePublication():

import type { ISmartDbHeldPublicationReceipt } from '@push.rocks/smartdb';

const held = await server.importDatabase({
  databaseName: 'tenant_a',
  source: snapshot,
  username: 'tenant_a_user',
  fence: {
    version: 1,
    scopeId: 'corestore-node-1.tenant-a',
    token: 42,
    mutationId: 'restore-42',
    payloadSha256: controlPlanePayloadSha256,
  },
  holdPublication: true,
});

const receipt = held.resourceFence as ISmartDbHeldPublicationReceipt;
await persistAndFsyncDownstreamState(receipt);
const released = await server.commitDatabasePublication({
  databaseName: 'tenant_a',
  resourceFence: receipt,
});

The held barrier survives restart and blocks wire commands, transactions, startup recovery, compaction, and index restoration for that database while unrelated databases remain available. Commit is exact and idempotent. A successful commit removes the raw capability from durable state and retains only protected verification material. Provider/root identity, durable fenced-mode markers, and bounded startup validation make copied, missing, corrupt, or legacy-active publication state fail closed. A higher fencing token compacts resolved older receipts, so long-lived coordinators do not exhaust receipt capacity.

Durable database publication holds and allocation fencing remain Linux-only. The macOS support described under stopped storage-root relocation does not enable these runtime mutation features.

Coordinators that do not yet have their own durable record can call getDatabaseResourceFenceState({ databaseName }) before creating it. File-backed SmartDB returns the durable scope, highest token, current publication phase, and allocation lifecycle/identity when the name is allocation-managed, or null only when both fence state and its durable marker are absent. The inspection never returns mutation IDs, mutation payload digests, publication receipts, or publication capabilities. Treat any active phase, identity mismatch, corrupt state, unsupported storage, or unsafe token as a hard stop rather than creating coordinator state.

Authoritative Database Allocation

allocateDatabaseTenant() is the create-only control-plane API for permanently allocation-managed database names. It is available only when allocationFencingVersion === 1, allocationFencingSupported === true, and allocationFencingRequiresDrain === true. The request requires literal expectedAbsent: true, an ISmartDbResourceFence, and the tenant database, username, and password. roles is optional and defaults to ['readWrite', 'dbAdmin']. SmartDB proves database-root and auth-principal absence while holding the database resource lock, maintenance write gate, and a bounded cross-process reservation on the durable auth store, then returns an ISmartDbAllocateDatabaseTenantResult; its .allocation field is the durable ISmartDbDatabaseAllocationIdentity. Exact fence replay returns the same allocation ID, generation, principal ID, and receipt digests.

Pass the returned allocation identity to every later ensureDatabaseTenant(), importDatabase(), and deleteDatabaseTenant() request for that name. Managed import and delete requests also require the exact allocation username; the identity alone is insufficient. Each distinct managed mutation requires a strictly newer fence token; the current token is accepted only for exact receipt replay or continuation, and older receipts become explicitly stale after advancement. Held publication receipts carry the allocation into commitDatabasePublication(). Managed requests without it and unmanaged requests with it fail closed. Deprovision persists deprovisioning, durably deletes the exact marked root, atomically removes only the exact principal, then persists a permanent deprovisioned tombstone. A later allocation requires a newer fence token, increments generation, and receives a fresh allocation and principal identity. Wire data access is available only while active and only to the current allocation principal; database/user ownership-changing wire commands are rejected.

The provider marker .__rustdb_allocation.json is created and preserved by SmartDB, never accepted from an import payload, and never included in logical exports. The resource lock uses the distinct smartdb-resource-allocation-managed-v1 marker. Pre-allocation binaries, including 2.18.1, interpret that lock marker as invalid and fail closed. Consequently, N-1 rollback of a storage root after allocation fencing has touched a name is intentionally unavailable; restore the allocation-aware binary rather than removing or editing provider metadata.

Read-Only Tenant Attestation

attestDatabaseTenant() verifies that an existing database is exclusively owned by the exact username, role set, and candidate password. It returns only { matches: boolean }; it never returns a URI, credential, principal identity, or mismatch detail. Missing databases or users, conflicting ownership, role differences, allocation-principal drift, and password mismatch all return false through the same constant-work credential path.

Attestation always holds read-only local and maintenance guards. File-backed storage independently supplies the cross-process publication guard. A durable usersPath independently supplies a shared auth guard while current durable auth state is read. Memory storage and auth without a usersPath use their configured in-process providers. Attestation does not refresh process caches or call tenant ensure, create, rotate, import, delete, fence repair, publication-epoch observation, or cache invalidation. Corrupt, held, deprovisioned, disabled, or busy provider state fails as an operational error instead of returning a match result.

const result = await server.attestDatabaseTenant({
  databaseName: 'tenant_a',
  username: 'tenant_a_user',
  roles: ['readWrite'],
  password: candidatePassword,
});

if (!result.matches) {
  throw new Error('Existing tenant authority does not match');
}

Allocation-Bound Read Access Grants

Scratch diagnostics can use issueDatabaseAllocationReadAccessGrant() to create one auxiliary read principal for an exact active allocation. The capability is advertised only when databaseAllocationReadAccessGrantVersion === 1 and databaseAllocationReadAccessGrantSupported === true; this requires file storage, enabled authentication, a durable usersPath bound to the storage root, and ready allocation fencing. The caller supplies databaseName, a bounded grantId, a bounded unique username, password, and the complete ISmartDbDatabaseAllocationIdentity. Roles and expiry are not caller-controlled. SmartDB always assigns exactly ['read'] and a fixed, non-renewing 12-minute expiry.

const grant = await server.issueDatabaseAllocationReadAccessGrant({
  databaseName: allocation.databaseName,
  grantId: 'scratch-diagnostic-2026-08-08',
  username: 'scratch_diagnostic_reader',
  password: generatedSecret,
  allocation,
});

const diagnosticClient = new MongoClient(grant.mongodbUri!, {
  directConnection: true,
});
await diagnosticClient.connect();

Exact issue replay validates the candidate password against the persisted SCRAM credential and returns the original principal generation, issuedAt, and expiresAt only while that grant remains active and unexpired; it never renews the grant. Expired or revoked grantId values are rejected. SmartDB permits at most one active read grant per allocation. Revoked and expired grantId values remain in a per-allocation anti-replay set capped at 1,024 entries, with no eviction while the allocation exists. Capacity exhaustion fails closed. Allocation deprovision uses a two-phase fail-closed sequence: any active grant is first durably retired and tombstoned, then the database is durably deleted, and finally the exact owner plus that allocation's replay state are removed in one auth rewrite. Interruption before completion leaves the grant retired rather than usable.

Grant-specific rejections from the issue and revoke methods are normalized automatically as SmartDbAllocationReadAccessGrantError with stable EGRANT_INVALID_REQUEST, EGRANT_CREDENTIAL_MISMATCH, EGRANT_CONFLICT, EGRANT_REPLAY_CAPACITY, or EGRANT_STATE_UNAVAILABLE codes. The companion exports TSmartDbAllocationReadAccessGrantErrorCode, smartDbAllocationReadAccessGrantErrorCodes, and normalizeSmartDbAllocationReadAccessGrantError(error, code) support type-safe handling and explicit normalization of structured bridge errors. Allocation and resource-fence failures retain their existing SmartDbResourceFenceError codes. Callers should classify errors by these exported classes and codes rather than by message text.

Active allocation read grants never survive a SmartDB engine restart or a new attachment to the durable auth store. Startup durably retires and tombstones them before accepting work; diagnostics must issue a new grantId, username, and credential after restart.

Grant principals can read only their exact allocation database. Reads, collection/index metadata, and read-only transactions are supported. Writes, DDL, user/admin operations, writing aggregates ($out and $merge), cross-database targets, and allocation ownership changes are denied. Revocation, expiry cleanup, and allocation deprovision abort matching logical sessions and transactions, release retained database permits, and remove matching cursors. Stale sockets fail their next command.

revokeDatabaseAllocationReadAccessGrant() takes databaseName, grantId, and the complete allocation identity, returns { revoked: true } without a secret, and is idempotent. Revoke-before-issue records anti-replay state only while a read permit proves that exact allocation is current. A delayed revoke after deprovision returns success without recreating auth metadata.

exportDatabase() drains live MongoDB wire commands for the selected database and holds an exclusive local and cross-process database lease while producing the snapshot. The export is therefore consistent across all collections. It enforces server ceilings while visiting documents instead of first materializing an unbounded database; callers can request lower maxEncodedBytes, maxCollections, maxDocuments, and maxIndexes limits. The result emits canonical MongoDB Extended JSON so every BSON type, including 64-bit integers, survives the JSON management channel exactly. Restricted views are carried only through the physical system.views catalog and are never materialized. importDatabase() accepts canonical or relaxed Extended JSON, validates a candidate catalog before mutation, and remaps exact system.views._id source-database prefixes when importing under another database name. Database migration clients should preserve the exported objects as JSON values and must not coerce Extended JSON numeric wrappers into JavaScript numbers.

getDatabaseContentDigest() computes a bounded, database-name-independent SHA-256 over collection names, exact BSON document bytes, and persisted index specifications. Collection, document, and index enumeration order is normalized; BSON field order and compound-index key order remain significant. The physical system.views catalog is included, with only its already-validated deterministic database prefix removed from _id for hashing so a cross-name import retains the same digest. ISmartDbGetDatabaseContentDigestInput accepts databaseName and optional ISmartDbDatabaseContentDigestLimits fields maxScannedBsonBytes, maxCollections, maxDocuments, and maxIndexes. Callers may lower the server ceilings of 96 MiB scanned BSON, 10,000 collections, 1,000,000 documents, and 100,000 indexes. The ISmartDbDatabaseContentDigest result reports format smartdb.database.content-digest.v1, algorithm sha256, the lowercase digest, and exact scan counters.

SmartdbServer.start() and the lifecycle-critical health, fence-state, digest, export, import, publication-commit, tenant-allocation, tenant-attestation, tenant-ensure, tenant-delete, read-grant issue, and read-grant revoke methods accept optional ISmartDbManagementOperationOptions with signal?: AbortSignal and timeoutMs?: number. timeoutMs, when provided, must be a positive safe integer no greater than 2,147,483,647. Startup applies both cancellation and one deadline across legacy storage migration, sidecar spawn, auth metadata migration, and database readiness. Cancellation and deadlines are fail-stop once the sidecar is owned: SmartDB attempts to terminate the Rust engine before rejecting. Once termination is confirmed, the operation cannot continue after the caller releases ownership. If termination itself fails, the rejection retains bridge ownership and the service owner must retry stop() until it succeeds. After cancellation or deadline termination, restart SmartDB before accepting more traffic.

const controller = new AbortController();
const exported = await server.exportDatabase(
  { databaseName: 'myapp' },
  { signal: controller.signal, timeoutMs: 5 * 60_000 },
);

Basic user management commands are available for authenticated users with root or userAdmin privileges:

await client.db('admin').command({
  createUser: 'reader',
  pwd: 'readpass',
  roles: [{ role: 'read', db: 'myapp' }],
});

await client.db('admin').command({ usersInfo: 'reader' });

Methods & Properties

Method / Property	Type	Description
start(options?)	Promise<void>	Start under one optional cancellation/deadline budget; join partial sidecars before rejection or retain ownership for stop() retry if termination fails
stop()	Promise<void>	Wait for active startup work, then stop the server and confirm Rust bridge cleanup, including partial startup cleanup
getConnectionUri()	string	Get the active mongodb:// URI; before start and after stop, port 0 remains unresolved
running	boolean	Whether the server is currently running
port	number	Actual bound port while running; otherwise the configured port (TCP mode)
host	string	Configured host (TCP mode)
socketPath	string | undefined	Socket path (socket mode)
processId	number | undefined	Operating-system pid of the spawned Rust engine while it runs as a child process (for crash and kill tests)
getMetrics()	Promise<ISmartDbMetrics>	Server metrics (db/collection counts, sessions, transactions, auth, uptime)
getOpLog(params?)	Promise<IOpLogResult>	Query oplog entries with optional filters
getOpLogStats()	Promise<IOpLogStats>	Aggregate oplog statistics
revertToSeq(seq, dryRun?)	Promise<IRevertResult>	Revert to a specific oplog sequence (must be within retained oplog history)
getCollections(db?)	Promise<ICollectionInfo[]>	List all collections with counts
getDocuments(db, coll, limit?, skip?)	Promise<IDocumentsResult>	Browse documents with pagination
getHealth(options?)	Promise<ISmartDbHealth>	Read readiness and explicit publication-hold capability fields with optional fail-stop cancellation/deadline ownership
allocateDatabaseTenant(params, options?)	Promise<ISmartDbAllocateDatabaseTenantResult>	Authoritatively allocate an absent file-backed tenant and return its durable allocation identity; supports fail-stop cancellation and deadlines
attestDatabaseTenant(params, options?)	Promise<ISmartDbAttestDatabaseTenantResult>	Verify exact existing tenant ownership, roles, and candidate credential without mutating provider, auth, fence, or runtime cache state
issueDatabaseAllocationReadAccessGrant(params, options?)	Promise<ISmartDbIssueDatabaseAllocationReadAccessGrantResult>	Issue or exactly replay one fixed 12-minute read grant bound to the complete active allocation identity
revokeDatabaseAllocationReadAccessGrant(params, options?)	Promise<ISmartDbRevokeDatabaseAllocationReadAccessGrantResult>	Idempotently revoke/tombstone one exact allocation read grant and invalidate its live runtime state
ensureDatabaseTenant(params, options?)	Promise<ISmartDbEnsureDatabaseTenantResult>	Idempotently ensure an exact fenced tenant; supports fail-stop cancellation and deadlines
getDatabaseResourceFenceState(params, options?)	Promise<ISmartDbDatabaseResourceFenceState | null>	Safely inspect a file-backed database fence high-water mark and publication phase without exposing receipts or capability material; supports fail-stop cancellation and deadlines
getDatabaseContentDigest(params, options?)	Promise<ISmartDbDatabaseContentDigest>	Compute a bounded, database-name-independent digest over exact BSON documents and persisted index specifications; supports fail-stop cancellation and deadlines
exportDatabase(params, options?)	Promise<ISmartDbDatabaseExport>	Export one database as lossless canonical MongoDB Extended JSON; supports fail-stop cancellation and deadlines
importDatabase(params, options?)	Promise<ISmartDbImportDatabaseResult>	Durably replace one database from canonical or relaxed MongoDB Extended JSON, optionally leaving publication held; supports fail-stop cancellation and deadlines
deleteDatabaseTenant(params, options?)	Promise<ISmartDbDeleteDatabaseTenantResult>	Durably delete an exact tenant database/user, optionally leaving publication held; supports fail-stop cancellation and deadlines
commitDatabasePublication(params, options?)	Promise<TSmartDbCommitDatabasePublicationResult>	Idempotently release an exact held publication receipt; supports fail-stop cancellation and deadlines

LocalSmartDb

Zero-config wrapper around SmartdbServer. Uses Unix sockets and file-based persistence.

Constructor Options (ILocalSmartDbOptions)

import { LocalSmartDb } from '@push.rocks/smartdb';

const db = new LocalSmartDb({
  folderPath: './data',                  // Required: data storage directory
  socketPath: '/tmp/custom.sock',        // Optional: custom socket (default: auto-generated)
  binaryPath: '/opt/spark/engines/smartdb/rustdb_linux_amd64', // Optional: exact engine
  fileStorageStartup: 'current-format-only', // Optional; default is 'migrate'
});

binaryPath has the same strict selection contract as SmartdbServer and is passed through unchanged on every start. Supply an absolute path because the underlying server and bridge are constructed by LocalSmartDb.start().

Methods & Properties

Method / Property	Type	Description
start()	Promise<ILocalSmartDbConnectionInfo>	Start and return connection info
stop()	Promise<void>	Stop the server
getConnectionInfo()	ILocalSmartDbConnectionInfo	Get current connection info
getConnectionUri()	string	Get the connection URI
getServer()	SmartdbServer	Access the underlying server
running	boolean	Whether the server is running
LocalSmartDb.relocateStoppedStorageRoot(input, options?)	Promise<ILocalSmartDbStoppedStorageRootRelocationReceipt>	Atomically relocate one eligible stopped Linux or macOS file-storage root and retain a source receipt
LocalSmartDb.inspectOfflineStringValue(input, options?)	Promise<TLocalSmartDbOfflineStringValueInspectionResult>	Read one exact top-level string value from stopped file storage without starting or mutating the engine
LocalSmartDb.inspectOfflinePhysicalNamespaces(input, options?)	Promise<ILocalSmartDbOfflinePhysicalNamespaceInspectionResult>	Validate stopped physical file storage and return only deterministic database and collection names

Stopped Storage-Root Relocation

relocateStoppedStorageRoot() moves an eligible stopped file-storage root without starting RustDb, binding a listener, running a TypeScript migration, initializing storage or auth, or performing WAL recovery. It supports Linux on the existing qualified local-filesystem set and macOS on writable local APFS volumes with ownership, persistent object identities, flock, and atomic exclusive/swap rename support. The source root, source parent, and destination parent must be trusted descriptor-safe objects on the same filesystem. The supplied source and destination must be absolute and resolve to distinct, non-root, non-nested physical paths, and the destination must be absent. macOS also rejects case-folded, Unicode-normalized, and ancestry aliases before creating relocation state. The trusted parents may differ.

import { LocalSmartDb } from '@push.rocks/smartdb';

const receipt = await LocalSmartDb.relocateStoppedStorageRoot({
  sourceFolderPath: '/var/lib/myapp/harness-controller/smartdb',
  destinationFolderPath: '/var/lib/myapp/hcon/smartdb',
  relocationId: 'move-2026-08-17-1',
}, {
  timeoutMs: 30_000,
});

// Start only the destination. The source is now the retained receipt file.
const db = new LocalSmartDb({ folderPath: receipt.destinationFolderPath });
await db.start();

The input must be a plain object with exactly the own enumerable data fields sourceFolderPath, destinationFolderPath, and relocationId. Each supplied path must already be absolute, is normalized before sidecar creation, is limited to 1 through 4095 UTF-8 bytes, and cannot contain control characters. The normalized destination path may not be longer in UTF-8 bytes than the normalized source path, so every root-relative path accepted by SmartDB 5.0.1 at the source remains no longer after relocation. relocationId is a 1 through 256-byte ASCII identifier beginning with an alphanumeric character and otherwise using only alphanumerics, ., _, :, @, or -. Management timeoutMs and signal use the same validation and one-deadline behavior as other SmartDB management operations.

The returned exact receipt has this shape:

interface ILocalSmartDbStoppedStorageRootRelocationReceipt {
  format: 'smartdb.storage-root-relocation.receipt.v1';
  version: 1;
  relocationId: string;
  sourceFolderPath: string;
  destinationFolderPath: string;
  providerRootId: string;
  storageRootDevice: string; // canonical decimal u64
  storageRootInode: string;  // canonical decimal u64
  receiptSha256: string;
  sourceReceiptRetained: true;
}

On success, the destination directory is the original storage-root inode and the source path is an exact mode-0600 regular-file receipt. SmartDB does not remove that source receipt. Data, WAL, hint, and index bytes are not copied or rewritten. providerRootId, root device/inode, and every provider/sentinel field are preserved; only storageRootSha256 changes to the destination's canonical path digest. This leaves a completed destination compatible with SmartDB 5.0.1.

Relocation deliberately supports only the plain, unbound LocalSmartDb topology. The existing owner lock, provider-identity.json, and resource-fencing sentinel must all be present; the operation never creates missing ownership metadata. A provider identity containing authUsersPathSha256 is unsupported. One narrow terminal transaction topology is also eligible: each retained internal transaction journal must be in the exact cleaned phase, contain no workspace debris or allocation binding, and match the exact transaction-publication marker in its database. Canonically named, empty retired legacy fence locks are accepted only when their resource digest belongs to one of those terminal transactions and their encoded inode matches the retained file. These terminal artifacts move unchanged with the root and continue the existing transaction token sequence after restart. Any unfinished or mismatched transaction, durable database-mutation journal, active fence receipt or capability, allocation state, unrelated or malformed retired lock, allocation/general publication marker, symlink, unknown entry, unsafe mode/owner/link count, exhausted scan bound, active owner, or ambiguous identity fails closed.

The operation journals phases preparing, prepared, exchanged, providerPublished, sentinelPublished, and cleaned. If a call returns RECOVERY_REQUIRED, stop all startup attempts and retry the exact same normalized source path, destination path, and relocationId with the same SmartDB version that began the relocation or a newer version. Startup rejects any relocation-workspace occupancy and does not mutate or recover it. Do not delete, rename, edit, or replace either path between attempts. Linux relocation state created by SmartDB 5.1.0 remains recoverable by 5.1.0 or newer; macOS recovery requires a macOS-capable release. A completed root remains 5.0.1-compatible.

An exact completed replay returns the same receipt only while the destination remains the current root. A later successful relocation leaves another receipt at that destination path and supersedes the earlier replay topology; retry the later operation instead. Manual copy or rename remains unsupported and continues to fail provider path/inode fencing.

Failures are exposed as LocalSmartDbStorageRootRelocationError with a code-only union: INVALID_REQUEST, UNSUPPORTED, BUSY, NOT_FOUND, DESTINATION_CONFLICT, IDENTITY_MISMATCH, STATE_CORRUPT, IO, or RECOVERY_REQUIRED. Error messages and causes do not include supplied paths. Timeout, abort, write, exit, or malformed-response failures after command dispatch are always RECOVERY_REQUIRED; exact retry is the only recovery action.

Offline String-Value Inspection

inspectOfflineStringValue() is a Linux-only operational API for reading one metadata value while the owning LocalSmartDb engine is stopped. Other platforms reject the call because the reader requires Linux openat2 descriptor traversal. It starts only the Rust management sidecar: it does not start a database listener, run storage migrations, repair tails, replay or truncate the WAL, compact data, or persist hints.

import { LocalSmartDb } from '@push.rocks/smartdb';

const result = await LocalSmartDb.inspectOfflineStringValue({
  folderPath: '/var/lib/myapp/smartdb',
  databaseName: 'myapp',
  collectionName: 'MetadataDoc',
  match: {
    field: 'key',
    value: 'migrationStateV1',
  },
  valueField: 'value',
  limits: {
    maximumDataFileBytes: 16 * 1024 * 1024,
    maximumWalFileBytes: 16 * 1024 * 1024,
    maximumRecords: 100_000,
    maximumRecordBytes: 1024 * 1024,
    maximumResultBytes: 1024 * 1024,
  },
}, {
  timeoutMs: 5000,
});

if (result.status === 'found') {
  console.log(result.value);
}

The result is only { status: 'not-found' } or { status: 'found', value: string }. No other document fields cross the management boundary. The reader validates descriptor-safe paths, the current data/WAL format, CRCs and live-record semantics, applies any bounded uncommitted WAL overlay in memory, and rejects ambiguous matches, corruption, exceeded limits, symlinks, an engine-owned storage root, or files that change during inspection.

The TypeScript API enforces these bounds before creating the sidecar or serializing the IPC request:

Input	Enforced range
folderPath	1 to 4096 UTF-8 bytes before and after absolute resolution; no control characters
databaseName, collectionName	1 to 255 UTF-8 bytes; one canonical path component; no controls, slash, or backslash
match.field, valueField	1 to 255 UTF-8 bytes; no control characters
match.value	0 to 1,048,576 UTF-8 bytes and no larger than maximumRecordBytes
Complete serialized request	At most 2,097,152 bytes
timeoutMs	1 to 2,147,483,647 milliseconds
maximumDataFileBytes	64 to 268,435,456 bytes
maximumWalFileBytes	64 to 67,108,864 bytes
maximumRecords	1 to 1,000,000 data/WAL items
maximumRecordBytes	22 to 17,825,792 bytes
maximumResultBytes	1 to 16,777,216 bytes

All numeric limits must be positive safe integers. The caller must stop and retain external ownership of the LocalSmartDb daemon for the full call. Current storage roots additionally enforce this through SmartDB's storage-owner lock. Roots created before that lock existed rely on the caller's external single-owner coordination plus descriptor and stability validation. One timeoutMs budget covers sidecar lookup/spawn, readiness, and the command. Cancellation or timeout kills and reaps the disposable sidecar before the call rejects.

Offline Physical Namespace Inspection

inspectOfflinePhysicalNamespaces() is a Linux-only management API for validating and listing the physical database and collection namespaces in a stopped SmartDB file-storage root. It uses Linux openat2 descriptor traversal and starts only the disposable Rust management sidecar. It never calls SmartdbServer.start(), opens a database listener, initializes the storage adapter, runs migration or recovery, repairs a tail, truncates a WAL, compacts data, or persists a hint.

import { LocalSmartDb } from '@push.rocks/smartdb';

const namespaces = await LocalSmartDb.inspectOfflinePhysicalNamespaces({
  folderPath: '/var/lib/myapp/smartdb',
  limits: {
    maximumEntries: 100_000,
    maximumDatabases: 4096,
    maximumCollections: 10_000,
    maximumIndexes: 100_000,
    maximumDataFileBytes: 256 * 1024 * 1024,
    maximumWalFileBytes: 64 * 1024 * 1024,
    maximumIndexFileBytes: 16 * 1024 * 1024,
    maximumTotalFileBytes: 1024 * 1024 * 1024 * 1024,
    maximumRecordsPerCollection: 1_000_000,
    maximumRecordBytes: 17 * 1024 * 1024,
    maximumResultBytes: 16 * 1024 * 1024,
  },
}, {
  timeoutMs: 30_000,
});

// {
//   schemaVersion: 1,
//   databases: [
//     { name: 'myapp', collections: ['MetadataDoc', 'system.views'] },
//   ],
// }

The result is exactly { schemaVersion: 1, databases: Array<{ name, collections }> }. Databases and each database's collections are sorted by physical name, empty database directories are retained, and no documents, index definitions, record identifiers, or values cross the management boundary.

The accepted root must contain the current .__rustdb_internal/storage-owner.lock; legacy roots without that lock are rejected. The sidecar takes the shared owner lock, so an active engine causes the call to fail. Every reported collection must have the canonical complete profile of regular data.rdb, wal.rdb, and indexes.json files. A regular keydir.hint is optional and is treated only as an allowed legacy cache file; runtime recovery ignores its contents. Missing canonical files, unexpected collection entries, malformed index metadata, invalid data or WAL records, symlinks, cross-filesystem traversal, resource-limit exhaustion, and changes observed during inspection fail the complete operation. No partial result is returned and the storage tree is not mutated.

Inspection performs two full bounded passes and returns only if their namespace results and retained filesystem observations agree. The entry, database, collection, index, file-byte, and record limits are enforced independently on each pass, so total validation work is at most twice the corresponding caller cap. The result-size cap applies to the single serialized result. This is a contract for the current owner-locked canonical SmartDB file-storage profile, not a claim of compatibility with arbitrary MongoDB layouts, legacy SmartDB roots, or unspecified future storage formats.

All limits are required; this API supplies no defaults or compatibility aliases. TypeScript validates the following bounds before creating the sidecar and revalidates folderPath after resolving it to an absolute path:

Input	Enforced range
folderPath	1 to 4096 UTF-8 bytes before and after absolute resolution; no control characters
Complete serialized request	At most 2,097,152 bytes
timeoutMs	1 to 2,147,483,647 milliseconds
maximumEntries	1 to 100,000 directory entries per pass
maximumDatabases	1 to 4,096 databases per pass
maximumCollections	1 to 10,000 collections per pass
maximumIndexes	1 to 100,000 index definitions per pass
maximumDataFileBytes	64 to 268,435,456 bytes per data file
maximumWalFileBytes	64 to 67,108,864 bytes per WAL file
maximumIndexFileBytes	2 to 16,777,216 bytes per index file
maximumTotalFileBytes	1 to 1,099,511,627,776 aggregate canonical collection-file bytes per pass
maximumRecordsPerCollection	1 to 1,000,000 data records plus WAL records/commit markers per collection per pass
maximumRecordBytes	22 to 17,825,792 bytes per record
maximumResultBytes	32 to 16,777,216 serialized result bytes

Every numeric value must be a safe integer. One optional timeoutMs budget and AbortSignal cover sidecar lookup/spawn, readiness, both inspection passes, and response delivery; timeout or cancellation kills and reaps the sidecar before rejection. For a point-in-time ownership proof, the caller must exclude every engine start and other storage lifecycle change continuously from before invocation through acceptance or handoff of the returned result. The owner lock proves that the engine was stopped during inspection, but it cannot replace that caller-held lifecycle exclusion after the sidecar releases its lock.

SmartdbDebugServer

Web-based debug dashboard served via @api.global/typedserver. Import from the debugserver subpath:

import { SmartdbDebugServer } from '@push.rocks/smartdb/debugserver';

const debugServer = new SmartdbDebugServer(server, { port: 4000 });
await debugServer.start();
// Dashboard at http://localhost:4000

await debugServer.stop();

The UI is bundled as base64-encoded content (via @git.zone/tsbundle) and served from memory — no static file directory needed.

SmartdbDebugUi (Web Component)

For embedding the debug UI directly into your own web application, import the <smartdb-debugui> web component:

import { SmartdbDebugUi } from '@push.rocks/smartdb/debugui';

// In your HTML/lit template:
// <smartdb-debugui .server=${mySmartdbServer}></smartdb-debugui>
//
// Or in HTTP mode (when served by SmartdbDebugServer):
// <smartdb-debugui apiBaseUrl=""></smartdb-debugui>

---

Supported Operations

SmartDB supports the core operations through the wire protocol. Use the standard mongodb driver — these all work:

CRUD

// Insert
await collection.insertOne({ name: 'Bob' });
await collection.insertMany([{ a: 1 }, { a: 2 }]);

// Find
const doc = await collection.findOne({ name: 'Bob' });
const docs = await collection.find({ age: { $gte: 18 } }).toArray();

// Update
await collection.updateOne({ name: 'Bob' }, { $set: { age: 25 } });
await collection.updateMany({ active: false }, { $set: { archived: true } });

// Delete
await collection.deleteOne({ name: 'Bob' });
await collection.deleteMany({ archived: true });

// Replace
await collection.replaceOne({ _id: id }, { name: 'New Bob', age: 30 });

// Find and Modify
await collection.findOneAndUpdate({ name: 'Bob' }, { $inc: { visits: 1 } }, { returnDocument: 'after' });
await collection.findOneAndDelete({ expired: true });
await collection.findOneAndReplace({ _id: id }, { name: 'Replaced' }, { returnDocument: 'after' });

Query Operators

// Comparison
{ age: { $eq: 25 } }     { age: { $ne: 25 } }
{ age: { $gt: 18 } }     { age: { $lt: 65 } }
{ age: { $gte: 18 } }    { age: { $lte: 65 } }
{ status: { $in: ['active', 'pending'] } }
{ status: { $nin: ['deleted'] } }

// Logical
{ $and: [{ age: { $gte: 18 } }, { active: true }] }
{ $or: [{ status: 'active' }, { admin: true }] }
{ $not: { status: 'deleted' } }

// Element
{ email: { $exists: true } }
{ type: { $type: 'string' } }

// Array
{ tags: { $all: ['mongodb', 'database'] } }
{ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }
{ tags: { $size: 3 } }

// Regex
{ name: { $regex: /^Al/i } }

Update Operators

{ $set: { name: 'New Name' } }
{ $unset: { tempField: '' } }
{ $inc: { count: 1 } }
{ $mul: { price: 1.1 } }
{ $min: { low: 50 } }        { $max: { high: 100 } }
{ $push: { tags: 'new' } }   { $pull: { tags: 'old' } }
{ $addToSet: { tags: 'unique' } }
{ $pop: { queue: 1 } }       // Remove last
{ $pop: { queue: -1 } }      // Remove first
{ $rename: { old: 'new' } }
{ $currentDate: { lastModified: true } }

Aggregation Pipeline

const results = await collection.aggregate([
  { $match: { status: 'active' } },
  { $group: { _id: '$category', total: { $sum: '$amount' } } },
  { $sort: { total: -1 } },
  { $limit: 10 },
  { $project: { category: '$_id', total: 1, _id: 0 } },
]).toArray();

Supported stages: $match, $project, $group, $sort, $limit, $skip, $unwind, $lookup, $addFields, $count, $facet, $replaceRoot, $set, $unionWith, $out, $merge

$lookup supports the equality form and a bounded correlated pipeline form with let, an optional $match using $expr/$and/$eq, and an optional following $limit. Correlated string equality predicates can use a full-key or subset foreign equality index to reduce candidates when available. Candidates are paged, the complete expression remains authoritative, and $limit counts only exact expression matches. Other BSON value shapes use the bounded scan path. Mixed localField/foreignField plus pipeline lookups and other inner stages are rejected explicitly. Complete filter trees are validated before data is read, and query nesting plus aggregation work are bounded.

A leading $match narrows the pipeline's source load instead of filtering after it: only matching documents are materialized, using an equality index when one serves the filter and a bounded scan otherwise. Filter shapes the query matcher cannot pre-compile fall back to the unnarrowed load, so results are unchanged. The materialization limit therefore applies to the matched set, which is what lets countDocuments(filter) run against collections larger than it.

Group accumulators: $sum, $avg, $min, $max, $first, $last, $push, $addToSet, $count

Indexes

await collection.createIndex({ email: 1 }, { unique: true });
await collection.createIndex({ name: 1, age: -1 });    // compound
await collection.createIndex({ field: 1 }, { sparse: true });
const indexes = await collection.listIndexes().toArray();
await collection.dropIndex('email_1');
await collection.dropIndexes();  // drop all except _id

🛡️ Unique indexes are enforced at the engine level. Duplicate values are rejected with a DuplicateKey error (code 11000) before the document is written to disk — on insertOne, updateOne, findAndModify, and upserts. Index definitions are persisted to indexes.json and automatically restored on restart.

SmartDB supports ascending and descending index keys with name, unique, sparse, and expireAfterSeconds options. It accepts background as a validated no-op and index version v: 2. Unsupported options, including partialFilterExpression, collation, and hidden, are rejected before any index in the request is created. TTL metadata is retained in the catalog; automatic TTL expiry is not implemented.

find() and collection aggregate() accept hint as an index name (including _id_) or an exact ascending/descending key pattern. Key-pattern order matters; if several indexes share a key pattern, use a name. The selected active scalar index supplies every candidate, using safe full-key or leading-prefix equality bounds, or a full scan of that index. The complete predicate still runs, and explicit sorting, projection, skip, and limit retain their normal semantics. Index traversal uses bounded pages under the namespace lock. Streaming reads release that lock between pages and client requests. Unsorted results have no guaranteed BSON value order.

await collection.createIndex({ organizationId: 1, userId: 1 }, { name: 'organization_user' });
const page = await collection.find({ organizationId: 'org-1' }, {
  hint: 'organization_user', sort: { userId: 1, _id: 1 }, skip: 20, limit: 10,
}).toArray();
const count = await collection.countDocuments({ organizationId: 'org-1' }, {
  hint: 'organization_user',
});

Unknown or quarantined indexes, malformed/ambiguous patterns, $natural, and hints on missing collections or views fail explicitly. Forced multikey traversal is unsupported and rejects; unhinted array queries keep their existing scan behavior. Sparse hints intentionally exclude documents absent from that index, including for empty filters and counts. A hint is never silently replaced by a collection scan or another index.

Hinted transaction reads construct the selected index from the existing bounded transaction snapshot plus buffered writes; they never fetch candidate documents from live storage. Read-only aggregate source pipelines also use that transaction view. Foreign-collection stages ($lookup, $unionWith) and writing stages ($out, $merge), including stages inside $facet, reject within transactions. Nontransaction execution uses the streaming and bounded-resource paths below.

Scalable query execution

Collections have no fixed total-document ceiling on the nontransaction find, count, distinct, update, delete, and findAndModify paths. Result cursors read bounded pages instead of retaining the whole result. Multi-document writes visit each source document once, including when updates move indexed values. Index publication stages changed keys instead of copying the complete collection index engine for each write.

Nontransaction aggregation pipelines composed of $match, $project, $unset, $set, $addFields, $replaceRoot, $replaceWith, $skip, $limit, $sort, $group, $count, $unwind, $lookup, $unionWith, and $facet also operate beyond 10,000 input or output documents. Counts use exact collection metadata where applicable, otherwise stream matching records. A limit bounds the work needed to find the requested number of matches. Without a limit, exact counts continue to EOF. Hints, deadlines and empty-result shapes remain supported; signed BSON count overflow and resource exhaustion reject explicitly. Exact counts never substitute estimates, and transaction counts retain the snapshot admission limits described below. Grouping accumulates one group at a time after an external sort; constant-key groups stream directly. Group accumulator semantics follow the documented local expression surface; this does not add the complete MongoDB aggregation language. Unwind retains one input while expanding arrays across batches. Lookup scans bounded foreign pages or indexed candidates, then assembles one byte-bounded result document. Unindexed joins can still require repeated foreign scans. Facet branches replay a shared input buffer or temporary spool; their combined result must fit the BSON document budget.

Scalar ordered indexes can satisfy complete sort keys after equality fields, including reverse traversal and mixed compound directions. This path currently requires distinct full index keys to preserve stable equal-key ordering. Arrays, unsupported BSON index types, and equal-key data use the general sort executor. Numeric comparison and scalar equality indexes share exact Int32, Int64, Double, and Decimal128 ordering, including large integers, signed zero, and NaNs.

Small sorted limits retain top-k candidates. Larger sorts and groups use bounded runs in unnamed temporary files, with automatic cleanup on exhaustion, cursor closure, cancellation, errors, and process exit. Configure the shared resources through SmartdbServer:

const server = new SmartdbServer({
  storage: 'file',
  storagePath: '/path/to/database',
  query: {
    sortBufferBytes: 4 * 1024 * 1024,
    maxSortSpillBytes: 4 * 1024 * 1024 * 1024,
    maxConcurrentSorts: 4,
  },
});

sortBufferBytes is an encoded-input target, not an exact allocator heap limit; a single BSON document may exceed the target. The spill budget includes retained cursor files, facet input spools, and simultaneous merge input/output. Exhausting resource admission returns an explicit error instead of truncating results. Cursor continuations retain at most 16 MiB each and share a 256 MiB encoded-byte budget. distinct results and individual group documents must fit their BSON response/document byte budgets. The 10,000-operation wire batch limit still applies to each request; the driver can split larger input batches.

Streaming collection reads are not snapshots: concurrent updates can affect later pages, and a dropped or replaced collection/index invalidates its continuation. Ownership checks, namespace gates, byte budgets, and maxTimeMS remain active through getMore and cleanup.

Remaining storage and pipeline limits: transactions still materialize their base snapshot and retain the 10,000-document / 32 MiB admission cap. $out and $merge can consume large source pipelines, but their final write batch retains the legacy 10,000-document / 32 MiB admission limit. Oversized output is rejected before changing the target. File transaction publication still rewrites a database snapshot. Removing those storage costs requires the owned Storage VNext commit/MVCC path and explicit offline migration; its recovered prototype is not selected for serving existing databases.

Comparing performance with MongoDB

The repository includes benchmark/run.ts, which starts fresh, loopback-only SmartDB and original MongoDB processes with isolated temporary storage. Supply an original mongod executable; the harness does not download one or use a MongoDB-compatible replacement as the reference server.

SMARTDB_RUST_BINARY="$PWD/rust/target/release/rustdb" \
  MONGODB_BINARY=/path/to/original/mongod \
  tsx benchmark/run.ts --engine both --sizes 1000,10001,100000 \
  --concurrency 1,16 --repetitions 3 --samples 250 --warmup 50 \
  > .nogit/scaling-comparison.jsonl

Run on Linux after building the current binary and stopping competing benchmark or build work. Both engines use the same driver and data, w:1, j:true, local reads, snapshot transactions, disabled retryable writes, and one connection per worker. MongoDB runs as a single-voter replica set. Repetitions alternate engine order. --id-type string covers string identifiers; --payload-bytes changes record size. Full scans and transactions use the separate --scan-samples count (default 3) because they are substantially more expensive.

JSON-lines output records binary hashes, host/version metadata, correctness failures, seed/index time, latency percentiles, throughput, process RSS and peak RSS, CPU ticks, and physical read/write bytes. Temporary database directories are retained for inspection and their paths are recorded. Failed operations remain visible; they must not be compared as successful fast queries. Small sample counts and a shared development host are diagnostic evidence, not a general claim of superiority over MongoDB. The recorded comparison includes both measured improvements and remaining gaps.

Database & Admin

await db.listCollections().toArray();
await db.createCollection('new');
await db.dropCollection('old');
await db.dropDatabase();
await db.stats();

const admin = client.db().admin();
await admin.listDatabases();
await admin.ping();
await admin.serverStatus();

SmartDB creates ordinary collections and one deliberately restricted view form. Capped collections, validators, collation, time-series collections, clustered indexes, encrypted fields, and all general view pipelines remain unsupported and are rejected before the database or namespace is created.

Restricted Deny-All Views

The only accepted view definition is the exact deny-all pipeline below. This is intended for compatibility barriers such as renaming a legacy source and leaving its old namespace present but permanently empty:

await db.collection('secrets').rename('secrets_v2');
await db.createCollection('secrets', {
  viewOn: 'secrets_v2',
  pipeline: [{ $match: { $expr: { $eq: [1, 0] } } }],
});

Both viewOn and pipeline are required. The pipeline must contain exactly one $match stage with exactly $expr: { $eq: [1, 0] }; empty pipelines, numeric or structural variants, additional stages, and additional collection-definition options are rejected with InvalidOptions (code 72) before mutation. Command metadata such as $db, lsid, comment, and API-version fields remains accepted. viewOn must identify an existing same-database collection or restricted view at creation time. General MongoDB views are not supported.

The logical view has no physical collection. find, count, and distinct return empty results. aggregate starts with an empty source and still applies the client pipeline; $lookup and $unionWith resolve the view as empty. Transactions first reading a current view record an explicit empty snapshot; a transaction that captured a populated collection before it is renamed and replaced by a view keeps that original snapshot for find, count, and distinct until commit conflict handling. Inserts, updates, deletes, findAndModify, indexes, $out, $merge, rename, and other data mutations fail with CommandNotSupportedOnView (code 166). dropCollection is the supported DDL exception for removing a logical restricted view.

Definitions are persisted in the physical system.views catalog. As in MongoDB, listCollections lists system.views as a physical storage collection and the logical namespace as type view with { viewOn, pipeline }, read-only metadata, and no idIndex. system.views is reserved from direct wire reads, writes, index operations, create, drop, rename, and aggregation output. Whole database drop and trusted database export/import/digest operations handle it as physical catalog data. Export never materializes a view; cross-database import remaps the exact catalog _id database prefix. Startup validates databases whose publication lease can be acquired before listener bind; a publication-held database is validated on its first permit after release. Import validates before mutation. Malformed, oversized, noncanonical, duplicate, reserved, or colliding catalog state fails closed.

system.views was not interpreted by earlier SmartDB releases. This reservation is a major-version compatibility boundary: before upgrading, operators must use the older release to rename or remove any unrelated physical system.views collection. Once upgraded, direct catalog rename or drop is intentionally rejected with code 166.

Bulk Operations

const result = await collection.bulkWrite([
  { insertOne: { document: { name: 'Bulk1' } } },
  { updateOne: { filter: { name: 'X' }, update: { $set: { bulk: true } } } },
  { deleteOne: { filter: { name: 'Expired' } } },
]);

Count & Distinct

const count = await collection.countDocuments({ status: 'active' });
const estimated = await collection.estimatedDocumentCount();
const names = await collection.distinct('name');

---

Wire Protocol Commands

Category	Commands
Handshake	hello, isMaster, ismaster
CRUD	find, insert, update, delete, findAndModify, getMore, killCursors
Aggregation	aggregate, count, distinct
Indexes	createIndexes, dropIndexes, listIndexes
Sessions	startSession, endSessions, killSessions
Transactions	startTransaction, commitTransaction, abortTransaction through driver sessions
Admin	ping, listDatabases, listCollections, drop, dropDatabase, create, serverStatus, dbStats, collStats, connectionStatus, renameCollection

Advertises MongoDB wire protocol versions 0–21. The documented command surface is tested with the official mongodb Node.js driver version 7.5.x.

OP_MSG parsing rejects unknown required flags, invalid CRC-32C checksums, duplicate BSON keys, noncanonical array indexes, malformed section boundaries, multiple body sections, duplicate or colliding document-sequence identifiers, and missing or non-string $db fields. Early command-envelope validation then accepts document sequences only as insert.documents, update.updates, or delete.deletes; empty, oversized, extra, mismatched, and body-colliding forms reject before authentication or command resources are acquired. All three accepted forms are consumed as the corresponding bulk command argument. Unknown optional flags are ignored. A message is limited to 48,000,000 bytes, each BSON document to 16,777,216 bytes, and BSON nesting to 100 levels. One message may contain at most 128 sections, 10,000 document-sequence documents, 100,000 BSON elements, and 100,000 BSON document or array containers. The advertised maxWriteBatchSize is therefore 10,000.

The moreToCome request flag is accepted only for the exact endSessions or killSessions cleanup envelope: an admin command body containing only the session array, writeConcern: { w: 0 }, and $db: "admin", with no document sequence. It executes without a transport response and leaves the connection reusable. Every other moreToCome envelope closes before command routing or response-owned resource creation.

Raw receive buffers and parsed requests are charged separately against a process-wide 256 MiB encoded-byte admission budget. A partial frame is charged by its incrementally reserved receive capacity, not by the length in its prefix. Receive leases retain their high-water charge until the buffer is dropped or successfully compacted, and every allocation growth is reconciled before another socket read. Large unread pipelines remain charged and are compacted only at geometric utilization thresholds; the budget is an encoded-input bound rather than a claim about exact allocator heap usage.

Read and Write Concerns and Wire Deadlines

SmartDB serves every database from exactly one voting node and states its concern profile in those terms instead of ignoring or blanket-rejecting the fields:

• Write concern. Every acknowledged write is fsynced to the write-ahead log and the data file before the response is sent, so w: 1, w: "majority" (the majority of one voter is that voter), and j: true are all satisfied by the same durable commit. w: 0 is acknowledged the same way whenever a response is expected; the exact writeConcern: { w: 0 } endSessions and killSessions envelopes still run without a transport response. wtimeout bounds a replication wait that does not exist on one node and has no additional effect. w: N for N > 1 fails with UnsatisfiableWriteConcern (code 100), tag-set modes fail with UnknownReplWriteConcern (code 79), and j: true fails with BadValue (code 2) on the in-memory storage backend, which does not journal. Write concern is accepted only on commands that write (including DDL, user management, commitTransaction, and abortTransaction) and is rejected with InvalidOptions (code 72) inside a multi-statement transaction, where it belongs on the commit.
• Read concern. local, available, majority, and linearizable all observe the durably committed state, because nothing is acknowledged before it is durable and reads never observe uncommitted transaction buffers. snapshot is accepted only on the first statement of a multi-document transaction and is served by the database-wide transaction snapshot described below. Read concern is accepted on find, count, distinct, and aggregate only, and afterClusterTime, atClusterTime, and afterOpTime are rejected with InvalidOptions because a single node issues no cluster time.
• Deadlines. maxTimeMS attaches a cooperative deadline to the command. It is checked at admission, during namespace-lock and maintenance-gate waits, before find, count, and distinct materialize documents, between documents in insert and delete batches, and immediately before a transaction commit is handed to storage; expiry fails with MaxTimeMSExpired (code 50). Database-permit waits keep their fixed internal timeout, and a single storage step or an in-progress update batch runs to completion before the next check, so a write that expires after its first documents were published reports the expiry while those documents remain durable, exactly like an interrupted MongoDB batch. The official driver transmits a transaction's maxCommitTimeMS as maxTimeMS on commitTransaction; a commit that expires before its batch is handed to storage publishes nothing, records the transaction as aborted, and carries the UnknownTransactionCommitResult label; the driver's commit retry then sees NoSuchTransaction with TransientTransactionError and restarts the whole transaction. getMore rejects maxTimeMS with BadValue because SmartDB has no awaitData cursors, and a literal maxCommitTimeMS field is rejected with InvalidOptions.

Callers may therefore rely on a successful SmartDB response as a durable, single-node acknowledgement. They must not read it as a replica-backed acknowledgement: majority semantics across several voters arrive with replication.

Unsupported Administrative Commands

SmartDB returns CommandNotFound (code 59) for the wire commands validate, explain, legacy authenticate, buildInfo/buildinfo, hostInfo, whatsmyuri, getLog, getCmdLineOpts, getParameter, setFreeMonitoring, currentOp, killOp, top, profile, compact, reIndex, fsync, and connPoolSync. These commands previously had placeholder success responses but do not yet implement their advertised MongoDB semantics. Authentication and authorization errors retain precedence; otherwise rejection occurs before database publication permits, session or transaction creation, and command dispatch. This restriction applies to the wire validate command, not the supported offline --validate-data tool. getFreeMonitoringStatus remains available and reports SmartDB's fixed disabled state.

---

Rust Crate Architecture 🦀

The Rust engine is organized as a Cargo workspace with 12 focused crates:

Crate	Purpose
rustdb	Binary entry point: TCP/Unix listener, management IPC, CLI
rustdb-config	Server configuration types (serde, camelCase JSON)
rustdb-wire	Wire protocol parser/encoder (OP_MSG, OP_QUERY, OP_REPLY)
rustdb-query	Query matcher, update engine, aggregation, sort, projection
rustdb-state	Internal deterministic Storage VNext commands, stable retry identities, and durable receipts
rustdb-kernel	Internal non-serving Storage VNext committed-log kernel with stable timelines, contiguous LSNs, strict recovery, and exact receipt replay
rustdb-replication	Internal non-serving OpenRaft 0.9.25 one-voter fresh-root foundation; not linked into the released rustdb server
rustdb-storage	Storage backends (memory, file), OpLog with point-in-time replay
rustdb-index	B-tree/hash indexes, query planner (IXSCAN/COLLSCAN)
rustdb-txn	Transaction + session management with snapshot isolation
rustdb-auth	SCRAM-SHA-256 credential handling, user metadata persistence, RBAC checks
rustdb-commands	40+ command handlers wiring everything together

Storage VNext is an internal, non-serving Linux storage path. Descriptor-relative access, local-filesystem validation and owner leases protect its roots. Its K2 kernel stores immutable paged roots, typed BSON records and checksummed commit markers in one append stream. Snapshots pin a root; commits append changed pages. Recovery, checkpoints and bounded generation compaction preserve pinned reads and conflict frontiers. Ambiguous publication fences the live owner until recovery.

Typed storage and index owners publish catalog, document, count, natural-order, index-entry and unique-owner changes together. Bounded exact-index pages avoid whole-collection candidate sets. The older bounded prototype and its persisted fixtures remain separate; the one-voter OpenRaft foundation is also non-serving. Custom index builds, general/reverse index ranges, auth/outcome integration, transaction overlays, offline conversion, serving resource admission and orphan-record reclamation remain integration work. Existing file/memory engines still serve requests, including their documented transaction limits. These foundations make no HA or overall MongoDB performance claim. macOS and other non-Linux VNext root opens remain unavailable; the serving legacy binaries are built for the targets below.

Cross-compiled for linux_amd64, linux_arm64, macos_amd64, and macos_arm64 via @git.zone/tsrust.

Multi-host release artifacts

Consumers obtain the matching engines from the same versioned @push.rocks/smartdb npm tarball on npmjs or Verdaccio. It includes dist_rust/rustdb_linux_amd64 and dist_rust/rustdb_linux_arm64, each accompanied by <binary>.tsrust-build.json. These provenance files contain binarySha256, the package version, Git commit and target. Verify the tarball's registry integrity and each executable's SHA-256 against that matching provenance file before installation. Both Linux artifacts are statically linked. Spark can extract and install these files independently of its compiled JavaScript bundle; SmartDB does not download engines at startup.

Normal tsrust builds select only the current host's partition from targetsByHost. Releases use tsrust matrix to coordinate both partitions after the version commit exists, then publish only after every exact-commit binary has executed and passed strict provenance assembly.

The release coordinator must be Linux amd64 with aarch64-linux-gnu-gcc and a registered Linux arm64 binfmt interpreter. Set these environment variables to an authorized Apple Silicon builder with Rosetta and a dedicated temporary root:

export SMARTDB_RELEASE_MACOS_SSH="release-user@mac-builder.example"
export SMARTDB_RELEASE_MACOS_TEMP_ROOT="/absolute/dedicated/smartdb-release"

gitzone release first runs pnpm run release:check-builders, before creating a version commit or tag. It verifies the project-local tsrust version, compiles and executes the required architecture probes, and rejects missing cross-toolchains or emulation, an unsuitable macOS host, missing tools, or an unsafe remote temporary root. The release build then:

1. Builds the TypeScript package and debug UI bundle from the version commit.
2. Streams an exact Git bundle into isolated local Linux and remote macOS worker workspaces and installs each frozen lockfile.
3. Builds and executes the Linux amd64, Linux arm64, macOS arm64, and macOS amd64 binaries on their assigned workers.
4. Retrieves bounded artifact sets and transactionally publishes the complete provenance-checked matrix into dist_rust/.

No host address or path is stored in the repository. A failed run retains its unique local diagnostics and owner-marked remote diagnostics, then prints their paths. Remote cleanup is an owner-validated publication gate, and an incomplete matrix never replaces the prior dist_rust/.

Storage Engine Reliability 🔒

The Bitcask-style file storage engine includes several reliability features:

• Write-ahead log (WAL) — every nontransactional point-document insert, update, or delete is logged before its data.rdb record is applied; bounded restart scanning validates the exact header, strictly increasing record sequences, nonduplicated commit references, operation shape, lengths, and CRCs, rejects complete or interior corruption, and repairs only an incomplete final record-header or commit-marker prefix. Recovery considers only the highest-sequence operation for each document key and replays it only when markerless, so an older intent cannot supersede a later acknowledged update, delete, or reinsert. If authoritative data is fsynced but its commit marker fails, that mutation succeeds; later document/index access and every mutating, destructive, replacement, cache-invalidation, or batch-publication path touching the collection reject until restart, while maintenance leaves the fence intact. Once committed entries grow past 16 MiB, header-preserving checkpoint truncation prevents unbounded normal growth.
• CRC32 checksums — every record is integrity-checked on read
• Automatic compaction — dead records are reclaimed when they exceed 50% of file size, runs on startup and after every write
• Authoritative streamed recovery — collection initialization rebuilds KeyDir from the complete data.rdb record stream, verifies each CRC without allocating the declared value size, and never trusts legacy hint files as runtime state; publication-held databases defer this work until first access after release
• Conservative torn-header repair — collection initialization truncates only a final sequence shorter than the fixed record header whose available record-magic bytes match; checksum, magic, length, UTF-8 key, and full-header short-payload failures stop initialization without changing data.rdb
• Stale socket cleanup — orphaned /tmp/smartdb-*.sock files from crashed instances are automatically cleaned up on startup

Data Integrity CLI 🔍

The Rust binary includes an offline integrity checker:

# Check all collections in a data directory
./dist_rust/rustdb_linux_amd64 --validate-data /path/to/data

# Output:
# === SmartDB Data Integrity Report ===
#
# Database: mydb
#   Collection: users
#     Header:       OK
#     Records:      1234 (1200 live, 34 tombstones)
#     Data size:    2097152 bytes
#     Duplicates:   0
#     CRC errors:   0
#     Hint file:    legacy cache (ignored)

Checks file headers, record CRC32 checksums, and duplicate _id entries. Existing hint files are reported as ignored legacy caches and do not determine data integrity. Exit code 1 if any authoritative-data errors are found.

---

Testing Example

import { expect, tap } from '@git.zone/tstest/tapbundle';
import { SmartdbServer } from '@push.rocks/smartdb';
import { MongoClient } from 'mongodb';

let server: SmartdbServer;
let client: MongoClient;

tap.test('setup', async () => {
  server = new SmartdbServer({ port: 27117 });
  await server.start();
  client = new MongoClient('mongodb://127.0.0.1:27117', { directConnection: true });
  await client.connect();
});

tap.test('should insert and find', async () => {
  const col = client.db('test').collection('items');
  await col.insertOne({ name: 'Widget', price: 9.99 });
  const item = await col.findOne({ name: 'Widget' });
  expect(item?.price).toEqual(9.99);
});

tap.test('should track changes in oplog', async () => {
  const oplog = await server.getOpLog();
  expect(oplog.entries.length).toBeGreaterThan(0);
  expect(oplog.entries[0].op).toEqual('insert');
});

tap.test('teardown', async () => {
  await client.close();
  await server.stop();
});

export default tap.start();

---

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

Company Information

Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.