@lossless.org/client

One TypeScript client for lossless.org NoSQLDB, SQLDB and ObjectStorage, with MongoDB, MariaDB, ClickHouse and S3 protocol adapters. LosslessOrgClient owns named connections and provides nosqldb(), sqldb() and objectstorage() interfaces. Server engines remain separate packages; this client never starts a database or creates a bucket implicitly.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install and connect

The mongodb dependency currently pins @tempfix/mongodb@7.6.0 through an npm alias to correct illegal getMore.maxTimeMS in bounded transactions without disabling their deadlines. This temporary driver is published only on https://verdaccio.lossless.digital. Configure the @tempfix scope before installation if your registry is not already Verdaccio:

Npm publication is paused while the tempfix alias exists; restore both registries when returning to the official driver after PR #5076 is released. This client version and later versions publish only to Verdaccio until then, so install the client from that registry:

pnpm config set @tempfix:registry https://verdaccio.lossless.digital --location=project
pnpm add @lossless.org/client --registry=https://verdaccio.lossless.digital

Supported runtime: Node.js. The test suites run on Node.js only.

Imports remain mongodb. The upstream correction is MongoDB driver PR #5076; return to the official driver after it is merged and a released version passes the transaction-pagination regressions.

With that registry configured, document access needs no additional driver install: mongodb is a dependency because the NoSQLDB/MongoDB family is the package baseline. The relational and object drivers are optional peer dependencies, so install the ones your families use:

Family	Additional install
nosqldb connections, @lossless.org/client/nosqldb, @lossless.org/client/testsupport	none
sqldb connections with backend sqldb or mariadb, @lossless.org/client/sqldb/mariadb	pnpm add mariadb
sqldb connections with backend clickhouse, @lossless.org/client/sqldb/clickhouse	pnpm add @clickhouse/client
objectstorage connections with backend s3	pnpm add @aws-sdk/client-s3

The aggregate entry point imports a family only when the configuration names it, so an absent driver stays absent until it is used; connect() then fails with driver_missing, naming the package, the entry point and the install command. The two relational families also have an entry point each: @lossless.org/client/sqldb/mariadb needs mariadb only and @lossless.org/client/sqldb/clickhouse needs @clickhouse/client only, while @lossless.org/client/sqldb is their union and needs both, because it exports SqlConnection and ClickHouseConnection as values. Import the family you use. A direct family import resolves its driver as the module loads, so an absent driver fails there as Node's ERR_MODULE_NOT_FOUND naming the package; driver_missing is the aggregate entry point's refusal, and it names the family entry point that needs the driver. optionalDrivers exposes the same table to tooling.

import { LosslessOrgClient } from '@lossless.org/client';

const client = new LosslessOrgClient({
  nosqldb: {
    app: { backend: 'mongodb', url: process.env.MONGODB_URL!, database: 'app' },
  },
  sqldb: {
    primary: { backend: 'mariadb', host: process.env.SQL_HOST!, database: 'app',
      user: process.env.SQL_USER!, password: process.env.SQL_PASSWORD! },
    analytics: { backend: 'clickhouse', url: process.env.CLICKHOUSE_URL!, database: 'app',
      username: process.env.CLICKHOUSE_USER!, password: process.env.CLICKHOUSE_PASSWORD! },
  },
  objectstorage: {
    assets: { backend: 's3', endpoint: process.env.S3_ENDPOINT!, region: 'us-east-1',
      credentials: { accessKeyId: process.env.S3_ACCESS_KEY!, secretAccessKey: process.env.S3_SECRET_KEY! },
      readinessBucket: 'existing-assets' },
  },
});

await client.connect({ timeoutMs: 30_000 });
try {
  const primary = client.sqldb('primary'); // SqlConnection
  const analytics = client.sqldb('analytics'); // ClickHouseConnection
  const rows = await primary.query<{ id: bigint }>({
    sql: 'SELECT id FROM accounts WHERE email = ?', values: ['alice@example.com'],
  }, { maxRows: 100 });
  for await (const row of analytics.stream<{ temperature: number }>({
    sql: 'SELECT temperature FROM readings WHERE device = {device:String}',
    values: { device: 'ssd-1' },
  })) {
    // Process each row before requesting more; filtering happens on the server.
  }
  const readiness = await client.ready();
} finally {
  await client.close();
}

Select nosqldb, sqldb or objectstorage as the backend for the corresponding lossless.org server. An engine profile declares capabilities; it does not translate unsupported SQL or add backend features. SQLDB requires engine version 0.2.3 or later for pooled connection reset and parameterless prepared statements. It has a smaller SQL/type subset than MariaDB, and does not currently support TLS, savepoints or ALTER TABLE. ClickHouse uses its HTTP protocol and SQL dialect. There are no cross-backend transactions or automatic replication.

Literal connection names and backend discriminators determine the return types. Each client owns the connections it constructs. connect() probes databases without changing their schemas. S3 construction is local; ready() checks an explicitly configured existing bucket and returns readiness_bucket_required if none was supplied. close() is idempotent, stops admission, cancels owned SQL operations, closes pools and joins final metrics-writer flushes. Create a new aggregate client after failure or close.

Interfaces

Import	API	Peer its declarations need
@lossless.org/client	LosslessOrgClient, configuration, capabilities, readiness, LosslessClientError	@aws-sdk/client-s3
@lossless.org/client/nosqldb	Migrated SmartData models, decorators, collections, cursors, sessions, exact persistence and administration; NoSqlConnection	none
@lossless.org/client/sqldb	Both relational families: SqlConnection, SqlTransaction, SqlTable, ClickHouseConnection, SmartClickHouseDb, tables, query builders, TimeDataTable, MetricWriter	none
@lossless.org/client/sqldb/mariadb	SqlConnection, SqlTransaction, SqlTable, their options and quoteIdentifier	none
@lossless.org/client/sqldb/clickhouse	ClickHouseConnection, SmartClickHouseDb, tables, query builders, TimeDataTable, MetricWriter, their options and quoteClickHouseIdentifier	none
@lossless.org/client/objectstorage	Migrated SmartBucket, buckets, directories, files, metadata, watchers, exact operations; ObjectStorageConnection	@aws-sdk/client-s3
@lossless.org/client/testsupport	Explicit disposable-database testing helpers	none

The last column is about type-checking, not loading: an entry point's declarations name an optional peer only where its API is typed against that driver. @lossless.org/client/objectstorage exposes the S3 client, SmartBucket.storageClient and the outputs the SDK defines, so a consumer type-checks it only with @aws-sdk/client-s3 installed, and the aggregate entry point reaches those declarations through client.objectstorage(). Every other entry point type-checks with no optional peer installed and without skipLibCheck; both relational families own every type in their public API and resolve their driver when the module loads. test/client/test.entrypoints.node.ts type-checks each entry point in a disposable consumer project that installed neither mariadb nor @aws-sdk/client-s3 and holds that table to it.

Family imports preserve the established SmartData and SmartBucket names and constructors. Their implementations live here; the client does not depend on the old packages. The aggregate loads only configured families. Direct family entry points do not initialize unrelated connections.

@lossless.org/client/nosqldb addition	API
Monotonic counters and timestamps	$max / $min in ISmartdataAtomicUpdate, for declared numeric and date fields
Race-free registration on a second unique key	$setOnInsert may seed a declared @unI() identity when upsert: true
Bounded plural upsert	Model.atomicUpsertMany(operations, opts?) — 1..1000 per-document filter/update pairs in one unordered round trip
Content-addressed primary keys	model option identityAsDocumentId on defineCollectionModel() and @Collection()
Identities on a migrated unique index	@unI({ indexName }) names the index that backs the identity instead of <field>_1
Typed null predicates	read and atomic filters accept null for a field declared optional, so a value stored as an explicit null stays addressable
Owned sessions on cursors	getCursor() takes the handle from db.createSession() and keeps it leased until the cursor is closed
Owned sessions on instance writes	instance.save({ session }) and instance.delete({ session }) take the handle from db.createSession(), leased for the one write like every other model API
Typed connection pool options	new SmartdataDb(options: ISmartdataDbOptions) types maxPoolSize, maxIdleTimeMS, serverSelectionTimeoutMS and socketTimeoutMS beside the descriptor fields, so no cast through IMongoDescriptor is needed
Named plural selectors	a non-empty $in anchors atomicUpdateMany(), atomicDeleteMany() and atomicUpsertMany(); singular operations are unchanged
Fields only atomic writes may update	@svDb({ atomicOnly: true }) and atomicOnlyFields — seeded by every write that creates the document, never overwritten by a later save()
Defaults inside $expr	{ $ifNull: ['$field', literal] } operands, so a guard covers rows written before the field existed
Fenced instance saves	instance.saveIf(fence, opts?) — one non-upserting updateOne; matchedCount === 0 means another writer moved the document first
Identity rotation	Model.atomicFindOneAndReplace(filter, replacement, opts?) — the one write that moves an @unI() identity, under the matched document's _id
Fail-fast on unprepared collections	a session-carrying call inside a transaction refuses instead of stalling on lazy index DDL until the client deadline
Migration-owned indexes	toleratedIndexNames on the model — named undeclared indexes are observed, never created, dropped or verified, and no longer make the topology divergent
Partial index filters in inspection	inspectNamespace(), inspectCollectionTopology() and getIndexInfo() report a partial index's partialFilterExpression exactly as the backend lists it, as a frozen snapshot present only on a partial index
Key-value store readiness	EasyStore.ensureInitialized() — installs the store's declared index at startup, outside any transaction window, so a first readKey()/writeKey() needs no index DDL
Key-value store copies	EasyStore readKey()/readAll() return copies and writeKey()/writeAll()/replace() store copies, in the persisted BSON form, so a read result is a stable snapshot and a caller-side mutation never reaches the store
Client deadlines	timeoutMS (1..120000) on every ordinary model read and write, instance save()/saveIf()/delete() included, on every exact call, and session.withTransaction(callback, { timeoutMS }) for a whole transaction, retries and commit included; expiry rejects with SmartdataPersistenceError code timeout, and an exact write interrupted in flight with ambiguous_write
Bounded database close	db.close({ timeoutMs }) bounds the whole close — session cleanup, the optional database removal and the driver client's close — and rejects at its deadline with SmartdataPersistenceError code timeout, leaving the database close_failed; the driver's close keeps running until it settles on its own, and the next close() joins it
Transactions ended by close	a session.withTransaction() whose session db.close() ends while it runs rejects with SmartdataPersistenceError code unsupported_operation — "nothing was committed" before the commit, "the commit outcome is unknown" during it — instead of resolving with the callback's result
Transaction conflicts	a session.withTransaction() whose every attempt the database aborted as transient (a write conflict or another error labelled TransientTransactionError) until the driver's 120-second retry window closed rejects with SmartdataTransactionConflictError (a SmartdataPersistenceError of code transaction_conflict) carrying attempts, elapsedMS and the last attempt's error as cause: nothing was committed and the transaction is safe to re-run. With timeoutMS it rejects with timeout instead. Ordinary model writes that run in their own transaction reject the same way
Transaction commit outcome	SmartdataSession.getCommitOutcome(error) states for every value a session.withTransaction() rejected with whether the transaction could have committed: not_committed (nothing was committed) or unknown (a commit was sent and never confirmed); the rejection itself keeps its identity
Cancellation	signal on every exact call, atomicDeleteMany() and session.withTransaction(callback, { signal }), which interrupts the session's operation in flight, rolls the attempt back and never commits after an abort
Nested array paths	@svDb({ arrayPaths: [...] }) declares nested arrays below a persisted root; $push, $addToSet and $pull in atomicUpdate(), atomicUpdateMany(), atomicFindOneAndUpdate() and atomicUpsertMany() accept them, while undeclared and non-array nested paths stay refused with invalid_argument
Grouped totals over nested paths	getGroupedTotals() / getGroupedTotalsPage() group by and sum nested paths declared through atomicPaths or arrayPaths, and unwind: path groups by each element of one declared array
Grouped totals over array elements	with unwind: 'lines', groupBy and sums name declared fields of the unwound element — groupBy: ['lines.accountNumber', 'lines.side'], sums: ['lines.amountCents'] — so per-account debit and credit totals need no raw aggregation; undeclared element fields and an unwind that holds no groupBy or sums path stay refused with invalid_argument
Grouped totals element filter	elementFilter: { 'lines.accountNumber': '1200' } beside unwind: 'lines' counts and sums only the matching elements — one account's balance per side — with the atomic-filter operators on the unwound array and its declared element paths; any other path or operator is refused with invalid_argument
Grouped totals paging	rows come ordered by group key; getGroupedTotalsPage() returns nextCursor while hasMore, cursor continues after it, and iterateGroupedTotals() reads every group page by page — exact across pages for null, number, string and boolean group values (proven on NoSQLDB 8.0.2, 10.9.0 and MongoDB; ObjectId and date values are ordered by BSON type order but not yet covered by the paging tests), refused with unsupported_operation for other values, and a cursor from another query is refused with invalid_argument
Query-plan inspection	Model.inspectQueryPlan(db, { filter, sort?, projection?, limit?, timeoutMS?, signal? }) — the winning plan's stages (stage, depth, indexName, direction) and nReturned/totalKeysExamined/totalDocsExamined, read-only and bounded, so a consumer proves a model query uses its intended index without raw driver access
Per-instance model binding	bindDocToManager(manager, Model) returns a subclass of a model declared with @managed() and no manager argument, named like the model and bound to that manager, so several databases use the same models at once; any other binding is refused with invalid_configuration
Subclasses need their own binding	a subclass of a bound model that declares decorators or a persistence policy of its own and no @Collection(), @managed() or defineCollectionModel() binding is refused with invalid_configuration on every read and write, so it never writes through the parent's schema; an undecorated subclass keeps the inherited binding
Observing this process's writes	db.observeWrites({ fields, listener }) — one frozen report per acknowledged model write, { collection, kind, documents, anchors } with the requested top-level fields of the written documents and of the selector's equality anchors; writes inside a db.createSession() transaction are reported after the commit, never after a rollback; works on NoSQLDB, where watch() is unsupported
Following this process's reads	db.trackReads(fn) resolves { result, reads }; reads.changed turns true once a write this process makes through the database's models could have changed what a model read of fn returned, judged by the read's query and the ids it returned, and a write it cannot place counts as one that did, so a cache keyed by reads rebuilds only what a write touched; works on NoSQLDB
Following every process's writes	with new SmartdataDb({ ..., writeRevisions: { partitionField: 'organizationId' } }) in every writer, the awaited reads.isCurrent() counts changes per collection and partition in the write's transaction; confirmed writes of this database that leave reads.changed false receive exact revision allowances, while other writers and unknown outcomes stay conservative; a revision snapshot overlapping a local commit that touches its keys is unknowable; db.invalidateTrackedReads() states raw writes and migrations; overlapping transactions of one process bump separate documents of a partition's counter, so they rarely conflict on it, a check reads the counters in one query plus one for those with shards in use, and every process of a database upgrades together; needs an engine identified as NoSQLDB, elsewhere isCurrent() is always false
Following committed writes in commit order	with new SmartdataDb({ ..., writeLog: { fields: ['id', 'organizationId'] } }) in every writer, db.followWrites({ fields, onWrite, onGap }) delivers each committed transaction once with its write reports: this instance's own commits at once (origin: 'local'), every other commit from the engine's commit journal in commit order (origin: 'journal', with its stamp), within 250 ms while active and a second when idle; rollbacks and aborted retry attempts deliver nothing, raw writes name their collections in unplaced, and a follower that falls out of the journal's 15-minute retention or meets a restarted memory engine calls onGap and continues from the head; needs NoSQLDB 10.9.0, elsewhere the option is inert and following is refused with unsupported_operation
Disposable test databases on a unix socket	SmartdataTestDatabase.create({ connection: { socketPath } }) from @lossless.org/client/testsupport reaches an engine that listens only on a unix-domain socket, fault helpers included; the loopback host/port form is unchanged

docs/source/smartdata/readme.md documents each one with its refusals.

@lossless.org/client/objectstorage addition	API
Listing with size and ETag	bucket.listObjectEntriesPage({ prefix?, limit, startAfter?, signal? }) — one keyset page of { key, size, etag, lastModified? }; bucket.listAllObjectEntries(prefix?, { pageSize?, signal? }) iterates every page
Bucket lookup for bucket-scoped credentials	smartBucket.getBucketByName(name, { signal? }) and smartBucket.bucketExists(name, { signal? }) send one HeadBucket; createBucket(name, { signal? }) and removeBucket(name, { signal? }) take the same options
Byte-range downloads	bucket.fastGetRangeStream({ path, range: { start, end? }, signal? }, typeArg?) — one ranged GetObject resolving { stream, range, size }; range failures reject with ObjectRangeError code invalid_argument, range_not_satisfiable or invalid_response
Reads of objects that may be missing	bucket.fastGetIfPresent({ path, signal? }) — one GetObject resolving the object's bytes, or null when the key does not exist
Bounded requests	new SmartBucket(config, { connectionTimeoutMs?, requestTimeoutMs?, maxAttempts? }) and the same options on new ObjectStorageConnection({ ... }) — every request of the client a SmartBucket builds ends within smartBucket.requestBounds.requestDeadlineMs, retries included, or rejects with ObjectStorageRequestDeadlineError
Signals on writes, moves and listings	bucket.fastPut({ ..., signal? }), bucket.fastMove({ ..., signal? }), bucket.listObjectKeysPage({ ..., signal? }), bucket.listAllObjects(prefix?, { signal? }) and bucket.listAllObjectsArray(prefix?, { signal? }); fastGet, fastPutStream, fastCopy and fastRemove already took one
Copies of large objects	bucket.fastCopy({ sourcePath, destinationPath?, targetBucket?, nativeMetadata?, deleteExistingNativeMetadata?, signal? }) copies a source above 5 GiB, which one CopyObject cannot copy, part by part with UploadPartCopy
Missing-bucket readiness	smartBucket.checkReadiness(name, { timeoutMs?, signal? }) and connection.ready() report reason: 'bucket-missing' when the provider answers that the bucket does not exist; a denied probe stays unavailable

checkReadiness() reports bucket-missing only for a definitive answer to its HeadBucket: NoSuchBucket or HTTP 404. A denied probe (HTTP 403), any other provider answer and a transport failure stay unavailable, and a probe that ends at its deadline stays timeout, so a credential problem never reads as a missing bucket. The result carries no provider error or cause. Before bucket-missing, a missing bucket reported unavailable. TSmartBucketReadinessFailureReason is an open union: code that switches over it keeps a default branch, because new reasons can arrive in minor releases.

A bucket lookup sends one HeadBucket for the named bucket, never ListBuckets, so a credential scoped to its bucket can open it without the account-wide list permission, and a lookup does not depend on which buckets a listing returns. A missing bucket (404) keeps its answer: getBucketByName() rejects with Bucket '<name>' not found. and bucketExists() resolves false. A bucket the credentials may not access (403) rejects both with an error that names the denial and keeps the provider error as cause; S3 does not say whether such a bucket exists. Any other provider failure is rethrown unchanged. signal aborts the in-flight request and rejects with the signal's reason. Lookups, creations and removals run as lifecycle-managed operations: SmartBucket.close() aborts and awaits them, and the storage client cannot be replaced while one is active.

listObjectEntriesPage() pages exactly like listObjectKeysPage(): limit is 1 to 1000, startAfter must be a non-empty key under the prefix, entries arrive in strict UTF-8 byte order after it, and nextStartAfter is present only while the provider reports more. Keyset pages stay valid when earlier keys are deleted between requests. Every field comes from the one ListObjectsV2 response, so no HEAD is sent per object. size is the listed Size, the byte length HEAD reports as ContentLength. etag is the listed ETag with its surrounding double quotes removed; HEAD returns the same entity tag, quoted. An entity tag identifies an object version and is not a content digest: AWS S3 derives it from the content MD5 only for single-part uploads without SSE-KMS or SSE-C, and a multipart tag ends in -<parts>. lastModified is present when the provider sends LastModified. Both values describe the object when the page was listed. A page with a missing or non-integer size, a missing or malformed ETag, or any key-page violation is refused rather than returned partially. signal aborts the in-flight request and rejects with the signal's reason; the iterator issues one request per page and none while the caller holds an entry, so breaking out ends the listing.

fastGetRangeStream() sends one GetObject with Range: bytes=start-end. start and end are zero-based and inclusive, as in HTTP; without end the range runs to the last byte. The stream carries exactly the served bytes, and range (the served range, end clamped to the last byte) and size (the whole object's length) come from the response's Content-Range, so a download route answers 206 Partial Content with Content-Range: bytes ${range.start}-${range.end}/${size} without a second HEAD. A start that is not a non-negative safe integer, or an end that is not a safe integer or lies below start, is refused with invalid_argument before any request. A range that starts at or beyond the end of the object — every range of an empty object included — rejects with range_not_satisfiable, the provider's 416 as cause. The 416 from S3 carries no object size, so a route that answers its own 416 with Content-Range: bytes */size reads the size with fastStat(). A response without a Content-Range that matches the request rejects with invalid_response and its body is destroyed, so a provider that ignored the header never hands out the whole object. signal and SmartBucket.close() cancel the request and the body as they do for fastGetStream().

fastGetIfPresent() answers "read it, or tell me it is gone" with one GetObject, so an object removed between an existence check and a read can no longer turn into a failed read: a provider answer that the key does not exist — NoSuchKey, NotFound or any other HTTP 404 — resolves null. Every other failure rejects as fastGet() does, the provider error unchanged: a missing bucket (NoSuchBucket), a denied request, a failure while the body streams, and an abort through signal or SmartBucket.close(), which rejects with the abort reason even when the provider answers not-found after it. fastExists() and the create-only check of fastPutStream() classify a missing object the same way.

fastCopy() reads the source with one HeadObject. A source of up to 5 GiB, the largest object one CopyObject may copy, is copied with one CopyObject, as before. A larger source, which failed before, is copied with a multipart upload of 256 MiB parts (larger when the object needs more than 10,000 parts, up to the 5 TiB S3 maximum), each part an UploadPartCopy of its byte range that requires the source to still carry the ETag the HeadObject read, so a source replaced during the copy fails the copy with the provider's 412 instead of mixing two versions. Each part is one request with its own requestTimeoutMs bound. The new object gets the properties CopyObject gives it: the source's metadata and content headers with deleteExistingNativeMetadata unset, only the given metadata with it set. Object tags, the storage class, server-side encryption settings and a website redirect are not carried by a part-by-part copy; the new object takes the target bucket's defaults for them. Any failure or abort, signal and SmartBucket.close() included, sends AbortMultipartUpload and never completes the upload; a create that answered after cancellation has its upload aborted as well. A copy above 5 GiB needs a provider with UploadPartCopy and x-amz-copy-source-if-match; ObjectStorage has them from 10.7.0.

Every request of the S3 client a SmartBucket builds is bounded. An attempt whose socket does not connect within connectionTimeoutMs (default 10 s, at most 60 s), or whose response headers do not arrive within requestTimeoutMs of its start (default 60 s, at most 1 h; the upload of the request body counts), fails with the SDK's TimeoutError; the SDK retries it up to maxAttempts attempts in all (default 3, at most 10, standard retry mode, AWS_MAX_ATTEMPTS and shared config files ignored). Each value must be a positive safe integer; anything else is refused with a TypeError before a client is built. smartBucket.requestBounds (and connection.requestBounds) reads the frozen bounds, including requestDeadlineMs = maxAttempts × requestTimeoutMs + (maxAttempts − 1) × 20 000 ms + 1 000 ms — the attempts, the longest wait the retry strategy takes before a retry (a provider's retry-after hint included), and a second for the client's own work. 221 000 ms with the defaults. A request still unsettled at that deadline, a stalled error body included, rejects with ObjectStorageRequestDeadlineError, its attempt in flight is destroyed and no further attempt of it starts, so a consumer can hold the client to an upper bound of its own (requestDeadlineMs < bound). The bound does not undo a request: a PutObject the provider received completely before the deadline or an abort may still take effect. It covers requests, not the time a caller takes to read a download it received: a response body streams at the caller's pace until it is consumed, the caller's signal aborts or the SmartBucket closes. While a client assigned through storageClient or replaceStorageClient() is in use, requestBounds is undefined and that client's own configuration applies. A single-request upload (fastPut, fastPutStream) must finish sending its body within requestTimeoutMs; large or slow bodies use fastPutStreamExact(), whose parts are separate requests, each bounded on its own (5 MiB unless requestedPartSize asks for more or the length needs more than 10,000 parts).

fastPutStream() and Directory.fastPutStream() send one PutObject, which must declare the body's length. Without contentLength only a file stream (fs.ReadStream) uploads, sized by the S3 client from the file; a Web ReadableStream or any other Readable is refused with ObjectStreamLengthError before any request and before the source is read, and the caller keeps the source. Such a stream failed before with a TypeError for the header x-amz-decoded-content-length: undefined. A stream of unknown length uploads with fastPutStreamExact({ maxContentLength }). File.create() and File.updateWithContents() take no length, so their stream form accepts only file streams.

A signal aborts the request in flight, stops its retries and rejects with the signal's reason; an already aborted signal sends nothing. A signal of a bucket read, write, copy, move, removal, existence check or listing that is not an AbortSignal, a Proxy of one included, is refused with a TypeError before any request. A per-call time limit is a signal as well, AbortSignal.timeout(ms) or an own AbortController.

Document APIs that need a MongoDB backend

The document model runs on both NoSQLDB and MongoDB, but several APIs do not mean the same thing on both. The NoSQLDB engine is qualified at 8.0.2 and 10.9.0 (see Verification); the two releases part on the grouped-totals and guarded-$expr rows below — 10.9.0 groups composite keys exactly and evaluates the guard, 8.0.2 does neither — on whether a migration can create a partial index or a collection validator at all, on when a transaction learns of a write conflict, and on whether $elemMatch applies query operators to scalar array elements.

init() asks the connected engine what it is with one buildInfo command and records the answer on the database as engineIdentity ({ product, version }). NoSQLDB answers with its own nosqldb identity document from 10.3.0 onward; a MongoDB server answers buildInfo without one; an engine that refuses the command — NoSQLDB 8.0.2 answers CommandNotFound — stays { product: 'unknown', version: null }. The identity is what states capabilities on a plain SmartdataDb, and an unidentified engine states none at all, so every capability-gated call refuses. NoSqlConnection keeps stating the capabilities of its declared backend from construction on.

Document API	On the NoSQLDB engine	On MongoDB
getGroupedTotals() / getGroupedTotalsPage() / iterateGroupedTotals() with one groupBy field	exact, including sums, declared nested paths, unwind of a declared array and declared fields of its element, elementFilter, group key order and cursor paging, on 8.0.2 and 10.9.0 alike	exact
the same with two groupBy fields	exact, sums, nested paths, unwind, two element fields of one unwound array (lines.accountNumber with lines.side), elementFilter and cursor paging included, on an engine that identifies itself as NoSQLDB 10.5.0 or newer: that release evaluates the composite group key per document. Below it the call is refused with unsupported_operation naming both fields and the single-field form, because the engine leaves the composite group key unresolved and answers with a single row whose group values are the literal field paths ('$owner') and whose count is the whole filtered collection; the refusal is the guard against that wrong number, not a fix for it	exact; the client wraps each key member in $ifNull, because MongoDB leaves a missing member out of the key and would otherwise report it as a second null group
guarded $expr atomic filters ($add, $lte, $ifNull)	exact on 10.6.0 and newer: the engine evaluates the client's fail-closed compilation ($cond, $let, $isNumber, $type), so the guard admits within the limit, refuses beyond it and never matches a stored value that is not a finite number. 8.0.2 rejects the compilation with invalid argument: $cond and leaves the document untouched	exact
watch() change streams	unsupported. NoSqlConnection refuses with unsupported_capability, and so does a plain SmartdataDb whose engine identified itself; against an engine that answers no identity the refusal instead arrives from the engine as CommandNotFound once the change stream is read. db.observeWrites() reports the process's own model writes on both backends	supported on replica sets
model-declared partial indexes (partialFilterExpression)	not expressible: the index-option type rejects it and the runtime refuses it with invalid_configuration. A client decision, not an engine limit on every release — 10.6.0 and newer create partial indexes, 8.0.2 refuses the option with InvalidOptions (72) — because one model declaration has to hold on every qualified backend. A migration may create one on 10.9.0; toleratedIndexNames accepts it and inspection reports its filter	not expressible, but a migration may own one; inspection reports its filter
collection validators (validator, validationLevel, validationAction)	no client API; a migration owns a validator through the driver. 10.7.0 and newer create the collection, list the options exactly as given and enforce them on every model write: a violating insert() or atomicUpdate() rejects with the engine's DocumentValidationFailure (121) and its errInfo, which the client passes through unchanged rather than mapping to a SmartdataPersistenceError code, and stores nothing. Models initialize on a validated collection like on any other. 8.0.2 refuses the option with InvalidOptions (72) and creates no collection, so a validator is never silently dropped	enforced; not exercised by the client's MongoDB suite
write conflicts between transactions, and writes outside a transaction to a document a transaction holds	10.9.0 (since 10.6.1) detects a conflict at the statement, as MongoDB does: the first writer wins, and the later transaction's statement rejects with the engine's WriteConflict (112) labelled TransientTransactionError, which the client passes through so session.withTransaction() re-runs that callback. A write outside a transaction waits until the transaction holding its document ends, then applies on top; the wait is bounded by the write's own timeoutMS and by the engine's fixed 60-second transaction lifetime, after which a waiting writer aborts the holder. Awaiting such a write inside the holding transaction's callback therefore stalls that transaction until one of the two expires. 8.0.2 detects the conflict at commit: the first transaction to commit wins, a write outside a transaction applies at once, and the transaction that wrote first re-runs. Both writes land on either engine; a callback must be safe to re-run and must not swallow the statement error	first writer wins at the statement and outside writes wait, MongoDB's documented rule; not exercised by the client's MongoDB suite
value-form $elemMatch in read filters ({ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }, { labels: { $not: { $elemMatch: { $nin: runnerLabels } } } })	exact on 10.7.1 and newer: the operators apply to each array element as a whole, so the range matches only a document with one score inside it and the claim selects a job only when every one of its labels is a runner label. 8.0.2 parses every operand as a query over embedded documents and refuses the value form with TypeMismatch (14). A non-document operand is refused on both, with MongoDB's BadValue (2) on 10.7.1 and newer and 14 on 8.0.2. The client passes the operand through unchanged; the filter type admits the value form over scalar elements and the document form over embedded documents. Atomic filters (atomicUpdate() and the other atomic writes) accept no $elemMatch on any backend: the client refuses it with invalid_argument before any backend is contacted	exact
Model.inspectQueryPlan()	explains on 10.9.0: the summary names the winning plan's stages and its execution totals. 8.0.2 answers explain with CommandNotFound (59), and the call rejects with unsupported_operation ("The connected engine does not explain queries.") rather than returning a plan it cannot know. Stage shapes are the engine's own: on 10.9.0 a limited sort on an unindexed field is LIMIT over SORT over COLLSCAN, an equality on a unique single-field index is FETCH over IXSCAN, and an $or whose branches each have an index is a COLLSCAN	explains: the same limited sort is SORT over COLLSCAN, the limit folded into the sort; the equality is EXPRESS_IXSCAN on the unique index; the $or is SUBPLAN over FETCH over OR over one IXSCAN per branch
aggregation-pipeline (array) updates	refused by the client with invalid_argument before any backend is contacted. A client-owned refusal, not an engine limit: both qualified engines apply a pipeline update themselves	same client refusal

test/nosqldb/smartdb/test.engine-capability-boundary.node.ts asserts the identity, grouped-totals, guarded-$expr, partial-index, collection-validator, write-conflict, $elemMatch and pipeline-update rows against both qualified engines, so a boundary that moves shows up as a failing suite rather than as a changed number in a consumer. The two halves of the grouped-totals, guarded-$expr, write-conflict and $elemMatch rows are separate tests, each bound to the engine that answers it, and the engine's own composite-key answer stays under assertion through the raw aggregateGroupedTotals() boundary — one literal row on 8.0.2, the two exact pairs on 10.9.0. The same boundary pins the rest of the engine's $group stage: a non-document accumulator is refused with TypeMismatch (14) on 10.9.0 while 8.0.2 answers it, and a $cond inside a composite key is evaluated per document on 10.9.0 while 8.0.2 answers its single literal row. It also pins whether a migration can create a partial index — 10.9.0 creates it and lists the filter, 8.0.2 refuses with 72 — and a collection validator — 10.9.0 creates and enforces it against model inserts and updates, 8.0.2 refuses with 72 — which writer a conflict sends back: on 10.9.0 the later transaction fails at its statement and re-runs while a write outside a transaction waits for the holder's commit, on 8.0.2 the later transaction commits first and the holder re-runs — and what $elemMatch answers through getInstances(): 10.9.0 matches the documented range and the $not/$elemMatch/$nin claim exactly as the MongoDB filter suite asserts them, 8.0.2 refuses both with 14, and a non-document operand is refused with 2 on 10.9.0 and 14 on 8.0.2. The watch() row is the contract refusal ts/nosqldb/classes.collection.ts states; the engine suite opens no change stream.

Relational SQL

SqlConnection uses the official MariaDB connector for parameters, protocol, TLS and decoding. This client owns cancellable admission and TCP sockets from authentication onward. A transaction holds one physical session; releasing a session resets its state before reuse.

• execute({ sql, values }, options) returns affectedRows, optional exact insertId: bigint, and completion: 'acknowledged'. Use it for statements that return an update result.
• query<Row>(statement, options) materializes at most 10,000 rows and 16 MiB by default. Set maxRows and maxBytes explicitly to change those limits. Overflow rejects with result_limit and closes the stream.
• stream<Row>(statement, options) iterates rows with backpressure. Early return, timeout, cancellation and connection shutdown close its owned stream/socket.
• insert(table, iterable, options) accepts iterable or async-iterable rows with the same columns, batching at 500 rows/4 MiB by default. Batches commit independently. A later failure reports partial_write with acknowledged rows, or ambiguous_write if the current batch outcome is unknown. Use a transaction with explicit statements for atomic multi-statement writes.
• transaction(async tx => ..., options) joins an outstanding final transaction operation before commit. Await each operation; concurrent operations on one transaction are rejected. A thrown callback rolls back when the session is still connected. An interrupted commit has an unknown outcome.
• table<Row>(name) handles an existing table with bound equality/null selectors, typed query, stream, insert, update and delete. Empty mutation selectors and undefined selector values are rejected. Schema creation is explicit SQL.

Values use placeholders; identifiers use a separate quoting function. Raw SQL text is trusted application code. A result generic is a caller-declared shape, not static validation of arbitrary SQL.

MariaDB returns BIGINT as bigint, DECIMAL as a string, binary columns as Buffer, SQL null as null, and dates/times as strings. Fractional timestamp strings retain server precision. JavaScript Date inputs bind in UTC with millisecond precision; MariaDB sessions use UTC. Exact fractional values beyond milliseconds should be supplied as strings. Unsafe integer numbers, non-finite numbers, invalid dates and unsupported parameter objects are rejected; use bigint or decimal strings for exact large values. SQLDB's supported scalar types follow its engine contract.

ClickHouse and metrics

ClickHouseConnection uses the official streaming HTTP connector. stream() consumes JSONEachRow batches without buffering the complete response. query() applies the same materialization limits as relational SQL. insert() streams an iterable with a default 4 MiB per-row limit and backpressure. It does not make a large insert atomic; a failed insert may have stored part or all of the input. JavaScript JSON row values must be serializable; represent 64-bit input integers as decimal strings. Int64/UInt64 and decimal results are returned as strings; decimal formatting follows the server and may omit trailing zeros. Binary data needs an explicit application encoding. Date/time strings follow the column's ClickHouse type and timezone.

execute() waits for server response completion and requests synchronous mutations, returning a query ID and acknowledged completion. It does not invent relational transactions for ClickHouse. Named parameters use ClickHouse syntax, such as {device:String}. Identifiers are quoted separately, including literal dots and backslashes.

connection.metrics exposes the migrated table/query features. Canonical metrics.createTable() defaults autoSchemaEvolution to false; preparation and schema changes are explicit. Standalone SmartClickHouseDb retains its existing opt-in startup/schema behavior for migrating applications.

const table = await client.sqldb('analytics').metrics.createTable<{ id: number; temperature: number }>({
  tableName: 'readings', orderBy: 'id',
  columns: [{ name: 'id', type: 'UInt32' }, { name: 'temperature', type: 'Float64' }],
});
const writer = table.createInsertStream({ batchSize: 500, maxBatchBytes: 4 * 1024 * 1024 });
await writer.write({ id: 1, temperature: 38.5 });
await writer.close(); // Joins the final server acknowledgement; rejects on failure.

Await every writer write() for backpressure. flush() and close() propagate errors, including background flush failures. Client close flushes owned writers before closing the connection. Timestamp-only polling watches cannot guarantee delivery of equal-timestamp or late rows and explicitly report unsupported_capability.

SQLDB advanced object metrics and disk-resident analytical scans remain unavailable. The client does not download objects to discover nested paths or aggregate them. Automatic deep-path indexing, correlated array predicates, expiry-driven path retirement and terabyte-scale storage require the corresponding qualified engine capability.

Budgets, errors and capabilities

New SQL methods accept signal and a total timeoutMs (30 seconds by default, including pool admission). Read cancellation closes owned transport resources. User-provided iterators receive return() on cancellation; their own pending I/O must also cooperate with the caller's signal. JavaScript cannot forcibly interrupt an arbitrary promise or transaction callback. Expiry revokes the transaction handle, so a callback resuming later cannot dispatch a write.

LosslessClientError provides a safe outer message, a code, optional backendCode, retryability, outcome and acknowledged-row evidence. cause preserves the original error and can contain statement values; do not log it indiscriminately. The taxonomy includes invalid arguments, unsupported capability, a missing optional driver, authentication, conflict, timeout, cancellation, result limits, backend failure, partial writes and unknown write outcomes. There are no automatic write retries. A transient hint never establishes that retrying an ambiguous write is safe.

Migrated document and object APIs retain their existing typed errors and exact-operation evidence. Their established per-operation timeout/ownership contracts continue to apply. Ordinary document calls and every exact call take a client-side timeoutMS that the driver enforces from server selection to the last reply, and session.withTransaction(callback, { timeoutMS }) bounds a whole transaction; an expired deadline rejects with SmartdataPersistenceError code timeout, and a write or commit that expired has an unknown outcome, which an exact write reports as ambiguous_write. The same exact calls, atomicDeleteMany() and withTransaction() take an AbortSignal; an aborted transaction is rolled back and never committed, while a commit already in flight runs to its own outcome. docs/source/smartdata/readme.md describes both, including the two driver cleanups that run on a fresh deadline of the same length after the deadline expired — the closing abort of a transaction whose callback stalled, and the killCursors after a stalled getMore of a read that needs more than one batch — so either settles within about twice timeoutMS. capabilities distinguishes available, unsupported and unknown; MongoDB transaction/change-stream availability still depends on deployment topology. NoSQLDB change streams are explicitly unsupported. capabilities.compositeGrouping is the one member with its own vocabulary — exact or unsupported — and states whether two-field grouped totals are evaluated exactly; MongoDB and NoSQLDB 10.5.0 and newer are exact, every earlier or unidentified engine and every relational or object connection is unsupported. On a plain SmartdataDb the whole statement is undefined until init() identifies the engine, and an absent statement means "not available". S3 exact-operation capability tests remain explicit and require owned disposable resources.

Migrating existing applications

Replace imports and the corresponding manifest dependency:

Previous package	New import
@push.rocks/smartdata	@lossless.org/client/nosqldb
@push.rocks/smartdata/testsupport	@lossless.org/client/testsupport
@push.rocks/smartbucket	@lossless.org/client/objectstorage
@push.rocks/smartclickhouse	@lossless.org/client/sqldb

SmartData persisted identities, decorator symbols, BSON/null/undefined behavior and exact persistence contracts are retained. Existing normal application access stays through those public model APIs; direct driver access belongs to this foundation or explicit versioned migrations.

ObjectStorage retains companion <key>.metadata objects and .trash/<encoded-original-key> layouts. Existing buffer/replay helpers, list-array methods and watcher state materialize data and are outside the new streaming-memory guarantee. Metadata locks are advisory, not atomic distributed locks; directory move remains unsupported. The getStorageClient() migration escape hatch is retained. No readiness call creates or deletes a bucket.

SmartClickHouse migration changes: replace RxJS next/complete insertion with awaited writer write/close; join database close during application shutdown. Materialized SQL reads now have explicit limits. Query-builder toSQL() contains placeholders; use toStatement() or pass its parameters with the SQL. Watch APIs reject the unsupported continuation guarantee. Insert errors always reach the caller. Existing stored dotted-column names remain unchanged. Original API documentation and licenses are preserved in the repository under docs/source.

nosqldb lineage

@lossless.org/client/nosqldb is a fork of @push.rocks/smartdata 11.14.2 (commit 9868817a63de72caef787bd8e68bac3deb9d6131), taken on 2026-09-10; this repository continues that package's git history, so every upstream fix up to and including 11.14.2 is present here. nosqldbLineage, exported from the family entry point, carries the same baseline for runtime and tooling checks, and docs/source/smartdata keeps the upstream readme, hints and changelog.

The fork departs from that baseline in six places: NoSqlConnection adds the client backend, capability and readiness contract; the family entry point exports it together with nosqldbLineage; watch() refuses to open a change stream when the connected backend reports change streams as unsupported; init() identifies the engine and states the capabilities that identity implies; a failed init() rejects with SmartdataPersistenceError (unsupported_operation) instead of the driver's own error, and without a deadline carries that error as cause, copied with every connection credential redacted, so code that matched the driver's error class (MongoServerSelectionError, …) reads cause.name (a cleanup that fails as well makes the rejection an AggregateError of both); and two-field grouped totals are refused on a backend that does not group composite keys exactly. The package version file was dropped, because versioning belongs to @lossless.org/client. Document-model capabilities added after the fork — $max/$min, identity seeding through $setOnInsert, atomicUpsertMany() and identityAsDocumentId — are this package's own additions and have no upstream counterpart.

Fixes land here. @push.rocks/smartdata is no longer a release channel for this implementation, so a later upstream version is not a source to merge from and not an upgrade path for consumers of this package.

Verification

pnpm build builds production declarations. pnpm test checks aggregate lifecycle and writers. pnpm run test:nosqldb, test:nosqldb:qualified, test:mongodb, test:objectstorage and test:sqldb exercise isolated backends. pnpm run test:qualification runs the aggregate, both engine suites and the MongoDB suite in one sequence; it is what the release preflight runs. pnpm run test:package packs the package and resolves every entry point; pnpm run test:install installs the packed tarball into a scratch project and proves that a document-only consumer receives no relational or object driver, and that the aggregate client refuses a missing one by name. Wrappers own and remove their disposable servers, containers and data directories. Never supply a production database or bucket to these destructive suites.

Qualified engines:

Component	Qualified versions
NoSQLDB engine	8.0.2 and 10.9.0 — the same test/nosqldb/smartdb/ suite runs against both, through the single engine seam in test/nosqldb/helpers/smartdb.ts; NOSQLDB_ENGINE=qualified selects 10.9.0
MongoDB	8.0.26 replica sets
ObjectStorage	10.3.1
SQLDB	0.2.3
MariaDB	11.8
ClickHouse	25.8

Both engine versions run the whole document suite. They part on asserted boundaries — 10.9.0 groups composite keys exactly and answers two-field grouped totals, evaluates guarded $expr filters, creates migration-owned partial indexes and collection validators, reports a transaction write conflict at the statement and matches value-form $elemMatch, 8.0.2 does none of these, reports the conflict at commit, and the client refuses the two-field form there — which Document APIs that need a MongoDB backend states row by row. Text search, the Lucene adapter and the broad read-filter surface are exercised against MongoDB only. pnpm run test:minio uses a digest-pinned MinIO image: sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e. That MinIO version enforces the tested conditional uploads but does not enforce conditional deletion; its exact-purge capability is correctly unavailable. ObjectStorage passes both live probes. AWS S3 has not been qualified against a live account in this migration; no universal S3 exact-operation guarantee is inferred from the SDK.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

Company Information

Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.