One TypeScript client for lossless.org NoSQLDB, SQLDB and ObjectStorage, with MongoDB, MariaDB, ClickHouse and S3 protocol adapters. LosslessOrgClient owns named connections and provides nosqldb(), sqldb() and objectstorage() interfaces. Server engines remain separate packages; this client never starts a database or creates a bucket implicitly.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install and connect

pnpm add @lossless.org/client

Document access works immediately: mongodb is a dependency because the NoSQLDB/MongoDB family is the package baseline. The relational and object drivers are optional peer dependencies, so install the ones your families use:

Family	Additional install
`nosqldb` connections, `@lossless.org/client/nosqldb`, `@lossless.org/client/testsupport`	none
`sqldb` connections with backend `sqldb` or `mariadb`, `@lossless.org/client/sqldb/mariadb`	`pnpm add mariadb`
`sqldb` connections with backend `clickhouse`, `@lossless.org/client/sqldb/clickhouse`	`pnpm add @clickhouse/client`
`objectstorage` connections with backend `s3`	`pnpm add @aws-sdk/client-s3`

The aggregate entry point imports a family only when the configuration names it, so an absent driver stays absent until it is used; connect() then fails with driver_missing, naming the package, the entry point and the install command. The two relational families also have an entry point each: @lossless.org/client/sqldb/mariadb needs mariadb only and @lossless.org/client/sqldb/clickhouse needs @clickhouse/client only, while @lossless.org/client/sqldb is their union and needs both, because it exports SqlConnection and ClickHouseConnection as values. Import the family you use. A direct family import resolves its driver as the module loads, so an absent driver fails there as Node's ERR_MODULE_NOT_FOUND naming the package; driver_missing is the aggregate entry point's refusal, and it names the family entry point that needs the driver. optionalDrivers exposes the same table to tooling.

import { LosslessOrgClient } from '@lossless.org/client';

const client = new LosslessOrgClient({
  nosqldb: {
    app: { backend: 'mongodb', url: process.env.MONGODB_URL!, database: 'app' },
  },
  sqldb: {
    primary: { backend: 'mariadb', host: process.env.SQL_HOST!, database: 'app',
      user: process.env.SQL_USER!, password: process.env.SQL_PASSWORD! },
    analytics: { backend: 'clickhouse', url: process.env.CLICKHOUSE_URL!, database: 'app',
      username: process.env.CLICKHOUSE_USER!, password: process.env.CLICKHOUSE_PASSWORD! },
  },
  objectstorage: {
    assets: { backend: 's3', endpoint: process.env.S3_ENDPOINT!, region: 'us-east-1',
      credentials: { accessKeyId: process.env.S3_ACCESS_KEY!, secretAccessKey: process.env.S3_SECRET_KEY! },
      readinessBucket: 'existing-assets' },
  },
});

await client.connect({ timeoutMs: 30_000 });
try {
  const primary = client.sqldb('primary'); // SqlConnection
  const analytics = client.sqldb('analytics'); // ClickHouseConnection
  const rows = await primary.query<{ id: bigint }>({
    sql: 'SELECT id FROM accounts WHERE email = ?', values: ['alice@example.com'],
  }, { maxRows: 100 });
  for await (const row of analytics.stream<{ temperature: number }>({
    sql: 'SELECT temperature FROM readings WHERE device = {device:String}',
    values: { device: 'ssd-1' },
  })) {
    // Process each row before requesting more; filtering happens on the server.
  }
  const readiness = await client.ready();
} finally {
  await client.close();
}

Select nosqldb, sqldb or objectstorage as the backend for the corresponding lossless.org server. An engine profile declares capabilities; it does not translate unsupported SQL or add backend features. SQLDB requires engine version 0.2.3 or later for pooled connection reset and parameterless prepared statements. It has a smaller SQL/type subset than MariaDB, and does not currently support TLS, savepoints or ALTER TABLE. ClickHouse uses its HTTP protocol and SQL dialect. There are no cross-backend transactions or automatic replication.

Literal connection names and backend discriminators determine the return types. Each client owns the connections it constructs. connect() probes databases without changing their schemas. S3 construction is local; ready() checks an explicitly configured existing bucket and returns readiness_bucket_required if none was supplied. close() is idempotent, stops admission, cancels owned SQL operations, closes pools and joins final metrics-writer flushes. Create a new aggregate client after failure or close.

Interfaces

Import	API	Peer its declarations need
`@lossless.org/client`	`LosslessOrgClient`, configuration, capabilities, readiness, `LosslessClientError`	`@aws-sdk/client-s3`
`@lossless.org/client/nosqldb`	Migrated SmartData models, decorators, collections, cursors, sessions, exact persistence and administration; `NoSqlConnection`	none
`@lossless.org/client/sqldb`	Both relational families: `SqlConnection`, `SqlTransaction`, `SqlTable`, `ClickHouseConnection`, `SmartClickHouseDb`, tables, query builders, `TimeDataTable`, `MetricWriter`	none
`@lossless.org/client/sqldb/mariadb`	`SqlConnection`, `SqlTransaction`, `SqlTable`, their options and `quoteIdentifier`	none
`@lossless.org/client/sqldb/clickhouse`	`ClickHouseConnection`, `SmartClickHouseDb`, tables, query builders, `TimeDataTable`, `MetricWriter`, their options and `quoteClickHouseIdentifier`	none
`@lossless.org/client/objectstorage`	Migrated `SmartBucket`, buckets, directories, files, metadata, watchers, exact operations; `ObjectStorageConnection`	`@aws-sdk/client-s3`
`@lossless.org/client/testsupport`	Explicit disposable-database testing helpers	none

The last column is about type-checking, not loading: an entry point's declarations name an optional peer only where its API is typed against that driver. @lossless.org/client/objectstorage exposes the S3 client, SmartBucket.storageClient and the outputs the SDK defines, so a consumer type-checks it only with @aws-sdk/client-s3 installed, and the aggregate entry point reaches those declarations through client.objectstorage(). Every other entry point type-checks with no optional peer installed and without skipLibCheck; both relational families own every type in their public API and resolve their driver when the module loads. test/client/test.entrypoints.node.ts type-checks each entry point in a disposable consumer project that installed neither mariadb nor @aws-sdk/client-s3 and holds that table to it.

Family imports preserve the established SmartData and SmartBucket names and constructors. Their implementations live here; the client does not depend on the old packages. The aggregate loads only configured families. Direct family entry points do not initialize unrelated connections.

`@lossless.org/client/nosqldb` addition	API
Monotonic counters and timestamps	`$max` / `$min` in `ISmartdataAtomicUpdate`, for declared numeric and date fields
Race-free registration on a second unique key	`$setOnInsert` may seed a declared `@unI()` identity when `upsert: true`
Bounded plural upsert	`Model.atomicUpsertMany(operations, opts?)` — 1..1000 per-document filter/update pairs in one unordered round trip
Content-addressed primary keys	model option `identityAsDocumentId` on `defineCollectionModel()` and `@Collection()`
Identities on a migrated unique index	`@unI({ indexName })` names the index that backs the identity instead of `<field>_1`
Typed `null` predicates	read and atomic filters accept `null` for a field declared optional, so a value stored as an explicit null stays addressable
Owned sessions on cursors	`getCursor()` takes the handle from `db.createSession()` and keeps it leased until the cursor is closed
Owned sessions on instance writes	`instance.save({ session })` and `instance.delete({ session })` take the handle from `db.createSession()`, leased for the one write like every other model API
Typed connection pool options	`new SmartdataDb(options: ISmartdataDbOptions)` types `maxPoolSize`, `maxIdleTimeMS`, `serverSelectionTimeoutMS` and `socketTimeoutMS` beside the descriptor fields, so no cast through `IMongoDescriptor` is needed
Named plural selectors	a non-empty `$in` anchors `atomicUpdateMany()`, `atomicDeleteMany()` and `atomicUpsertMany()`; singular operations are unchanged
Fields only atomic writes may update	`@svDb({ atomicOnly: true })` and `atomicOnlyFields` — seeded by every write that creates the document, never overwritten by a later `save()`
Defaults inside `$expr`	`{ $ifNull: ['$field', literal] }` operands, so a guard covers rows written before the field existed
Fenced instance saves	`instance.saveIf(fence, opts?)` — one non-upserting `updateOne`; `matchedCount === 0` means another writer moved the document first
Identity rotation	`Model.atomicFindOneAndReplace(filter, replacement, opts?)` — the one write that moves an `@unI()` identity, under the matched document's `_id`
Fail-fast on unprepared collections	a session-carrying call inside a transaction refuses instead of stalling on lazy index DDL until the client deadline
Migration-owned indexes	`toleratedIndexNames` on the model — named undeclared indexes are observed, never created, dropped or verified, and no longer make the topology divergent
Partial index filters in inspection	`inspectNamespace()`, `inspectCollectionTopology()` and `getIndexInfo()` report a partial index's `partialFilterExpression` exactly as the backend lists it, as a frozen snapshot present only on a partial index
Key-value store readiness	`EasyStore.ensureInitialized()` — installs the store's declared index at startup, outside any transaction window, so a first `readKey()`/`writeKey()` needs no index DDL
Key-value store copies	`EasyStore` `readKey()`/`readAll()` return copies and `writeKey()`/`writeAll()`/`replace()` store copies, in the persisted BSON form, so a read result is a stable snapshot and a caller-side mutation never reaches the store
Client deadlines	`timeoutMS` (1..120000) on every ordinary model read and write, instance `save()`/`saveIf()`/`delete()` included, on every exact call, and `session.withTransaction(callback, { timeoutMS })` for a whole transaction, retries and commit included; expiry rejects with `SmartdataPersistenceError` code `timeout`, and an exact write interrupted in flight with `ambiguous_write`
Bounded database close	`db.close({ timeoutMs })` bounds the whole close — session cleanup, the optional database removal and the driver client's close — and rejects at its deadline with `SmartdataPersistenceError` code `timeout`, leaving the database `close_failed`; the driver's close keeps running until it settles on its own, and the next `close()` joins it
Transactions ended by close	a `session.withTransaction()` whose session `db.close()` ends while it runs rejects with `SmartdataPersistenceError` code `unsupported_operation` — "nothing was committed" before the commit, "the commit outcome is unknown" during it — instead of resolving with the callback's result
Transaction commit outcome	`SmartdataSession.getCommitOutcome(error)` states for every value a `session.withTransaction()` rejected with whether the transaction could have committed: `not_committed` (nothing was committed) or `unknown` (a commit was sent and never confirmed); the rejection itself keeps its identity
Cancellation	`signal` on every exact call, `atomicDeleteMany()` and `session.withTransaction(callback, { signal })`, which interrupts the session's operation in flight, rolls the attempt back and never commits after an abort
Nested array paths	`@svDb({ arrayPaths: [...] })` declares nested arrays below a persisted root; `$push`, `$addToSet` and `$pull` in `atomicUpdate()`, `atomicUpdateMany()`, `atomicFindOneAndUpdate()` and `atomicUpsertMany()` accept them, while undeclared and non-array nested paths stay refused with `invalid_argument`
Grouped totals over nested paths	`getGroupedTotals()` / `getGroupedTotalsPage()` group by and sum nested paths declared through `atomicPaths` or `arrayPaths`, and `unwind: path` groups by each element of one declared array
Grouped totals over array elements	with `unwind: 'lines'`, `groupBy` and `sums` name declared fields of the unwound element — `groupBy: ['lines.accountNumber', 'lines.side']`, `sums: ['lines.amountCents']` — so per-account debit and credit totals need no raw aggregation; undeclared element fields and an `unwind` that holds no `groupBy` or `sums` path stay refused with `invalid_argument`
Grouped totals element filter	`elementFilter: { 'lines.accountNumber': '1200' }` beside `unwind: 'lines'` counts and sums only the matching elements — one account's balance per side — with the atomic-filter operators on the unwound array and its declared element paths; any other path or operator is refused with `invalid_argument`
Grouped totals paging	rows come ordered by group key; `getGroupedTotalsPage()` returns `nextCursor` while `hasMore`, `cursor` continues after it, and `iterateGroupedTotals()` reads every group page by page — exact across pages for null, number, string and boolean group values (proven on NoSQLDB 8.0.2, 10.7.2 and MongoDB; ObjectId and date values are ordered by BSON type order but not yet covered by the paging tests), refused with `unsupported_operation` for other values, and a cursor from another query is refused with `invalid_argument`
Query-plan inspection	`Model.inspectQueryPlan(db, { filter, sort?, projection?, limit?, timeoutMS?, signal? })` — the winning plan's stages (`stage`, `depth`, `indexName`, `direction`) and `nReturned`/`totalKeysExamined`/`totalDocsExamined`, read-only and bounded, so a consumer proves a model query uses its intended index without raw driver access
Per-instance model binding	`bindDocToManager(manager, Model)` returns a subclass of a model declared with `@managed()` and no manager argument, named like the model and bound to that manager, so several databases use the same models at once; any other binding is refused with `invalid_configuration`
Subclasses need their own binding	a subclass of a bound model that declares decorators or a persistence policy of its own and no `@Collection()`, `@managed()` or `defineCollectionModel()` binding is refused with `invalid_configuration` on every read and write, so it never writes through the parent's schema; an undecorated subclass keeps the inherited binding
Disposable test databases on a unix socket	`SmartdataTestDatabase.create({ connection: { socketPath } })` from `@lossless.org/client/testsupport` reaches an engine that listens only on a unix-domain socket, fault helpers included; the loopback `host`/`port` form is unchanged

docs/source/smartdata/readme.md documents each one with its refusals.

`@lossless.org/client/objectstorage` addition	API
Listing with size and ETag	`bucket.listObjectEntriesPage({ prefix?, limit, startAfter?, signal? })` — one keyset page of `{ key, size, etag, lastModified? }`; `bucket.listAllObjectEntries(prefix?, { pageSize?, signal? })` iterates every page
Bucket lookup for bucket-scoped credentials	`smartBucket.getBucketByName(name, { signal? })` and `smartBucket.bucketExists(name, { signal? })` send one HeadBucket; `createBucket(name, { signal? })` and `removeBucket(name, { signal? })` take the same options
Byte-range downloads	`bucket.fastGetRangeStream({ path, range: { start, end? }, signal? }, typeArg?)` — one ranged GetObject resolving `{ stream, range, size }`; range failures reject with `ObjectRangeError` code `invalid_argument`, `range_not_satisfiable` or `invalid_response`

A bucket lookup sends one HeadBucket for the named bucket, never ListBuckets, so a credential scoped to its bucket can open it without the account-wide list permission, and a lookup does not depend on which buckets a listing returns. A missing bucket (404) keeps its answer: getBucketByName() rejects with Bucket '<name>' not found. and bucketExists() resolves false. A bucket the credentials may not access (403) rejects both with an error that names the denial and keeps the provider error as cause; S3 does not say whether such a bucket exists. Any other provider failure is rethrown unchanged. signal aborts the in-flight request and rejects with the signal's reason. Lookups, creations and removals run as lifecycle-managed operations: SmartBucket.close() aborts and awaits them, and the storage client cannot be replaced while one is active.

listObjectEntriesPage() pages exactly like listObjectKeysPage(): limit is 1 to 1000, startAfter must be a non-empty key under the prefix, entries arrive in strict UTF-8 byte order after it, and nextStartAfter is present only while the provider reports more. Keyset pages stay valid when earlier keys are deleted between requests. Every field comes from the one ListObjectsV2 response, so no HEAD is sent per object. size is the listed Size, the byte length HEAD reports as ContentLength. etag is the listed ETag with its surrounding double quotes removed; HEAD returns the same entity tag, quoted. An entity tag identifies an object version and is not a content digest: AWS S3 derives it from the content MD5 only for single-part uploads without SSE-KMS or SSE-C, and a multipart tag ends in -<parts>. lastModified is present when the provider sends LastModified. Both values describe the object when the page was listed. A page with a missing or non-integer size, a missing or malformed ETag, or any key-page violation is refused rather than returned partially. signal aborts the in-flight request and rejects with the signal's reason; the iterator issues one request per page and none while the caller holds an entry, so breaking out ends the listing.

fastGetRangeStream() sends one GetObject with Range: bytes=start-end. start and end are zero-based and inclusive, as in HTTP; without end the range runs to the last byte. The stream carries exactly the served bytes, and range (the served range, end clamped to the last byte) and size (the whole object's length) come from the response's Content-Range, so a download route answers 206 Partial Content with Content-Range: bytes ${range.start}-${range.end}/${size} without a second HEAD. A start that is not a non-negative safe integer, or an end that is not a safe integer or lies below start, is refused with invalid_argument before any request. A range that starts at or beyond the end of the object — every range of an empty object included — rejects with range_not_satisfiable, the provider's 416 as cause. The 416 from S3 carries no object size, so a route that answers its own 416 with Content-Range: bytes */size reads the size with fastStat(). A response without a Content-Range that matches the request rejects with invalid_response and its body is destroyed, so a provider that ignored the header never hands out the whole object. signal and SmartBucket.close() cancel the request and the body as they do for fastGetStream().

Document APIs that need a MongoDB backend

The document model runs on both NoSQLDB and MongoDB, but several APIs do not mean the same thing on both. The NoSQLDB engine is qualified at 8.0.2 and 10.7.2 (see Verification); the two releases part on the grouped-totals and guarded-$expr rows below — 10.7.2 groups composite keys exactly and evaluates the guard, 8.0.2 does neither — on whether a migration can create a partial index or a collection validator at all, on when a transaction learns of a write conflict, and on whether $elemMatch applies query operators to scalar array elements.

init() asks the connected engine what it is with one buildInfo command and records the answer on the database as engineIdentity ({ product, version }). NoSQLDB answers with its own nosqldb identity document from 10.3.0 onward; a MongoDB server answers buildInfo without one; an engine that refuses the command — NoSQLDB 8.0.2 answers CommandNotFound — stays { product: 'unknown', version: null }. The identity is what states capabilities on a plain SmartdataDb, and an unidentified engine states none at all, so every capability-gated call refuses. NoSqlConnection keeps stating the capabilities of its declared backend from construction on.

Document API	On the NoSQLDB engine	On MongoDB
`getGroupedTotals()` / `getGroupedTotalsPage()` / `iterateGroupedTotals()` with one `groupBy` field	exact, including `sums`, declared nested paths, `unwind` of a declared array and declared fields of its element, `elementFilter`, group key order and `cursor` paging, on 8.0.2 and 10.7.2 alike	exact
the same with two `groupBy` fields	exact, `sums`, nested paths, `unwind`, two element fields of one unwound array (`lines.accountNumber` with `lines.side`), `elementFilter` and `cursor` paging included, on an engine that identifies itself as NoSQLDB 10.5.0 or newer: that release evaluates the composite group key per document. Below it the call is refused with `unsupported_operation` naming both fields and the single-field form, because the engine leaves the composite group key unresolved and answers with a single row whose group values are the literal field paths (`'$owner'`) and whose count is the whole filtered collection; the refusal is the guard against that wrong number, not a fix for it	exact; the client wraps each key member in `$ifNull`, because MongoDB leaves a missing member out of the key and would otherwise report it as a second null group
guarded `$expr` atomic filters (`$add`, `$lte`, `$ifNull`)	exact on 10.6.0 and newer: the engine evaluates the client's fail-closed compilation (`$cond`, `$let`, `$isNumber`, `$type`), so the guard admits within the limit, refuses beyond it and never matches a stored value that is not a finite number. 8.0.2 rejects the compilation with `invalid argument: $cond` and leaves the document untouched	exact
`watch()` change streams	unsupported. `NoSqlConnection` refuses with `unsupported_capability`, and so does a plain `SmartdataDb` whose engine identified itself; against an engine that answers no identity the refusal instead arrives from the engine as `CommandNotFound` once the change stream is read	supported on replica sets
model-declared partial indexes (`partialFilterExpression`)	not expressible: the index-option type rejects it and the runtime refuses it with `invalid_configuration`. A client decision, not an engine limit on every release — 10.6.0 and newer create partial indexes, 8.0.2 refuses the option with `InvalidOptions` (72) — because one model declaration has to hold on every qualified backend. A migration may create one on 10.7.2; `toleratedIndexNames` accepts it and inspection reports its filter	not expressible, but a migration may own one; inspection reports its filter
collection validators (`validator`, `validationLevel`, `validationAction`)	no client API; a migration owns a validator through the driver. 10.7.0 and newer create the collection, list the options exactly as given and enforce them on every model write: a violating `insert()` or `atomicUpdate()` rejects with the engine's `DocumentValidationFailure` (121) and its `errInfo`, which the client passes through unchanged rather than mapping to a `SmartdataPersistenceError` code, and stores nothing. Models initialize on a validated collection like on any other. 8.0.2 refuses the option with `InvalidOptions` (72) and creates no collection, so a validator is never silently dropped	enforced; not exercised by the client's MongoDB suite
write conflicts between transactions, and writes outside a transaction to a document a transaction holds	10.7.2 (since 10.6.1) detects a conflict at the statement, as MongoDB does: the first writer wins, and the later transaction's statement rejects with the engine's `WriteConflict` (112) labelled `TransientTransactionError`, which the client passes through so `session.withTransaction()` re-runs that callback. A write outside a transaction waits until the transaction holding its document ends, then applies on top; the wait is bounded by the write's own `timeoutMS` and by the engine's fixed 60-second transaction lifetime, after which a waiting writer aborts the holder. Awaiting such a write inside the holding transaction's callback therefore stalls that transaction until one of the two expires. 8.0.2 detects the conflict at commit: the first transaction to commit wins, a write outside a transaction applies at once, and the transaction that wrote first re-runs. Both writes land on either engine; a callback must be safe to re-run and must not swallow the statement error	first writer wins at the statement and outside writes wait, MongoDB's documented rule; not exercised by the client's MongoDB suite
value-form `$elemMatch` in read filters (`{ scores: { $elemMatch: { $gte: 80, $lt: 90 } } }`, `{ labels: { $not: { $elemMatch: { $nin: runnerLabels } } } }`)	exact on 10.7.1 and newer: the operators apply to each array element as a whole, so the range matches only a document with one score inside it and the claim selects a job only when every one of its labels is a runner label. 8.0.2 parses every operand as a query over embedded documents and refuses the value form with `TypeMismatch` (14). A non-document operand is refused on both, with MongoDB's `BadValue` (2) on 10.7.1 and newer and 14 on 8.0.2. The client passes the operand through unchanged; the filter type admits the value form over scalar elements and the document form over embedded documents. Atomic filters (`atomicUpdate()` and the other atomic writes) accept no `$elemMatch` on any backend: the client refuses it with `invalid_argument` before any backend is contacted	exact
`Model.inspectQueryPlan()`	explains on 10.7.2: the summary names the winning plan's stages and its execution totals. 8.0.2 answers `explain` with `CommandNotFound` (59), and the call rejects with `unsupported_operation` ("The connected engine does not explain queries.") rather than returning a plan it cannot know. Stage shapes are the engine's own: on 10.7.2 a limited sort on an unindexed field is `LIMIT` over `SORT` over `COLLSCAN`, an equality on a unique single-field index is `FETCH` over `IXSCAN`, and an `$or` whose branches each have an index is a `COLLSCAN`	explains: the same limited sort is `SORT` over `COLLSCAN`, the limit folded into the sort; the equality is `EXPRESS_IXSCAN` on the unique index; the `$or` is `SUBPLAN` over `FETCH` over `OR` over one `IXSCAN` per branch
aggregation-pipeline (array) updates	refused by the client with `invalid_argument` before any backend is contacted. A client-owned refusal, not an engine limit: both qualified engines apply a pipeline update themselves	same client refusal

test/nosqldb/smartdb/test.engine-capability-boundary.node.ts asserts the identity, grouped-totals, guarded-$expr, partial-index, collection-validator, write-conflict, $elemMatch and pipeline-update rows against both qualified engines, so a boundary that moves shows up as a failing suite rather than as a changed number in a consumer. The two halves of the grouped-totals, guarded-$expr, write-conflict and $elemMatch rows are separate tests, each bound to the engine that answers it, and the engine's own composite-key answer stays under assertion through the raw aggregateGroupedTotals() boundary — one literal row on 8.0.2, the two exact pairs on 10.7.2. The same boundary pins the rest of the engine's $group stage: a non-document accumulator is refused with TypeMismatch (14) on 10.7.2 while 8.0.2 answers it, and a $cond inside a composite key is evaluated per document on 10.7.2 while 8.0.2 answers its single literal row. It also pins whether a migration can create a partial index — 10.7.2 creates it and lists the filter, 8.0.2 refuses with 72 — and a collection validator — 10.7.2 creates and enforces it against model inserts and updates, 8.0.2 refuses with 72 — which writer a conflict sends back: on 10.7.2 the later transaction fails at its statement and re-runs while a write outside a transaction waits for the holder's commit, on 8.0.2 the later transaction commits first and the holder re-runs — and what $elemMatch answers through getInstances(): 10.7.2 matches the documented range and the $not/$elemMatch/$nin claim exactly as the MongoDB filter suite asserts them, 8.0.2 refuses both with 14, and a non-document operand is refused with 2 on 10.7.2 and 14 on 8.0.2. The watch() row is the contract refusal ts/nosqldb/classes.collection.ts states; the engine suite opens no change stream.

Relational SQL

SqlConnection uses the official MariaDB connector for parameters, protocol, TLS and decoding. This client owns cancellable admission and TCP sockets from authentication onward. A transaction holds one physical session; releasing a session resets its state before reuse.

execute({ sql, values }, options) returns affectedRows, optional exact insertId: bigint, and completion: 'acknowledged'. Use it for statements that return an update result.
query<Row>(statement, options) materializes at most 10,000 rows and 16 MiB by default. Set maxRows and maxBytes explicitly to change those limits. Overflow rejects with result_limit and closes the stream.
stream<Row>(statement, options) iterates rows with backpressure. Early return, timeout, cancellation and connection shutdown close its owned stream/socket.
insert(table, iterable, options) accepts iterable or async-iterable rows with the same columns, batching at 500 rows/4 MiB by default. Batches commit independently. A later failure reports partial_write with acknowledged rows, or ambiguous_write if the current batch outcome is unknown. Use a transaction with explicit statements for atomic multi-statement writes.
transaction(async tx => ..., options) joins an outstanding final transaction operation before commit. Await each operation; concurrent operations on one transaction are rejected. A thrown callback rolls back when the session is still connected. An interrupted commit has an unknown outcome.
table<Row>(name) handles an existing table with bound equality/null selectors, typed query, stream, insert, update and delete. Empty mutation selectors and undefined selector values are rejected. Schema creation is explicit SQL.

Values use placeholders; identifiers use a separate quoting function. Raw SQL text is trusted application code. A result generic is a caller-declared shape, not static validation of arbitrary SQL.

MariaDB returns BIGINT as bigint, DECIMAL as a string, binary columns as Buffer, SQL null as null, and dates/times as strings. Fractional timestamp strings retain server precision. JavaScript Date inputs bind in UTC with millisecond precision; MariaDB sessions use UTC. Exact fractional values beyond milliseconds should be supplied as strings. Unsafe integer numbers, non-finite numbers, invalid dates and unsupported parameter objects are rejected; use bigint or decimal strings for exact large values. SQLDB's supported scalar types follow its engine contract.

ClickHouse and metrics

ClickHouseConnection uses the official streaming HTTP connector. stream() consumes JSONEachRow batches without buffering the complete response. query() applies the same materialization limits as relational SQL. insert() streams an iterable with a default 4 MiB per-row limit and backpressure. It does not make a large insert atomic; a failed insert may have stored part or all of the input. JavaScript JSON row values must be serializable; represent 64-bit input integers as decimal strings. Int64/UInt64 and decimal results are returned as strings; decimal formatting follows the server and may omit trailing zeros. Binary data needs an explicit application encoding. Date/time strings follow the column's ClickHouse type and timezone.

execute() waits for server response completion and requests synchronous mutations, returning a query ID and acknowledged completion. It does not invent relational transactions for ClickHouse. Named parameters use ClickHouse syntax, such as {device:String}. Identifiers are quoted separately, including literal dots and backslashes.

connection.metrics exposes the migrated table/query features. Canonical metrics.createTable() defaults autoSchemaEvolution to false; preparation and schema changes are explicit. Standalone SmartClickHouseDb retains its existing opt-in startup/schema behavior for migrating applications.

const table = await client.sqldb('analytics').metrics.createTable<{ id: number; temperature: number }>({
  tableName: 'readings', orderBy: 'id',
  columns: [{ name: 'id', type: 'UInt32' }, { name: 'temperature', type: 'Float64' }],
});
const writer = table.createInsertStream({ batchSize: 500, maxBatchBytes: 4 * 1024 * 1024 });
await writer.write({ id: 1, temperature: 38.5 });
await writer.close(); // Joins the final server acknowledgement; rejects on failure.

Await every writer write() for backpressure. flush() and close() propagate errors, including background flush failures. Client close flushes owned writers before closing the connection. Timestamp-only polling watches cannot guarantee delivery of equal-timestamp or late rows and explicitly report unsupported_capability.

SQLDB advanced object metrics and disk-resident analytical scans remain unavailable. The client does not download objects to discover nested paths or aggregate them. Automatic deep-path indexing, correlated array predicates, expiry-driven path retirement and terabyte-scale storage require the corresponding qualified engine capability.

Budgets, errors and capabilities

New SQL methods accept signal and a total timeoutMs (30 seconds by default, including pool admission). Read cancellation closes owned transport resources. User-provided iterators receive return() on cancellation; their own pending I/O must also cooperate with the caller's signal. JavaScript cannot forcibly interrupt an arbitrary promise or transaction callback. Expiry revokes the transaction handle, so a callback resuming later cannot dispatch a write.

LosslessClientError provides a safe outer message, a code, optional backendCode, retryability, outcome and acknowledged-row evidence. cause preserves the original error and can contain statement values; do not log it indiscriminately. The taxonomy includes invalid arguments, unsupported capability, a missing optional driver, authentication, conflict, timeout, cancellation, result limits, backend failure, partial writes and unknown write outcomes. There are no automatic write retries. A transient hint never establishes that retrying an ambiguous write is safe.

Migrated document and object APIs retain their existing typed errors and exact-operation evidence. Their established per-operation timeout/ownership contracts continue to apply. Ordinary document calls and every exact call take a client-side timeoutMS that the driver enforces from server selection to the last reply, and session.withTransaction(callback, { timeoutMS }) bounds a whole transaction; an expired deadline rejects with SmartdataPersistenceError code timeout, and a write or commit that expired has an unknown outcome, which an exact write reports as ambiguous_write. The same exact calls, atomicDeleteMany() and withTransaction() take an AbortSignal; an aborted transaction is rolled back and never committed, while a commit already in flight runs to its own outcome. docs/source/smartdata/readme.md describes both, including the two driver cleanups that run on a fresh deadline of the same length after the deadline expired — the closing abort of a transaction whose callback stalled, and the killCursors after a stalled getMore of a read that needs more than one batch — so either settles within about twice timeoutMS. capabilities distinguishes available, unsupported and unknown; MongoDB transaction/change-stream availability still depends on deployment topology. NoSQLDB change streams are explicitly unsupported. capabilities.compositeGrouping is the one member with its own vocabulary — exact or unsupported — and states whether two-field grouped totals are evaluated exactly; MongoDB and NoSQLDB 10.5.0 and newer are exact, every earlier or unidentified engine and every relational or object connection is unsupported. On a plain SmartdataDb the whole statement is undefined until init() identifies the engine, and an absent statement means "not available". S3 exact-operation capability tests remain explicit and require owned disposable resources.

Migrating existing applications

Replace imports and the corresponding manifest dependency:

Previous package	New import
`@push.rocks/smartdata`	`@lossless.org/client/nosqldb`
`@push.rocks/smartdata/testsupport`	`@lossless.org/client/testsupport`
`@push.rocks/smartbucket`	`@lossless.org/client/objectstorage`
`@push.rocks/smartclickhouse`	`@lossless.org/client/sqldb`

SmartData persisted identities, decorator symbols, BSON/null/undefined behavior and exact persistence contracts are retained. Existing normal application access stays through those public model APIs; direct driver access belongs to this foundation or explicit versioned migrations.

ObjectStorage retains companion <key>.metadata objects and .trash/<encoded-original-key> layouts. Existing buffer/replay helpers, list-array methods and watcher state materialize data and are outside the new streaming-memory guarantee. Metadata locks are advisory, not atomic distributed locks; directory move remains unsupported. The getStorageClient() migration escape hatch is retained. No readiness call creates or deletes a bucket.

SmartClickHouse migration changes: replace RxJS next/complete insertion with awaited writer write/close; join database close during application shutdown. Materialized SQL reads now have explicit limits. Query-builder toSQL() contains placeholders; use toStatement() or pass its parameters with the SQL. Watch APIs reject the unsupported continuation guarantee. Insert errors always reach the caller. Existing stored dotted-column names remain unchanged. Original API documentation and licenses are preserved in the repository under docs/source.

nosqldb lineage

@lossless.org/client/nosqldb is a fork of @push.rocks/smartdata 11.14.2 (commit 9868817a63de72caef787bd8e68bac3deb9d6131), taken on 2026-09-10; this repository continues that package's git history, so every upstream fix up to and including 11.14.2 is present here. nosqldbLineage, exported from the family entry point, carries the same baseline for runtime and tooling checks, and docs/source/smartdata keeps the upstream readme, hints and changelog.

The fork departs from that baseline in five places: NoSqlConnection adds the client backend, capability and readiness contract; the family entry point exports it together with nosqldbLineage; watch() refuses to open a change stream when the connected backend reports change streams as unsupported; init() identifies the engine and states the capabilities that identity implies; and two-field grouped totals are refused on a backend that does not group composite keys exactly. The package version file was dropped, because versioning belongs to @lossless.org/client. Document-model capabilities added after the fork — $max/$min, identity seeding through $setOnInsert, atomicUpsertMany() and identityAsDocumentId — are this package's own additions and have no upstream counterpart.

Fixes land here. @push.rocks/smartdata is no longer a release channel for this implementation, so a later upstream version is not a source to merge from and not an upgrade path for consumers of this package.

Verification

pnpm build builds production declarations. pnpm test checks aggregate lifecycle and writers. pnpm run test:nosqldb, test:nosqldb:qualified, test:mongodb, test:objectstorage and test:sqldb exercise isolated backends. pnpm run test:qualification runs the aggregate, both engine suites and the MongoDB suite in one sequence; it is what the release preflight runs. pnpm run test:package packs the package and resolves every entry point; pnpm run test:install installs the packed tarball into a scratch project and proves that a document-only consumer receives no relational or object driver, and that the aggregate client refuses a missing one by name. Wrappers own and remove their disposable servers, containers and data directories. Never supply a production database or bucket to these destructive suites.

Qualified engines:

Component	Qualified versions
NoSQLDB engine	8.0.2 and 10.7.2 — the same `test/nosqldb/smartdb/` suite runs against both, through the single engine seam in `test/nosqldb/helpers/smartdb.ts`; `NOSQLDB_ENGINE=qualified` selects 10.7.2
MongoDB	8.0.26 replica sets
ObjectStorage	10.3.1
SQLDB	0.2.3
MariaDB	11.8
ClickHouse	25.8

Both engine versions run the whole document suite. They part on asserted boundaries — 10.7.2 groups composite keys exactly and answers two-field grouped totals, evaluates guarded $expr filters, creates migration-owned partial indexes and collection validators, reports a transaction write conflict at the statement and matches value-form $elemMatch, 8.0.2 does none of these, reports the conflict at commit, and the client refuses the two-field form there — which Document APIs that need a MongoDB backend states row by row. Text search, the Lucene adapter and the broad read-filter surface are exercised against MongoDB only. pnpm run test:minio uses a digest-pinned MinIO image: sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e. That MinIO version enforces the tested conditional uploads but does not enforce conditional deletion; its exact-purge capability is correctly unavailable. ObjectStorage passes both live probes. AWS S3 has not been qualified against a live account in this migration; no universal S3 exact-operation guarantee is inferred from the SDK.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

readme.md

@lossless.org/client