One TypeScript client for lossless.org NoSQLDB, SQLDB and ObjectStorage, with MongoDB, MariaDB, ClickHouse and S3 protocol adapters. LosslessOrgClient owns named connections and provides nosqldb(), sqldb() and objectstorage() interfaces. Server engines remain separate packages; this client never starts a database or creates a bucket implicitly.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install and connect

pnpm add @lossless.org/client

Document access works immediately: mongodb is a dependency because the NoSQLDB/MongoDB family is the package baseline. The relational and object drivers are optional peer dependencies, so install the ones your families use:

Family	Additional install
`nosqldb` connections, `@lossless.org/client/nosqldb`, `@lossless.org/client/testsupport`	none
`sqldb` connections with backend `sqldb` or `mariadb`, `@lossless.org/client/sqldb/mariadb`	`pnpm add mariadb`
`sqldb` connections with backend `clickhouse`, `@lossless.org/client/sqldb/clickhouse`	`pnpm add @clickhouse/client`
`objectstorage` connections with backend `s3`	`pnpm add @aws-sdk/client-s3`

The aggregate entry point imports a family only when the configuration names it, so an absent driver stays absent until it is used; connect() then fails with driver_missing, naming the package, the entry point and the install command. The two relational families also have an entry point each: @lossless.org/client/sqldb/mariadb needs mariadb only and @lossless.org/client/sqldb/clickhouse needs @clickhouse/client only, while @lossless.org/client/sqldb is their union and needs both, because it exports SqlConnection and ClickHouseConnection as values. Import the family you use. A direct family import resolves its driver as the module loads, so an absent driver fails there as Node's ERR_MODULE_NOT_FOUND naming the package; driver_missing is the aggregate entry point's refusal, and it names the family entry point that needs the driver. optionalDrivers exposes the same table to tooling.

import { LosslessOrgClient } from '@lossless.org/client';

const client = new LosslessOrgClient({
  nosqldb: {
    app: { backend: 'mongodb', url: process.env.MONGODB_URL!, database: 'app' },
  },
  sqldb: {
    primary: { backend: 'mariadb', host: process.env.SQL_HOST!, database: 'app',
      user: process.env.SQL_USER!, password: process.env.SQL_PASSWORD! },
    analytics: { backend: 'clickhouse', url: process.env.CLICKHOUSE_URL!, database: 'app',
      username: process.env.CLICKHOUSE_USER!, password: process.env.CLICKHOUSE_PASSWORD! },
  },
  objectstorage: {
    assets: { backend: 's3', endpoint: process.env.S3_ENDPOINT!, region: 'us-east-1',
      credentials: { accessKeyId: process.env.S3_ACCESS_KEY!, secretAccessKey: process.env.S3_SECRET_KEY! },
      readinessBucket: 'existing-assets' },
  },
});

await client.connect({ timeoutMs: 30_000 });
try {
  const primary = client.sqldb('primary'); // SqlConnection
  const analytics = client.sqldb('analytics'); // ClickHouseConnection
  const rows = await primary.query<{ id: bigint }>({
    sql: 'SELECT id FROM accounts WHERE email = ?', values: ['alice@example.com'],
  }, { maxRows: 100 });
  for await (const row of analytics.stream<{ temperature: number }>({
    sql: 'SELECT temperature FROM readings WHERE device = {device:String}',
    values: { device: 'ssd-1' },
  })) {
    // Process each row before requesting more; filtering happens on the server.
  }
  const readiness = await client.ready();
} finally {
  await client.close();
}

Select nosqldb, sqldb or objectstorage as the backend for the corresponding lossless.org server. An engine profile declares capabilities; it does not translate unsupported SQL or add backend features. SQLDB requires engine version 0.2.3 or later for pooled connection reset and parameterless prepared statements. It has a smaller SQL/type subset than MariaDB, and does not currently support TLS, savepoints or ALTER TABLE. ClickHouse uses its HTTP protocol and SQL dialect. There are no cross-backend transactions or automatic replication.

Literal connection names and backend discriminators determine the return types. Each client owns the connections it constructs. connect() probes databases without changing their schemas. S3 construction is local; ready() checks an explicitly configured existing bucket and returns readiness_bucket_required if none was supplied. close() is idempotent, stops admission, cancels owned SQL operations, closes pools and joins final metrics-writer flushes. Create a new aggregate client after failure or close.

Interfaces

Import	API	Peer its declarations need
`@lossless.org/client`	`LosslessOrgClient`, configuration, capabilities, readiness, `LosslessClientError`	`@aws-sdk/client-s3`
`@lossless.org/client/nosqldb`	Migrated SmartData models, decorators, collections, cursors, sessions, exact persistence and administration; `NoSqlConnection`	none
`@lossless.org/client/sqldb`	Both relational families: `SqlConnection`, `SqlTransaction`, `SqlTable`, `ClickHouseConnection`, `SmartClickHouseDb`, tables, query builders, `TimeDataTable`, `MetricWriter`	none
`@lossless.org/client/sqldb/mariadb`	`SqlConnection`, `SqlTransaction`, `SqlTable`, their options and `quoteIdentifier`	none
`@lossless.org/client/sqldb/clickhouse`	`ClickHouseConnection`, `SmartClickHouseDb`, tables, query builders, `TimeDataTable`, `MetricWriter`, their options and `quoteClickHouseIdentifier`	none
`@lossless.org/client/objectstorage`	Migrated `SmartBucket`, buckets, directories, files, metadata, watchers, exact operations; `ObjectStorageConnection`	`@aws-sdk/client-s3`
`@lossless.org/client/testsupport`	Explicit disposable-database testing helpers	none

The last column is about type-checking, not loading: an entry point's declarations name an optional peer only where its API is typed against that driver. @lossless.org/client/objectstorage exposes the S3 client, SmartBucket.storageClient and the outputs the SDK defines, so a consumer type-checks it only with @aws-sdk/client-s3 installed, and the aggregate entry point reaches those declarations through client.objectstorage(). Every other entry point type-checks with no optional peer installed and without skipLibCheck; both relational families own every type in their public API and resolve their driver when the module loads. test/client/test.entrypoints.node.ts type-checks each entry point in a disposable consumer project that installed neither mariadb nor @aws-sdk/client-s3 and holds that table to it.

Family imports preserve the established SmartData and SmartBucket names and constructors. Their implementations live here; the client does not depend on the old packages. The aggregate loads only configured families. Direct family entry points do not initialize unrelated connections.

`@lossless.org/client/nosqldb` addition	API
Monotonic counters and timestamps	`$max` / `$min` in `ISmartdataAtomicUpdate`, for declared numeric and date fields
Race-free registration on a second unique key	`$setOnInsert` may seed a declared `@unI()` identity when `upsert: true`
Bounded plural upsert	`Model.atomicUpsertMany(operations, opts?)` — 1..1000 per-document filter/update pairs in one unordered round trip
Content-addressed primary keys	model option `identityAsDocumentId` on `defineCollectionModel()` and `@Collection()`
Identities on a migrated unique index	`@unI({ indexName })` names the index that backs the identity instead of `<field>_1`
Typed `null` predicates	read and atomic filters accept `null` for a field declared optional, so a value stored as an explicit null stays addressable
Owned sessions on cursors	`getCursor()` takes the handle from `db.createSession()` and keeps it leased until the cursor is closed
Named plural selectors	a non-empty `$in` anchors `atomicUpdateMany()`, `atomicDeleteMany()` and `atomicUpsertMany()`; singular operations are unchanged
Fields only atomic writes may update	`@svDb({ atomicOnly: true })` and `atomicOnlyFields` — seeded by every write that creates the document, never overwritten by a later `save()`
Defaults inside `$expr`	`{ $ifNull: ['$field', literal] }` operands, so a guard covers rows written before the field existed
Fenced instance saves	`instance.saveIf(fence, opts?)` — one non-upserting `updateOne`; `matchedCount === 0` means another writer moved the document first
Identity rotation	`Model.atomicFindOneAndReplace(filter, replacement, opts?)` — the one write that moves an `@unI()` identity, under the matched document's `_id`
Fail-fast on unprepared collections	a session-carrying call inside a transaction refuses instead of stalling on lazy index DDL until the client deadline
Migration-owned indexes	`toleratedIndexNames` on the model — named undeclared indexes are observed, never created, dropped or verified, and no longer make the topology divergent
Key-value store readiness	`EasyStore.ensureInitialized()` — installs the store's declared index at startup, outside any transaction window, so a first `readKey()`/`writeKey()` needs no index DDL

docs/source/smartdata/readme.md documents each one with its refusals.

Document APIs that need a MongoDB backend

The document model runs on both NoSQLDB and MongoDB, but four APIs do not mean the same thing on both. The NoSQLDB engine is qualified at 8.0.2 and 10.5.0 (see Verification); the two releases part on one row below — 10.5.0 groups composite keys exactly, 8.0.2 does not — and answer every other row alike.

init() asks the connected engine what it is with one buildInfo command and records the answer on the database as engineIdentity ({ product, version }). NoSQLDB answers with its own nosqldb identity document from 10.3.0 onward; a MongoDB server answers buildInfo without one; an engine that refuses the command — NoSQLDB 8.0.2 answers CommandNotFound — stays { product: 'unknown', version: null }. The identity is what states capabilities on a plain SmartdataDb, and an unidentified engine states none at all, so every capability-gated call refuses. NoSqlConnection keeps stating the capabilities of its declared backend from construction on.

Document API	On the NoSQLDB engine	On MongoDB
`getGroupedTotals()` / `getGroupedTotalsPage()` with one `groupBy` field	exact, including `sums`	exact
the same with two `groupBy` fields	exact, `sums` included, on an engine that identifies itself as NoSQLDB 10.5.0 or newer: that release evaluates the composite group key per document. Below it the call is refused with `unsupported_operation` naming both fields and the single-field form, because the engine leaves the composite group key unresolved and answers with a single row whose group values are the literal field paths (`'$owner'`) and whose count is the whole filtered collection; the refusal is the guard against that wrong number, not a fix for it	exact
guarded `$expr` atomic filters (`$add`, `$lte`, `$ifNull`)	refused by the engine: the fail-closed compilation uses `$cond`, which the engine rejects with `invalid argument: $cond`. The document is left untouched	exact
`watch()` change streams	unsupported. `NoSqlConnection` refuses with `unsupported_capability`, and so does a plain `SmartdataDb` whose engine identified itself; against an engine that answers no identity the refusal instead arrives from the engine as `CommandNotFound` once the change stream is read	supported on replica sets
model-declared partial indexes (`partialFilterExpression`)	not expressible: the index-option type rejects it and the runtime refuses it with `invalid_configuration`; the engine cannot create one either and refuses the option itself with `InvalidOptions` (72) on both qualified releases. Migration-owned partial indexes stay reachable through `toleratedIndexNames`	not expressible, but a migration may own one
aggregation-pipeline (array) updates	refused by the client with `invalid_argument` before any backend is contacted. A client-owned refusal, not an engine limit: both qualified engines apply a pipeline update themselves	same client refusal

test/nosqldb/smartdb/test.engine-capability-boundary.node.ts asserts the identity, grouped-totals, guarded-$expr, partial-index and pipeline-update rows against both qualified engines, so a boundary that moves shows up as a failing suite rather than as a changed number in a consumer. The two halves of the grouped-totals row are separate tests, each bound to the engine that answers it, and the engine's own composite-key answer stays under assertion through the raw aggregateGroupedTotals() boundary — one literal row on 8.0.2, the two exact pairs on 10.5.0. The same boundary pins what 10.5.0 gained with it, a validated $group stage: a $cond inside a composite key and a non-document accumulator are refused with TypeMismatch (14) there, while 8.0.2 answers both. The watch() row is the contract refusal ts/nosqldb/classes.collection.ts states; the engine suite opens no change stream.

Relational SQL

SqlConnection uses the official MariaDB connector for parameters, protocol, TLS and decoding. This client owns cancellable admission and TCP sockets from authentication onward. A transaction holds one physical session; releasing a session resets its state before reuse.

execute({ sql, values }, options) returns affectedRows, optional exact insertId: bigint, and completion: 'acknowledged'. Use it for statements that return an update result.
query<Row>(statement, options) materializes at most 10,000 rows and 16 MiB by default. Set maxRows and maxBytes explicitly to change those limits. Overflow rejects with result_limit and closes the stream.
stream<Row>(statement, options) iterates rows with backpressure. Early return, timeout, cancellation and connection shutdown close its owned stream/socket.
insert(table, iterable, options) accepts iterable or async-iterable rows with the same columns, batching at 500 rows/4 MiB by default. Batches commit independently. A later failure reports partial_write with acknowledged rows, or ambiguous_write if the current batch outcome is unknown. Use a transaction with explicit statements for atomic multi-statement writes.
transaction(async tx => ..., options) joins an outstanding final transaction operation before commit. Await each operation; concurrent operations on one transaction are rejected. A thrown callback rolls back when the session is still connected. An interrupted commit has an unknown outcome.
table<Row>(name) handles an existing table with bound equality/null selectors, typed query, stream, insert, update and delete. Empty mutation selectors and undefined selector values are rejected. Schema creation is explicit SQL.

Values use placeholders; identifiers use a separate quoting function. Raw SQL text is trusted application code. A result generic is a caller-declared shape, not static validation of arbitrary SQL.

MariaDB returns BIGINT as bigint, DECIMAL as a string, binary columns as Buffer, SQL null as null, and dates/times as strings. Fractional timestamp strings retain server precision. JavaScript Date inputs bind in UTC with millisecond precision; MariaDB sessions use UTC. Exact fractional values beyond milliseconds should be supplied as strings. Unsafe integer numbers, non-finite numbers, invalid dates and unsupported parameter objects are rejected; use bigint or decimal strings for exact large values. SQLDB's supported scalar types follow its engine contract.

ClickHouse and metrics

ClickHouseConnection uses the official streaming HTTP connector. stream() consumes JSONEachRow batches without buffering the complete response. query() applies the same materialization limits as relational SQL. insert() streams an iterable with a default 4 MiB per-row limit and backpressure. It does not make a large insert atomic; a failed insert may have stored part or all of the input. JavaScript JSON row values must be serializable; represent 64-bit input integers as decimal strings. Int64/UInt64 and decimal results are returned as strings; decimal formatting follows the server and may omit trailing zeros. Binary data needs an explicit application encoding. Date/time strings follow the column's ClickHouse type and timezone.

execute() waits for server response completion and requests synchronous mutations, returning a query ID and acknowledged completion. It does not invent relational transactions for ClickHouse. Named parameters use ClickHouse syntax, such as {device:String}. Identifiers are quoted separately, including literal dots and backslashes.

connection.metrics exposes the migrated table/query features. Canonical metrics.createTable() defaults autoSchemaEvolution to false; preparation and schema changes are explicit. Standalone SmartClickHouseDb retains its existing opt-in startup/schema behavior for migrating applications.

const table = await client.sqldb('analytics').metrics.createTable<{ id: number; temperature: number }>({
  tableName: 'readings', orderBy: 'id',
  columns: [{ name: 'id', type: 'UInt32' }, { name: 'temperature', type: 'Float64' }],
});
const writer = table.createInsertStream({ batchSize: 500, maxBatchBytes: 4 * 1024 * 1024 });
await writer.write({ id: 1, temperature: 38.5 });
await writer.close(); // Joins the final server acknowledgement; rejects on failure.

Await every writer write() for backpressure. flush() and close() propagate errors, including background flush failures. Client close flushes owned writers before closing the connection. Timestamp-only polling watches cannot guarantee delivery of equal-timestamp or late rows and explicitly report unsupported_capability.

SQLDB advanced object metrics and disk-resident analytical scans remain unavailable. The client does not download objects to discover nested paths or aggregate them. Automatic deep-path indexing, correlated array predicates, expiry-driven path retirement and terabyte-scale storage require the corresponding qualified engine capability.

Budgets, errors and capabilities

New SQL methods accept signal and a total timeoutMs (30 seconds by default, including pool admission). Read cancellation closes owned transport resources. User-provided iterators receive return() on cancellation; their own pending I/O must also cooperate with the caller's signal. JavaScript cannot forcibly interrupt an arbitrary promise or transaction callback. Expiry revokes the transaction handle, so a callback resuming later cannot dispatch a write.

LosslessClientError provides a safe outer message, a code, optional backendCode, retryability, outcome and acknowledged-row evidence. cause preserves the original error and can contain statement values; do not log it indiscriminately. The taxonomy includes invalid arguments, unsupported capability, a missing optional driver, authentication, conflict, timeout, cancellation, result limits, backend failure, partial writes and unknown write outcomes. There are no automatic write retries. A transient hint never establishes that retrying an ambiguous write is safe.

Migrated document and object APIs retain their existing typed errors and exact-operation evidence. Their established per-operation timeout/ownership contracts continue to apply. capabilities distinguishes available, unsupported and unknown; MongoDB transaction/change-stream availability still depends on deployment topology. NoSQLDB change streams are explicitly unsupported. capabilities.compositeGrouping is the one member with its own vocabulary — exact or unsupported — and states whether two-field grouped totals are evaluated exactly; MongoDB and NoSQLDB 10.5.0 and newer are exact, every earlier or unidentified engine and every relational or object connection is unsupported. On a plain SmartdataDb the whole statement is undefined until init() identifies the engine, and an absent statement means "not available". S3 exact-operation capability tests remain explicit and require owned disposable resources.

Migrating existing applications

Replace imports and the corresponding manifest dependency:

Previous package	New import
`@push.rocks/smartdata`	`@lossless.org/client/nosqldb`
`@push.rocks/smartdata/testsupport`	`@lossless.org/client/testsupport`
`@push.rocks/smartbucket`	`@lossless.org/client/objectstorage`
`@push.rocks/smartclickhouse`	`@lossless.org/client/sqldb`

SmartData persisted identities, decorator symbols, BSON/null/undefined behavior and exact persistence contracts are retained. Existing normal application access stays through those public model APIs; direct driver access belongs to this foundation or explicit versioned migrations.

ObjectStorage retains companion <key>.metadata objects and .trash/<encoded-original-key> layouts. Existing buffer/replay helpers, list-array methods and watcher state materialize data and are outside the new streaming-memory guarantee. Metadata locks are advisory, not atomic distributed locks; directory move remains unsupported. The getStorageClient() migration escape hatch is retained. No readiness call creates or deletes a bucket.

SmartClickHouse migration changes: replace RxJS next/complete insertion with awaited writer write/close; join database close during application shutdown. Materialized SQL reads now have explicit limits. Query-builder toSQL() contains placeholders; use toStatement() or pass its parameters with the SQL. Watch APIs reject the unsupported continuation guarantee. Insert errors always reach the caller. Existing stored dotted-column names remain unchanged. Original API documentation and licenses are preserved in the repository under docs/source.

nosqldb lineage

@lossless.org/client/nosqldb is a fork of @push.rocks/smartdata 11.14.2 (commit 9868817a63de72caef787bd8e68bac3deb9d6131), taken on 2026-09-10; this repository continues that package's git history, so every upstream fix up to and including 11.14.2 is present here. nosqldbLineage, exported from the family entry point, carries the same baseline for runtime and tooling checks, and docs/source/smartdata keeps the upstream readme, hints and changelog.

The fork departs from that baseline in five places: NoSqlConnection adds the client backend, capability and readiness contract; the family entry point exports it together with nosqldbLineage; watch() refuses to open a change stream when the connected backend reports change streams as unsupported; init() identifies the engine and states the capabilities that identity implies; and two-field grouped totals are refused on a backend that does not group composite keys exactly. The package version file was dropped, because versioning belongs to @lossless.org/client. Document-model capabilities added after the fork — $max/$min, identity seeding through $setOnInsert, atomicUpsertMany() and identityAsDocumentId — are this package's own additions and have no upstream counterpart.

Fixes land here. @push.rocks/smartdata is no longer a release channel for this implementation, so a later upstream version is not a source to merge from and not an upgrade path for consumers of this package.

Verification

pnpm build builds production declarations. pnpm test checks aggregate lifecycle and writers. pnpm run test:nosqldb, test:nosqldb:qualified, test:mongodb, test:objectstorage and test:sqldb exercise isolated backends. pnpm run test:qualification runs the aggregate, both engine suites and the MongoDB suite in one sequence; it is what the release preflight runs. pnpm run test:package packs the package and resolves every entry point; pnpm run test:install installs the packed tarball into a scratch project and proves that a document-only consumer receives no relational or object driver, and that the aggregate client refuses a missing one by name. Wrappers own and remove their disposable servers, containers and data directories. Never supply a production database or bucket to these destructive suites.

Qualified engines:

Component	Qualified versions
NoSQLDB engine	8.0.2 and 10.5.0 — the same `test/nosqldb/smartdb/` suite runs against both, through the single engine seam in `test/nosqldb/helpers/smartdb.ts`; `NOSQLDB_ENGINE=qualified` selects 10.5.0
MongoDB	8.0.26 replica sets
ObjectStorage	10.1.0
SQLDB	0.2.3
MariaDB	11.8
ClickHouse	25.8

Both engine versions run the whole document suite. They part on one asserted boundary — 10.5.0 groups composite keys exactly and answers two-field grouped totals, 8.0.2 does neither and the client refuses there — which Document APIs that need a MongoDB backend states row by row. Text search, the Lucene adapter and the broad read-filter surface are exercised against MongoDB only. pnpm run test:minio uses a digest-pinned MinIO image: sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e. That MinIO version enforces the tested conditional uploads but does not enforce conditional deletion; its exact-purge capability is correctly unavailable. ObjectStorage passes both live probes. AWS S3 has not been qualified against a live account in this migration; no universal S3 exact-operation guarantee is inferred from the SDK.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

readme.md

@lossless.org/client