serve.zone/corestore
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
162 Commits 2 Branches 51 Tags
main
T
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

readme.md

@serve.zone/corestore

Corestore is the node-local storage provider for serve.zone workloads. One process starts a NoSQLDB database endpoint, one ObjectStorage S3-compatible endpoint per configured object-storage pool, a Coreflow-facing control API, and a Docker VolumeDriver plugin. Its storage API reconciles backend-neutral named filesystem and object-storage bindings.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

What It Provides

Corestore packages the storage primitives that a workload usually needs on a serve.zone node:

  • Database: @lossless.org/nosqldb 10.3.0 provides authoritative allocation fencing, allocation-bound read grants, and restricted persistent views. NoSQLDB is the database server and exposes the MongoDB wire protocol on port 27017.
  • Object storage: @lossless.org/objectstorage exposes one S3-compatible endpoint per physical pool; the default local pool uses port 9000.
  • Control API: HTTP JSON API on port 3000 for named storage reconciliation, service-resource provisioning, metrics, snapshots, restores, and archive object replication.
  • Contract version: the installed @serve.zone/interfaces release is the contract version. Corestore 32.0.0 requires interfaces 32; no request, response, record or digest input carries a version field, and the control paths carry no version segment.
  • Credential bindings: Interfaces 32 value-free database and object-storage requests are reconciled under publication grants; plaintext is derived only for authenticated, non-cacheable material delivery.
  • Docker volumes: Docker's VolumeDriver API over /run/docker/plugins/corestore.sock.
  • Backups: @serve.zone/containerarchive stores deduplicated volume snapshots under the Corestore data directory.

Corestore is deliberately node-local. Volume data lives on the node where Docker mounts it, and the Docker driver reports Scope: local.

Moving To NoSQLDB And ObjectStorage

Corestore uses the published @lossless.org/nosqldb 10.3.0 and @lossless.org/objectstorage 10.0.1 APIs. NoSQLDB 10 reads supported existing stores without a startup or read-time format rewrite. Actual writes can introduce its certified document-ordinal records; after those writes, reopening requires NoSQLDB 10 or later. Preserve a pre-upgrade backup when a binary downgrade is required. Startup accepts only the owning engine's supported current format; older, mixed, incomplete allocation or unsupported roots remain rejected. Authentication is stored in the native database; the removed auth.usersPath option is no longer passed and no legacy user file is imported or deleted. This upgrade does not convert unsupported data automatically. Follow the owning NoSQLDB migration and admission documentation before deploying against existing storage.

ObjectStorage 10 requires GNU/Linux amd64 or arm64, glibc 2.34 or later, the matching GNU loader, libm.so.6 and libgcc_s.so.1. Musl-only hosts are unsupported. The service image uses the GNU ht-docker-node:lts base. Existing deployments require a separately qualified image rollout and any explicit database format conversion before their persisted runtime image identity is changed.

Both image stages pin the qualified ht-docker-node 7.0.0 LTS digest. Its native package-manager launcher selects this project's exact pnpm 12.4.1 pin. Release tooling uses tsdocker 3.6.0 and tstest 6.1; configured image tests start the compiled Corestore provider and verify database persistence across restart.

Provider values smartdb and smartstorage, versioned smartdb.* and smartstorage.* formats, hash domains, export item types, retention metadata, existing data directories and the operator migration command retain their stored identities. Do not rename those values or remove private metadata to force startup. The ObjectStorage migration documentation preserves the corresponding storage and cluster protocols.

Earlier SmartDB 5 Admission Boundary

SmartDB 5 reserves the physical system.views namespace. Before upgrading an existing Corestore installation, inspect every managed database while the older Corestore/SmartDB release is still running and rename or remove any unrelated or noncanonical physical system.views collection. Corestore 6 startup fails closed rather than interpreting such data as a view catalog. This is a compatibility boundary; do not bypass the validation or delete unreviewed database contents during the upgrade.

Workload Database Queries

Corestore embeds NoSQLDB 10.3.0 and persists its own control records with the @lossless.org/client 1.0.0 NoSQL client (@lossless.org/client/nosqldb). Workloads can use the same public client APIs for declared index hints, bounded counts, sorted offset pages, and owned transactions. Hinted reads and counts inside a transaction use its snapshot plus buffered writes. Sparse hints intentionally omit documents absent from the selected index, including when counting an empty filter.

Non-null database fence observations must include the current engine's canonical providerRootId. Corestore validates that identity alongside the exact fence-state schema before continuing a mutation or recovering a durable allocation; missing, malformed or unknown fields fail closed.

NoSQLDB accepts scalar index names and exact ascending/descending key patterns. Missing or ambiguous indexes, $natural, hints on views, and forced multikey traversal fail explicitly. The NoSQL client accepts model-declared index names and _id_; use its finite getOffsetPage() API for pages in an owned transaction. inspectNamespace() provides bounded catalog and document-count observations without creating collections or indexes. Set documentCountMode: 'exact' for unrestricted counts, including collections above 10,000 records; existing work and time budgets still apply. Its result is observational, so callers still need fencing or quiescence when they require a stable namespace.

Runtime Layout

Default data root:

/data/corestore

Inside that root Corestore creates:

Path Purpose
smartdb/ NoSQLDB file storage, user data, and the authoritative binding and database-mutation registries Corestore writes through its NoSQL client.
smartstorage/ S3-compatible object data.
volumes/ Docker volume mountpoints.
volume-archive/ ContainerArchive repository for volume snapshots.
restore-staging/ Bounded archive objects staged for isolated restores.
restore-control-locks/ Durable isolated-restore lease and fencing records.
restore-control-state/ Durable isolated-restore state and mutation receipts.
resource-mutation-locks/ Per-target resource fencing records.
resource-mutation-state/ Durable target-resource mutation state.
database-export-package-locks/ Durable offline-package attempt leases and fencing records.
database-export-package-state/ Fsynced offline-package preparing, completion, and release state.
database-export-package-artifacts/ Private per-attempt repositories, inputs, and retained closures.
database-export-package-capacity/ Durable fixed-slot reservations that bound concurrent and retained package artifacts.
corestore-manifest.json Service-resource, volume, and snapshot manifest. It is not authoritative for named storage bindings.
corestore-secret.json Persisted master secret and derived admin credentials.

Corestore requires its data root and the package lock, state, artifact, and capacity roots to be real directories owned by the effective process user. It hardens accepted roots to mode 0700 and refuses symlinks, foreign ownership, or paths that change identity during startup validation.

The master secret is generated on first start unless CORESTORE_MASTER_SECRET is provided. Database tenant credentials are derived from the service id; named object-storage credentials are derived independently from each stable binding id and are never persisted in binding records.

Configuration

Env var Default Purpose
CORESTORE_DATA_DIR /data/corestore Persistent data root.
CORESTORE_BIND_ADDRESS 0.0.0.0 Bind address for control, S3, and DB endpoints.
CORESTORE_PUBLIC_HOST corestore Hostname written into generated service credentials.
CORESTORE_CONTROL_PORT 3000 Control API port.
CORESTORE_S3_PORT 9000 S3 endpoint port.
CORESTORE_OBJECT_STORAGE_POOLS_JSON unset Strict JSON for additional local or host-mounted NFS object-storage pools.
CORESTORE_DB_PORT 27017 NoSQLDB MongoDB-wire endpoint port.
CORESTORE_REGION us-east-1 Region value for S3 credentials.
CORESTORE_API_TOKEN unset Required 32–4096 byte control token; configure this or CORESTORE_API_TOKEN_FILE, never both.
CORESTORE_API_TOKEN_FILE unset Alternative file containing the control token, with one optional trailing newline.
CORESTORE_RESTORE_GRANT_KEYRING_FILE unset JSON keyring containing trusted RSA public keys for isolated-restore grants.
CORESTORE_RESTORE_GRANT_ISSUER unset Exact expected iss claim for isolated-restore grants.
CORESTORE_RESTORE_GRANT_AUDIENCE unset Exact expected aud claim for isolated-restore grants.
CORESTORE_CLUSTER_ID unset Cluster identity to which isolated-restore grants must be bound.
CORESTORE_NODE_NAME unset Node identity to which isolated-restore grants must be bound.
CORESTORE_ISOLATED_RESTORE_LEASE_MS 3600000 Durable restore/resource mutation lease duration in milliseconds.
CORESTORE_DATABASE_EXPORT_PACKAGE_LEASE_MS 3600000 Durable offline-package attempt lease duration in milliseconds.
CORESTORE_DATABASE_EXPORT_PACKAGE_MAX_RETAINED_BYTES 1073741824 Aggregate package admission ceiling; it must be a positive safe integer yielding between 1 and 4,096 full 256 MiB slots (minimum 268435456). A remainder below one full slot is discarded.
CORESTORE_MASTER_SECRET generated and persisted Seed for deterministic tenant credentials.
CORESTORE_DB_ROOT_USER corestore_root NoSQLDB root username.
CORESTORE_DB_ROOT_PASSWORD derived or persisted NoSQLDB root password override.
CORESTORE_S3_ADMIN_ACCESS_KEY_ID derived or persisted ObjectStorage admin access key override.
CORESTORE_S3_ADMIN_SECRET_ACCESS_KEY derived or persisted ObjectStorage admin secret override.
CORESTORE_VOLUME_PLUGIN_SOCKET /run/docker/plugins/corestore.sock Docker VolumeDriver socket path.
CORESTORE_ARCHIVE_PASSPHRASE unset Optional ContainerArchive encryption passphrase.
CORESTORE_VERBOSE false Enables verbose ObjectStorage logging when set to true.

Corestore refuses to start unless exactly one control-token source is configured. Tokens must contain no whitespace. The file form must be a regular, non-symlink file and is preferable when the orchestrator can mount secrets. Every TCP control API endpoint except GET /health requires exactly one Authorization: Bearer <token> header; the legacy x-corestore-token header is rejected.

The five isolated-restore settings must be configured together to enable signed restore grants. Partial configuration prevents startup; leaving all five unset keeps ordinary Corestore APIs available but makes isolated-restore operations unavailable. The keyring has this exact shape:

{
  "keys": [
    {
      "kid": "restore-key-1",
      "publicPem": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----\n"
    }
  ]
}

Object-storage pools

The implicit default pool preserves the existing local ObjectStorage identity at <CORESTORE_DATA_DIR>/smartstorage and CORESTORE_S3_PORT. Additional pools are configured by CORESTORE_OBJECT_STORAGE_POOLS_JSON. Each pool has a unique endpoint, physical root, backend identity, and one or more portable policy classes. Corestore starts and reconciles one ObjectStorage provider per pool.

Example host-mounted NFS capacity pool:

{
  "pools": [
    {
      "id": "archive-hdd",
      "port": 9001,
      "directory": "/storage-pools/archive-hdd",
      "backend": {
        "kind": "mountedFs",
        "expectedFilesystemType": "nfs",
        "expectedSource": "192.0.2.10:/volume1/onebox"
      },
      "classes": [
        {
          "classId": "onebox.object-storage.capacity.v1",
          "revision": "1",
          "kind": "objectStorage",
          "performanceTiers": ["capacity"],
          "durabilities": ["persistent"],
          "topologies": ["singleNode"],
          "hardQuota": false,
          "snapshotModes": ["none"],
          "backup": false,
          "encryptedInTransit": false,
          "accessModes": ["readWrite"],
          "versioning": false,
          "retention": true
        }
      ]
    }
  ]
}

The host must mount NFS before Corestore starts and bind-mount the exact mount root into the Corestore container at the configured directory. Corestore does not mount remote filesystems itself. Mounted pools require an exact numeric IPv4 server:/absolute/export source and the Linux nosymfollow VFS option. NFS server-coordinated locking must remain enabled: nolock and local_lock=all|flock are rejected, while an absent local_lock option, local_lock=none, or local_lock=posix is accepted. ObjectStorage validates the mount type, source, mountpoint, remote locking, symlink policy, and publication primitives before accepting traffic. A missing, replaced, or mismatched mount fails startup instead of writing into the underlying local directory.

Pool ids, ports, directories, class revisions, and backend identities are persisted as an immutable catalog. Once a pool is adopted, its definition cannot be changed or removed, even when no live binding currently refers to it; adding a class revision to that existing pool also changes its immutable identity and is rejected. Add a new pool containing the new class revision and use the fenced migration protocol when physical placement must change.

Starting Corestore

Install the published package in a consumer:

pnpm add @serve.zone/corestore @serve.zone/interfaces@32.3.0
pnpm exec corestore

For a repository checkout:

pnpm install
pnpm build
pnpm start

For TypeScript development:

pnpm run startTs

Programmatic startup:

import { CoreStore } from '@serve.zone/corestore';

const corestore = new CoreStore({
  dataDir: '/var/lib/serve.zone/corestore',
  apiTokenFile: '/run/secrets/corestore-api-token',
  restoreGrantKeyringFile: '/run/secrets/corestore-restore-keyring.json',
  restoreGrantIssuer: 'https://cloudly.example',
  restoreGrantAudience: 'corestore.restore',
  clusterId: 'cluster-1',
  nodeName: 'node-1',
});

await corestore.start();

Offline MongoDB-wire migration

The image includes one explicit operator migration that copies a stopped workload's database from a MongoDB-wire-compatible source into Corestore's NoSQLDB backend. The migration is not part of normal startup. First provision the target through Corestore's normal database resource API while Corestore is running, but do not deliver its credentials to the workload. Then stop Corestore and fence every source writer. The migration requires that exact NoSQLDB tenant and its already-durable manifest entry, reads the manifest without rewriting it, replaces an empty target or resumes an exact replay behind NoSQLDB's durable publication hold, releases publication, and verifies canonical BSON documents and complete index definitions. The source database may contain no collections. The migration persists through Corestore's normal database-mutation coordinator, so provision, migration, later snapshot/restore, and deprovision remain in one durable fence lineage. An exact replay with the same migration id and fencing token returns the same released result.

The migration rejects collection options and index kinds or options that NoSQLDB cannot enforce. It also fails closed above a 96 MiB serialized export, 10,000 collections, 1,000,000 documents, or 100,000 indexes. Fix and release a missing NoSQLDB capability before retrying; split a larger migration through an explicitly reviewed process instead of weakening this preflight.

If the running Corestore uses CORESTORE_CLUSTER_ID and CORESTORE_NODE_NAME, the offline migration requires those same values so its NoSQLDB fence remains in the existing resource-mutation scope. Configure both or neither.

CORESTORE_MIGRATION_SOURCE_FENCED=true \
CORESTORE_MIGRATION_CORESTORE_STOPPED=true \
CORESTORE_MIGRATION_TARGET_CREDENTIALS_UNDELIVERED=true \
CORESTORE_MIGRATION_SOURCE_MONGODB_URI='<protected MongoDB URI>' \
CORESTORE_MIGRATION_SOURCE_DATABASE='<source database>' \
CORESTORE_MIGRATION_SERVICE_ID='<stable service id>' \
CORESTORE_MIGRATION_SERVICE_NAME='<service name>' \
CORESTORE_MIGRATION_ID='<stable migration id>' \
CORESTORE_MIGRATION_FENCING_TOKEN='<positive integer>' \
pnpm run migrate:legacy-mongodb-to-smartdb

CORESTORE_DATA_DIR, CORESTORE_PUBLIC_HOST, and CORESTORE_DB_PORT select the target Corestore instance and keep their normal defaults. Never place the source URI in logs or committed configuration.

Whole-owner SmartDB 5 to NoSQLDB 10 conversion

migration.js also supports a versioned whole-owner engine conversion for a Corestore data root written by SmartDB 5. It is an offline operator operation, never an automatic startup fallback. engine-source-inventory reads the protected manifest and MongoDB-wire namespace list and reports only owned database names and allocation status. It requires an exact match between corestore_control, every manifest-owned database, and the wire inventory; an unowned database or an admin collection refuses conversion. engine-source-export exports the control database and every ordinary tenant through bounded read-only wire operations into a new mode-0600 artifact. It preserves canonical BSON numeric widths and complete supported index specifications, including TTL seconds. A preliminary export with CORESTORE_ENGINE_SOURCE_FENCED=false is for rehearsal only.

For an authoritative export, stop all workload writers, establish the orchestrator's source fence, and set CORESTORE_ENGINE_SOURCE_FENCED=true. Keep the old Corestore wire endpoint available for that export, then stop Corestore. Preserve a separate verified checkpoint. Copy the complete non-database Corestore data root to an isolated stage, including corestore-manifest.json, corestore-secret.json, object storage, volumes, and coordinator state; leave smartdb absent in the stage. The export artifact and staged root contain private data. Do not deliver target credentials or activate the target during conversion. The stage command refuses any allocation-managed database; finish its owning operation or use its exact supported deprovision/transfer contract first.

Generic and preliminary wire exports retain a database-wide snapshot transaction. Only an authoritative whole-owner export with both CORESTORE_ENGINE_SOURCE_FENCED=true and a validated source digest proof uses bounded nontransactional pages. SmartDB 5 intentionally caps the bytes and documents materialized by one find, so the converter retries only that exact capacity refusal with a strictly smaller page and discards the complete failed page attempt. It never falls back from a snapshot error. The fenced page order is valid only while the old physical engine is immutable; the converter requires equal collection counts before and after each collection, complete page totals, an unchanged collection/index catalog, and exact owner-digest totals. The orchestrator must still prove the authenticated old-owner digest is identical before and after export.

pnpm run migrate:engine-source-inventory
pnpm run migrate:engine-source-export
pnpm run migrate:engine-stage
pnpm run migrate:engine-activate

Inventory and export require CORESTORE_ENGINE_SOURCE_MANIFEST_PATH and CORESTORE_ENGINE_SOURCE_SECRET_PATH in the same read-only mounted source root, plus CORESTORE_ENGINE_SOURCE_HOST and CORESTORE_ENGINE_SOURCE_DB_PORT. Set CORESTORE_ENGINE_SOURCE_DB_ROOT_USER only when the source used a non-default root user. The converter derives the protected wire URI in memory; never pass credentials through Docker environment or command arguments. Export additionally requires an absolute, initially nonexistent CORESTORE_ENGINE_EXPORT_PATH and explicit CORESTORE_ENGINE_SOURCE_FENCED=true or false. Stage reads that private artifact and requires absolute CORESTORE_ENGINE_STAGE_DATA_DIR and initially nonexistent CORESTORE_ENGINE_STAGE_RECEIPT_PATH, stable CORESTORE_ENGINE_MIGRATION_ID, CORESTORE_PUBLIC_HOST, CORESTORE_DB_PORT, CORESTORE_S3_PORT, CORESTORE_REGION, and all three assertions CORESTORE_ENGINE_SOURCE_FENCED=true, CORESTORE_ENGINE_CORESTORE_STOPPED=true, and CORESTORE_ENGINE_TARGET_CREDENTIALS_UNDELIVERED=true. Supply the runtime's exact CORESTORE_CLUSTER_ID/CORESTORE_NODE_NAME pair and CORESTORE_OBJECT_STORAGE_POOLS_JSON when configured.

SmartDB 5 can expose an empty physical system.views catalog while refusing wire listIndexes and reads on it. The converter preserves that namespace and its canonical _id_ index only with an orchestrator-owned source digest proof. Set CORESTORE_ENGINE_SOURCE_DIGEST_PROOF_PATH to the private, exact mode-0600 proof file and CORESTORE_ENGINE_MIGRATION_ID to its migration identity for engine-source-export. The proof has format onebox.corestore-engine.source-digest-proof.v1, binds the checkpoint, source container/image, manifest hash and database names, and contains authenticated old-owner smartdb.database.content-digest.v1 entries for every ordinary tenant. The orchestrator must obtain those entries through the fenced old Corestore owner's digest endpoint, journal the exact proof bytes and SHA-256 before export, then repeat the digest reads and require equality afterward. Corestore verifies the proof's shape and exact inventory and independently checks collection, document and index totals for every proved ordinary tenant, including one without system.views; it cannot authenticate proof authorship by itself. A missing, nonempty, changed or uncertain catalog refuses conversion. The private export artifact retains sourceDigestProof: { proof, sha256 }; its value-free stdout summary reports sourceDigestProofSha256 for the orchestrator's journal check.

The old engine stores the catalog _id_ index as {name,key} but its wire listCollections.idIndex reports {v:2,key,name}. The converter uses that canonical wire metadata and compares the complete transformed collection set against the held target readback. It does not compare old-owner and target content-digest SHA values across this metadata normalization; the orchestrator instead requires identical authenticated old-owner digests before and after export.

Stage imports corestore_control into a fresh NoSQLDB root, verifies it, runs the ordered Corestore control-data migrations, then ensures every ordinary tenant and imports it through Corestore's durable database-mutation coordinator. It independently reads each tenant through the exact held-publication capability and compares canonical BSON documents and index definitions with the fenced source. The stage command writes each value-free held receipt and digest into the new private mode-0600 receipt file and returns its SHA-256 in the stdout summary; the orchestrator durably journals that hash before proceeding. Stage refuses any control-database TTL index because control migrations require its data plane while tenant holds remain closed. A failed or interrupted stage is discarded in full and recreated from the checkpoint and fenced export; never activate a partial stage.

engine-activate requires the same staged root, fenced export, migration identity, runtime endpoint and cluster identity, plus CORESTORE_ENGINE_STAGE_RECEIPT_PATH pointing at that exact private result and CORESTORE_ENGINE_ACTIVATE=true. It reopens the same physical NoSQLDB root, checks the migrated control digest and independently re-exports every still-held tenant before releasing any tenant. Released tenants from an interrupted activation must match their durable held-receipt hash; exact retry resumes the remaining holds. Activation is the deliberate publication step; only then may the orchestrator start the isolated Corestore runtime, qualify native tenant ownership and object access, and reconcile resource consumers. The converter does not copy object bytes itself or rotate previously delivered credentials.

Each database export is limited to 96 MiB, 10,000 collections, 1,000,000 documents, and 100,000 indexes; the whole owner is limited to 256 MiB of serialized database export. Collection options, index kinds or options that the target engine cannot enforce, active allocations, or larger owners require an owning migration capability before conversion. These refusals never authorize dropping data or weakening equality checks.

Upgrading from 7.0.0

This release renames every version-bearing token Corestore owns, so values it already wrote have to be rewritten before the runtime can read them. Corestore does that itself: the first start of 32.0.0 runs ts_migration/runner.ts after the control database opens and before any coordinator reads its on-disk state. Every step is idempotent, its completion is recorded in the CoreStoreDataMigration collection, and a step that meets state it cannot rewrite refuses start-up by name without changing anything. A second start is a no-op; an interrupted start resumes where it stopped.

What migrates automatically:

Step What it rewrites
storage-binding-fields Drops schemaVersion from the record and its resolved binding, renames the storage class id and the object-storage file delivery format, and recomputes requestDigest, specDigest, intentDigest and serviceRequestKey with the runtime's own digest functions.
credential-binding-fields Drops schemaVersion from the record, the binding request and the publication grant, recomputes bindingRequestDigest, and carries it into the grant and into the object-storage retention evidence. That evidence loses the version this release added to its capability evidence, and its intentSha256 is re-derived because the digest input dropped a version member; the provider's own sentinel version is kept verbatim.
database-mutation-fields Drops schemaVersion and renames the portable payload format. It deliberately leaves requestSha256, replaceHistory[].requestSha256 and smartDbFence untouched — see below.
storage-pool-catalog-fields Drops schemaVersion, renames the class keys and adopts the pool's current configDigest, but only while the pool's durable providerIdentity still matches the configuration.
filesystem-volume-options Drops version from corestore-manifest.json and rewrites every Corestore-owned volume's servezone.storage.* options to the migrated binding's digest.
coordinator-protocol-trees Collects the lease trees 7.0.0 wrote. Its claims, release markers, commit receipts and export-package capacity reservations carry a version member this release refuses, and the export-package coordinator reads them on every start, so a data directory that keeps them cannot start at all.

Preconditions, all of them checked before the first step writes anything:

  • Quiesce database mutations. Commit or release every held publication before stopping 7.0.0. A held publication is a NoSQLDB barrier only the issuing release can clear: its held receipt is rebuilt from requestSha256 and the payload format, both of which this release states differently.
  • No isolated restore still owns its staging directory. A restore's coordinator configuration and resource-lease identities both change, so a staged run cannot be resumed across the upgrade, and collecting the 7.0.0 lease trees leaves the start-up sweep no state to read. Only a staging directory whose latest durable state is present and not cleaned refuses; the named recovery is to finish that restore under 7.0.0, or to remove the named directory under restore-staging. Directories the release already cleaned, and the lock directories left behind under resource-mutation-locks, are garbage the migration collects; they are never converted.
  • No object-storage migration records. A live object-storage migration is a two-party protocol: its consumer holds migrationDigest, candidateDigest and the snapshot digests and sends them back. Complete or abort every migration and clear its records first.
  • 7.0.0 database backup receipts are re-exported, not converted. Their format is now corestore.database.backup, which the receipt reader does not recognise. Take a fresh export with this release instead; a re-issued receipt for the same backup replays onto the same restoreId.
  • Clients request new read grants after the restart. Nothing about a read-access grant is persisted, NoSQLDB retires every active grant when its engine restarts, and the credentials derive from a renamed HMAC domain. Ask for a new grantId; do not replay the old one.

The refusals read as the precondition that was missed, with the records that caused them:

corestore-migration-database-mutation-held-publication: 1 database mutation(s) hold a NoSQLDB
publication that only the issuing release can commit or release; commit or release them before
upgrading (<targetResourceId>)

corestore-migration-isolated-restore-in-flight: an isolated restore is staged by a release whose
coordinator configuration and resource lease identities this upgrade renames; finish the restore
under that release, or remove its staging directory under <dataDir>/restore-staging (<restoreId>)

corestore-migration-object-storage-migration-records-present: 1 object-storage migration record(s)
and 0 migration object record(s) were written by a release whose digests this upgrade renames;
complete or abort every object-storage migration and clear its records before upgrading

corestore-migration-storage-binding-unmigratable, corestore-migration-credential-binding-unmigratable, corestore-migration-credential-operation-claim-unmigratable, corestore-migration-database-mutation-unmigratable and corestore-migration-filesystem-volume-unowned name a single record that does not validate in its 32 shape after the rewrite; each carries the record's identity and the underlying contract error as its cause.

Why the database fence is not rewritten: NoSQLDB pins a database's durable fence scope at the record's current token, and the record reader binds smartDbFence to the provider receipt inside heldResult / releasedResult. Rewriting the scope would either be refused at the current token or require forging that receipt. Left as written, the next mutation takes token + 1, and the strictly newer token carries the new scope. The cost is that replaying a pre-upgrade restoreId answers 409 database backup restore identity does not match durable state, because the history entry keeps only the digest and not the request it was computed from.

Control API

GET /health is always unauthenticated:

curl http://corestore:3000/health

All other control endpoints require exactly one Authorization: Bearer <token> header.

Useful endpoints:

Method Path Purpose
GET /health/details Returns authenticated component-level health details.
GET /metrics Returns database, object storage, and volume metrics.
GET /control/storage/capabilities Advertises enforced portable storage guarantees and exact policy revisions.
POST /control/credential-bindings/reconcile Reconciles one Interfaces 32 value-free credential binding under its exact publication grant.
GET /control/credential-bindings?serviceId=<id> Lists secret-free credential binding status, optionally for one service.
GET /control/credential-bindings/<id> Returns one secret-free credential binding status.
POST /control/credential-bindings/<id>/material Returns grant-bound plaintext material with Cache-Control: no-store and Pragma: no-cache.
POST /control/credential-bindings/<id>/retain Permanently marks an omitted or disabled binding retained without deleting its provider resource.
POST /control/ordinary-credentials/rotate Rotates one legacy manifest resource under a durable generation and returns a value-free receipt.
POST /control/ordinary-credentials/material Returns current, provider-attested resource environment material for an exact active rotation receipt; never cache it.
POST /control/storage/bindings/reconcile Idempotently reconciles one stable named binding.
GET /control/storage/bindings?serviceId=<id> Lists canonical secret-free bindings for a service.
GET /control/storage/bindings/<id> Gets one canonical secret-free binding.
POST /control/storage/bindings/<id>/credentials Returns object-storage credential material to the authenticated adapter with Cache-Control: no-store.
POST /control/storage/bindings/<id>/release Applies the binding's declared retain/delete policy and persists a tombstone.
GET /control/storage/bindings/<id>/migration-snapshot Captures the exact unfenced source binding snapshot required to prepare a migration.
POST /control/storage/migrations/prepare Fences the source and prepares a hidden destination binding on a different pool.
GET /control/storage/migrations/<id> Returns the durable migration phase and required consumer action.
POST /control/storage/migrations/<id>/quiesce Accepts bound quiesce evidence, seals the source, verifies the destination, and performs the durable cutover.
POST /control/storage/migrations/<id>/activation Accepts bound evidence that the consumer started on the destination.
POST /control/storage/migrations/<id>/cleanup Performs fenced source cleanup after destination activation.
POST /control/storage/migrations/<id>/abort Aborts a pre-cutover migration through its exact recovery contract.
POST /control/storage/migrations/<id>/resume Resumes a retryable phase without bypassing its durable fence.
GET /volumes Lists managed Docker volumes.
GET /volumes/snapshots?name=<volume> Lists snapshots for one volume or all volumes.
POST /volumes/create Creates or updates a named managed volume.
POST /volumes/remove Removes an unmounted volume.
POST /volumes/snapshot Creates a ContainerArchive snapshot of a volume.
POST /volumes/restore Restores a snapshot into a volume.
GET /resources Lists provisioned per-service DB/S3 resources, but fails closed when any database is allocation-managed or quarantined because this broad route cannot carry an expected reference.
POST /resources/status Returns exact service-resource status and is allocation-aware through expectedDatabaseAllocation.
POST /resources/provision Provisions database and/or object storage for a service.
POST /resources/deprovision Deletes provisioned DB/S3 resources for a service.
POST /resources/database/digest Computes a bounded canonical content digest for exactly one service-owned NoSQLDB database.
POST /resources/database/read-access-grant/issue Issues or exactly replays one fixed-expiry read-only credential for an allocated database.
POST /resources/database/read-access-grant/revoke Idempotently revokes one allocation-bound read-only credential.
POST /resources/database/export Exports exactly one service-owned NoSQLDB database as a bounded portable payload.
POST /resources/database/import Replaces a pre-provisioned service database under a durable fence and returns a held-publication receipt.
POST /resources/database/import/commit Releases the exact held database only after the orchestrator durably records its receipt.
POST /resources/database/backup/export Streams one selected service database as a bounded portable ContainerArchive closure.
POST /resources/database/backup/package Packages one exact offline NoSQLDB database export into a durable, replayable selected closure.
POST /resources/database/backup/package/release Releases one completed package only after the caller reports fsynced closure and receipt identities.
POST /resources/database/backup/restore Verifies a selected closure in isolated staging and durably restores it into a pre-provisioned target database.
POST /resources/snapshot Snapshots service DB/S3 resources.
POST /resources/restore Restores service DB/S3 resources.
POST /archive/manifest Returns a manifest for the local archive repository.
POST /archive/object/read Reads an archive object as base64 with size and SHA-256.
POST /archive/object/write Writes a validated archive object from base64.
POST /archive/prune Prunes archive data under configured retention and free-space bounds.
POST /isolated-restore/prepare Validates an operation-scoped grant, mappings, and archive manifest, then creates durable fenced state.
POST /isolated-restore/archive/object/write Writes one bounded archive object chunk under a write-object grant.
POST /isolated-restore/execute Restores only the grant-authorized scratch resources.
POST /isolated-restore/status Returns sanitized durable progress under a status grant.
POST /isolated-restore/cleanup Cleans staged artifacts under a cleanup grant while preserving durable fencing evidence.

Ordinary credential rotation accepts {serviceId, capability, operationId, expectedGeneration} for a manifest-owned database or objectstorage resource. A resource with no rotation record starts at generation 0. An exact operation retry returns the same value-free receipt; the separate material route accepts {receipt} and returns the current provider-attested environment only while that receipt is active. Deprovision retires the generation before deleting the provider resource, recovers either side of manifest publication on restart, and revokes old material receipts. Reprovision retains the last generation so it cannot silently return to generation-0 credentials.

Provision resources for a service:

curl -X POST http://corestore:3000/resources/provision \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer <CORESTORE_API_TOKEN>' \
  -d '{"serviceId":"svc-123","serviceName":"api","capabilities":["database","objectstorage"]}'

The response includes service-specific environment variables such as MONGODB_URI, S3_BUCKET, AWS_ACCESS_KEY_ID, and AWS_ENDPOINT_URL.

MONGODB_URI, MONGODB_URL, and the related MONGO_* names are intentional: NoSQLDB is the database server, and workloads connect to it through the MongoDB wire protocol. They do not imply that a MongoDB server is running.

Scratch database allocations

Scratch and rehearsal databases are allocated through the database-only /resources/provision path. The provisioning body adds databaseAllocation with exactly mode: "expected-absent", purpose: "scratch-rehearsal", and a stable rehearsalId; it does not accept allocationId or an expected reference. Corestore requires NoSQLDB's authoritative allocation-fencing health contract before recording prepared state, asks NoSQLDB to allocate the tenant, durably fsyncs the manifest binding, and only then returns credentials. The response returns the collision-resistant allocationId as part of the safe allocation reference. If the allocation becomes durable but manifest publication fails, provisioning returns HTTP 503 with an explicit restart-recovery instruction; before opening the control API, startup resumes released allocations, prepared allocations whose NoSQLDB lifecycle is still allocating or active, and completed deletes. Allocation-ownership conflicts quarantine only the affected database and return HTTP 409, while provider unavailability or malformed provider state aborts startup. An exact retry after response loss or restart replays the same allocation; a new rehearsal for the same service advances the allocation generation after the prior tenant has been deprovisioned. Reuse of an allocation or rehearsal identity is rejected.

Orchestrators should import Corestore's canonical allocation normalizers and identity helpers instead of reproducing the protocol rules:

import {
  coreStoreDatabaseAllocationReferencesEqual,
  getCoreStoreDatabaseIdentity,
  normalizeCoreStoreDatabaseAllocationReference,
  normalizeCoreStoreDatabaseAllocationRequest,
} from '@serve.zone/corestore';

These helpers validate the exact allocation request/reference schemas, compare every safe reference field, and derive the same database and username identity used by Corestore provisioning. They do not expose credentials or provider-private allocation evidence.

The package root also exports validateCoreStorePersistedDatabaseAllocation, getCoreStoreDatabaseAllocationReference, and coreStoreSmartDbAllocationIdentitiesEqual for Corestore-aware provider integrations that already hold the complete persisted allocation record or NoSQLDB allocation identity. General orchestrators should use only the safe request, reference, equality, and database-identity helpers shown above; provider-state helpers accept private allocation structures and must not be used to widen API responses or logs.

{
  "serviceId": "Service:scratch-123",
  "serviceName": "scratch-123",
  "capabilities": ["database"],
  "databaseAllocation": {
    "mode": "expected-absent",
    "purpose": "scratch-rehearsal",
    "rehearsalId": "rehearsal-123"
  }
}

The safe allocation reference contains only purpose, rehearsalId, allocationId, generation, bindingSha256, and receiptSha256. It intentionally excludes the NoSQLDB principal, publication capability, database URI, password, and provider evidence. Later status, digest, portable export/import/commit, snapshot, ordinary restore, selected database backup export/restore, and deprovision requests use expectedDatabaseAllocation and require the exact returned reference whenever the database is allocated. Export operations require the source allocation reference; import, commit, ordinary restore, and backup restore require the target allocation reference. Omitted, malformed, stale, or mismatched references fail before NoSQLDB reads or writes. Existing ordinary databases without allocation metadata retain their ordinary behavior, but Corestore checks NoSQLDB's durable allocation state and quarantines a manifest that omits, corrupts, or conflicts with an active provider allocation instead of treating it as legacy.

Allocation read access grants

The grant endpoints accept at most 16 KiB of UTF-8 JSON with exactly serviceId, grantId, and the active allocation's safe expectedDatabaseAllocation reference. They are available only for allocation-managed databases and require the normal bearer control token. Corestore derives the NoSQLDB username and password with domain-separated HMAC-SHA256 over its protected persistent master secret and the exact service, allocation, and grant identity. It does not persist or log those credentials.

Issue returns serviceId, grantId, the safe databaseAllocation reference, databaseName, username, authSource, fixed roles: ["read"], issuedAt, expiresAt, and a Corestore-public-host mongodbUri. The URI is the only response field containing the password. An active exact replay returns the same grant; each grant expires after 12 minutes and is not renewable. Revoke returns only serviceId, grantId, the safe allocation reference, and revoked: true.

Issue, revoke, authentication failures, and all error responses use Cache-Control: no-store. Grant operations share the database mutation lock with allocation, import, restore, ensure, and deletion. Corestore maps NoSQLDB's exported EGRANT_* codes without parsing messages: invalid requests return 400, credential/conflict/replay-capacity failures return 409, and provider-state failures return 503. Unknown or malformed provider failures are sanitized to 503. Request disconnect, shutdown, or the five-minute provider deadline aborts the NoSQLDB management operation; NoSQLDB terminates its engine fail-stop, leaving Corestore unready until the orchestrator restarts it. NoSQLDB retains at most 1,024 retired grant IDs per allocation without eviction and rejects additional IDs when that replay set is full.

Isolated restore consumes that reference. @serve.zone/interfaces binds the safe allocation reference into the restore mapping as IIsolatedRestoreDatabaseMapping.target.databaseAllocation, and because the mapping is what the restore grant's canonical digest covers, the reference is signed authority rather than a request parameter. /isolated-restore/prepare and /isolated-restore/execute require a database mapping whose target.databaseAllocation is exactly the allocation this Corestore holds whenever the target database is allocation-managed. A missing reference returns HTTP 422 resourceMappings[<id>].target.databaseAllocation is required for an allocation-managed database; a reference naming another allocation returns HTTP 422 ... does not match the allocation this corestore holds; a reference on a database that is not allocation-managed returns HTTP 422 ... names an allocation on a database that is not allocation-managed. There is no unallocated fallback for an allocated target. The check runs again at execute time for each mapping, so an allocation rotated between prepare and execute is refused before any NoSQLDB mutation. The restore itself runs through the same per-database mutation lock and the same durable NoSQLDB ownership checks as allocation, ordinary restore, ensure, and deletion: the reference is carried into the portable import and its publication commit, and after the commit Corestore re-asserts the claim against both the manifest resource and NoSQLDB's durable allocation state, and refuses the restore if the restored database does not hold the allocation the restore was authorized against. A quarantined target service or quarantined allocation metadata still returns HTTP 409 before any of this. Isolated restore status, archive writes, cleanup, and ordinary unallocated execution are unchanged. Native /resources/restore and selected database backup restore support allocated targets as before.

Portable database handoff

The portable database endpoints are the database backup and migration boundary. Export resolves the source database only from the requested manifest serviceId; import resolves the target database and tenant user only from the pre-provisioned targetServiceId. A caller cannot select a target database name or username. For an allocated database, every source descriptor, target, held receipt, and released response carries the safe allocation reference rather than raw provider allocation evidence.

/resources/database/digest accepts serviceId and, for an allocated database, requires its exact expectedDatabaseAllocation in a body of at most 16 KiB. Portable export follows the same rule. Both resolve the database through the manifest and NoSQLDB durable-ownership boundary. Digest returns the resolved service/resource/database source descriptor, and NoSQLDB's smartdb.database.content-digest.v1 result. The digest covers collection names, exact BSON document bytes, and persisted index specifications while normalizing enumeration order only. BSON field order and compound-index key order remain significant. Scans are capped at 96 MiB of BSON, 10,000 collections, 1,000,000 documents, and 100,000 indexes. Compare digest.sha256 and the counters when proving equal content across source and restored databases; do not compare the complete digest objects because digest.databaseName is intentionally service-specific.

The portable payload contains canonical UTF-8 JSON bytes for smartdb.database.export.v1, encoded as base64 with an exact byte count and SHA-256. Corestore rejects non-canonical base64, invalid UTF-8, a non-canonical JSON encoding, descriptor mismatches, payloads above 16 MiB, and complete portable request bodies above 24 MiB. NoSQLDB export is additionally capped at 10,000 collections, 1,000,000 documents, and 100,000 indexes.

Corestore-native database snapshots created by /resources/snapshot and consumed by ordinary or isolated resource restore use a separate 96 MiB payload ceiling. This matches the explicit offline MongoDB-wire migration envelope without weakening the 16 MiB portable API boundary. The archive reader admits only the single trailing newline written by Corestore in addition to that payload.

Import requires a stable restoreId and positive orchestrationFence. Portable-import fences are monotonic within the portable API's own ordering domain; internal snapshot and isolated-restore attempts use a separate ordering domain and cannot make the next valid portable fence stale. Corestore durably binds the request to one NoSQLDB fence through the client's exact persistence before mutating the database. When no control record exists yet, Corestore first inspects NoSQLDB's durable released high-water mark and continues at the next token; an active publication, identity mismatch, corrupt state, or exhausted token fails before any coordination record or provider mutation is created. An exact retry after response loss or restart returns the same held receipt; after commit it returns the released receipt. Reusing a completed restoreId, submitting a stale portable orchestration fence, or conflicting with an active mutation returns 409. The imported database remains unavailable until publication is committed.

The orchestrator must follow this order:

  1. Call /resources/database/import and receive the held receipt.
  2. Persist the complete receipt and migration checkpoint durably.
  3. Call /resources/database/import/commit with that exact receipt and, for an allocated target, the separate top-level expectedDatabaseAllocation target reference.

Digest, import, export, commit, and their error responses use Cache-Control: no-store. Corestore admits only one memory-bounded portable database operation at a time, rejects concurrent mutations for the same database without queueing them, aborts response delivery on shutdown or disconnect, applies a delivery deadline, and gives lifecycle-critical NoSQLDB management and health calls a five-minute fail-stop deadline. It retains bounded monotonic restore history for each target and ordering domain. Provision, restore, isolated restore, and deprovision share the same per-database resource-mutation namespace; there is no unfenced database mutation path in normal Corestore operation.

Selected database backup closures

The database backup endpoints are the operator-scoped backup transport for database payloads larger than the 16 MiB portable handoff envelope. Export accepts a JSON object of at most 16 KiB with serviceId and backupId; it also requires the selected source database's exact expectedDatabaseAllocation when allocated. Backup restore requires the target database's exact reference inside the canonical x-corestore-database-backup-restore header. Corestore creates one database snapshot with controlled ownership tags, exports the exact snapshot closure through ContainerArchive, and returns the opaque closure with a non-cacheable receipt header on the bearer-authenticated response. The receipt's SHA-256 fields bind identity and integrity; they are not an independent signature or MAC. MongoDB wire protocol remains the workload interface before and after backup; the closure transports NoSQLDB's canonical database snapshot representation.

The underlying NoSQLDB snapshot payload is capped at the 96 MiB Corestore-native database envelope. Closure transfer is capped at 256 MiB, 4,096 immutable archive objects, 128 MiB of verified selected plaintext, a 16 KiB canonical receipt header, and five minutes without transfer progress. Restore requires one exact Content-Length, identity transfer encoding, no content encoding, and byte length equal to the receipt. It imports only into a new private disposable staging repository, validates the exact one-snapshot ownership and item contract, restores into a pre-provisioned target service database through the normal durable NoSQLDB fence, and removes staging before returning success.

The restore identity binds the backup, source and target resource identities, snapshot ID, and closure digest. Omitting restoreAttemptId preserves the original restore identity exactly. A caller can provide a canonical restoreAttemptId to select a new fenced restore attempt: retries with the same value resume or replay the same mutation, while a different value intentionally reapplies the backup to the same exact target allocation. The attempt identifier cannot change or bypass expectedDatabaseAllocation.

Selected closure is a semantic backup boundary, not a byte-level tenant-confidentiality boundary. ContainerArchive excludes unrelated snapshot manifests and unselected global-index entries, but immutable reachable packs can contain physical bytes or sidecar descriptions shared with other snapshots. Keep these artifacts inside the trusted operator backup boundary. Onebox continues to back up object storage through the streaming S3 API rather than this database-only closure endpoint.

The repository-wide /archive/manifest and /archive/object/* endpoints are legacy node-replication surfaces used by the current Coreflow backup path. They are not safe service-selected backup APIs and must not be used by new consumers; they remain only until Coreflow moves to selected closures.

Offline database export packaging

/resources/database/backup/package accepts one exact canonical smartdb.database.export.v1 file, including its single trailing LF, without reading or mutating the selected NoSQLDB database contents. It still reads current NoSQLDB ownership and allocation state to validate the source fence. The request body uses exact application/json, one exact positive Content-Length, identity transfer encoding, and no content encoding. Canonical package metadata from @serve.zone/interfaces/runtime is encoded as unpadded base64url in x-corestore-database-export-package-request. Metadata is capped at 16 KiB before encoding, input at 96 MiB plus its LF, and the resulting closure at 256 MiB and 4,096 immutable objects. Input and response delivery each fail after five minutes without progress. The control server keeps a bounded 64 KiB aggregate request-header budget so the encoded metadata and authorization header reach the route without weakening the contract limit.

Corestore validates the input length, SHA-256, UTF-8, canonical JSON round trip, database name, collection and index uniqueness, current source ownership, and exact allocation expectation. It ingests the unchanged bytes as the sole database.json / smartdb-database-export item in an attempt-private ContainerArchive repository using the configured archive encryption profile. A successful response has runtime.corestoreDatabaseBackupMediaType (application/vnd.serve-zone.corestore-database-backup), the exact closure Content-Length, and the raw opaque closure as its body. The normal closure receipt is returned in x-corestore-database-backup-receipt; x-corestore-database-export-package-response carries the canonical package completion and replay flag. Both headers use unpadded base64url canonical JSON. All package responses are non-cacheable, and failed requests close the connection so an unread rejected body cannot be reused as another HTTP request.

Each packageAttemptId has a cross-restart lease and fsynced state. Before binding a new attempt, Corestore atomically publishes a durable fixed-slot reservation for the full 256 MiB per-closure maximum. Once admitted canonical metadata is durably bound, a failed or truncated input leaves that attempt and reservation in preparing; the caller must retry the same attempt and metadata rather than minting replacement IDs. The default ceiling provides four slots, bounding admitted incomplete and completed attempts across crashes. Full capacity rejects a new attempt before creating its lease or state with HTTP 507 and PACKAGE_CAPACITY_EXHAUSTED; exact preparing/completed retries and releases remain available. Corestore fsyncs and hashes the retained closure before recording completion, then reopens, pins, and revalidates that same descriptor before every delivery. An exact retry must resend and pass validation for the same input bytes and receives the identical retained closure; conflicting reuse returns ATTEMPT_CONFLICT. A disconnected response leaves the completed attempt replayable.

The caller must fsync the received closure and the exact canonical receipt bytes produced by encodeCorestoreDatabaseBackupReceipt() before posting the canonical release request to /resources/database/backup/package/release. Release requires exact application/json, one exact positive Content-Length, identity transfer encoding, no content encoding, and canonical JSON within the 16 KiB control limit. It returns a JSON release response. Corestore durably records release intent before deleting the attempt-private repository and closure, fsyncs the artifact root, and retains a small terminal state for exact release replay. Package retries after successful release return HTTP 410. The bearer control token is the transport authority; Corestore binds but does not independently authenticate upstream settlement receipt digests carried by the package request.

Package clients can import coreStoreDatabaseExportPackageRequestHeader, coreStoreDatabaseExportPackageResponseHeader, coreStoreDatabaseExportPackageReleasePath, and coreStoreDatabaseExportPackageMaximumHeaderBytes from @serve.zone/corestore instead of duplicating the public transport names and header budget.

The following Node.js outline shows the complete request shapes and the required canonical encoding, verification, durable write, and release order. Values such as allocation references and upstream settlement digests must come from the orchestrator's approved operation; they are not invented by the package client.

import { createHash } from 'node:crypto';
import { open } from 'node:fs/promises';
import {
  coreStoreDatabaseExportPackageRequestHeader,
  coreStoreDatabaseExportPackageResponseHeader,
  coreStoreDatabaseExportPackageReleasePath,
  type ICoreStoreDatabaseExportPackageRequest,
  type ICoreStoreDatabaseExportPackageReleaseRequest,
} from '@serve.zone/corestore';
import * as runtime from '@serve.zone/interfaces/runtime';

const sha256 = (bytes: Uint8Array) => createHash('sha256').update(bytes).digest('hex');
const databaseExportBytes = Buffer.from(`${canonicalSmartDbExportJson}\n`, 'utf8');
const packageRequest: ICoreStoreDatabaseExportPackageRequest = {
  kind: 'corestore-database-export-package-request',
  packageAttemptId,
  packageBackupId,
  input: {
    format: 'smartdb.database.export.v1',
    databaseName,
    byteLength: databaseExportBytes.length,
    sha256: sha256(databaseExportBytes),
    lineFeedTerminator: 'single-lf',
  },
  approvedSourceBackup: {
    backupId: approvedBackupId,
    sourceServiceId,
    snapshotId: approvedSnapshotId,
    backupReceiptSha256: approvedBackupReceiptSha256,
  },
  completeOutputReceiptSha256,
  allocationExpectation: {
    expectation: 'exact',
    allocation: {
      purpose: 'scratch-rehearsal',
      rehearsalId,
      allocationId,
      generation,
      bindingSha256,
      receiptSha256: allocationReceiptSha256,
    },
  },
};
const packageRequestBytes = Buffer.from(
  runtime.encodeCorestoreDatabaseExportPackageRequest(packageRequest),
);
const packageHttpResponse = await fetch(`${corestoreOrigin}/resources/database/backup/package`, {
  method: 'POST',
  headers: {
    authorization: `Bearer ${corestoreToken}`,
    'content-type': 'application/json',
    'content-length': String(databaseExportBytes.length),
    [coreStoreDatabaseExportPackageRequestHeader]: packageRequestBytes.toString('base64url'),
  },
  body: new Uint8Array(databaseExportBytes),
});
if (!packageHttpResponse.ok) throw new Error(`package failed: ${packageHttpResponse.status}`);

const encodedPackageResponse = packageHttpResponse.headers.get(
  coreStoreDatabaseExportPackageResponseHeader,
);
const encodedReceipt = packageHttpResponse.headers.get('x-corestore-database-backup-receipt');
if (!encodedPackageResponse || !encodedReceipt) throw new Error('package headers are missing');
const packageResponseBytes = Buffer.from(encodedPackageResponse, 'base64url');
if (packageResponseBytes.toString('base64url') !== encodedPackageResponse) {
  throw new Error('package response header is not canonical base64url');
}
const packageResponse = runtime.normalizeCorestoreDatabaseExportPackageResponseForRequest(
  JSON.parse(packageResponseBytes.toString('utf8')),
  packageRequest,
);
const closureBytes = Buffer.from(await packageHttpResponse.arrayBuffer());
if (
  closureBytes.length !== packageResponse.completion.closureByteLength
  || sha256(closureBytes) !== packageResponse.completion.closureSha256
) {
  throw new Error('package closure identity does not match its completion');
}
const receiptBytes = Buffer.from(
  runtime.encodeCorestoreDatabaseBackupReceipt(packageResponse.completion.receipt),
);
if (receiptBytes.toString('base64url') !== encodedReceipt) {
  throw new Error('package receipt header is not canonical or does not match the completion');
}

for (const [fileName, bytes] of [
  ['database.closure', closureBytes],
  ['receipt.json', receiptBytes],
] as const) {
  const handle = await open(fileName, 'wx', 0o600);
  try {
    await handle.writeFile(bytes);
    await handle.sync();
  } finally {
    await handle.close();
  }
}
const outputDirectory = await open('.', 'r');
try {
  await outputDirectory.sync();
} finally {
  await outputDirectory.close();
}

const releaseRequest: ICoreStoreDatabaseExportPackageReleaseRequest = {
  kind: 'corestore-database-export-package-release-request',
  packageAttemptId,
  requestSha256: packageResponse.completion.requestSha256,
  completionSha256: sha256(
    runtime.encodeCorestoreDatabaseExportPackageCompletion(packageResponse.completion),
  ),
  callerFsyncedClosureByteLength: closureBytes.length,
  callerFsyncedClosureSha256: sha256(closureBytes),
  callerFsyncedReceiptByteLength: receiptBytes.length,
  callerFsyncedReceiptSha256: sha256(receiptBytes),
  callerFsyncedAt: Math.max(Date.now(), packageResponse.completion.completedAt),
};
const releaseBytes = Buffer.from(
  runtime.encodeCorestoreDatabaseExportPackageReleaseRequest(releaseRequest),
);
const releaseRequestHandle = await open('release-request.json', 'wx', 0o600);
try {
  await releaseRequestHandle.writeFile(releaseBytes);
  await releaseRequestHandle.sync();
} finally {
  await releaseRequestHandle.close();
}
const releaseDirectory = await open('.', 'r');
try {
  await releaseDirectory.sync();
} finally {
  await releaseDirectory.close();
}
const releaseHttpResponse = await fetch(`${corestoreOrigin}${coreStoreDatabaseExportPackageReleasePath}`, {
  method: 'POST',
  headers: {
    authorization: `Bearer ${corestoreToken}`,
    'content-type': 'application/json',
    'content-length': String(releaseBytes.length),
  },
  body: new Uint8Array(releaseBytes),
});
if (!releaseHttpResponse.ok) throw new Error(`release failed: ${releaseHttpResponse.status}`);

After the first release POST, retries must read and resend the exact fsynced release-request.json bytes. Do not regenerate callerFsyncedAt after an ambiguous response because any byte change is conflicting reuse and returns 409 ATTEMPT_CONFLICT.

Named storage bindings

The storage-binding API consumes the shared @serve.zone/interfaces App Store request/class shapes and returns platform.TResolvedStorageBinding. The request contains portable policy only. Provider names, host paths, network shares, authentication mechanisms, mount options, endpoint overrides, and bucket names are rejected as unknown physical fields.

Example object-storage reconciliation:

{
  "bindingId": "binding:backup-archive",
  "serviceId": "Service:example",
  "generation": 1,
  "policy": {
    "classId": "corestore.object-storage.standard",
    "revision": "1"
  },
  "storageClass": {
    "kind": "objectStorage",
    "purpose": "backup",
    "required": {
      "durability": "persistent"
    }
  },
  "request": {
    "id": "backup-archive",
    "kind": "objectStorage",
    "storageClass": "archiveStorage",
    "reclaimPolicy": "retain",
    "accessMode": "readWrite",
    "delivery": {
      "type": "launcher-environment",
      "keys": {
        "endpoint": "ARCHIVE_S3_ENDPOINT",
        "bucket": "ARCHIVE_S3_BUCKET",
        "region": "ARCHIVE_S3_REGION",
        "accessKeyId": "ARCHIVE_S3_ACCESS_KEY_ID",
        "secretAccessKey": "ARCHIVE_S3_SECRET_ACCESS_KEY"
      }
    }
  },
  "credentialManagementScope": "platform:object-storage-binding"
}

The request digest is the contract's value, not a Corestore-local one: it is derived with the interfaces createStorageRequestSha256, a bare lowercase hex SHA-256 over the canonical request alone, so a consumer that re-derives it from the request it declared reaches the same digest. Callers cannot supply one. A request is only admitted if that function can digest it, because the request subtree is normalized by the interfaces normalizeStorageRequest, the very judgement createStorageRequestSha256 derives the digest from: delivery environment keys match ^[A-Z_][A-Z0-9_]{0,255}$, a declared protection block carries at least one member, and no member may be present with an undefined value. A contract refusal is answered as a malformed request (400) carrying the contract's own message, which names the failing field. On top of the contract Corestore applies only the rules it owns: the request id ^[a-z][a-z0-9-]{0,62}$ and class key ^[A-Za-z][A-Za-z0-9-]{0,62}$ that name derived volumes, buckets and credential resources, a single credential file directly below /run/secrets that is never the platform's own /run/secrets/secret.json, and sessionToken delivery refused as an unsupported capability (422) because Corestore issues long-lived pool credentials and has no session-token source. Stable (serviceId, request.id) and binding identities are unique. The same generation and intent is idempotent, stale generations fail, and changed intent requires an explicit migration. Binding records are strictly revalidated on every control-record read and write, updated through revision compare-and-swap, and fenced by operation leases. The value-free credentialManagementScope describes platform ownership without storing or coupling the binding to a secret record.

Release is a durable replay-safe lifecycle: the release intent is recorded before provider deletion, provider deletion is idempotently retried after a crash, and terminal retained or released tombstones prevent allocation-id reuse. Reconcile cannot resume after release starts. Provider and persistence details are reduced to fixed failure messages; binding responses and persisted failures never expose provider credential material or raw provider errors.

Object-storage placement is the selected policy class revision: portable workload requirements never contain host paths, NFS exports, SMB shares, or provider endpoint overrides. Pool catalog records, binding placement, migration ownership, and migration state are authoritative control records and survive process restarts.

Changing a live object's class is an explicit migration, never an in-place reconcile. Corestore creates a hidden destination, copies every object with exact create-only publication and preserved supported metadata, asks the orchestrator to quiesce the consumer, establishes a source seal, verifies the destination manifest, publishes the durable binding cutover, waits for consumer activation evidence, and only then performs fenced source cleanup. Abort is allowed only before the commit point; after cutover, recovery rolls forward.

The configured classes deliberately advertise only guarantees the named lifecycle enforces:

  • Filesystem: persistent, single-node, ReadWriteOnce, standard performance.
  • Default object storage: persistent, single-node, read-write, standard performance, with optional compliance retention.
  • Additional object-storage pools may advertise exactly one of standard, high-IOPS, or capacity performance when the operator maps that class to a matching physical backend.
  • No hard quotas, snapshot/backup guarantee, versioning, read-only object policy, multi-node topology, or transport-encryption guarantee yet.
  • Every hard capacity quantity is rejected until Corestore can reserve and report an enforced byte limit. Session-token delivery is rejected until Corestore can issue actual session credentials.

Reconcile, get, and list responses never contain provider credential values. The separate credentials endpoint requires the current service id and generation, returns semantic credential material, and is non-cacheable. The fulfillment adapter maps that material to the App Store request's launcher-environment or file delivery contract.

Runtime credential bindings

The credential-binding API is the Coreflow handoff for @serve.zone/interfaces 32. POST /control/credential-bindings/reconcile accepts only the exact value-free binding request — every object-storage request bound to its durable bucketName — and its current publication grant. GET /control/credential-bindings, optionally filtered by serviceId, and GET /control/credential-bindings/<id> return status and provider authority without credential values. POST /control/credential-bindings/<id>/material requires the exact current grant and is the only route that returns plaintext; successful, authentication-failure, and error responses are non-cacheable.

Legacy ordinary resources with credentials already delivered through /resources/provision use a separate, explicit rotation API. POST /control/ordinary-credentials/rotate accepts { "serviceId": "...", "capability": "database" | "objectstorage", "operationId": "...", "expectedGeneration": 0 }; absence of an ordinary rotation record means generation zero. The response is { "receipt": { "serviceId": "...", "capability": "...", "resourceName": "...", "operationId": "...", "previousGeneration": 0, "generation": 1, "phase": "active" } }, without credentials. Exact operation replay returns the same current receipt; stale generations, conflicting operation identities, allocation-managed databases and binding-managed resources fail closed. Corestore claims the operation through SmartData CAS, verifies a fenced provider ensure without deleting documents or objects, persists the new generation, updates its manifest, and recovers interrupted phases on startup. For object storage the bucket access-key ID stays stable while the secret changes; the old key/secret pair stops authenticating.

Only after durably storing that receipt should a consumer call authenticated POST /control/ordinary-credentials/material with { "receipt": <exact receipt> }. It returns { "material": { "capability": "...", "generation": 1, "env": { ... } } } with the canonical database or S3 environment values. Corestore attests the current provider credential and exact manifest before delivery. Both routes send Cache-Control: no-store and Pragma: no-cache. A consumer must durably refresh its own resource record and workload secrets before restarting writers; Corestore does not mutate consumer state.

Corestore derives database and object-storage credentials from its protected master secret and never persists them in binding records. Every active material read replays the exact NoSQLDB ensure fence to reapply and prove the expected database password, username, roles, and receipt, or validates the exact ObjectStorage bucket credential and retention evidence. Generic provision, import, restore, migration cleanup, and deprovision paths are rejected when their target is credential-managed.

Interfaces 32 defines publication grants but no deletion grant. Corestore therefore has retain-only credential lifecycle authority: /retain writes a permanent tombstone and leaves the database or bucket intact. Omission from desired state, disablement, grant expiry, process restart, and retained status never authorize provider deletion or credential revocation. A retained resource remains protected from generic mutation and cannot be reconciled or have material delivered again.

Each isolated-restore request body carries a restoreGrant compact JWT in addition to its exact request fields. Grants use RS256, identify a configured key by kid, authorize exactly one operation, expire within five minutes by default, bind the target cluster and node, and bind the canonical resource-mapping and archive-manifest hashes. Corestore rejects a grant when any bound request value differs.

Snapshot a volume:

curl -X POST http://corestore:3000/volumes/snapshot \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer <CORESTORE_API_TOKEN>' \
  -d '{"name":"sz-api-data-abc123","snapshotName":"before-deploy"}'

Restore a volume snapshot:

curl -X POST http://corestore:3000/volumes/restore \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer <CORESTORE_API_TOKEN>' \
  -d '{"name":"sz-api-data-abc123","snapshotId":"<snapshot-id>","clear":true}'

Docker Volume Driver

Corestore implements these Docker VolumeDriver endpoints over its Unix socket:

  • /Plugin.Activate
  • /VolumeDriver.Capabilities
  • /VolumeDriver.Create
  • /VolumeDriver.Remove
  • /VolumeDriver.Mount
  • /VolumeDriver.Unmount
  • /VolumeDriver.Path
  • /VolumeDriver.Get
  • /VolumeDriver.List

The Corestore service must bind mount /run/docker/plugins from the host so Docker can discover /run/docker/plugins/corestore.sock.

Volume mountpoints are real host directories under:

<CORESTORE_DATA_DIR>/volumes/<safe-volume-name>-<hash>/data

Docker bind-mounts those paths into workload containers. Corestore tracks mount ids, service metadata, backup flags, and snapshot history in corestore-manifest.json.

Coreflow Integration

The intended platform behavior is:

  • Coreflow deploys Corestore as a global service so each workload node has a local storage provider.
  • New portable filesystem and object-storage requests are matched against /control/storage/capabilities and reconciled through stable v2 binding ids.
  • Coreflow obtains object-storage credential values only from the authenticated non-cacheable credentials endpoint, then applies the declared launcher-environment or file delivery contract.
  • Coreflow reconciles Interfaces 32 runtime credential bindings through /control/credential-bindings/reconcile, observes them through the list/status routes, obtains values only from /<id>/material, and calls /<id>/retain for omission or disablement; it must not infer deletion authority from an expired or absent grant.
  • The resolved filesystem resourceRef is handed to the runtime volume attachment; physical attachment details do not enter App Store manifests.
  • Workload platform bindings for database and objectstorage call /resources/provision.
  • First-class workload volumes use Docker DriverConfig.Name = 'corestore' by default.
  • Backup orchestration snapshots volumes through /volumes/snapshot and service resources through /resources/snapshot; portable database transfer uses /resources/database/export.
  • Ordinary local restore orchestration uses /volumes/restore, /resources/restore, and archive object read/write endpoints. Database restore is internally routed through the same durable import/commit fence as portable handoff.
  • Authorized scratch-environment recovery uses the signed /isolated-restore/* prepare, object-write, execute, status, and cleanup sequence. Each phase receives a new operation-scoped grant from the control plane.

Docker Image

pnpm run build:docker

The image exposes 3000, 9000, and 27017 and stores runtime data under /data/corestore unless CORESTORE_DATA_DIR overrides it.

The image is built from Dockerfile_##version##, so tsdocker publishes it under the release version tag only — code.foss.global/serve.zone/corestore:<version> with the version from package.json. From 32.0.0 on Corestore publishes no latest tag; the existing corestore:latest stays frozen at the 7.0.0 build for consumers that still pull it by that name, so pin Corestore by version. Both version labels stay out of the Dockerfile on purpose: tsdocker injects version and org.opencontainers.image.version from package.json on every build path, and a literal here would drift from what actually shipped.

Development

Common commands:

pnpm install
pnpm build
pnpm test
pnpm run watch

Important files:

Path Purpose
ts/index.ts CLI startup wrapper exporting CoreStore, runCli, and stop.
ts/corestore.classes.corestore.ts Main runtime, control API, VolumeDriver API, provisioning, snapshots, and archive replication.
ts/corestore.interfaces.ts Request, response, manifest, resource, and snapshot types.
ts/corestore.storage.ts Strict portable v2 contract normalization, capability negotiation, digests, and collision checks.
ts/corestore.classes.storagebindingstore.ts Authoritative named-binding records and unique identities, persisted through the NoSQL client.
ts/corestore.objectstoragepools.ts Strict physical pool configuration, portable class mapping, and immutable pool identities.
ts/corestore.plugins.ts Centralized dependency imports.
ts/corestore.classes.controlauth.ts Strict bearer-token and token-file validation.
ts/corestore.classes.restoregrant.ts Signed isolated-restore grant verification and node binding.
ts/corestore.classes.isolatedrestorecoordinator.ts Durable leases, fencing, receipts, reconciliation, and crash recovery.
ts/corestore.classes.safetar.ts Bounded fail-closed archive extraction.
ts_migration/runner.ts Ordered, idempotent persisted-data migrations run at start-up, with completion recorded in the control database.
ts_migration/legacy-mongodb-to-smartdb.ts Explicit offline MongoDB-wire to NoSQLDB migration with compatibility preflight, publication fencing, and exact verification.
migration.js Operator-only migration entrypoint included in the container image and package.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in license.md.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

Company Information

Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

Reference in New Issue View Git Blame Copy Permalink
S
Description
Node-local SmartDB, S3-compatible object storage, volume, snapshot, and restore provider.
Readme
9.5 MiB
Languages
TypeScript 91.5%
HTML 6.4%
JavaScript 1.9%
Shell 0.2%