smartmigration/readme.hints.md

readme.hints.md

Internal development notes for @push.rocks/smartmigration. This file is not published to npm.

Architecture

• The ledger is the source of truth. Two backends: mongo (via smartdata EasyStore) and S3 (via bucket.fastPut/fastGet).
• Step execution is sequential and registration-ordered. The from/to fields exist for chain validation, not for DAG resolution.
• Drivers (mongodb.Db, S3Client) are exposed via ctx, not wrapped. smartmigration writes no SQL or S3 wrappers of its own.

Locking caveats

• The mongo lock uses a read-modify-write pattern over EasyStore (which writes the entire document each time). This is a last-writer-wins CAS, not a true atomic CAS. It's adequate for v1; v2 may move to findOneAndUpdate against the underlying mongo collection.
• The S3 lock is best-effort (S3 has no conditional writes without versioning). S3-only deployments should not run multiple instances against the same ledger without external coordination.

Why EasyStore (and not a custom collection)?

• Already exists, lazy collection setup, single-document storage that fits the entire ledger naturally.
• Avoids polluting the user's DB with smartmigration-internal classes.
• See /mnt/data/lossless/push.rocks/smartdata/ts/classes.easystore.ts for the implementation.

Testing notes

• Use @push.rocks/smartmongo's SmartMongo.createAndStart() for an in-memory replica set.
• After createAndStart(), call getMongoDescriptor() and feed the result to new SmartdataDb(...).
• Always call db.close() and smartMongo.stop() in test cleanup.