feat(smartmigration): add initial smartmigration package with MongoDB and S3 migration runner
This commit is contained in:
26
readme.hints.md
Normal file
26
readme.hints.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# readme.hints.md
|
||||
|
||||
Internal development notes for `@push.rocks/smartmigration`. This file is not published to npm.
|
||||
|
||||
## Architecture
|
||||
|
||||
- The ledger is the source of truth. Two backends: mongo (via smartdata `EasyStore`) and S3 (via `bucket.fastPut`/`fastGet`).
|
||||
- Step execution is **sequential and registration-ordered**. The `from`/`to` fields exist for chain validation, not for DAG resolution.
|
||||
- Drivers (`mongodb.Db`, `S3Client`) are exposed via `ctx`, not wrapped. smartmigration writes no SQL or S3 wrappers of its own.
|
||||
|
||||
## Locking caveats
|
||||
|
||||
- The mongo lock uses a read-modify-write pattern over `EasyStore` (which writes the entire document each time). This is a last-writer-wins CAS, not a true atomic CAS. It's adequate for v1; v2 may move to `findOneAndUpdate` against the underlying mongo collection.
|
||||
- The S3 lock is best-effort (S3 has no conditional writes without versioning). S3-only deployments should not run multiple instances against the same ledger without external coordination.
|
||||
|
||||
## Why EasyStore (and not a custom collection)?
|
||||
|
||||
- Already exists, lazy collection setup, single-document storage that fits the entire ledger naturally.
|
||||
- Avoids polluting the user's DB with smartmigration-internal classes.
|
||||
- See `/mnt/data/lossless/push.rocks/smartdata/ts/classes.easystore.ts` for the implementation.
|
||||
|
||||
## Testing notes
|
||||
|
||||
- Use `@push.rocks/smartmongo`'s `SmartMongo.createAndStart()` for an in-memory replica set.
|
||||
- After `createAndStart()`, call `getMongoDescriptor()` and feed the result to `new SmartdataDb(...)`.
|
||||
- Always call `db.close()` and `smartMongo.stop()` in test cleanup.
|
||||
Reference in New Issue
Block a user