feat(listing): Add memory-efficient listing APIs: async generator, RxJS observable, and cursor pagination; export ListCursor and Minimatch; add minimatch dependency; bump to 4.2.0
This commit is contained in:
55
changelog.md
55
changelog.md
@@ -1,5 +1,60 @@
|
||||
# Changelog
|
||||
|
||||
## 2025-11-20 - 4.3.0 - feat(listing)
|
||||
Add memory-efficient listing APIs: async generator, RxJS observable, and cursor pagination; export ListCursor and Minimatch; add minimatch dependency; bump to 4.2.0
|
||||
|
||||
- Added memory-efficient listing methods on Bucket: listAllObjects (async generator), listAllObjectsObservable (RxJS Observable), createCursor (returns ListCursor) and listAllObjectsArray (convenience array collector).
|
||||
- New ListCursor class (ts/classes.listcursor.ts) providing page-based iteration: next(), hasMore(), reset(), getToken()/setToken().
|
||||
- Added glob matching helper findByGlob(pattern) using minimatch (exported via plugins.Minimatch).
|
||||
- Exported ListCursor from ts/index.ts and exported Minimatch via ts/plugins.ts.
|
||||
- Added minimatch dependency in package.json and bumped package version to 4.2.0; increased test timeout to 120s.
|
||||
- Updated tests to read S3_SECRETKEY, S3_PORT and to assert bucket name from env (test/test.node+deno.ts, test/test.trash.node+deno.ts).
|
||||
- No breaking changes: new APIs are additive and existing behavior preserved.
|
||||
|
||||
## 2025-11-20 - 4.2.0 - feat(listing)
|
||||
Add memory-efficient listing with async generators, RxJS observables, and cursor pagination for huge buckets
|
||||
|
||||
**New Memory-Efficient Listing Methods:**
|
||||
|
||||
**Async Generator (Recommended for most use cases):**
|
||||
- `Bucket.listAllObjects(prefix?)` - Stream object keys one at a time using `for await...of`
|
||||
- `Bucket.findByGlob(pattern)` - Find objects matching glob patterns (e.g., `**/*.json`, `npm/packages/*/index.json`)
|
||||
- Memory efficient, supports early termination, composable
|
||||
|
||||
**RxJS Observable (For complex reactive pipelines):**
|
||||
- `Bucket.listAllObjectsObservable(prefix?)` - Emit keys as Observable for use with RxJS operators (filter, map, take, etc.)
|
||||
- Perfect for complex data transformations and reactive architectures
|
||||
|
||||
**Cursor Pattern (For manual pagination control):**
|
||||
- `Bucket.createCursor(prefix?, options?)` - Create cursor for explicit page-by-page iteration
|
||||
- `ListCursor.next()` - Fetch next page of results
|
||||
- `ListCursor.hasMore()` - Check if more results available
|
||||
- `ListCursor.reset()` - Reset to beginning
|
||||
- `ListCursor.getToken()` / `ListCursor.setToken()` - Save/restore pagination state
|
||||
- Ideal for UI pagination and resumable operations
|
||||
|
||||
**Convenience Methods:**
|
||||
- `Bucket.listAllObjectsArray(prefix?)` - Collect all keys into array (WARNING: loads all into memory)
|
||||
|
||||
**Benefits:**
|
||||
- ✅ Memory-efficient streaming for buckets with millions of objects
|
||||
- ✅ Three patterns for different use cases (generators, observables, cursors)
|
||||
- ✅ Support for early termination and incremental processing
|
||||
- ✅ Glob pattern matching with minimatch
|
||||
- ✅ Full TypeScript support with proper types
|
||||
- ✅ Zero breaking changes - all new methods
|
||||
|
||||
**Dependencies:**
|
||||
- Added `minimatch` for glob pattern support
|
||||
|
||||
**Files Changed:**
|
||||
- `ts/classes.bucket.ts` - Added all listing methods
|
||||
- `ts/classes.listcursor.ts` - NEW: Cursor implementation
|
||||
- `ts/plugins.ts` - Export Minimatch
|
||||
- `ts/index.ts` - Export ListCursor
|
||||
- `test/test.listing.node+deno.ts` - NEW: Comprehensive listing tests
|
||||
- `package.json` - Added minimatch dependency
|
||||
|
||||
## 2025-11-20 - 4.1.0 - feat(core)
|
||||
Add S3 endpoint normalization, directory pagination, improved metadata checks, trash support, and related tests
|
||||
|
||||
|
||||
Reference in New Issue
Block a user