Files

Juergen Kunz 1f4b7319d3 feat(core): Add S3 endpoint normalization, directory pagination, improved metadata checks, trash support, and related tests

2025-11-20 13:58:02 +00:00

14 KiB

Raw Blame History

Changelog

2025-11-20 - 4.1.0 - feat(core)

Add S3 endpoint normalization, directory pagination, improved metadata checks, trash support, and related tests

Add normalizeS3Descriptor helper to sanitize and normalize various S3 endpoint formats and emit warnings for mismatches (helpers.ts).
Use normalized endpoint and credentials when constructing S3 client in SmartBucket (classes.smartbucket.ts).
Implement paginated listing helper listObjectsV2AllPages in Directory and use it for listFiles and listDirectories to aggregate Contents and CommonPrefixes across pages (classes.directory.ts).
Improve MetaData.hasMetaData to catch NotFound errors and return false instead of throwing (classes.metadata.ts).
Export metadata and trash modules from index (ts/index.ts) and add a Trash class with utilities for trashed files and key encoding (classes.trash.ts).
Enhance Bucket operations: fastCopy now preserves or replaces native metadata correctly, cleanAllContents supports paginated deletion, and improved fastExists error handling (classes.bucket.ts).
Fix Directory.getSubDirectoryByName to construct new Directory instances with the correct parent directory reference.
Add tests covering metadata absence and pagination behavior (test/test.local.node+deno.ts).

2025-11-20 - 4.0.1 - fix(plugins)

Use explicit node: imports for native path and stream modules in ts/plugins.ts

Replaced imports of 'path' and 'stream' with 'node:path' and 'node:stream' in ts/plugins.ts.
Ensures correct ESM resolution of Node built-ins when package.json type is 'module' and avoids accidental conflicts with userland packages.

2025-11-20 - 4.0.0 - BREAKING CHANGE(core)

Make API strict-by-default: remove *Strict variants, throw on not-found/exists conflicts, add explicit exists() methods, update docs/tests and bump deps

Breaking: Core API methods are strict by default and now throw errors instead of returning null when targets are missing or already exist (e.g. getBucketByName, getFile, getSubDirectoryByName, fastPut, fastPutStream).
Removed *Strict variants: fastPutStrict, getBucketByNameStrict, getFileStrict, getSubDirectoryByNameStrict — use the base methods which are now strict.
Added explicit existence checks: bucketExists (SmartBucket), fileExists (Directory/fileExists), directoryExists (Directory.directoryExists), and fastExists (Bucket.fastExists) to allow non-throwing checks before operations.
Return type updates: fastPut now returns Promise (no null), getBucketByName/getFile/getSubDirectoryByName now return the respective objects or throw.
Improved error messages to guide callers (e.g. suggest setting overwrite:true on fastPut when object exists).
Updated README, changelog and tests to reflect the new strict semantics and usage patterns.
Developer/runtime dependency bumps: @git.zone/tsbuild, @git.zone/tsrun, @git.zone/tstest, @aws-sdk/client-s3, @push.rocks/smartstring, @tsclass/tsclass (version bumps recorded in package.json).
Major version bump to 4.0.0 to reflect breaking API changes.

2025-11-20 - 4.0.0 - BREAKING: Strict by default + exists methods

Complete API overhaul: all methods throw by default, removed all *Strict variants, added dedicated exists methods

Breaking Changes:

Putters (Write Operations):

fastPut: Return type Promise<File | null> → Promise<File>, throws when file exists and overwrite is false
fastPutStream: Now throws when file exists and overwrite is false (previously returned silently)
fastPutStrict: Removed - use fastPut directly

Getters (Read Operations):

getBucketByName: Return type Promise<Bucket | null> → Promise<Bucket>, throws when bucket not found
getBucketByNameStrict: Removed - use getBucketByName directly
getFile: Return type Promise<File | null> → Promise<File>, throws when file not found
getFileStrict: Removed - use getFile directly
getSubDirectoryByName: Return type Promise<Directory | null> → Promise<Directory>, throws when directory not found
getSubDirectoryByNameStrict: Removed - use getSubDirectoryByName directly

New Methods (Existence Checks):

bucket.fastExists({ path }) - ✅ Already existed
directory.fileExists({ path }) - NEW - Check if file exists
directory.directoryExists(name) - NEW - Check if subdirectory exists
smartBucket.bucketExists(name) - NEW - Check if bucket exists

Benefits:

✅ Simpler API: Removed 4 redundant *Strict methods
✅ Type-safe: No nullable returns - Promise<T> not Promise<T | null>
✅ Fail-fast: Errors throw immediately with precise stack traces
✅ Consistent: All methods behave the same way
✅ Explicit: Use exists() to check, then get() to retrieve
✅ Better debugging: Error location is always precise

Migration Guide:

// ============================================
// Pattern 1: Check then Get (Recommended)
// ============================================

// Before (v3.x):
const bucket = await smartBucket.getBucketByName('my-bucket');
if (bucket) {
  // use bucket
}

// After (v4.0):
if (await smartBucket.bucketExists('my-bucket')) {
  const bucket = await smartBucket.getBucketByName('my-bucket'); // guaranteed to exist
  // use bucket
}

// ============================================
// Pattern 2: Try/Catch
// ============================================

// Before (v3.x):
const file = await directory.getFile({ path: 'file.txt' });
if (!file) {
  // Handle not found
}

// After (v4.0):
try {
  const file = await directory.getFile({ path: 'file.txt' });
  // use file
} catch (error) {
  // Handle not found
}

// ============================================
// Pattern 3: Remove *Strict calls
// ============================================

// Before (v3.x):
const file = await directory.getFileStrict({ path: 'file.txt' });

// After (v4.0):
const file = await directory.getFile({ path: 'file.txt' }); // already strict

// ============================================
// Pattern 4: Write Operations
// ============================================

// Before (v3.x):
const file = await bucket.fastPutStrict({ path: 'file.txt', contents: 'data' });

// After (v4.0):
const file = await bucket.fastPut({ path: 'file.txt', contents: 'data' }); // already strict

2025-08-18 - 3.3.10 - fix(helpers)

Normalize and robustly parse S3 endpoint configuration; use normalized descriptor in SmartBucket and update dev tooling

Add normalizeS3Descriptor to ts/helpers.ts: robust endpoint parsing, coercion of useSsl/port, sanitization, warnings for dropped URL parts, and canonical endpoint URL output.
Update SmartBucket (ts/classes.smartbucket.ts) to use the normalized endpoint, region, credentials and forcePathStyle from normalizeS3Descriptor.
Adjust dev tooling: bump @git.zone/tsbuild -> ^2.6.7, @git.zone/tstest -> ^2.3.4, @push.rocks/qenv -> ^6.1.3 and update test script to run tstest with --verbose --logfile --timeout 60.
Add .claude/settings.local.json containing local assistant/CI permission settings (local config only).

2025-08-15 - 3.3.9 - fix(docs)

Revise README with detailed usage examples and add local Claude settings

Revamped README: reorganized content, added emojis and clearer headings for install, getting started, bucket/file/directory operations, streaming, metadata, trash/recovery, locking, and advanced configuration.
Added many concrete code examples for SmartBucket, Bucket, Directory, File, streaming (node/web), RxJS replay subjects, metadata handling, trash workflow, file locking, magic-bytes detection, JSON operations, and cleaning bucket contents.
Included testing instructions (pnpm test) and a Best Practices section with recommendations for strict mode, streaming, metadata, trash usage, and locking.
Added .claude/settings.local.json to include local Claude configuration and tool permissions.
No source code or public API changes; documentation and local tooling config only.

2025-08-15 - 3.3.8 - fix(tests)

Update tests to use @git.zone/tstest, upgrade dependencies, remove GitLab CI and add local CI/workspace config

Tests: replace imports from @push.rocks/tapbundle with @git.zone/tstest/tapbundle and switch tap.start() to export default tap.start()
Dependencies: bump @aws-sdk/client-s3 and several @push.rocks packages; upgrade @tsclass/tsclass to a newer major
DevDependencies: upgrade @git.zone/tsbuild, @git.zone/tstest, @push.rocks/qenv, and @push.rocks/tapbundle
CI/config: remove .gitlab-ci.yml, add .claude/settings.local.json
Workspace: add pnpm-workspace.yaml and packageManager field in package.json

2024-12-02 - 3.3.7 - fix(package)

Update author field in package.json

Corrected the author field from 'Lossless GmbH' to 'Task Venture Capital GmbH' in the package.json file.

2024-12-02 - 3.3.6 - fix(package)

Fix license field in package.json to reflect MIT licensing

2024-11-25 - 3.3.5 - fix(test)

Refactor trash test to improve metadata validation

Added new checks in trash tests to ensure metadata files are correctly moved to trash.
Validated the presence and integrity of metadata within trashed files.

2024-11-25 - 3.3.4 - fix(core)

Minor refactoring and cleanup of TypeScript source files for improved readability and maintainability.

2024-11-24 - 3.3.3 - fix(documentation)

Improved documentation accuracy and consistency

Updated the project description to reflect the cloud-agnostic nature and advanced capabilities
Enhanced the README with detailed explanations and code examples for advanced features like trash management
Clarified the handling and importance of metadata using the MetaData utility

2024-11-24 - 3.3.2 - fix(documentation)

Updated keywords and description for clarity and consistency.

Modified keywords and description in package.json and npmextra.json.
Enhanced readme.md file structure and examples

2024-11-24 - 3.3.1 - fix(File)

Fixed issue with file restore metadata operations.

Corrected the order of operations in the file restore function to ensure custom metadata is appropriately deleted after moving the file.

2024-11-24 - 3.3.0 - feat(core)

Enhanced directory handling and file restoration from trash

Refined getSubDirectoryByName to handle file paths treated as directories.
Introduced file restoration function from trash to original or specified paths.

2024-11-24 - 3.2.2 - fix(core)

Refactor Bucket class for improved error handling

Ensured safe access using non-null assertions when finding a bucket.
Enhanced fastPut method by adding fastPutStrict for safer operations.
Added explicit error handling and type checking in fastExists method.

2024-11-24 - 3.2.1 - fix(metadata)

Fix metadata handling for deleted files

Ensured metadata is correctly stored and managed when files are deleted into the trash.

2024-11-24 - 3.2.0 - feat(bucket)

Enhanced SmartBucket with trash management and metadata handling

Added functionality to move files to a trash directory.
Introduced methods to handle file metadata more robustly.
Implemented a method to clean all contents from a bucket.
Enhanced directory retrieval to handle non-existent directories with options.
Improved handling of file paths and metadata within the storage system.

2024-11-18 - 3.1.0 - feat(file)

Added functionality to retrieve magic bytes from files and detect file types using magic bytes.

Introduced method getMagicBytes in File and Bucket classes to retrieve a specific number of bytes from a file.
Enhanced file type detection by utilizing magic bytes in MetaData class.
Updated dependencies for better performance and compatibility.

2024-11-18 - 3.0.24 - fix(metadata)

Fix metadata handling to address type assertion and data retrieval.

Fixed type assertion issues in MetaData class properties with type non-null assertions.
Corrected the handling of JSON data retrieval in MetaData.storeCustomMetaData function.

2024-10-16 - 3.0.23 - fix(dependencies)

Update package dependencies for improved functionality and security.

Updated @aws-sdk/client-s3 to version ^3.670.0 for enhanced S3 client capabilities.
Updated @push.rocks/smartstream to version ^3.2.4.
Updated the dev dependency @push.rocks/tapbundle to version ^5.3.0.

2024-07-28 - 3.0.22 - fix(dependencies)

Update dependencies and improve bucket retrieval logging

Updated @aws-sdk/client-s3 to ^3.620.0
Updated @git.zone/tsbuild to ^2.1.84
Updated @git.zone/tsrun to ^1.2.49
Updated @push.rocks/smartpromise to ^4.0.4
Updated @tsclass/tsclass to ^4.1.2
Added a log for when a bucket is not found by name in getBucketByName method

2024-07-04 - 3.0.21 - fix(test)

Update endpoint configuration in tests to use environment variable

Modified qenv.yml to include S3_ENDPOINT as a required environment variable.
Updated test files to fetch S3_ENDPOINT from environment instead of hardcoding.

2024-06-19 - 3.0.20 - Fix and Stability Updates

Improved overall stability and consistency.