Compare commits

...

10 Commits

9 changed files with 140 additions and 55 deletions

View File

@ -1,5 +1,31 @@
# Changelog
## 2024-12-02 - 3.3.7 - fix(package)
Update author field in package.json
- Corrected the author field from 'Lossless GmbH' to 'Task Venture Capital GmbH' in the package.json file.
## 2024-12-02 - 3.3.6 - fix(package)
Fix license field in package.json to reflect MIT licensing
## 2024-11-25 - 3.3.5 - fix(test)
Refactor trash test to improve metadata validation
- Added new checks in trash tests to ensure metadata files are correctly moved to trash.
- Validated the presence and integrity of metadata within trashed files.
## 2024-11-25 - 3.3.4 - fix(core)
Minor refactoring and cleanup of TypeScript source files for improved readability and maintainability.
## 2024-11-24 - 3.3.3 - fix(documentation)
Improved documentation accuracy and consistency
- Updated the project description to reflect the cloud-agnostic nature and advanced capabilities
- Enhanced the README with detailed explanations and code examples for advanced features like trash management
- Clarified the handling and importance of metadata using the MetaData utility
## 2024-11-24 - 3.3.2 - fix(documentation)
Updated keywords and description for clarity and consistency.

19
license Normal file
View File

@ -0,0 +1,19 @@
Copyright (c) 2014 Task Venture Capital GmbH (hello@task.vc)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -8,32 +8,30 @@
"githost": "code.foss.global",
"gitscope": "push.rocks",
"gitrepo": "smartbucket",
"description": "A TypeScript library facilitating cloud-agnostic object storage with capabilities such as bucket management, file operations, directory management, and advanced data streaming functionalities.",
"description": "A TypeScript library providing a cloud-agnostic interface for managing object storage with functionalities like bucket management, file and directory operations, and advanced features such as metadata handling and file locking.",
"npmPackagename": "@push.rocks/smartbucket",
"license": "MIT",
"keywords": [
"TypeScript",
"cloud storage",
"cloud agnostic",
"object storage",
"bucket management",
"file operations",
"directory management",
"data streaming",
"multi-cloud",
"S3",
"minio",
"API",
"unified storage",
"multi-cloud",
"file locking",
"metadata management",
"buffer handling",
"access control",
"cloud agnostic",
"data streaming",
"environment configuration",
"unified storage",
"bucket policies",
"trash management",
"file transfer",
"data management",
"streaming",
"environment configuration"
"streaming"
]
}
},

4
package-lock.json generated
View File

@ -1,12 +1,12 @@
{
"name": "@push.rocks/smartbucket",
"version": "3.3.2",
"version": "3.3.7",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@push.rocks/smartbucket",
"version": "3.3.2",
"version": "3.3.7",
"license": "UNLICENSED",
"dependencies": {
"@push.rocks/smartpath": "^5.0.18",

View File

@ -1,12 +1,12 @@
{
"name": "@push.rocks/smartbucket",
"version": "3.3.2",
"description": "A TypeScript library facilitating cloud-agnostic object storage with capabilities such as bucket management, file operations, directory management, and advanced data streaming functionalities.",
"version": "3.3.7",
"description": "A TypeScript library providing a cloud-agnostic interface for managing object storage with functionalities like bucket management, file and directory operations, and advanced features such as metadata handling and file locking.",
"main": "dist_ts/index.js",
"typings": "dist_ts/index.d.ts",
"type": "module",
"author": "Lossless GmbH",
"license": "UNLICENSED",
"author": "Task Venture Capital GmbH",
"license": "MIT",
"scripts": {
"test": "(tstest test/)",
"build": "(tsbuild --web --allowimplicitany)"
@ -47,27 +47,25 @@
],
"keywords": [
"TypeScript",
"cloud storage",
"cloud agnostic",
"object storage",
"bucket management",
"file operations",
"directory management",
"data streaming",
"multi-cloud",
"S3",
"minio",
"API",
"unified storage",
"multi-cloud",
"file locking",
"metadata management",
"buffer handling",
"access control",
"cloud agnostic",
"data streaming",
"environment configuration",
"unified storage",
"bucket policies",
"trash management",
"file transfer",
"data management",
"streaming",
"environment configuration"
"streaming"
],
"homepage": "https://code.foss.global/push.rocks/smartbucket",
"repository": {

View File

@ -1 +1,3 @@
* The project uses the official s3 client, not the minio client.
* notice the difference between *Strict methods and the normal methods.
* metadata is handled though the MetaData class. Important!

View File

@ -1,7 +1,7 @@
```markdown
# @push.rocks/smartbucket
A TypeScript library offering simple and cloud-agnostic object storage with advanced features like bucket creation, file and directory management, and data streaming.
A comprehensive TypeScript library for cloud-agnostic object storage offering bucket management, file operations, and advanced data streaming.
## Install
@ -15,6 +15,10 @@ This command will add `@push.rocks/smartbucket` to your project's dependencies a
## Usage
### Introduction
`@push.rocks/smartbucket` provides a robust set of features to manage cloud storage operations in a cloud-agnostic manner. By leveraging this library, you can seamlessly interact with object storage services like AWS S3, without being tied to any vendor-specific implementations. This library not only abstracts basic file operations but also integrates advanced capabilities such as metadata management, data streaming, file locking, and bucket policies, all through a simplified API.
### Table of Contents
1. [Setting Up](#setting-up)
@ -34,11 +38,12 @@ This command will add `@push.rocks/smartbucket` to your project's dependencies a
- [Bucket Policies](#bucket-policies)
- [Metadata Management](#metadata-management)
- [File Locking](#file-locking)
- [Trash Management](#trash-management)
6. [Cloud Agnosticism](#cloud-agnosticism)
### Setting Up
Start by setting up `@push.rocks/smartbucket` in a TypeScript file, ensuring your project uses ECMAScript modules:
Begin by importing the necessary classes from the `@push.rocks/smartbucket` package into your TypeScript file. Create an instance of `SmartBucket` with your storage configuration:
```typescript
import {
@ -57,13 +62,13 @@ const mySmartBucket = new SmartBucket({
});
```
Replace `"yourAccessKey"`, `"yourSecretKey"`, and `"yourEndpointURL"` with appropriate values for your cloud storage service.
Replace `"yourAccessKey"`, `"yourSecretKey"`, and `"yourEndpointURL"` with actual data specific to your cloud provider.
### Working with Buckets
#### Creating a New Bucket
To create a new bucket, use the `createBucket` method. Remember that bucket names must be unique across the storage service:
Creating a bucket involves invoking the `createBucket` method. Note that bucket names are unique and follow the rules of the cloud provider:
```typescript
async function createBucket(bucketName: string) {
@ -80,11 +85,11 @@ createBucket("myNewBucket");
#### Listing Buckets
SmartBucket allows you to manage buckets but relies on the cloud provider's SDK for listing them.
While the library uses cloud-provider capabilities like AWS SDK to list existing buckets, `smartbucket` is aimed at simplifying content management within them.
#### Deleting Buckets
You can delete a bucket using the `removeBucket` method:
To delete a bucket, simply call the `removeBucket` function:
```typescript
async function deleteBucket(bucketName: string) {
@ -96,20 +101,22 @@ async function deleteBucket(bucketName: string) {
}
}
deleteBucket("myNewBucket");
deleteBucket("anotherBucketName");
```
### File Operations in Buckets
SmartBucket offers a unified API to execute file-based operations efficiently.
#### Uploading Files
To upload a file to a bucket, use the `fastPut` method:
Upload a file using the `fastPut` method, specifying the bucket name, file path, and content:
```typescript
async function uploadFile(bucketName: string, filePath: string, fileContent: Buffer | string) {
const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
await bucket.fastPut({ path: filePath, contents: fileContent });
console.log(`File uploaded to ${bucketName} at ${filePath}`);
console.log(`File uploaded to ${filePath}`);
}
uploadFile("myBucket", "example.txt", "This is a sample file content.");
@ -117,7 +124,7 @@ uploadFile("myBucket", "example.txt", "This is a sample file content.");
#### Downloading Files
Retrieve files using the `fastGet` method:
Download files using `fastGet`. It retrieves the file content as a buffer:
```typescript
async function downloadFile(bucketName: string, filePath: string) {
@ -131,7 +138,7 @@ downloadFile("myBucket", "example.txt");
#### Streaming Files
For large files, use streams:
For large-scale applications, stream files without loading them fully into memory:
```typescript
async function streamFile(bucketName: string, filePath: string) {
@ -146,13 +153,13 @@ streamFile("myBucket", "largefile.txt");
#### Deleting Files
Remove files with the `fastRemove` method:
Delete files with precision using `fastRemove`:
```typescript
async function deleteFile(bucketName: string, filePath: string) {
const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
await bucket.fastRemove({ path: filePath });
console.log(`File ${filePath} deleted from ${bucketName}.`);
console.log(`File ${filePath} deleted.`);
}
deleteFile("myBucket", "example.txt");
@ -160,9 +167,11 @@ deleteFile("myBucket", "example.txt");
### Directory Operations
Leverage directory functionalities to better organize and manage files within buckets.
#### Listing Directories and Files
You can navigate and list files in directories within a bucket:
Listing contents showcases a directorys structure and file contents:
```typescript
async function listDirectory(bucketName: string, directoryPath: string) {
@ -182,7 +191,7 @@ listDirectory("myBucket", "path/to/directory");
#### Managing Files in Directories
Upload, download, and manage files using directory abstractions:
Additional functionalities allow file management, inclusive of handling sub-directories:
```typescript
async function manageFilesInDirectory(bucketName: string, directoryPath: string, fileName: string, content: string) {
@ -202,21 +211,20 @@ manageFilesInDirectory("myBucket", "myDir", "example.txt", "File content here");
### Advanced Features
The librarys advanced features streamline intricate cloud storage workflows.
#### Bucket Policies
SmartBucket facilitates bucket policy management, depending on the cloud SDK's capabilities.
The library offers tools for maintaining consistent bucket policies across storage providers, assisting in defining access roles and permissions.
#### Metadata Management
You can retrieve and manipulate object metadata, employing it for additional data storage:
Easily manage and store metadata by using the `MetaData` utility:
```typescript
async function handleMetadata(bucketName: string, filePath: string) {
const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
const meta = await bucket.smartbucketRef.s3Client.send(new plugins.s3.HeadObjectCommand({
Bucket: bucket.name,
Key: filePath,
}));
const meta = await bucket.fastStat({ path: filePath });
console.log("Metadata:", meta.Metadata);
}
@ -225,7 +233,7 @@ handleMetadata("myBucket", "example.txt");
#### File Locking
Lock files to prevent changes:
Prevent accidental writes by locking files:
```typescript
async function lockFile(bucketName: string, filePath: string) {
@ -238,14 +246,34 @@ async function lockFile(bucketName: string, filePath: string) {
lockFile("myBucket", "example.txt");
```
### Cloud Agnosticism
#### Trash Management
`@push.rocks/smartbucket` supports multiple cloud providers, enhancing flexibility in cloud strategies without significant code changes. Adjust configurations as necessary for different providers, as services like AWS S3 or Google Cloud Storage might offer unique features beyond SmartBucket's unified interface.
SmartBucket enables a safe deletion mode where files can be moved to a recycling bin, allowing for restoration:
This guide demonstrates various operations with `@push.rocks/smartbucket`. Always refer to the comprehensive documentation and cloud provider details to fully leverage the library's capabilities.
```typescript
async function trashAndRestoreFile(bucketName: string, filePath: string) {
const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
const file: File = await bucket.getBaseDirectory().getFileStrict({ path: filePath });
// Move the file to trash
await file.delete({ mode: 'trash' });
console.log(`File ${filePath} moved to trash.`);
// Retrieve the file from the trash
const trashFile = await bucket.getTrash().getTrashedFileByOriginalName({ path: filePath });
await trashFile.restore();
console.log(`File ${filePath} restored from trash.`);
}
trashAndRestoreFile("myBucket", "example.txt");
```
This readme provides detailed documentation on using the `@push.rocks/smartbucket` module, demonstrating its capabilities through comprehensive examples and use cases. Each section is designed to guide a user through basic to more complex operations, ensuring a complete presentation of the library's features.
### Cloud Agnosticism
`@push.rocks/smartbucket` supports a multitude of cloud providers, enhancing flexibility in adopting different cloud strategies without the need for extensive code rewrite. It offers a uniform interface allowing to perform operations seamlessly between different storage solutions such as AWS S3, Google Cloud Storage, and more. This aspect empowers organizations to align their storage decisions with business needs rather than technical constraints.
By following this guide, you should be well-equipped to handle cloud storage operations using the `@push.rocks/smartbucket` library. Diligently constructed code examples elucidate the extensive functionalities offered by the library, aligned with best practices in cloud storage. For a deeper dive into any specific feature, refer to the comprehensive documentation provided with the library and the official documentation of the cloud providers you are integrating with.
```
## License and Legal Information

View File

@ -52,7 +52,21 @@ tap.test('should put a file into the trash', async () => {
console.log(fileMetadata.toString());
expect(await file.getMetaData().then((meta) => meta.metadataFile.getJsonData())).toEqual({});
await file.delete({ mode: 'trash' });
jestExpect(await file.getMetaData().then((meta) => meta.metadataFile.getJsonData())).toEqual({
const getTrashContents = async () => {
const trash = await myBucket.getTrash();
const trashDir = await trash.getTrashDir();
return await trashDir.listFiles();
}
const trashedFiles = await getTrashContents();
expect(trashedFiles.length).toEqual(2);
const trashedMetaFile = trashedFiles.find(file => file.name.endsWith('.metadata'));
expect(trashedMetaFile).toBeDefined();
expect(trashedMetaFile).toBeInstanceOf(smartbucket.File);
jestExpect(await trashedMetaFile!.getJsonData()).toEqual({
custom_recycle: {
deletedAt: jestExpect.any(Number),
originalPath: "trashtest/trashme.txt",

View File

@ -3,6 +3,6 @@
*/
export const commitinfo = {
name: '@push.rocks/smartbucket',
version: '3.3.2',
description: 'A TypeScript library facilitating cloud-agnostic object storage with capabilities such as bucket management, file operations, directory management, and advanced data streaming functionalities.'
version: '3.3.7',
description: 'A TypeScript library providing a cloud-agnostic interface for managing object storage with functionalities like bucket management, file and directory operations, and advanced features such as metadata handling and file locking.'
}