smartbucket/readme.md

# @push.rocks/smartbucket

A comprehensive TypeScript library for cloud-agnostic object storage offering bucket management, file operations, and advanced data streaming.

## Install

To install `@push.rocks/smartbucket`, ensure you have Node.js and npm installed. Then, run the following command in your project directory:

```bash
npm install @push.rocks/smartbucket --save

This command will add @push.rocks/smartbucket to your project's dependencies and install it along with its requirements in the node_modules directory.

Usage

Introduction

@push.rocks/smartbucket provides a robust set of features to manage cloud storage operations in a cloud-agnostic manner. By leveraging this library, you can seamlessly interact with object storage services like AWS S3, without being tied to any vendor-specific implementations. This library not only abstracts basic file operations but also integrates advanced capabilities such as metadata management, data streaming, file locking, and bucket policies, all through a simplified API.

Table of Contents

1. Setting Up
2. Working with Buckets
   • Creating a New Bucket
   • Listing Buckets
   • Deleting Buckets
3. File Operations in Buckets
   • Uploading Files
   • Downloading Files
   • Streaming Files
   • Deleting Files
4. Directory Operations
   • Listing Directories and Files
   • Managing Files in Directories
5. Advanced Features
   • Bucket Policies
   • Metadata Management
   • File Locking
   • Trash Management
6. Cloud Agnosticism

Setting Up

Begin by importing the necessary classes from the @push.rocks/smartbucket package into your TypeScript file. Create an instance of SmartBucket with your storage configuration:

import {
  SmartBucket,
  Bucket,
  Directory,
  File
} from '@push.rocks/smartbucket';

const mySmartBucket = new SmartBucket({
  accessKey: "yourAccessKey",
  accessSecret: "yourSecretKey",
  endpoint: "yourEndpointURL",
  port: 443,
  useSsl: true
});

Replace "yourAccessKey", "yourSecretKey", and "yourEndpointURL" with actual data specific to your cloud provider.

Working with Buckets

Creating a New Bucket

Creating a bucket involves invoking the createBucket method. Note that bucket names are unique and follow the rules of the cloud provider:

async function createBucket(bucketName: string) {
  try {
    const newBucket: Bucket = await mySmartBucket.createBucket(bucketName);
    console.log(`Bucket ${bucketName} created successfully.`);
  } catch (error) {
    console.error("Error creating bucket:", error);
  }
}

createBucket("myNewBucket");

Listing Buckets

While the library uses cloud-provider capabilities like AWS SDK to list existing buckets, smartbucket is aimed at simplifying content management within them.

Deleting Buckets

To delete a bucket, simply call the removeBucket function:

async function deleteBucket(bucketName: string) {
  try {
    await mySmartBucket.removeBucket(bucketName);
    console.log(`Bucket ${bucketName} deleted successfully.`);
  } catch (error) {
    console.error("Error deleting bucket:", error);
  }
}

deleteBucket("anotherBucketName");

File Operations in Buckets

SmartBucket offers a unified API to execute file-based operations efficiently.

Uploading Files

Upload a file using the fastPut method, specifying the bucket name, file path, and content:

async function uploadFile(bucketName: string, filePath: string, fileContent: Buffer | string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  await bucket.fastPut({ path: filePath, contents: fileContent });
  console.log(`File uploaded to ${filePath}`);
}

uploadFile("myBucket", "example.txt", "This is a sample file content.");

Downloading Files

Download files using fastGet. It retrieves the file content as a buffer:

async function downloadFile(bucketName: string, filePath: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  const content: Buffer = await bucket.fastGet({ path: filePath });
  console.log("Downloaded content:", content.toString());
}

downloadFile("myBucket", "example.txt");

Streaming Files

For large-scale applications, stream files without loading them fully into memory:

async function streamFile(bucketName: string, filePath: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  const stream = await bucket.fastGetStream({ path: filePath }, "nodestream");
  stream.on('data', chunk => console.log("Chunk:", chunk.toString()));
  stream.on('end', () => console.log("Download completed."));
}

streamFile("myBucket", "largefile.txt");

Deleting Files

Delete files with precision using fastRemove:

async function deleteFile(bucketName: string, filePath: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  await bucket.fastRemove({ path: filePath });
  console.log(`File ${filePath} deleted.`);
}

deleteFile("myBucket", "example.txt");

Directory Operations

Leverage directory functionalities to better organize and manage files within buckets.

Listing Directories and Files

Listing contents showcases a directory’s structure and file contents:

async function listDirectory(bucketName: string, directoryPath: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  const baseDirectory: Directory = await bucket.getBaseDirectory();
  const targetDirectory = await baseDirectory.getSubDirectoryByName(directoryPath);

  console.log('Directories:');
  (await targetDirectory.listDirectories()).forEach(dir => console.log(dir.name));

  console.log('Files:');
  (await targetDirectory.listFiles()).forEach(file => console.log(file.name));
}

listDirectory("myBucket", "path/to/directory");

Managing Files in Directories

Additional functionalities allow file management, inclusive of handling sub-directories:

async function manageFilesInDirectory(bucketName: string, directoryPath: string, fileName: string, content: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  const baseDirectory: Directory = await bucket.getBaseDirectory();
  const directory = await baseDirectory.getSubDirectoryByName(directoryPath) ?? baseDirectory;

  await directory.fastPut({ path: fileName, contents: content });
  console.log(`File ${fileName} created in ${directoryPath}`);

  const fileContent = await directory.fastGet({ path: fileName });
  console.log(`Content of ${fileName}: ${fileContent.toString()}`);
}

manageFilesInDirectory("myBucket", "myDir", "example.txt", "File content here");

Advanced Features

The library’s advanced features streamline intricate cloud storage workflows.

Bucket Policies

The library offers tools for maintaining consistent bucket policies across storage providers, assisting in defining access roles and permissions.

Metadata Management

Easily manage and store metadata by using the MetaData utility:

async function handleMetadata(bucketName: string, filePath: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  const meta = await bucket.fastStat({ path: filePath });
  console.log("Metadata:", meta.Metadata);
}

handleMetadata("myBucket", "example.txt");

File Locking

Prevent accidental writes by locking files:

async function lockFile(bucketName: string, filePath: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  const file: File = await bucket.getBaseDirectory().getFileStrict({ path: filePath });
  await file.lock({ timeoutMillis: 600000 }); // Lock for 10 minutes
  console.log(`File ${filePath} locked.`);
}

lockFile("myBucket", "example.txt");

Trash Management

SmartBucket enables a safe deletion mode where files can be moved to a recycling bin, allowing for restoration:

async function trashAndRestoreFile(bucketName: string, filePath: string) {
  const bucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
  const file: File = await bucket.getBaseDirectory().getFileStrict({ path: filePath });

  // Move the file to trash
  await file.delete({ mode: 'trash' });
  console.log(`File ${filePath} moved to trash.`);

  // Retrieve the file from the trash
  const trashFile = await bucket.getTrash().getTrashedFileByOriginalName({ path: filePath });
  await trashFile.restore();
  console.log(`File ${filePath} restored from trash.`);
}

trashAndRestoreFile("myBucket", "example.txt");

Cloud Agnosticism

@push.rocks/smartbucket supports a multitude of cloud providers, enhancing flexibility in adopting different cloud strategies without the need for extensive code rewrite. It offers a uniform interface allowing to perform operations seamlessly between different storage solutions such as AWS S3, Google Cloud Storage, and more. This aspect empowers organizations to align their storage decisions with business needs rather than technical constraints.

By following this guide, you should be well-equipped to handle cloud storage operations using the @push.rocks/smartbucket library. Diligently constructed code examples elucidate the extensive functionalities offered by the library, aligned with best practices in cloud storage. For a deeper dive into any specific feature, refer to the comprehensive documentation provided with the library and the official documentation of the cloud providers you are integrating with.

## License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository. 

**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

### Company Information

Task Venture Capital GmbH  
Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.