push.rocks/smartbucket
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(core): update

Browse Source
This commit is contained in:
Philipp Kunz 2024-05-17 19:24:52 +02:00
parent 2a15362ced
commit c70ee820d7
4 changed files with 223 additions and 48 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
npmextra.json
View File

@ -8,22 +8,25 @@
"githost": "code.foss.global",
"gitscope": "push.rocks",
"gitrepo": "smartbucket",
"description": "A TypeScript library for simple cloud independent object storage with support for buckets, directories, and files.",
"description": "A TypeScript library that offers simple, cloud-independent object storage with features like bucket creation, file management, and directory management.",
"npmPackagename": "@push.rocks/smartbucket",
"license": "MIT",
"keywords": [
"cloud storage",
"object storage",
"minio",
"S3",
"TypeScript",
"smartstream",
"smartpromise",
"smartpath",
"smartrx",
"buckets",
"files management",
"directories management"
"S3",
"minio",
"file management",
"directory management",
"bucket creation",
"data streaming",
"multi-cloud",
"API",
"unified storage",
"buffer handling",
"access key",
"secret key"
]
}
},

23
package.json
View File

@ -1,7 +1,7 @@
{
"name": "@push.rocks/smartbucket",
"version": "3.0.2",
"description": "A TypeScript library for simple cloud independent object storage with support for buckets, directories, and files.",
"description": "A TypeScript library that offers simple, cloud-independent object storage with features like bucket creation, file management, and directory management.",
"main": "dist_ts/index.js",
"typings": "dist_ts/index.d.ts",
"type": "module",
@ -45,15 +45,18 @@
"keywords": [
"cloud storage",
"object storage",
"minio",
"S3",
"TypeScript",
"smartstream",
"smartpromise",
"smartpath",
"smartrx",
"buckets",
"files management",
"directories management"
"S3",
"minio",
"file management",
"directory management",
"bucket creation",
"data streaming",
"multi-cloud",
"API",
"unified storage",
"buffer handling",
"access key",
"secret key"
]
}

219
readme.md
View File

@ -1,5 +1,5 @@
# @push.rocks/smartbucket
simple cloud independent object storage
A TypeScript library for simple cloud independent object storage with support for buckets, directories, and files.
## Install
@ -13,15 +13,28 @@ This command will download and install `@push.rocks/smartbucket` along with its
## Usage
`@push.rocks/smartbucket` is a module designed to provide simple cloud-independent object storage functionality. It wraps various cloud storage providers such as AWS S3, Google Cloud Storage, and others, offering a unified API to manage storage buckets and objects within those buckets.
`@push.rocks/smartbucket` is a TypeScript module designed to provide simple cloud-independent object storage functionality. It wraps various cloud storage providers such as AWS S3, Google Cloud Storage, and others, offering a unified API to manage storage buckets and objects within those buckets.
To use `@push.rocks/smartbucket` in your project, you'll need to follow these general steps:
In this guide, we will delve into the usage of SmartBucket, covering its full range of features from setting up the library to advanced usage scenarios.
### Table of Contents
1. [Setting Up](#setting-up)
2. [Creating a New Bucket](#creating-a-new-bucket)
3. [Listing Buckets](#listing-buckets)
4. [Working with Files](#working-with-files)
- [Uploading Files](#uploading-files)
- [Downloading Files](#downloading-files)
- [Deleting Files](#deleting-files)
- [Streaming Files](#streaming-files)
5. [Working with Directories](#working-with-directories)
6. [Advanced Features](#advanced-features)
- [Bucket Policies](#bucket-policies)
- [Object Metadata](#object-metadata)
- [Cloud Agnostic](#cloud-agnostic)
### Setting Up
First, ensure you are using ECMAScript modules (ESM) and TypeScript in your project for best compatibility.
Here's how to import and initialize smartbucket in a TypeScript file:
First, ensure you are using ECMAScript modules (ESM) and TypeScript in your project for best compatibility. Here's how to import and initialize SmartBucket in a TypeScript file:
```typescript
import {
@ -35,10 +48,12 @@ const mySmartBucket = new SmartBucket({
accessKey: "yourAccessKey",
accessSecret: "yourSecretKey",
endpoint: "yourEndpointURL",
port: 443, // Default is 443, could be customized for specific endpoint
useSsl: true // Defaults to true
});
```
Make sure to replace `"yourAccessKey"`, `"yourSecretKey"`, and `"yourEndpointURL"` with your actual credentials and endpoint URL.
Make sure to replace `"yourAccessKey"`, `"yourSecretKey"`, and `"yourEndpointURL"` with your actual credentials and endpoint URL. The `port` and `useSsl` options are optional and can be omitted if the defaults are acceptable.
### Creating a New Bucket
@ -58,19 +73,15 @@ async function createBucket(bucketName: string) {
createBucket("exampleBucket");
```
**Important:** Bucket names must be unique across the storage service.
Bucket names must be unique across the storage service.
### Listing Buckets
To list all buckets:
Currently, SmartBucket does not include a direct method to list all buckets, but you can access the underlying client provided by the cloud storage SDK to perform such operations, depending on the SDK's capabilities.
```typescript
// Currently, SmartBucket does not include a direct method to list all buckets,
// but you can access the underlying client provided by the cloud storage SDK
// to perform such operations, depending on the SDK's capabilities.
```
### Working with Files
### Uploading Objects to a Bucket
#### Uploading Files
To upload an object to a bucket:
@ -78,8 +89,10 @@ To upload an object to a bucket:
async function uploadFile(bucketName: string, filePath: string, fileContent: Buffer | string) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
await myBucket.fastStore(filePath, fileContent);
await myBucket.fastPut({ path: filePath, contents: fileContent });
console.log(`File uploaded to ${bucketName} at ${filePath}`);
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
@ -87,7 +100,7 @@ async function uploadFile(bucketName: string, filePath: string, fileContent: Buf
uploadFile("exampleBucket", "path/to/object.txt", "Hello, world!");
```
### Downloading Objects from a Bucket
#### Downloading Files
To download an object:
@ -95,8 +108,10 @@ To download an object:
async function downloadFile(bucketName: string, filePath: string) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
const fileContent: Buffer = await myBucket.fastGet(filePath);
const fileContent: Buffer = await myBucket.fastGet({ path: filePath });
console.log("Downloaded file content:", fileContent.toString());
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
@ -104,7 +119,7 @@ async function downloadFile(bucketName: string, filePath: string) {
downloadFile("exampleBucket", "path/to/object.txt");
```
### Deleting Objects
#### Deleting Files
To delete an object from a bucket:
@ -112,8 +127,10 @@ To delete an object from a bucket:
async function deleteFile(bucketName: string, filePath: string) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
await myBucket.fastRemove(filePath);
await myBucket.fastRemove({ path: filePath });
console.log(`File at ${filePath} deleted from ${bucketName}.`);
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
@ -121,19 +138,171 @@ async function deleteFile(bucketName: string, filePath: string) {
deleteFile("exampleBucket", "path/to/object.txt");
```
#### Streaming Files
SmartBucket allows you to work with file streams, which can be useful for handling large files.
To read a file as a stream:
```typescript
import { ReplaySubject } from '@push.rocks/smartrx';
async function readFileStream(bucketName: string, filePath: string) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
const fileStream: ReplaySubject<Buffer> = await myBucket.fastGetStream({ path: filePath });
fileStream.subscribe({
next(chunk: Buffer) {
console.log("Chunk received:", chunk.toString());
},
complete() {
console.log("File read completed.");
},
error(err) {
console.error("Error reading file stream:", err);
}
});
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
// Use the function
readFileStream("exampleBucket", "path/to/object.txt");
```
To write a file as a stream:
```typescript
import { Readable } from 'stream';
async function writeFileStream(bucketName: string, filePath: string, readableStream: Readable) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
await myBucket.fastPutStream({ path: filePath, dataStream: readableStream });
console.log(`File streamed to ${bucketName} at ${filePath}`);
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
// Create a readable stream from a string
const readable = new Readable();
readable.push('Hello world streamed as a file!');
readable.push(null); // Indicates end of the stream
// Use the function
writeFileStream("exampleBucket", "path/to/streamedObject.txt", readable);
```
### Working with Directories
`@push.rocks/smartbucket` abstracts directories within buckets for easier object management. You can create, list, and delete directories using the `Directory` class.
### Additional Features
```typescript
async function listDirectoryContents(bucketName: string, directoryPath: string) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
const baseDirectory: Directory = await myBucket.getBaseDirectory();
const targetDirectory: Directory = await baseDirectory.getSubDirectoryByName(directoryPath);
console.log('Listing directories:');
const directories = await targetDirectory.listDirectories();
directories.forEach(dir => {
console.log(`- ${dir.name}`);
});
- **Bucket Policies:** Manage bucket policies to control access permissions.
- **Object Metadata:** Retrieve and modify object metadata.
- **Cloud-Agnostic:** Designed to work with multiple cloud providers, allowing for easier migration or multi-cloud strategies.
console.log('Listing files:');
const files = await targetDirectory.listFiles();
files.forEach(file => {
console.log(`- ${file.name}`);
});
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
// Use the function
listDirectoryContents("exampleBucket", "some/directory/path");
```
To create a file within a directory:
```typescript
async function createFileInDirectory(bucketName: string, directoryPath: string, fileName: string, fileContent: string) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
const baseDirectory: Directory = await myBucket.getBaseDirectory();
const targetDirectory: Directory = await baseDirectory.getSubDirectoryByName(directoryPath);
await targetDirectory.createEmptyFile(fileName); // Create an empty file
const file = new File({ directoryRefArg: targetDirectory, fileName });
await file.updateWithContents({ contents: fileContent });
console.log(`File created: ${fileName}`);
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
// Use the function
createFileInDirectory("exampleBucket", "some/directory", "newfile.txt", "Hello, world!");
```
### Advanced Features
#### Bucket Policies
Manage bucket policies to control access permissions. This feature is dependent on the policies provided by the storage service (e.g., AWS S3, MinIO).
#### Object Metadata
You can retrieve and modify object metadata. Metadata can be useful for storing additional information about an object.
To retrieve metadata:
```typescript
async function getObjectMetadata(bucketName: string, filePath: string) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
const metadata = await mySmartBucket.minioClient.statObject(bucketName, filePath);
console.log("Object metadata:", metadata);
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
// Use the function
getObjectMetadata("exampleBucket", "path/to/object.txt");
```
To update metadata:
```typescript
async function updateObjectMetadata(bucketName: string, filePath: string, newMetadata: { [key: string]: string }) {
const myBucket: Bucket = await mySmartBucket.getBucketByName(bucketName);
if (myBucket) {
await myBucket.copyObject({
objectKey: filePath,
nativeMetadata: newMetadata,
deleteExistingNativeMetadata: false,
});
console.log(`Metadata updated for ${filePath}`);
} else {
console.error(`Bucket ${bucketName} does not exist.`);
}
}
// Use the function
updateObjectMetadata("exampleBucket", "path/to/object.txt", {
customKey: "customValue"
});
```
#### Cloud Agnostic
`@push.rocks/smartbucket` is designed to work with multiple cloud providers, allowing for easier migration or multi-cloud strategies. This means you can switch from one provider to another with minimal changes to your codebase.
Remember, each cloud provider has specific features and limitations. `@push.rocks/smartbucket` aims to abstract common functionalities, but always refer to the specific cloud provider's documentation for advanced features or limitations.
> **Note:** This document focuses on basic operations to get you started with `@push.rocks/smartbucket`. For advanced usage, including streaming data, managing bucket policies, and handling large file uploads, refer to the detailed API documentation and examples.
This guide covers the basic to advanced scenarios of using `@push.rocks/smartbucket`. For further details, refer to the API documentation and examples.
## License and Legal Information

4
ts/00_commitinfo_data.ts
View File

@ -3,6 +3,6 @@
*/
export const commitinfo = {
name: '@push.rocks/smartbucket',
version: '3.0.2',
description: 'A TypeScript library for simple cloud independent object storage with support for buckets, directories, and files.'
version: '3.0.3',
description: 'A TypeScript library that offers simple, cloud-independent object storage with features like bucket creation, file management, and directory management.'
}