smartfile/readme.md

242 lines
8.2 KiB
Markdown
Raw Permalink Normal View History

2024-04-02 18:53:02 +00:00
# @push.rocks/smartfile
2024-04-03 13:39:26 +00:00
> Provides a robust suite of tools for managing files in Node.js using TypeScript.
2024-04-01 15:46:40 +00:00
## Install
2024-04-02 19:37:31 +00:00
To integrate `@push.rocks/smartfile` into your project, run:
2024-04-02 18:53:02 +00:00
```bash
npm install @push.rocks/smartfile
2024-04-01 15:46:40 +00:00
```
2020-02-07 21:00:54 +00:00
## Usage
2024-04-14 15:35:54 +00:00
`@push.rocks/smartfile` offers extensive file management utilities, enabling seamless file processing with TypeScript in a Node.js environment. Below are detailed examples showcasing various features of the module.
2024-04-01 15:46:40 +00:00
2024-04-14 15:35:54 +00:00
### Quick Start
2024-04-01 15:46:40 +00:00
2024-04-14 15:35:54 +00:00
First, ensure you're working in an environment that supports ECMAScript modules (ESM) and TypeScript. Heres how youd generally import and use `@push.rocks/smartfile`:
2024-04-01 15:46:40 +00:00
```typescript
2024-05-17 15:49:57 +00:00
import { SmartFile, StreamFile, VirtualDirectory, fs, memory, interpreter } from '@push.rocks/smartfile';
2024-04-01 15:46:40 +00:00
```
2024-04-14 15:35:54 +00:00
### Working with `SmartFile`
2024-04-01 15:46:40 +00:00
2024-04-02 19:37:31 +00:00
#### Reading Files
2024-04-01 15:46:40 +00:00
2024-04-14 15:35:54 +00:00
To read from a file and convert it to a `SmartFile` instance:
2024-04-02 18:53:02 +00:00
```typescript
2024-04-14 15:35:54 +00:00
const myJsonSmartFile: SmartFile = await SmartFile.fromFilePath('./somePath/data.json');
const jsonData = JSON.parse(myJsonSmartFile.contents.toString());
console.log(jsonData); // Assuming the file contains JSON content
2024-04-02 18:53:02 +00:00
```
2024-04-02 19:37:31 +00:00
#### Writing Files
2024-04-01 15:46:40 +00:00
2024-04-14 15:35:54 +00:00
To write data to a file through a `SmartFile`:
2024-04-01 15:46:40 +00:00
2024-04-02 18:53:02 +00:00
```typescript
2024-04-14 15:35:54 +00:00
const filePath: string = './output/outputData.json';
const content: string = JSON.stringify({ key: 'value' });
2024-04-02 19:37:31 +00:00
await memory.toFs(content, filePath);
2024-04-01 15:46:40 +00:00
```
2024-04-14 15:35:54 +00:00
### Streaming Large Files with `StreamFile`
2024-04-02 18:53:02 +00:00
2024-04-14 15:35:54 +00:00
When dealing with large files, you can use `StreamFile` to handle such files efficiently, minimizing memory usage:
2024-04-02 18:53:02 +00:00
2024-04-02 19:37:31 +00:00
```typescript
2024-04-14 15:35:54 +00:00
const largeFile: StreamFile = await StreamFile.fromPath('./largeInput/largeFile.mp4');
await largeFile.writeToDisk('./largeOutput/largeFileCopy.mp4');
2024-04-02 18:53:02 +00:00
```
2024-05-17 15:49:57 +00:00
### Working with Virtual Directories
2024-04-01 15:46:40 +00:00
2024-04-14 15:35:54 +00:00
Handling multiple files as if they were part of a file system:
2024-04-01 15:46:40 +00:00
2024-04-02 19:37:31 +00:00
```typescript
2024-05-17 15:49:57 +00:00
const virtualDir = await VirtualDirectory.fromFsDirPath('./data/inputDir');
await virtualDir.saveToDisk('./data/outputDir');
2024-04-14 15:35:54 +00:00
```
### File System Operations
`@push.rocks/smartfile` provides a suite of utilities for common file system operations such as copying, deleting, and listing files or directories.
#### Copying a File
```typescript
await fs.copy('./sourceFile.txt', './destinationFile.txt');
```
#### Deleting a Directory
```typescript
await fs.remove('./directoryToDelete');
2024-04-02 18:53:02 +00:00
```
2024-04-01 15:46:40 +00:00
2024-04-14 15:35:54 +00:00
#### Listing Files in a Directory
```typescript
const fileList: string[] = await fs.listFiles('./someDirectory');
console.log(fileList);
```
### Advanced File Management
For specialized file operations, such as editing the contents of a file or streaming files from URLs, `@push.rocks/smartfile` includes advanced management features.
#### Editing a Files Contents
```typescript
const smartFileToEdit: SmartFile = await SmartFile.fromFilePath('./editableFile.txt');
2024-05-17 15:49:57 +00:00
await smartFileToEdit.editContentAsString(async (content) => content.replace(/originalText/g, 'newText'));
2024-04-14 15:35:54 +00:00
await smartFileToEdit.write();
```
#### Streaming a File from a URL
```typescript
const streamedFile: StreamFile = await StreamFile.fromUrl('https://example.com/file.pdf');
await streamedFile.writeToDisk('./downloadedFiles/file.pdf');
```
### Working with File Buffers and Streams
`@push.rocks/smartfile` allows you to easily work with files using Buffers and Streams, facilitating operations like file transformations, uploads, and downloads.
#### Creating a SmartFile from a Buffer
```typescript
const buffer: Buffer = Buffer.from('Sample data');
const bufferSmartFile: SmartFile = await SmartFile.fromBuffer('./bufferFile.txt', buffer);
await bufferSmartFile.write();
```
2024-05-17 15:49:57 +00:00
### Using `VirtualDirectory` for Complex File Management
2024-04-14 15:35:54 +00:00
2024-05-17 15:49:57 +00:00
`VirtualDirectory` simplifies the management of multiple files that are otherwise scattered across different directories or created at runtime.
2024-04-14 15:35:54 +00:00
2024-05-17 15:49:57 +00:00
#### Creating a `VirtualDirectory`
2024-04-14 15:35:54 +00:00
2024-05-17 15:49:57 +00:00
To create a `VirtualDirectory` from an existing file directory:
```typescript
const virtualDirectory = await VirtualDirectory.fromFsDirPath('./someDirectory');
```
#### Adding More Files
You can add more `SmartFile` instances to your `VirtualDirectory`:
```typescript
const additionalFiles = [
await SmartFile.fromFilePath('./anotherDirectory/file1.txt'),
await SmartFile.fromFilePath('./anotherDirectory/file2.txt')
];
virtualDirectory.addSmartfiles(additionalFiles);
```
#### Saving `VirtualDirectory` to Disk
Save your entire `VirtualDirectory` to disk:
```typescript
await virtualDirectory.saveToDisk('./outputDirectory');
```
### Utilizing StreamFile for Efficient File Handling
Using `StreamFile` can be especially beneficial for large files or when performing streaming operations.
#### Streaming from a URL
`StreamFile` provides capabilities to stream files directly from URLs, making it easier to work with remote content.
```typescript
const urlStreamFile: StreamFile = await StreamFile.fromUrl('https://example.com/largefile.zip');
await urlStreamFile.writeToDisk('./downloadedFiles/largefile.zip');
```
### Combining Buffer and Stream Approaches
Create `StreamFile` from a buffer for efficient, multi-use streams.
```typescript
const buffer = Buffer.from('Streaming buffer content');
const bufferStreamFile = StreamFile.fromBuffer(buffer, 'bufferBasedStream.txt');
await bufferStreamFile.writeToDisk('./streams/bufferBasedStream.txt');
```
### Read and Write Operations with StreamFile
Perform read and write operations efficiently using `StreamFile`.
```typescript
const fileStream = await StreamFile.fromPath('./inputData/largeFile.data');
await fileStream.writeToDisk('./outputData/largeFileCopy.data');
```
Check for completeness of your read and write operations, ensuring the integrity of file content.
```typescript
const readBuffer = await fileStream.getContentAsBuffer();
console.log(readBuffer.toString());
```
### Ensuring File Readiness for Streaming
Ensure a file is ready for streaming or create a custom readable stream incorporating data transformation.
```typescript
const smartReadStream = new SmartReadStream('./incomingData/sample.data');
smartReadStream.on('data', (chunk) => {
console.log('New Data Chunk:', chunk.toString());
});
```
### File Transformation with SmartReadStream
Perform transformations on the stream of data while reading:
```typescript
smartReadStream.on('data', (chunk) => {
// Perform some transformation
const transformedChunk = chunk.toString().toUpperCase();
console.log('Transformed Data Chunk:', transformedChunk);
});
```
### Streaming with SmartReadStream
Stream data from a `SmartReadStream` to a file efficiently managing large datasets.
```typescript
const transformedWriteStream = fs.createWriteStream('./processedData/transformed.data');
smartReadStream.pipe(transformedWriteStream);
```
`@push.rocks/smartfile` significantly simplifies the handling of complex file operations in Node.js projects, making these tasks straightforward while maintaining efficiency and ease of use. Explore and leverage these features to enhance your project's file management capabilities.
2024-04-03 13:39:26 +00:00
2024-04-02 18:53:02 +00:00
## License and Legal Information
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
### Company Information
Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany
2020-02-07 21:00:54 +00:00
2024-04-02 18:53:02 +00:00
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
2020-02-07 21:00:54 +00:00
2024-04-14 15:35:54 +00:00
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.