push.rocks/smartfile
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update tsconfig

Browse Source
This commit is contained in:
Philipp Kunz 2024-04-14 17:35:54 +02:00
parent 21b3870a7d
commit 50cc3fa8bf
4 changed files with 73 additions and 210 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
npmextra.json
View File

@ -17,15 +17,13 @@
"files management",
"TypeScript",
"Node.js",
"read files",
"write files",
"copy files",
"file operations",
"file manipulation",
"file streaming",
"directories manipulation",
"virtual file system",
"virtual directory",
"filesystem utilities",
"ESM syntax",
"memory-efficient streaming"
"memory-efficient file handling",
"custom file operations"
]
}
},

14
package.json
View File

@ -19,15 +19,13 @@
"files management",
"TypeScript",
"Node.js",
"read files",
"write files",
"copy files",
"file operations",
"file manipulation",
"file streaming",
"directories manipulation",
"virtual file system",
"virtual directory",
"filesystem utilities",
"ESM syntax",
"memory-efficient streaming"
"memory-efficient file handling",
"custom file operations"
],
"author": "Lossless GmbH <hello@lossless.com> (https://lossless.com)",
"license": "MIT",
@ -75,4 +73,4 @@
"browserslist": [
"last 1 chrome versions"
]
}
}

1
readme.hints.md Normal file
View File

@ -0,0 +1 @@

256
readme.md
View File

@ -12,252 +12,118 @@ npm install @push.rocks/smartfile
## Usage
`@push.rocks/smartfile` offers a comprehensive set of utilities for handling files in Node.js projects using modern TypeScript and ESM syntax. It simplifies various file operations such as reading, writing, copying, and streaming files, as well as working with directories and virtual file systems.
`@push.rocks/smartfile` offers extensive file management utilities, enabling seamless file processing with TypeScript in a Node.js environment. Below are detailed examples showcasing various features of the module.
### Key Features and Classes
### Quick Start
- **`SmartFile`**: Facilitates reading from and writing to individual files, managing metadata.
- **`StreamFile`**: Optimizes memory usage by enabling efficient file streaming.
- **`VirtualDirectory`**: Allows manipulation of a group of files or directories as a virtual file system.
### Getting Started with ESM and TypeScript
First, ensure your project supports ESM syntax and TypeScript. Then, begin by importing the desired features from `@push.rocks/smartfile`:
First, ensure you're working in an environment that supports ECMAScript modules (ESM) and TypeScript. Here’s how you’d generally import and use `@push.rocks/smartfile`:
```typescript
import { SmartFile, StreamFile, VirtualDirectory, memory, fs as smartFs } from '@push.rocks/smartfile';
import { SmartFile, StreamFile, VirtualDirectory, fs, memory } from '@push.rocks/smartfile';
```
### Reading and Writing Files
### Working with `SmartFile`
#### Reading Files
Reading a JSON file:
To read from a file and convert it to a `SmartFile` instance:
```typescript
const myJsonFile: SmartFile = await SmartFile.fromFilePath('./data.json');
const jsonData = JSON.parse(myJsonFile.contents.toString());
console.log(jsonData);
const myJsonSmartFile: SmartFile = await SmartFile.fromFilePath('./somePath/data.json');
const jsonData = JSON.parse(myJsonSmartFile.contents.toString());
console.log(jsonData); // Assuming the file contains JSON content
```
#### Writing Files
Writing content to a file:
To write data to a file through a `SmartFile`:
```typescript
const filePath: string = './output.txt';
const content: string = 'Hello, SmartFile!';
const filePath: string = './output/outputData.json';
const content: string = JSON.stringify({ key: 'value' });
await memory.toFs(content, filePath);
console.log('File saved successfully.');
```
### Streaming Large Files
### Streaming Large Files with `StreamFile`
For large files, `StreamFile` provides a memory-efficient streaming solution:
When dealing with large files, you can use `StreamFile` to handle such files efficiently, minimizing memory usage:
```typescript
import { createReadStream } from 'fs';
const sourceStream = createReadStream('./large-video.mp4');
const myStreamFile = await StreamFile.fromStream(sourceStream, 'large-video.mp4');
await myStreamFile.writeToDir('./storage');
console.log('Large file streamed to disk successfully.');
const largeFile: StreamFile = await StreamFile.fromPath('./largeInput/largeFile.mp4');
await largeFile.writeToDisk('./largeOutput/largeFileCopy.mp4');
```
### Working with Virtual Directories
### Managing Virtual Directories
`VirtualDirectory` abstracts a collection of files allowing operations to be performed as if they were on disk:
Handling multiple files as if they were part of a file system:
```typescript
const virtualDir = new VirtualDirectory();
virtualDir.addSmartfiles([smartFile1, smartFile2]); // Assuming these are SmartFile instances
await virtualDir.saveToDisk('./virtual-output');
console.log('Virtual directory saved to disk.');
await virtualDir.addSmartfiles([smartFile1, smartFile2]);
await virtualDir.saveToDisk('./virtualDirOutput');
```
### Advanced File Operations
### File System Operations
`@push.rocks/smartfile` simplifies complex file operations, including:
- Copying directories and files
- Removing files or directories
- Listing files and directories with filters
- Reading file content directly into JavaScript objects
`@push.rocks/smartfile` provides a suite of utilities for common file system operations such as copying, deleting, and listing files or directories.
### Web File Handling
#### Copying a File
Handling files from HTTP requests:
`@push.rocks/smartfile` offers utilities to work with files from web sources, making it simpler to manage downloads and uploads.
```typescript
await fs.copy('./sourceFile.txt', './destinationFile.txt');
```
### Comprehensive File Management
#### Deleting a Directory
Whether you're dealing with local files, directories, or files over the internet, `@push.rocks/smartfile` provides a comprehensive set of tools to streamline your workflow and reduce the complexity of file management in your Node.js projects.
```typescript
await fs.remove('./directoryToDelete');
```
## API Reference
#### Listing Files in a Directory
### `SmartFile` Class
```typescript
const fileList: string[] = await fs.listFiles('./someDirectory');
console.log(fileList);
```
#### Static Methods
### Advanced File Management
- `SmartFile.fromFilePath(filePath: string, baseArg?: string): Promise<SmartFile>`
- Creates a `SmartFile` instance from a file path.
- `SmartFile.fromBuffer(filePath: string, contentBufferArg: Buffer, baseArg?: string): Promise<SmartFile>`
- Creates a `SmartFile` instance from a `Buffer`.
- `SmartFile.fromString(filePath: string, contentStringArg: string, encodingArg: 'utf8' | 'binary', baseArg?: string): Promise<SmartFile>`
- Creates a `SmartFile` instance from a string.
- `SmartFile.fromFoldedJson(foldedJsonArg: string): Promise<SmartFile>`
- Creates a `SmartFile` instance from a folded JSON string.
- `SmartFile.fromStream(stream: Readable, filePath: string, baseArg?: string): Promise<SmartFile>`
- Creates a `SmartFile` instance from a `Readable` stream.
- `SmartFile.fromUrl(urlArg: string): Promise<SmartFile>`
- Creates a `SmartFile` instance from a URL.
For specialized file operations, such as editing the contents of a file or streaming files from URLs, `@push.rocks/smartfile` includes advanced management features.
#### Instance Properties
#### Editing a File’s Contents
- `path: string`
- The relative path of the file.
- `parsedPath: path.ParsedPath`
- A parsed path object.
- `absolutePath: string`
- The absolute path of the file.
- `absoluteParsedPath: path.ParsedPath`
- A parsed absolute path object.
- `contentBuffer: Buffer`
- The content of the file as a `Buffer`.
- `base: string`
- The current working directory of the file.
- `sync: boolean`
- Indicates whether the file is synced with the disk.
```typescript
const smartFileToEdit: SmartFile = await SmartFile.fromFilePath('./editableFile.txt');
await smartFileToEdit.editContentAsString((content) => content.replace(/originalText/g, 'newText'));
await smartFileToEdit.write();
```
#### Instance Methods
#### Streaming a File from a URL
- `setContentsFromString(contentString: string, encodingArg?: 'utf8' | 'binary'): void`
- Sets the contents of the file from a string.
- `write(): Promise<void>`
- Writes the file to disk at its original location.
- `writeToDiskAtPath(filePathArg: string): Promise<void>`
- Writes the file to the specified path on disk.
- `writeToDir(dirPathArg: string): Promise<string>`
- Writes the file to a directory combined with the relative path portion.
- `read(): Promise<void>`
- Reads the file from disk.
- `delete(): Promise<void>`
- Deletes the file from disk at its original location.
- `getHash(typeArg?: 'path' | 'content' | 'all'): Promise<string>`
- Returns a hash of the file based on the specified type.
- `updateFileName(fileNameArg: string): void`
- Updates the file name of the `SmartFile` instance.
- `editContentAsString(editFuncArg: (fileStringArg: string) => Promise<string>): Promise<void>`
- Edits the content of the file as a string using the provided edit function.
- `getStream(): Readable`
- Returns a `Readable` stream from the file's content buffer.
```typescript
const streamedFile: StreamFile = await StreamFile.fromUrl('https://example.com/file.pdf');
await streamedFile.writeToDisk('./downloadedFiles/file.pdf');
```
### `StreamFile` Class
### Working with File Buffers and Streams
#### Static Methods
`@push.rocks/smartfile` allows you to easily work with files using Buffers and Streams, facilitating operations like file transformations, uploads, and downloads.
- `StreamFile.fromPath(filePath: string): Promise<StreamFile>`
- Creates a `StreamFile` instance from a file path.
- `StreamFile.fromUrl(url: string): Promise<StreamFile>`
- Creates a `StreamFile` instance from a URL.
- `StreamFile.fromBuffer(buffer: Buffer, relativeFilePath?: string): StreamFile`
- Creates a `StreamFile` instance from a `Buffer`.
- `StreamFile.fromStream(stream: Readable, relativeFilePath?: string, multiUse?: boolean): StreamFile`
- Creates a `StreamFile` instance from a `Readable` stream.
#### Creating a SmartFile from a Buffer
#### Instance Properties
```typescript
const buffer: Buffer = Buffer.from('Sample data');
const bufferSmartFile: SmartFile = await SmartFile.fromBuffer('./bufferFile.txt', buffer);
await bufferSmartFile.write();
```
- `relativeFilePath?: string`
- The relative file path of the `StreamFile`.
- `multiUse: boolean`
- Indicates whether the `StreamFile` can be used multiple times.
- `used: boolean`
- Indicates whether the `StreamFile` has been used.
#### Uploading a File
#### Instance Methods
Stream files or buffers can be integrated with web frameworks to handle file uploads efficiently, utilizing streams to reduce memory footprint.
- `createReadStream(): Promise<Readable>`
- Creates a new `Readable` stream from the source.
- `writeToDisk(filePathArg: string): Promise<void>`
- Writes the stream to the disk at the specified path.
- `writeToDir(dirPathArg: string): Promise<void>`
- Writes the stream to a directory combined with the relative path portion.
- `getContentAsBuffer(): Promise<Buffer>`
- Returns the content of the `StreamFile` as a `Buffer`.
- `getContentAsString(formatArg?: 'utf8' | 'binary'): Promise<string>`
- Returns the content of the `StreamFile` as a string.
### Conclusion
### `VirtualDirectory` Class
#### Static Methods
- `VirtualDirectory.fromFsDirPath(pathArg: string): Promise<VirtualDirectory>`
- Creates a `VirtualDirectory` instance from a file system directory path.
- `VirtualDirectory.fromVirtualDirTransferableObject(virtualDirTransferableObjectArg: VirtualDirTransferableObject): Promise<VirtualDirectory>`
- Creates a `VirtualDirectory` instance from a `VirtualDirTransferableObject`.
#### Instance Properties
- `smartfileArray: SmartFile[]`
- An array of `SmartFile` instances in the `VirtualDirectory`.
#### Instance Methods
- `addSmartfiles(smartfileArrayArg: SmartFile[]): void`
- Adds an array of `SmartFile` instances to the `VirtualDirectory`.
- `getFileByPath(pathArg: string): Promise<SmartFile | undefined>`
- Retrieves a `SmartFile` instance from the `VirtualDirectory` by its path.
- `toVirtualDirTransferableObject(): Promise<VirtualDirTransferableObject>`
- Converts the `VirtualDirectory` to a `VirtualDirTransferableObject`.
- `saveToDisk(dirArg: string): Promise<void>`
- Saves the `VirtualDirectory` to the disk at the specified directory.
- `shiftToSubdirectory(subDir: string): Promise<VirtualDirectory>`
- Shifts the `VirtualDirectory` to a subdirectory.
- `addVirtualDirectory(virtualDir: VirtualDirectory, newRoot: string): Promise<void>`
- Adds another `VirtualDirectory` to the current `VirtualDirectory` with a new root path.
### `fs` Module
The `fs` module provides various file system utility functions. Some notable functions include:
- `fileExistsSync(filePath: string): boolean`
- Checks if a file exists synchronously.
- `fileExists(filePath: string): Promise<boolean>`
- Checks if a file exists asynchronously.
- `listFoldersSync(pathArg: string, regexFilter?: RegExp): string[]`
- Lists folders in a directory synchronously.
- `listFolders(pathArg: string, regexFilter?: RegExp): Promise<string[]>`
- Lists folders in a directory asynchronously.
- `listFilesSync(pathArg: string, regexFilter?: RegExp): string[]`
- Lists files in a directory synchronously.
- `listFiles(pathArg: string, regexFilter?: RegExp): Promise<string[]>`
- Lists files in a directory asynchronously.
- `listFileTree(dirPathArg: string, miniMatchFilter: string, absolutePathsBool?: boolean): Promise<string[]>`
- Lists a file tree using a minimatch filter.
- `waitForFileToBeReady(filePathArg: string): Promise<void>`
- Waits for a file to be ready (exists and is not empty).
- `toFs(filePathArg: string, fileContentArg: string | Buffer | SmartFile | StreamFile, optionsArg?: { respectRelative?: boolean }): Promise<void>`
- Writes a string, `Buffer`, `SmartFile`, or `StreamFile` to the file system.
### `memory` Module
The `memory` module provides utility functions for working with files in memory. Some notable functions include:
- `toObject(fileStringArg: string, fileTypeArg: string): any`
- Converts a file string to an object based on the file type.
- `toFs(fileContentArg: string | Buffer | SmartFile | StreamFile, filePathArg: string, optionsArg?: IToFsOptions): Promise<void>`
- Writes a string, `Buffer`, `SmartFile`, or `StreamFile` to the file system.
- `toFsSync(fileArg: string, filePathArg: string): void`
- Writes a string to the file system synchronously.
- `smartfileArrayToFs(smartfileArrayArg: SmartFile[], dirArg: string): Promise<void>`
- Writes an array of `SmartFile` instances to the file system.
### `interpreter` Module
The `interpreter` module provides utility functions for interpreting file types and contents. Some notable functions include:
- `filetype(pathArg: string): string`
- Determines the file type based on the file path.
- `objectFile(fileStringArg: string, fileTypeArg: any): any`
- Converts a file string to an object based on the file type.
With `@push.rocks/smartfile`, managing files in Node.js using TypeScript becomes significantly more straightforward and efficient. Utilizing the provided classes and methods, you can handle a wide range of file operations, from basic read/write tasks to complex operations such as virtual directory management and file streaming, with minimal boilerplate code and maximum efficiency.
## License and Legal Information
@ -276,4 +142,4 @@ Registered at District court Bremen HRB 35230 HB, Germany
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.