initial
This commit is contained in:
445
readme.md
Normal file
445
readme.md
Normal file
@@ -0,0 +1,445 @@
|
||||
# @push.rocks/smartfs
|
||||
|
||||
Modern, pluggable filesystem module with fluent API, Web Streams support, and multiple storage backends.
|
||||
|
||||
## Features
|
||||
|
||||
- **🎯 Fluent API** - Action-last chainable interface for elegant code
|
||||
- **🔌 Pluggable Providers** - Support for multiple storage backends (Node.js fs, memory, S3, etc.)
|
||||
- **🌊 Web Streams** - Modern streaming with Web Streams API
|
||||
- **💾 Transactions** - Atomic multi-file operations with automatic rollback
|
||||
- **👀 File Watching** - Event-based file system monitoring
|
||||
- **⚡ Async-Only** - Modern async/await patterns throughout
|
||||
- **📦 Zero Dependencies** - Core functionality with minimal dependencies
|
||||
- **🎨 TypeScript** - Full type safety and IntelliSense support
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pnpm install @push.rocks/smartfs
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```typescript
|
||||
import { SmartFs, SmartFsProviderNode } from '@push.rocks/smartfs';
|
||||
|
||||
// Create a SmartFS instance with Node.js provider
|
||||
const fs = new SmartFs(new SmartFsProviderNode());
|
||||
|
||||
// Write and read files with fluent API
|
||||
await fs.file('/path/to/file.txt')
|
||||
.encoding('utf8')
|
||||
.write('Hello, World!');
|
||||
|
||||
const content = await fs.file('/path/to/file.txt')
|
||||
.encoding('utf8')
|
||||
.read();
|
||||
|
||||
console.log(content); // "Hello, World!"
|
||||
```
|
||||
|
||||
## API Overview
|
||||
|
||||
### File Operations
|
||||
|
||||
The fluent API uses **action-last pattern** - configure first, then execute:
|
||||
|
||||
```typescript
|
||||
// Read file
|
||||
const content = await fs.file('/path/to/file.txt')
|
||||
.encoding('utf8')
|
||||
.read();
|
||||
|
||||
// Write file
|
||||
await fs.file('/path/to/file.txt')
|
||||
.encoding('utf8')
|
||||
.mode(0o644)
|
||||
.write('content');
|
||||
|
||||
// Atomic write (write to temp, then rename)
|
||||
await fs.file('/path/to/file.txt')
|
||||
.atomic()
|
||||
.write('content');
|
||||
|
||||
// Append to file
|
||||
await fs.file('/path/to/file.txt')
|
||||
.encoding('utf8')
|
||||
.append('more content');
|
||||
|
||||
// Copy file
|
||||
await fs.file('/source.txt')
|
||||
.preserveTimestamps()
|
||||
.copy('/destination.txt');
|
||||
|
||||
// Move file
|
||||
await fs.file('/old.txt')
|
||||
.move('/new.txt');
|
||||
|
||||
// Delete file
|
||||
await fs.file('/path/to/file.txt')
|
||||
.delete();
|
||||
|
||||
// Check existence
|
||||
const exists = await fs.file('/path/to/file.txt').exists();
|
||||
|
||||
// Get stats
|
||||
const stats = await fs.file('/path/to/file.txt').stat();
|
||||
```
|
||||
|
||||
### Directory Operations
|
||||
|
||||
```typescript
|
||||
// Create directory
|
||||
await fs.directory('/path/to/dir').create();
|
||||
|
||||
// Create nested directories
|
||||
await fs.directory('/path/to/nested/dir')
|
||||
.recursive()
|
||||
.create();
|
||||
|
||||
// List directory
|
||||
const entries = await fs.directory('/path/to/dir').list();
|
||||
|
||||
// List recursively with filter
|
||||
const tsFiles = await fs.directory('/path/to/dir')
|
||||
.recursive()
|
||||
.filter('*.ts')
|
||||
.includeStats()
|
||||
.list();
|
||||
|
||||
// Filter with RegExp
|
||||
const files = await fs.directory('/path/to/dir')
|
||||
.filter(/\.txt$/)
|
||||
.list();
|
||||
|
||||
// Filter with function
|
||||
const largeFiles = await fs.directory('/path/to/dir')
|
||||
.includeStats()
|
||||
.filter(entry => entry.stats && entry.stats.size > 1024)
|
||||
.list();
|
||||
|
||||
// Delete directory
|
||||
await fs.directory('/path/to/dir')
|
||||
.recursive()
|
||||
.delete();
|
||||
|
||||
// Check existence
|
||||
const exists = await fs.directory('/path/to/dir').exists();
|
||||
```
|
||||
|
||||
### Streaming Operations
|
||||
|
||||
SmartFS uses **Web Streams API** for efficient handling of large files:
|
||||
|
||||
```typescript
|
||||
// Read stream
|
||||
const readStream = await fs.file('/large-file.bin')
|
||||
.chunkSize(64 * 1024)
|
||||
.readStream();
|
||||
|
||||
const reader = readStream.getReader();
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
// Process chunk (Uint8Array)
|
||||
console.log('Chunk size:', value.length);
|
||||
}
|
||||
|
||||
// Write stream
|
||||
const writeStream = await fs.file('/output.bin').writeStream();
|
||||
const writer = writeStream.getWriter();
|
||||
|
||||
await writer.write(new Uint8Array([1, 2, 3]));
|
||||
await writer.write(new Uint8Array([4, 5, 6]));
|
||||
await writer.close();
|
||||
|
||||
// Pipe streams
|
||||
const input = await fs.file('/input.txt').readStream();
|
||||
const output = await fs.file('/output.txt').writeStream();
|
||||
await input.pipeTo(output);
|
||||
```
|
||||
|
||||
### Transactions
|
||||
|
||||
Execute multiple file operations atomically with automatic rollback on failure:
|
||||
|
||||
```typescript
|
||||
// Simple transaction
|
||||
await fs.transaction()
|
||||
.file('/file1.txt').write('content 1')
|
||||
.file('/file2.txt').write('content 2')
|
||||
.file('/file3.txt').delete()
|
||||
.commit();
|
||||
|
||||
// Transaction with error handling
|
||||
const tx = fs.transaction()
|
||||
.file('/important.txt').write('critical data')
|
||||
.file('/backup.txt').copy('/backup-old.txt')
|
||||
.file('/temp.txt').delete();
|
||||
|
||||
try {
|
||||
await tx.commit();
|
||||
console.log('Transaction completed successfully');
|
||||
} catch (error) {
|
||||
console.error('Transaction failed and was rolled back:', error);
|
||||
// All operations are automatically reverted
|
||||
}
|
||||
```
|
||||
|
||||
### File Watching
|
||||
|
||||
Monitor filesystem changes with event-based watching:
|
||||
|
||||
```typescript
|
||||
// Watch a single file
|
||||
const watcher = await fs.watch('/path/to/file.txt')
|
||||
.onChange(event => {
|
||||
console.log('File changed:', event.path);
|
||||
})
|
||||
.start();
|
||||
|
||||
// Watch directory recursively
|
||||
const dirWatcher = await fs.watch('/path/to/dir')
|
||||
.recursive()
|
||||
.filter('*.ts')
|
||||
.debounce(100)
|
||||
.onChange(event => console.log('Changed:', event.path))
|
||||
.onAdd(event => console.log('Added:', event.path))
|
||||
.onDelete(event => console.log('Deleted:', event.path))
|
||||
.start();
|
||||
|
||||
// Stop watching
|
||||
await dirWatcher.stop();
|
||||
|
||||
// Watch with custom filter
|
||||
const customWatcher = await fs.watch('/path/to/dir')
|
||||
.recursive()
|
||||
.filter(path => path.endsWith('.ts') && !path.includes('test'))
|
||||
.onAll(event => {
|
||||
console.log(`${event.type}: ${event.path}`);
|
||||
})
|
||||
.start();
|
||||
```
|
||||
|
||||
## Providers
|
||||
|
||||
SmartFS supports multiple storage backends through providers:
|
||||
|
||||
### Node.js Provider
|
||||
|
||||
Uses Node.js `fs/promises` API for local filesystem operations:
|
||||
|
||||
```typescript
|
||||
import { SmartFs, SmartFsProviderNode } from '@push.rocks/smartfs';
|
||||
|
||||
const fs = new SmartFs(new SmartFsProviderNode());
|
||||
```
|
||||
|
||||
**Capabilities:**
|
||||
- ✅ File watching
|
||||
- ✅ Atomic writes
|
||||
- ✅ Transactions
|
||||
- ✅ Streaming
|
||||
- ✅ Symbolic links
|
||||
- ✅ File permissions
|
||||
|
||||
### Memory Provider
|
||||
|
||||
In-memory virtual filesystem, perfect for testing:
|
||||
|
||||
```typescript
|
||||
import { SmartFs, SmartFsProviderMemory } from '@push.rocks/smartfs';
|
||||
|
||||
const fs = new SmartFs(new SmartFsProviderMemory());
|
||||
|
||||
// All operations work in memory
|
||||
await fs.file('/virtual/file.txt').write('data');
|
||||
const content = await fs.file('/virtual/file.txt').read();
|
||||
|
||||
// Clear all data
|
||||
fs.provider.clear();
|
||||
```
|
||||
|
||||
**Capabilities:**
|
||||
- ✅ File watching
|
||||
- ✅ Atomic writes
|
||||
- ✅ Transactions
|
||||
- ✅ Streaming
|
||||
- ❌ Symbolic links
|
||||
- ✅ File permissions
|
||||
|
||||
### Custom Providers
|
||||
|
||||
Create your own provider by implementing `ISmartFsProvider`:
|
||||
|
||||
```typescript
|
||||
import type { ISmartFsProvider } from '@push.rocks/smartfs';
|
||||
|
||||
class MyCustomProvider implements ISmartFsProvider {
|
||||
public readonly name = 'custom';
|
||||
public readonly capabilities = {
|
||||
supportsWatch: true,
|
||||
supportsAtomic: true,
|
||||
supportsTransactions: true,
|
||||
supportsStreaming: true,
|
||||
supportsSymlinks: false,
|
||||
supportsPermissions: true,
|
||||
};
|
||||
|
||||
// Implement all required methods...
|
||||
async readFile(path: string, options?) { /* ... */ }
|
||||
async writeFile(path: string, content, options?) { /* ... */ }
|
||||
// ... etc
|
||||
}
|
||||
|
||||
const fs = new SmartFs(new MyCustomProvider());
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Encoding Options
|
||||
|
||||
```typescript
|
||||
// UTF-8 (default for text)
|
||||
await fs.file('/file.txt').encoding('utf8').write('text');
|
||||
|
||||
// Binary
|
||||
const buffer = Buffer.from([0x48, 0x65, 0x6c, 0x6c, 0x6f]);
|
||||
await fs.file('/file.bin').write(buffer);
|
||||
|
||||
// Base64
|
||||
await fs.file('/file.txt').encoding('base64').write('SGVsbG8=');
|
||||
|
||||
// Hex
|
||||
await fs.file('/file.txt').encoding('hex').write('48656c6c6f');
|
||||
```
|
||||
|
||||
### File Permissions
|
||||
|
||||
```typescript
|
||||
// Set file mode
|
||||
await fs.file('/script.sh')
|
||||
.mode(0o755)
|
||||
.write('#!/bin/bash\necho "Hello"');
|
||||
|
||||
// Set directory mode
|
||||
await fs.directory('/private')
|
||||
.mode(0o700)
|
||||
.create();
|
||||
```
|
||||
|
||||
### Complex Filtering
|
||||
|
||||
```typescript
|
||||
// Multiple conditions
|
||||
const files = await fs.directory('/src')
|
||||
.recursive()
|
||||
.includeStats()
|
||||
.filter(entry => {
|
||||
if (!entry.stats) return false;
|
||||
return entry.isFile &&
|
||||
entry.name.endsWith('.ts') &&
|
||||
entry.stats.size > 1024 &&
|
||||
entry.stats.mtime > new Date('2024-01-01');
|
||||
})
|
||||
.list();
|
||||
```
|
||||
|
||||
### Transaction Operations
|
||||
|
||||
```typescript
|
||||
// Complex transaction
|
||||
const tx = fs.transaction();
|
||||
|
||||
// Write multiple files
|
||||
tx.file('/data/file1.json').write(JSON.stringify(data1));
|
||||
tx.file('/data/file2.json').write(JSON.stringify(data2));
|
||||
|
||||
// Copy backups
|
||||
tx.file('/data/file1.json').copy('/backup/file1.json');
|
||||
tx.file('/data/file2.json').copy('/backup/file2.json');
|
||||
|
||||
// Delete old files
|
||||
tx.file('/data/old1.json').delete();
|
||||
tx.file('/data/old2.json').delete();
|
||||
|
||||
// Execute atomically
|
||||
await tx.commit();
|
||||
```
|
||||
|
||||
## Type Definitions
|
||||
|
||||
SmartFS is fully typed with TypeScript:
|
||||
|
||||
```typescript
|
||||
import type {
|
||||
IFileStats,
|
||||
IDirectoryEntry,
|
||||
IWatchEvent,
|
||||
ITransactionOperation,
|
||||
TEncoding,
|
||||
TFileMode,
|
||||
} from '@push.rocks/smartfs';
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
pnpm test
|
||||
|
||||
# Run specific test
|
||||
pnpm tstest test/test.memory.provider.ts --verbose
|
||||
|
||||
# Run with log output
|
||||
pnpm tstest test/test.node.provider.ts --logfile .nogit/testlogs/test.log
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
SmartFS throws descriptive errors:
|
||||
|
||||
```typescript
|
||||
try {
|
||||
await fs.file('/nonexistent.txt').read();
|
||||
} catch (error) {
|
||||
console.error(error.message);
|
||||
// "ENOENT: no such file or directory, open '/nonexistent.txt'"
|
||||
}
|
||||
|
||||
// Transactions automatically rollback on error
|
||||
try {
|
||||
await fs.transaction()
|
||||
.file('/file1.txt').write('data')
|
||||
.file('/file2.txt').write('data')
|
||||
.commit();
|
||||
} catch (error) {
|
||||
// All operations are reverted
|
||||
console.error('Transaction failed:', error);
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Use streaming** for large files (> 1MB)
|
||||
2. **Batch operations** with transactions
|
||||
3. **Use memory provider** for testing
|
||||
4. **Enable atomic writes** for critical data
|
||||
5. **Debounce watchers** to reduce event spam
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions welcome! Please ensure:
|
||||
- All tests pass
|
||||
- Code follows existing style
|
||||
- TypeScript types are complete
|
||||
- Documentation is updated
|
||||
|
||||
## License
|
||||
|
||||
MIT © [Lossless GmbH](https://lossless.gmbh)
|
||||
|
||||
---
|
||||
|
||||
For more information, visit [code.foss.global](https://code.foss.global/push.rocks/smartfs)
|
||||
Reference in New Issue
Block a user