smartfile/readme.hints.md

SmartFile Implementation Hints

Major Architectural Change (v12.0.0)

Overview

SmartFile has been refactored to focus exclusively on in-memory file representations (SmartFile, StreamFile, VirtualDirectory). All filesystem operations have been moved to or delegated to @push.rocks/smartfs.

Key Changes

1. Factory Pattern Introduction

   • New SmartFileFactory class introduced
   • Factory is bound to a SmartFs instance (from @push.rocks/smartfs)
   • All file instances are created through the factory
   • Factory methods: fromFilePath(), fromUrl(), fromBuffer(), fromString(), etc.

2. SmartFile, StreamFile, VirtualDirectory

   • Now accept optional smartFs parameter in constructor
   • Filesystem operations (write, read, delete) use smartFs if available
   • Fallback to legacy methods if smartFs not provided (for backward compatibility)
   • Static factory methods moved to SmartFileFactory

3. Separation of Concerns

   • SmartFile = In-memory file representation (path + content buffer)
   • StreamFile = Lazy-loaded streaming file representation
   • VirtualDirectory = Collection of SmartFiles in memory
   • SmartFs (from @push.rocks/smartfs) = Filesystem operations

Usage Pattern

import { SmartFileFactory } from '@push.rocks/smartfile';
import { SmartFs, SmartFsProviderNode } from '@push.rocks/smartfs';

// Create factory with SmartFs instance
const smartFs = new SmartFs(new SmartFsProviderNode());
const factory = new SmartFileFactory(smartFs);

// Or use default Node.js factory
const factory = SmartFileFactory.nodeFs();

// Create SmartFile through factory
const file = await factory.fromFilePath('./data.json');
await file.write(); // Uses bound smartFs instance

// Create StreamFile
const stream = await factory.streamFromPath('./large.zip');

// Create VirtualDirectory
const vdir = await factory.virtualDirectoryFromPath('./src');

What Belongs Where

SmartFile/StreamFile/VirtualDirectory (this package):

• ✅ In-memory file representation
• ✅ Content manipulation (edit, parse, transform)
• ✅ Loading content FROM sources (factory methods)
• ✅ Saving content TO destinations (write methods)
• ✅ Instance metadata (hash, size, mime type)
• ✅ Collection operations (for VirtualDirectory)

SmartFs (@push.rocks/smartfs):

• ✅ Filesystem queries (exists, stat)
• ✅ File operations without content loading (copy, move)
• ✅ Directory operations (list, create, delete)
• ✅ Streaming operations (readStream, writeStream)
• ✅ Provider abstraction (Node.js, memory, S3, etc.)

VirtualDirectory Collection Methods

VirtualDirectory now has comprehensive collection methods:

Queries (operate on in-memory collection):

• exists(path) / has(path) - Check if path exists in collection
• getFileByPath(path) - Get SmartFile from collection
• listFiles() - List all SmartFiles
• listDirectories() - List directory paths represented in collection
• filter(predicate) - Filter SmartFiles
• map(fn) - Transform SmartFiles
• find(predicate) - Find SmartFile
• size() - Number of files in collection
• isEmpty() - Check if collection is empty

Mutations:

• addSmartfiles(files) - Add files to collection
• addSmartfile(file) - Add single file
• removeByPath(path) - Remove from collection
• clear() - Empty collection
• merge(otherVDir) - Merge another VirtualDirectory

Backward Compatibility

• Legacy namespace exports (fs, memory, fsStream, interpreter) are deprecated
• They remain functional for transition period but marked with @deprecated
• Will be removed in future version
• Users should migrate to @push.rocks/smartfs and SmartFileFactory

Migration Path

Old (deprecated):

import * as smartfile from '@push.rocks/smartfile';

const file = await smartfile.SmartFile.fromFilePath('./file.txt');
await file.write();

const exists = await smartfile.fs.fileExists('./file.txt');
await smartfile.fs.copy('./a.txt', './b.txt');

New (recommended):

import { SmartFileFactory } from '@push.rocks/smartfile';
import { SmartFs, SmartFsProviderNode } from '@push.rocks/smartfs';

const factory = SmartFileFactory.nodeFs();
const file = await factory.fromFilePath('./file.txt');
await file.write();

const smartFs = new SmartFs(new SmartFsProviderNode());
const exists = await smartFs.file('./file.txt').exists();
await smartFs.file('./a.txt').copy('./b.txt');

Testing Considerations

• Tests should use SmartFileFactory.nodeFs() or create custom factory with memory provider
• VirtualDirectory tests can use collection methods without filesystem access
• Filesystem operations should be tested via @push.rocks/smartfs

Future Plans

• Remove deprecated namespace exports completely
• Full smartfs integration (remove fallback code)
• Potentially remove fs-extra, glob dependencies once smartfs is fully integrated

listFileTree Function Enhancement (ts/fs.ts:367-415)

Issue Fixed

The listFileTree function previously had inconsistent behavior with **/*.extension patterns across different systems and glob implementations. Some implementations would miss root-level files when using patterns like **/*.ts.

Solution Implemented

Modified the function to explicitly handle **/ patterns by:

1. Detecting when a pattern starts with **/
2. Extracting the file pattern after **/ (e.g., *.ts from **/*.ts)
3. Running both the original pattern and the extracted root pattern
4. Using a Set to deduplicate results and ensure consistent ordering

Key Benefits

• Guarantees consistent behavior across all systems
• Ensures both root-level and nested files are found with **/* patterns
• Maintains backward compatibility
• No performance degradation due to efficient deduplication

Test Coverage

Added comprehensive tests to verify:

• Both root and nested files are found with **/*.ts
• No duplicate entries in results
• Edge cases with various file extensions work correctly

This fix ensures tools like tsbuild check **/*.ts work reliably across all systems.