smartfile/readme.hints.md

# SmartFile Implementation Hints

## Major Architectural Change (v12.0.0)

### Overview

SmartFile has been refactored to focus exclusively on **in-memory file representations** (SmartFile, StreamFile, VirtualDirectory). All filesystem operations have been moved to or delegated to `@push.rocks/smartfs`.

### Key Changes

1. **Factory Pattern Introduction**
   - New `SmartFileFactory` class introduced
   - Factory is bound to a `SmartFs` instance (from `@push.rocks/smartfs`)
   - All file instances are created through the factory
   - Factory methods: `fromFilePath()`, `fromUrl()`, `fromBuffer()`, `fromString()`, etc.

2. **SmartFile, StreamFile, VirtualDirectory**
   - Now accept optional `smartFs` parameter in constructor
   - Filesystem operations (write, read, delete) use `smartFs` if available
   - Fallback to legacy methods if `smartFs` not provided (for backward compatibility)
   - Static factory methods moved to `SmartFileFactory`

3. **Separation of Concerns**
   - **SmartFile** = In-memory file representation (path + content buffer)
   - **StreamFile** = Lazy-loaded streaming file representation
   - **VirtualDirectory** = Collection of SmartFiles in memory
   - **SmartFs** (from @push.rocks/smartfs) = Filesystem operations

### Usage Pattern

```typescript
import { SmartFileFactory } from '@push.rocks/smartfile';
import { SmartFs, SmartFsProviderNode } from '@push.rocks/smartfs';

// Create factory with SmartFs instance
const smartFs = new SmartFs(new SmartFsProviderNode());
const factory = new SmartFileFactory(smartFs);

// Or use default Node.js factory
const factory = SmartFileFactory.nodeFs();

// Create SmartFile through factory
const file = await factory.fromFilePath('./data.json');
await file.write(); // Uses bound smartFs instance

// Create StreamFile
const stream = await factory.streamFromPath('./large.zip');

// Create VirtualDirectory
const vdir = await factory.virtualDirectoryFromPath('./src');
```

### What Belongs Where

**SmartFile/StreamFile/VirtualDirectory (this package)**:
- ✅ In-memory file representation
- ✅ Content manipulation (edit, parse, transform)
- ✅ Loading content FROM sources (factory methods)
- ✅ Saving content TO destinations (write methods)
- ✅ Instance metadata (hash, size, mime type)
- ✅ Collection operations (for VirtualDirectory)

**SmartFs (@push.rocks/smartfs)**:
- ✅ Filesystem queries (exists, stat)
- ✅ File operations without content loading (copy, move)
- ✅ Directory operations (list, create, delete)
- ✅ Streaming operations (readStream, writeStream)
- ✅ Provider abstraction (Node.js, memory, S3, etc.)

### VirtualDirectory Collection Methods

VirtualDirectory now has comprehensive collection methods:

**Queries** (operate on in-memory collection):
- `exists(path)` / `has(path)` - Check if path exists in collection
- `getFileByPath(path)` - Get SmartFile from collection
- `listFiles()` - List all SmartFiles
- `listDirectories()` - List directory paths represented in collection
- `filter(predicate)` - Filter SmartFiles
- `map(fn)` - Transform SmartFiles
- `find(predicate)` - Find SmartFile
- `size()` - Number of files in collection
- `isEmpty()` - Check if collection is empty

**Mutations**:
- `addSmartfiles(files)` - Add files to collection
- `addSmartfile(file)` - Add single file
- `removeByPath(path)` - Remove from collection
- `clear()` - Empty collection
- `merge(otherVDir)` - Merge another VirtualDirectory

### Backward Compatibility

- Legacy namespace exports (`fs`, `memory`, `fsStream`, `interpreter`) are **deprecated**
- They remain functional for transition period but marked with `@deprecated`
- Will be removed in future version
- Users should migrate to `@push.rocks/smartfs` and `SmartFileFactory`

### Migration Path

**Old (deprecated)**:
```typescript
import * as smartfile from '@push.rocks/smartfile';

const file = await smartfile.SmartFile.fromFilePath('./file.txt');
await file.write();

const exists = await smartfile.fs.fileExists('./file.txt');
await smartfile.fs.copy('./a.txt', './b.txt');
```

**New (recommended)**:
```typescript
import { SmartFileFactory } from '@push.rocks/smartfile';
import { SmartFs, SmartFsProviderNode } from '@push.rocks/smartfs';

const factory = SmartFileFactory.nodeFs();
const file = await factory.fromFilePath('./file.txt');
await file.write();

const smartFs = new SmartFs(new SmartFsProviderNode());
const exists = await smartFs.file('./file.txt').exists();
await smartFs.file('./a.txt').copy('./b.txt');
```

### Testing Considerations

- Tests should use `SmartFileFactory.nodeFs()` or create custom factory with memory provider
- VirtualDirectory tests can use collection methods without filesystem access
- Filesystem operations should be tested via `@push.rocks/smartfs`

### Future Plans

- Remove deprecated namespace exports completely
- Full smartfs integration (remove fallback code)
- Potentially remove fs-extra, glob dependencies once smartfs is fully integrated

## listFileTree Function Enhancement (ts/fs.ts:367-415)

### Issue Fixed

The `listFileTree` function previously had inconsistent behavior with `**/*.extension` patterns across different systems and glob implementations. Some implementations would miss root-level files when using patterns like `**/*.ts`.

### Solution Implemented

Modified the function to explicitly handle `**/` patterns by:

1. Detecting when a pattern starts with `**/`
2. Extracting the file pattern after `**/` (e.g., `*.ts` from `**/*.ts`)
3. Running both the original pattern and the extracted root pattern
4. Using a Set to deduplicate results and ensure consistent ordering

### Key Benefits

- Guarantees consistent behavior across all systems
- Ensures both root-level and nested files are found with `**/*` patterns
- Maintains backward compatibility
- No performance degradation due to efficient deduplication

### Test Coverage

Added comprehensive tests to verify:

- Both root and nested files are found with `**/*.ts`
- No duplicate entries in results
- Edge cases with various file extensions work correctly

This fix ensures tools like `tsbuild check **/*.ts` work reliably across all systems.