smartwatch/readme.hints.md

# smartchok - Technical Hints

## Native File Watching (v2.0.0+)

The module now uses native file watching APIs instead of chokidar, providing cross-runtime support for Node.js, Deno, and Bun.

### Exported Class

The package exports the `Smartwatch` class (not `Smartchok`):
```typescript
import { Smartwatch } from '@push.rocks/smartchok';
```

### Architecture

```
ts/
├── smartwatch.classes.smartwatch.ts   # Main Smartwatch class
├── smartwatch.plugins.ts              # Dependencies (smartenv, picomatch, etc.)
├── watchers/
│   ├── index.ts                       # Factory with runtime detection
│   ├── interfaces.ts                  # IWatcher interface and types
│   ├── watcher.node.ts                # Node.js/Bun implementation (fs.watch)
│   └── watcher.deno.ts                # Deno implementation (Deno.watchFs)
└── utils/
    └── write-stabilizer.ts            # awaitWriteFinish polling implementation
```

### Runtime Detection

Uses `@push.rocks/smartenv` v6.x for runtime detection:
- **Node.js/Bun**: Uses native `fs.watch()` with `{ recursive: true }`
- **Deno**: Uses `Deno.watchFs()` async iterable

### Dependencies

- **picomatch**: Glob pattern matching (zero deps, well-maintained)
- **@push.rocks/smartenv**: Runtime detection (Node.js, Deno, Bun)
- **@push.rocks/smartrx**: RxJS Subject/Observable management
- **@push.rocks/smartpromise**: Deferred promise utilities
- **@push.rocks/lik**: Stringmap for pattern storage

### Why picomatch?

Native file watching APIs don't support glob patterns. Picomatch provides glob pattern matching with:
- Zero dependencies
- 164M+ weekly downloads
- Excellent security profile
- Full glob syntax support

### Event Handling

Native events are normalized to a consistent interface:

| Node.js/Bun Event | Deno Event | Normalized Event |
|-------------------|------------|------------------|
| `rename` (file exists) | `create` | `add` |
| `rename` (file gone) | `remove` | `unlink` |
| `change` | `modify` | `change` |

### awaitWriteFinish Implementation

The `WriteStabilizer` class replaces chokidar's built-in write stabilization:
- Polls file size until stable (configurable threshold: 300ms default)
- Configurable poll interval (100ms default)
- Handles file deletion during write detection

### Platform Requirements

- **Node.js 20+**: Required for native recursive watching on all platforms
- **Deno**: Works on all versions with `Deno.watchFs()`
- **Bun**: Uses Node.js compatibility layer

### Architecture (v6.3.0+) - Chokidar-Inspired

The Node.js watcher has been refactored with elegant patterns inspired by [chokidar](https://github.com/paulmillr/chokidar):

**DirEntry Class:**
- Tracks directory contents with proper disposal
- Encapsulates file tracking and inode management
- `dispose()` method freezes object to catch use-after-cleanup bugs

**Path Normalization (v6.3.1+):**
- ALL paths are normalized to absolute at entry points
- Prevents relative/absolute path mismatch bugs
- `watchPath()`, `handleFsEvent()`, `scanDirectory()` all resolve paths

**Restart Rescan (v6.3.1+):**
- When watcher restarts, it rescans the directory
- Catches files created during the restart window
- Clears stale DirEntry data before rescan
- Clears pending unlink timeouts to prevent stale events

**ENOSPC Handling (v6.3.1+):**
- inotify limit errors now trigger watcher restart
- Previously only logged error without recovery

**Atomic Write Handling:**
- Unlink events are queued with 100ms delay
- If add event arrives for same path within delay, unlink is cancelled
- Emits single `change` event instead of `unlink` + `add`
- Handles editor atomic saves elegantly

**Closer Registry:**
- Maps watch paths to cleanup functions
- Ensures proper resource cleanup on stop
- `addCloser()` / `runClosers()` pattern

**Event Constants Object:**
```typescript
const EV = {
  ADD: 'add',
  CHANGE: 'change',
  UNLINK: 'unlink',
  ADD_DIR: 'addDir',
  UNLINK_DIR: 'unlinkDir',
  READY: 'ready',
  ERROR: 'error',
} as const;
```

**Configuration Constants:**
```typescript
const CONFIG = {
  MAX_RETRIES: 3,
  INITIAL_RESTART_DELAY: 1000,
  MAX_RESTART_DELAY: 30000,
  HEALTH_CHECK_INTERVAL: 30000,
  ATOMIC_DELAY: 100,
  TEMP_FILE_DELAY: 50,
} as const;
```

### Robustness Features (v6.1.0+)

The Node.js watcher includes automatic recovery mechanisms based on learnings from [chokidar](https://github.com/paulmillr/chokidar) and known [fs.watch issues](https://github.com/nodejs/node/issues/47058):

**Auto-restart on failure:**
- Watchers automatically restart when errors occur
- Exponential backoff (1s → 30s max)
- Maximum 3 retry attempts before giving up
- **v6.2.0+**: Race condition guards prevent orphan watchers when `stop()` is called during restart

**Inode tracking (critical for long-running watchers):**
- `fs.watch()` watches the **inode**, not the path!
- When directories are replaced (git checkout, atomic saves), the inode changes
- Health check detects inode changes and restarts the watcher
- **v6.2.0+**: File-level inode tracking detects delete+recreate (common editor save pattern)
- This is the most common cause of "watcher stops working after some time"

**Health check monitoring:**
- 30-second periodic health checks
- Detects when watched paths disappear
- Detects inode changes (directory replacement)
- Detects ENOSPC errors (inotify limit exceeded)
- **v6.2.0+**: Protected against dual-restart race conditions (health check + error handler)

**ENOSPC detection (Linux inotify limit):**
- Detects when `/proc/sys/fs/inotify/max_user_watches` is exceeded
- Logs fix command: `echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p`

**Error isolation:**
- Subscriber errors don't crash the watcher
- All events emitted via `safeEmit()` with try-catch

**Untracked file handling (v6.2.0+):**
- Files created after initial scan are properly detected
- Untracked file deletions emit `unlink` events instead of being silently dropped

**Event Deferral During Initial Scan (v6.2.2+):**
- Events are queued until initial scan completes
- Prevents race conditions where events arrive before `watchedFiles` is populated
- Deferred events are processed after scan completes

**Event Sequence Tracking (v6.2.2+):**
- Debounce now tracks ALL events in sequence, not just the last one
- Prevents losing intermediate events (e.g., add→change→delete no longer loses add)
- Intelligent processing of event sequences:
  - Delete+recreate with inode change → emits `unlink` then `add`
  - Rapid create+delete → emits both events
  - Multiple changes → single `change` event (debouncing)

**Post-Stop Event Guards (v6.2.2+):**
- `handleFsEvent()` returns early if watcher is stopped
- Pending emits are cleared BEFORE setting `_isWatching = false`
- Prevents orphaned timeouts and events after `stop()`

**Verbose logging:**
- All lifecycle events logged with `[smartwatch]` prefix
- Event sequences logged for debugging complex scenarios
- Helps debug watcher issues in production

Example log output:
```
[smartwatch] Starting watcher for 1 base path(s)...
[smartwatch] Started watching: ./test/assets/
[smartwatch] Starting health check (every 30s)
[smartwatch] Watcher started with 1 active watcher(s)
[smartwatch] Health check: 1 watchers active
[smartwatch] Processing event sequence for ./src/file.ts: [rename, rename, change]
[smartwatch] File inode changed (delete+recreate): ./src/file.ts
[smartwatch]   Previous inode: 12345, current: 67890
```

### Known fs.watch Limitations

1. **Watches inode, not path** - If a directory is replaced, watcher goes stale
2. **inotify limits on Linux** - Default `max_user_watches` (8192) may be too low
3. **No events for some atomic writes** - Some editors' save patterns may not trigger events
4. **Platform differences** - Linux uses inotify, macOS uses FSEvents/kqueue

### Testing

```bash
pnpm test
```

Test files:
- **test.basic.ts** - Core functionality (add, change, unlink events)
- **test.inode.ts** - Inode change detection, atomic writes
- **test.stress.ts** - Rapid modifications, many files, interleaved operations

Tests verify:
- Creating Smartwatch instance
- Adding glob patterns
- Receiving 'add', 'change', 'unlink' events
- Inode change detection (delete+recreate pattern)
- Atomic write pattern (temp file + rename)
- Rapid file modifications (debouncing)
- Many files created rapidly
- Interleaved add/change/delete operations
- Graceful shutdown

## Dev Dependencies

- Using `@git.zone/tstest` v3.x with tapbundle
- Import from `@git.zone/tstest/tapbundle`