Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ef2388b16f | |||
| 6f6868f2ad | |||
| ea57de06dd | |||
| 4894253e48 | |||
| 2f55f628f5 | |||
| 5dda689b4c | |||
| 8677f61da1 | |||
| 0c236d44d3 | |||
| 913c14bfcf | |||
| 2ce056f401 | |||
| 1c77a94b96 | |||
| 097ea96e99 | |||
| 5a6d9a2575 | |||
| 7a7ee041a3 |
71
changelog.md
71
changelog.md
@@ -1,5 +1,76 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 2025-12-11 - 6.2.2 - fix(watcher.node)
|
||||||
|
Defer events during initial scan, track full event sequences, and harden watcher shutdown
|
||||||
|
|
||||||
|
- Defer fs.watch events that arrive during the initial directory scan and process them after the scan completes to avoid race conditions where watchedFiles isn't populated.
|
||||||
|
- Debounce now tracks the full sequence of events per file (rename/change) instead of collapsing to the last event, preventing intermediate events from being lost.
|
||||||
|
- Detect delete+recreate via inode changes and emit unlink then add when appropriate; handle rapid create+delete sequences by emitting both events.
|
||||||
|
- During stop(), cancel pending debounced emits before flipping _isWatching and make handleFsEvent return early when watcher is stopped to prevent orphaned timeouts and post-stop emits.
|
||||||
|
- Add verbose logging of event sequences to aid debugging of complex fs event scenarios.
|
||||||
|
- Update tests to expect unlink + add for inode replacement scenarios.
|
||||||
|
- Version bump from 6.2.1 → 6.2.2
|
||||||
|
|
||||||
|
## 2025-12-10 - 6.2.1 - fix(watcher.node)
|
||||||
|
Handle fs.watch close without spurious restarts; add tests and improve test runner
|
||||||
|
|
||||||
|
- Prevent spurious restarts and noisy warnings on fs.watch 'close' by checking the internal isWatching flag before logging and restarting (ts/watchers/watcher.node.ts).
|
||||||
|
- Add comprehensive test suites covering basic operations, inode-change detection, atomic writes and stress scenarios (test/test.basic.ts, test/test.inode.ts, test/test.stress.ts).
|
||||||
|
- Remove outdated test (test/test.ts) and delete the test asset test/assets/hi.txt.
|
||||||
|
- Update test script in package.json to enable verbose logging, write a logfile and increase timeout to 120s to reduce flakiness in test runs.
|
||||||
|
|
||||||
|
## 2025-12-10 - 6.2.0 - feat(watchers)
|
||||||
|
Improve Node watcher robustness: file-level inode tracking, abortable restarts, restart race guards, and untracked-file handling
|
||||||
|
|
||||||
|
- Add file-level inode tracking to detect delete+recreate (editor atomic saves) and emit correct 'change'/'add' events
|
||||||
|
- Make restart delays abortable via AbortController so stop() cancels pending restarts and prevents orphan watchers
|
||||||
|
- Guard against concurrent/dual restarts with restartingPaths to avoid race conditions between health checks and error handlers
|
||||||
|
- Emit 'unlink' for deletions of previously untracked files (files created after initial scan) and clean up inode state
|
||||||
|
- Track file inodes during initial scan and update/cleanup inode state on events
|
||||||
|
- Improve logging for restart/inode/delete+recreate scenarios and update documentation/readme hints to v6.2.0+
|
||||||
|
|
||||||
|
## 2025-12-08 - 6.1.1 - fix(watchers/watcher.node)
|
||||||
|
Improve Node watcher robustness: inode tracking, ENOSPC detection, enhanced health checks and temp-file handling
|
||||||
|
|
||||||
|
- Track directory inodes (watchedInodes) and restart watchers if inode changes are detected (addresses stale watchers when directories are replaced).
|
||||||
|
- Health check now validates inode stability and explicitly detects ENOSPC (inotify max_user_watches) errors, emitting errors and logging a recommended fix command.
|
||||||
|
- Detect ENOSPC in FSWatcher error events and log guidance to increase inotify limits.
|
||||||
|
- Clear inode tracking state on watcher stop to avoid stale state across restarts.
|
||||||
|
- Improve temporary file handling and logging to avoid dropping events for atomic writes (only skip pure temp files and log skipped temp events).
|
||||||
|
- Documentation (readme.hints.md) updated with robustness notes, known fs.watch limitations, and example logs.
|
||||||
|
|
||||||
|
## 2025-12-08 - 6.1.0 - feat(watcher.node)
|
||||||
|
Add automatic restart, periodic health checks, and safe event emission to Node watcher; improve logging and stat handling
|
||||||
|
|
||||||
|
- NodeWatcher: introduced safeEmit() to isolate subscriber errors and prevent watcher crashes
|
||||||
|
- Auto-restart on failure with exponential backoff (1s → 30s) and up to 3 retry attempts per watched base path
|
||||||
|
- Periodic health checks (every 30s) to detect missing watched paths and trigger automatic restarts
|
||||||
|
- Handle unexpected FSWatcher 'close' events and restart watchers when they close silently
|
||||||
|
- Verbose lifecycle logging with `[smartwatch]` prefix for start/stop/health/restart events
|
||||||
|
- Clear restart tracking and stop health checks on watcher.stop() to ensure clean shutdown
|
||||||
|
- Improved statSafe() to normalize followSymlinks logic and log non-ENO errors as warnings
|
||||||
|
- Updated readme.hints.md documenting the new robustness features (v6.1.0+)
|
||||||
|
|
||||||
|
## 2025-12-08 - 6.0.0 - BREAKING CHANGE(watchers)
|
||||||
|
Replace polling-based write stabilization with debounce-based event coalescing and simplify watcher options
|
||||||
|
|
||||||
|
- Remove polling-based WriteStabilizer (ts/utils/write-stabilizer.ts) and related waitForWriteFinish logic
|
||||||
|
- Introduce debounce-based coalescing (debounceMs) for Node and Deno watchers (ts/watchers/watcher.node.ts, ts/watchers/watcher.deno.ts)
|
||||||
|
- Change IWatcherOptions: remove stabilityThreshold/pollInterval/maxWaitTime and add debounceMs
|
||||||
|
- Default watcher options updated to use debounceMs = 100
|
||||||
|
- Node/Deno watchers now skip editor temp files earlier, debounce duplicate events, and emit events directly (no size polling)
|
||||||
|
- Updated default watcher creation in Smartwatch to pass debounceMs
|
||||||
|
- Update package.json build script to run 'tsbuild tsfolders'
|
||||||
|
|
||||||
|
## 2025-12-08 - 5.1.0 - feat(watchers)
|
||||||
|
Improve write stabilization and ignore temporary editor files
|
||||||
|
|
||||||
|
- Add maxWaitTime option (ms) to IWatcherOptions and WriteStabilizer to cap how long stabilization will wait (default: 1000ms).
|
||||||
|
- WriteStabilizer: reduce default stabilityThreshold from 300ms to 100ms and track write start time to enforce maxWaitTime and avoid indefinite polling.
|
||||||
|
- Node and Deno watchers: detect and ignore common temporary/editor files (e.g. *.tmp.*, *.swp, *.swx, trailing ~, .#*) to prevent spurious events from atomic saves.
|
||||||
|
- Node watcher: treat rename (atomic-save) events as already-complete files and emit add/change immediately without stabilization.
|
||||||
|
- Deno watcher: use the configured maxWaitTime and polling-based stabilization for modify events to ensure consistent behavior across runtimes.
|
||||||
|
|
||||||
## 2025-11-30 - 5.0.0 - BREAKING CHANGE(@push.rocks/smartwatch)
|
## 2025-11-30 - 5.0.0 - BREAKING CHANGE(@push.rocks/smartwatch)
|
||||||
Rename package and update branding/docs: switch from @push.rocks/smartchok to @push.rocks/smartwatch, update repository/homepage/bugs URLs and author, and refresh README examples and install instructions.
|
Rename package and update branding/docs: switch from @push.rocks/smartchok to @push.rocks/smartwatch, update repository/homepage/bugs URLs and author, and refresh README examples and install instructions.
|
||||||
|
|
||||||
|
|||||||
@@ -1,14 +1,14 @@
|
|||||||
{
|
{
|
||||||
"name": "@push.rocks/smartwatch",
|
"name": "@push.rocks/smartwatch",
|
||||||
"version": "5.0.0",
|
"version": "6.2.2",
|
||||||
"private": false,
|
"private": false,
|
||||||
"description": "A cross-runtime file watcher with glob pattern support for Node.js, Deno, and Bun.",
|
"description": "A cross-runtime file watcher with glob pattern support for Node.js, Deno, and Bun.",
|
||||||
"main": "dist_ts/index.js",
|
"main": "dist_ts/index.js",
|
||||||
"typings": "dist_ts/index.d.ts",
|
"typings": "dist_ts/index.d.ts",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"test": "(npm run prepareTest && tstest test/)",
|
"test": "(npm run prepareTest && tstest test/ --verbose --logfile --timeout 120)",
|
||||||
"prepareTest": "(rm -f ./test/assets/hi.txt)",
|
"prepareTest": "(rm -f ./test/assets/hi.txt)",
|
||||||
"build": "tsbuild",
|
"build": "tsbuild tsfolders",
|
||||||
"buildDocs": "tsdoc"
|
"buildDocs": "tsdoc"
|
||||||
},
|
},
|
||||||
"repository": {
|
"repository": {
|
||||||
|
|||||||
@@ -71,6 +71,84 @@ The `WriteStabilizer` class replaces chokidar's built-in write stabilization:
|
|||||||
- **Deno**: Works on all versions with `Deno.watchFs()`
|
- **Deno**: Works on all versions with `Deno.watchFs()`
|
||||||
- **Bun**: Uses Node.js compatibility layer
|
- **Bun**: Uses Node.js compatibility layer
|
||||||
|
|
||||||
|
### Robustness Features (v6.1.0+)
|
||||||
|
|
||||||
|
The Node.js watcher includes automatic recovery mechanisms based on learnings from [chokidar](https://github.com/paulmillr/chokidar) and known [fs.watch issues](https://github.com/nodejs/node/issues/47058):
|
||||||
|
|
||||||
|
**Auto-restart on failure:**
|
||||||
|
- Watchers automatically restart when errors occur
|
||||||
|
- Exponential backoff (1s → 30s max)
|
||||||
|
- Maximum 3 retry attempts before giving up
|
||||||
|
- **v6.2.0+**: Race condition guards prevent orphan watchers when `stop()` is called during restart
|
||||||
|
|
||||||
|
**Inode tracking (critical for long-running watchers):**
|
||||||
|
- `fs.watch()` watches the **inode**, not the path!
|
||||||
|
- When directories are replaced (git checkout, atomic saves), the inode changes
|
||||||
|
- Health check detects inode changes and restarts the watcher
|
||||||
|
- **v6.2.0+**: File-level inode tracking detects delete+recreate (common editor save pattern)
|
||||||
|
- This is the most common cause of "watcher stops working after some time"
|
||||||
|
|
||||||
|
**Health check monitoring:**
|
||||||
|
- 30-second periodic health checks
|
||||||
|
- Detects when watched paths disappear
|
||||||
|
- Detects inode changes (directory replacement)
|
||||||
|
- Detects ENOSPC errors (inotify limit exceeded)
|
||||||
|
- **v6.2.0+**: Protected against dual-restart race conditions (health check + error handler)
|
||||||
|
|
||||||
|
**ENOSPC detection (Linux inotify limit):**
|
||||||
|
- Detects when `/proc/sys/fs/inotify/max_user_watches` is exceeded
|
||||||
|
- Logs fix command: `echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p`
|
||||||
|
|
||||||
|
**Error isolation:**
|
||||||
|
- Subscriber errors don't crash the watcher
|
||||||
|
- All events emitted via `safeEmit()` with try-catch
|
||||||
|
|
||||||
|
**Untracked file handling (v6.2.0+):**
|
||||||
|
- Files created after initial scan are properly detected
|
||||||
|
- Untracked file deletions emit `unlink` events instead of being silently dropped
|
||||||
|
|
||||||
|
**Event Deferral During Initial Scan (v6.2.2+):**
|
||||||
|
- Events are queued until initial scan completes
|
||||||
|
- Prevents race conditions where events arrive before `watchedFiles` is populated
|
||||||
|
- Deferred events are processed after scan completes
|
||||||
|
|
||||||
|
**Event Sequence Tracking (v6.2.2+):**
|
||||||
|
- Debounce now tracks ALL events in sequence, not just the last one
|
||||||
|
- Prevents losing intermediate events (e.g., add→change→delete no longer loses add)
|
||||||
|
- Intelligent processing of event sequences:
|
||||||
|
- Delete+recreate with inode change → emits `unlink` then `add`
|
||||||
|
- Rapid create+delete → emits both events
|
||||||
|
- Multiple changes → single `change` event (debouncing)
|
||||||
|
|
||||||
|
**Post-Stop Event Guards (v6.2.2+):**
|
||||||
|
- `handleFsEvent()` returns early if watcher is stopped
|
||||||
|
- Pending emits are cleared BEFORE setting `_isWatching = false`
|
||||||
|
- Prevents orphaned timeouts and events after `stop()`
|
||||||
|
|
||||||
|
**Verbose logging:**
|
||||||
|
- All lifecycle events logged with `[smartwatch]` prefix
|
||||||
|
- Event sequences logged for debugging complex scenarios
|
||||||
|
- Helps debug watcher issues in production
|
||||||
|
|
||||||
|
Example log output:
|
||||||
|
```
|
||||||
|
[smartwatch] Starting watcher for 1 base path(s)...
|
||||||
|
[smartwatch] Started watching: ./test/assets/
|
||||||
|
[smartwatch] Starting health check (every 30s)
|
||||||
|
[smartwatch] Watcher started with 1 active watcher(s)
|
||||||
|
[smartwatch] Health check: 1 watchers active
|
||||||
|
[smartwatch] Processing event sequence for ./src/file.ts: [rename, rename, change]
|
||||||
|
[smartwatch] File inode changed (delete+recreate): ./src/file.ts
|
||||||
|
[smartwatch] Previous inode: 12345, current: 67890
|
||||||
|
```
|
||||||
|
|
||||||
|
### Known fs.watch Limitations
|
||||||
|
|
||||||
|
1. **Watches inode, not path** - If a directory is replaced, watcher goes stale
|
||||||
|
2. **inotify limits on Linux** - Default `max_user_watches` (8192) may be too low
|
||||||
|
3. **No events for some atomic writes** - Some editors' save patterns may not trigger events
|
||||||
|
4. **Platform differences** - Linux uses inotify, macOS uses FSEvents/kqueue
|
||||||
|
|
||||||
### Testing
|
### Testing
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
HI
|
|
||||||
127
test/test.basic.ts
Normal file
127
test/test.basic.ts
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||||
|
import * as smartwatch from '../ts/index.js';
|
||||||
|
import * as smartfile from '@push.rocks/smartfile';
|
||||||
|
import * as smartrx from '@push.rocks/smartrx';
|
||||||
|
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import * as path from 'path';
|
||||||
|
|
||||||
|
// Skip in CI
|
||||||
|
if (process.env.CI) {
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
const TEST_DIR = './test/assets';
|
||||||
|
|
||||||
|
// Helper to delay
|
||||||
|
const delay = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));
|
||||||
|
|
||||||
|
// Helper to wait for an event with timeout
|
||||||
|
async function waitForEvent<T>(
|
||||||
|
observable: smartrx.rxjs.Observable<T>,
|
||||||
|
timeoutMs: number = 5000
|
||||||
|
): Promise<T> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const timeout = setTimeout(() => {
|
||||||
|
subscription.unsubscribe();
|
||||||
|
reject(new Error(`Timeout waiting for event after ${timeoutMs}ms`));
|
||||||
|
}, timeoutMs);
|
||||||
|
|
||||||
|
const subscription = observable.subscribe((value) => {
|
||||||
|
clearTimeout(timeout);
|
||||||
|
subscription.unsubscribe();
|
||||||
|
resolve(value);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let testSmartwatch: smartwatch.Smartwatch;
|
||||||
|
|
||||||
|
// ===========================================
|
||||||
|
// BASIC TESTS
|
||||||
|
// ===========================================
|
||||||
|
|
||||||
|
tap.test('should create a new instance', async () => {
|
||||||
|
testSmartwatch = new smartwatch.Smartwatch([]);
|
||||||
|
expect(testSmartwatch).toBeInstanceOf(smartwatch.Smartwatch);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should add paths and start watching', async () => {
|
||||||
|
testSmartwatch.add([`${TEST_DIR}/**/*.txt`]);
|
||||||
|
await testSmartwatch.start();
|
||||||
|
expect(testSmartwatch.status).toEqual('watching');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should detect ADD event for new files', async () => {
|
||||||
|
const addObservable = await testSmartwatch.getObservableFor('add');
|
||||||
|
const eventPromise = waitForEvent(addObservable);
|
||||||
|
|
||||||
|
// Create a new file
|
||||||
|
const testFile = path.join(TEST_DIR, 'add-test.txt');
|
||||||
|
await smartfile.memory.toFs('test content', testFile);
|
||||||
|
|
||||||
|
const [filePath] = await eventPromise;
|
||||||
|
expect(filePath).toInclude('add-test.txt');
|
||||||
|
|
||||||
|
// Cleanup
|
||||||
|
await fs.promises.unlink(testFile);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should detect CHANGE event for modified files', async () => {
|
||||||
|
// First create the file
|
||||||
|
const testFile = path.join(TEST_DIR, 'change-test.txt');
|
||||||
|
await smartfile.memory.toFs('initial content', testFile);
|
||||||
|
|
||||||
|
// Wait for add event to complete
|
||||||
|
await delay(200);
|
||||||
|
|
||||||
|
const changeObservable = await testSmartwatch.getObservableFor('change');
|
||||||
|
const eventPromise = waitForEvent(changeObservable);
|
||||||
|
|
||||||
|
// Modify the file
|
||||||
|
await smartfile.memory.toFs('modified content', testFile);
|
||||||
|
|
||||||
|
const [filePath] = await eventPromise;
|
||||||
|
expect(filePath).toInclude('change-test.txt');
|
||||||
|
|
||||||
|
// Cleanup
|
||||||
|
await fs.promises.unlink(testFile);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should detect UNLINK event for deleted files', async () => {
|
||||||
|
// First create the file
|
||||||
|
const testFile = path.join(TEST_DIR, 'unlink-test.txt');
|
||||||
|
await smartfile.memory.toFs('to be deleted', testFile);
|
||||||
|
|
||||||
|
// Wait for add event to complete
|
||||||
|
await delay(200);
|
||||||
|
|
||||||
|
const unlinkObservable = await testSmartwatch.getObservableFor('unlink');
|
||||||
|
const eventPromise = waitForEvent(unlinkObservable);
|
||||||
|
|
||||||
|
// Delete the file
|
||||||
|
await fs.promises.unlink(testFile);
|
||||||
|
|
||||||
|
const [filePath] = await eventPromise;
|
||||||
|
expect(filePath).toInclude('unlink-test.txt');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should stop the watch process', async () => {
|
||||||
|
await testSmartwatch.stop();
|
||||||
|
expect(testSmartwatch.status).toEqual('idle');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('cleanup: remove any remaining test files', async () => {
|
||||||
|
const files = await fs.promises.readdir(TEST_DIR);
|
||||||
|
for (const file of files) {
|
||||||
|
if (file.startsWith('add-') || file.startsWith('change-') || file.startsWith('unlink-')) {
|
||||||
|
try {
|
||||||
|
await fs.promises.unlink(path.join(TEST_DIR, file));
|
||||||
|
} catch {
|
||||||
|
// Ignore
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
export default tap.start();
|
||||||
135
test/test.inode.ts
Normal file
135
test/test.inode.ts
Normal file
@@ -0,0 +1,135 @@
|
|||||||
|
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||||
|
import * as smartwatch from '../ts/index.js';
|
||||||
|
import * as smartfile from '@push.rocks/smartfile';
|
||||||
|
import * as smartrx from '@push.rocks/smartrx';
|
||||||
|
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import * as path from 'path';
|
||||||
|
|
||||||
|
// Skip in CI
|
||||||
|
if (process.env.CI) {
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
const TEST_DIR = './test/assets';
|
||||||
|
|
||||||
|
// Helper to delay
|
||||||
|
const delay = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));
|
||||||
|
|
||||||
|
// Helper to wait for an event with timeout
|
||||||
|
async function waitForEvent<T>(
|
||||||
|
observable: smartrx.rxjs.Observable<T>,
|
||||||
|
timeoutMs: number = 5000
|
||||||
|
): Promise<T> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const timeout = setTimeout(() => {
|
||||||
|
subscription.unsubscribe();
|
||||||
|
reject(new Error(`Timeout waiting for event after ${timeoutMs}ms`));
|
||||||
|
}, timeoutMs);
|
||||||
|
|
||||||
|
const subscription = observable.subscribe((value) => {
|
||||||
|
clearTimeout(timeout);
|
||||||
|
subscription.unsubscribe();
|
||||||
|
resolve(value);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let testSmartwatch: smartwatch.Smartwatch;
|
||||||
|
|
||||||
|
// ===========================================
|
||||||
|
// INODE CHANGE DETECTION TESTS
|
||||||
|
// ===========================================
|
||||||
|
|
||||||
|
tap.test('setup: start watcher', async () => {
|
||||||
|
testSmartwatch = new smartwatch.Smartwatch([`${TEST_DIR}/**/*.txt`]);
|
||||||
|
await testSmartwatch.start();
|
||||||
|
expect(testSmartwatch.status).toEqual('watching');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should detect delete+recreate (inode change scenario)', async () => {
|
||||||
|
// This simulates what many editors do: delete file, create new file
|
||||||
|
const testFile = path.join(TEST_DIR, 'inode-test.txt');
|
||||||
|
|
||||||
|
// Create initial file
|
||||||
|
await smartfile.memory.toFs('initial content', testFile);
|
||||||
|
await delay(200);
|
||||||
|
|
||||||
|
// Get the initial inode
|
||||||
|
const initialStats = await fs.promises.stat(testFile);
|
||||||
|
const initialInode = initialStats.ino;
|
||||||
|
console.log(`[test] Initial inode: ${initialInode}`);
|
||||||
|
|
||||||
|
// With event sequence tracking, delete+recreate emits: unlink, then add
|
||||||
|
// This is more accurate than just emitting 'change'
|
||||||
|
const unlinkObservable = await testSmartwatch.getObservableFor('unlink');
|
||||||
|
const addObservable = await testSmartwatch.getObservableFor('add');
|
||||||
|
const unlinkPromise = waitForEvent(unlinkObservable, 3000);
|
||||||
|
const addPromise = waitForEvent(addObservable, 3000);
|
||||||
|
|
||||||
|
// Delete and recreate (this creates a new inode)
|
||||||
|
await fs.promises.unlink(testFile);
|
||||||
|
await smartfile.memory.toFs('recreated content', testFile);
|
||||||
|
|
||||||
|
// Check inode changed
|
||||||
|
const newStats = await fs.promises.stat(testFile);
|
||||||
|
const newInode = newStats.ino;
|
||||||
|
console.log(`[test] New inode: ${newInode}`);
|
||||||
|
expect(newInode).not.toEqual(initialInode);
|
||||||
|
|
||||||
|
// Should detect both unlink and add events for delete+recreate
|
||||||
|
const [[unlinkPath], [addPath]] = await Promise.all([unlinkPromise, addPromise]);
|
||||||
|
expect(unlinkPath).toInclude('inode-test.txt');
|
||||||
|
expect(addPath).toInclude('inode-test.txt');
|
||||||
|
console.log(`[test] Detected unlink + add events for delete+recreate`);
|
||||||
|
|
||||||
|
// Cleanup
|
||||||
|
await fs.promises.unlink(testFile);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should detect atomic write pattern (temp file + rename)', async () => {
|
||||||
|
// This simulates what Claude Code and many editors do:
|
||||||
|
// 1. Write to temp file (file.txt.tmp.12345)
|
||||||
|
// 2. Rename temp file to target file
|
||||||
|
const testFile = path.join(TEST_DIR, 'atomic-test.txt');
|
||||||
|
const tempFile = path.join(TEST_DIR, 'atomic-test.txt.tmp.12345');
|
||||||
|
|
||||||
|
// Create initial file
|
||||||
|
await smartfile.memory.toFs('initial content', testFile);
|
||||||
|
await delay(200);
|
||||||
|
|
||||||
|
const changeObservable = await testSmartwatch.getObservableFor('change');
|
||||||
|
const eventPromise = waitForEvent(changeObservable, 3000);
|
||||||
|
|
||||||
|
// Atomic write: create temp file then rename
|
||||||
|
await smartfile.memory.toFs('atomic content', tempFile);
|
||||||
|
await fs.promises.rename(tempFile, testFile);
|
||||||
|
|
||||||
|
// Should detect the change to the target file
|
||||||
|
const [filePath] = await eventPromise;
|
||||||
|
expect(filePath).toInclude('atomic-test.txt');
|
||||||
|
expect(filePath).not.toInclude('.tmp.');
|
||||||
|
|
||||||
|
// Cleanup
|
||||||
|
await fs.promises.unlink(testFile);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('teardown: stop watcher', async () => {
|
||||||
|
await testSmartwatch.stop();
|
||||||
|
expect(testSmartwatch.status).toEqual('idle');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('cleanup: remove test files', async () => {
|
||||||
|
const files = await fs.promises.readdir(TEST_DIR);
|
||||||
|
for (const file of files) {
|
||||||
|
if (file.startsWith('inode-') || file.startsWith('atomic-')) {
|
||||||
|
try {
|
||||||
|
await fs.promises.unlink(path.join(TEST_DIR, file));
|
||||||
|
} catch {
|
||||||
|
// Ignore
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
export default tap.start();
|
||||||
174
test/test.stress.ts
Normal file
174
test/test.stress.ts
Normal file
@@ -0,0 +1,174 @@
|
|||||||
|
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
||||||
|
import * as smartwatch from '../ts/index.js';
|
||||||
|
import * as smartfile from '@push.rocks/smartfile';
|
||||||
|
import * as smartrx from '@push.rocks/smartrx';
|
||||||
|
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import * as path from 'path';
|
||||||
|
|
||||||
|
// Skip in CI
|
||||||
|
if (process.env.CI) {
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
const TEST_DIR = './test/assets';
|
||||||
|
|
||||||
|
// Helper to delay
|
||||||
|
const delay = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));
|
||||||
|
|
||||||
|
// Helper to collect events
|
||||||
|
function collectEvents<T>(
|
||||||
|
observable: smartrx.rxjs.Observable<T>,
|
||||||
|
durationMs: number
|
||||||
|
): Promise<T[]> {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
const events: T[] = [];
|
||||||
|
const subscription = observable.subscribe((value) => {
|
||||||
|
events.push(value);
|
||||||
|
});
|
||||||
|
|
||||||
|
setTimeout(() => {
|
||||||
|
subscription.unsubscribe();
|
||||||
|
resolve(events);
|
||||||
|
}, durationMs);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let testSmartwatch: smartwatch.Smartwatch;
|
||||||
|
|
||||||
|
// ===========================================
|
||||||
|
// STRESS TESTS
|
||||||
|
// ===========================================
|
||||||
|
|
||||||
|
tap.test('setup: start watcher', async () => {
|
||||||
|
testSmartwatch = new smartwatch.Smartwatch([`${TEST_DIR}/**/*.txt`]);
|
||||||
|
await testSmartwatch.start();
|
||||||
|
expect(testSmartwatch.status).toEqual('watching');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('STRESS: rapid file modifications', async () => {
|
||||||
|
const testFile = path.join(TEST_DIR, 'stress-rapid.txt');
|
||||||
|
|
||||||
|
// Create initial file
|
||||||
|
await smartfile.memory.toFs('initial', testFile);
|
||||||
|
await delay(200);
|
||||||
|
|
||||||
|
const changeObservable = await testSmartwatch.getObservableFor('change');
|
||||||
|
|
||||||
|
// Rapidly modify the file 20 times
|
||||||
|
const RAPID_CHANGES = 20;
|
||||||
|
const eventCollector = collectEvents(changeObservable, 3000);
|
||||||
|
|
||||||
|
for (let i = 0; i < RAPID_CHANGES; i++) {
|
||||||
|
await smartfile.memory.toFs(`content ${i}`, testFile);
|
||||||
|
await delay(10); // 10ms between writes
|
||||||
|
}
|
||||||
|
|
||||||
|
const events = await eventCollector;
|
||||||
|
console.log(`[test] Rapid changes: sent ${RAPID_CHANGES}, received ${events.length} events`);
|
||||||
|
|
||||||
|
// Due to debouncing, we won't get all events, but we should get at least some
|
||||||
|
expect(events.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Cleanup
|
||||||
|
await fs.promises.unlink(testFile);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('STRESS: many files created rapidly', async () => {
|
||||||
|
const FILE_COUNT = 20;
|
||||||
|
const files: string[] = [];
|
||||||
|
|
||||||
|
const addObservable = await testSmartwatch.getObservableFor('add');
|
||||||
|
const eventCollector = collectEvents(addObservable, 5000);
|
||||||
|
|
||||||
|
// Create many files rapidly
|
||||||
|
for (let i = 0; i < FILE_COUNT; i++) {
|
||||||
|
const file = path.join(TEST_DIR, `stress-many-${i}.txt`);
|
||||||
|
files.push(file);
|
||||||
|
await smartfile.memory.toFs(`content ${i}`, file);
|
||||||
|
await delay(20); // 20ms between creates
|
||||||
|
}
|
||||||
|
|
||||||
|
const events = await eventCollector;
|
||||||
|
console.log(`[test] Many files: created ${FILE_COUNT}, detected ${events.length} events`);
|
||||||
|
|
||||||
|
// Should detect most or all files
|
||||||
|
expect(events.length).toBeGreaterThanOrEqual(FILE_COUNT * 0.8); // Allow 20% tolerance
|
||||||
|
|
||||||
|
// Cleanup all files
|
||||||
|
for (const file of files) {
|
||||||
|
try {
|
||||||
|
await fs.promises.unlink(file);
|
||||||
|
} catch {
|
||||||
|
// Ignore if already deleted
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('STRESS: interleaved add/change/delete operations', async () => {
|
||||||
|
const testFiles = [
|
||||||
|
path.join(TEST_DIR, 'stress-interleave-1.txt'),
|
||||||
|
path.join(TEST_DIR, 'stress-interleave-2.txt'),
|
||||||
|
path.join(TEST_DIR, 'stress-interleave-3.txt'),
|
||||||
|
];
|
||||||
|
|
||||||
|
// Create initial files
|
||||||
|
for (const file of testFiles) {
|
||||||
|
await smartfile.memory.toFs('initial', file);
|
||||||
|
}
|
||||||
|
await delay(300);
|
||||||
|
|
||||||
|
const addObservable = await testSmartwatch.getObservableFor('add');
|
||||||
|
const changeObservable = await testSmartwatch.getObservableFor('change');
|
||||||
|
const unlinkObservable = await testSmartwatch.getObservableFor('unlink');
|
||||||
|
|
||||||
|
const addEvents = collectEvents(addObservable, 3000);
|
||||||
|
const changeEvents = collectEvents(changeObservable, 3000);
|
||||||
|
const unlinkEvents = collectEvents(unlinkObservable, 3000);
|
||||||
|
|
||||||
|
// Interleaved operations
|
||||||
|
await smartfile.memory.toFs('changed 1', testFiles[0]); // change
|
||||||
|
await delay(50);
|
||||||
|
await fs.promises.unlink(testFiles[1]); // delete
|
||||||
|
await delay(50);
|
||||||
|
await smartfile.memory.toFs('recreated 1', testFiles[1]); // add (recreate)
|
||||||
|
await delay(50);
|
||||||
|
await smartfile.memory.toFs('changed 2', testFiles[2]); // change
|
||||||
|
await delay(50);
|
||||||
|
|
||||||
|
const [adds, changes, unlinks] = await Promise.all([addEvents, changeEvents, unlinkEvents]);
|
||||||
|
|
||||||
|
console.log(`[test] Interleaved: adds=${adds.length}, changes=${changes.length}, unlinks=${unlinks.length}`);
|
||||||
|
|
||||||
|
// Should have detected some events of each type
|
||||||
|
expect(changes.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Cleanup
|
||||||
|
for (const file of testFiles) {
|
||||||
|
try {
|
||||||
|
await fs.promises.unlink(file);
|
||||||
|
} catch {
|
||||||
|
// Ignore
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('teardown: stop watcher', async () => {
|
||||||
|
await testSmartwatch.stop();
|
||||||
|
expect(testSmartwatch.status).toEqual('idle');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('cleanup: remove stress test files', async () => {
|
||||||
|
const files = await fs.promises.readdir(TEST_DIR);
|
||||||
|
for (const file of files) {
|
||||||
|
if (file.startsWith('stress-')) {
|
||||||
|
try {
|
||||||
|
await fs.promises.unlink(path.join(TEST_DIR, file));
|
||||||
|
} catch {
|
||||||
|
// Ignore
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
export default tap.start();
|
||||||
50
test/test.ts
50
test/test.ts
@@ -1,50 +0,0 @@
|
|||||||
import { tap, expect } from '@git.zone/tstest/tapbundle';
|
|
||||||
import * as smartwatch from '../ts/index.js';
|
|
||||||
import * as smartfile from '@push.rocks/smartfile';
|
|
||||||
import * as smartpromise from '@push.rocks/smartpromise';
|
|
||||||
import * as smartrx from '@push.rocks/smartrx';
|
|
||||||
|
|
||||||
import * as fs from 'fs';
|
|
||||||
|
|
||||||
// the module to test
|
|
||||||
if (process.env.CI) {
|
|
||||||
process.exit(0);
|
|
||||||
}
|
|
||||||
|
|
||||||
let testSmartwatch: smartwatch.Smartwatch;
|
|
||||||
let testAddObservable: smartrx.rxjs.Observable<[string, fs.Stats]>;
|
|
||||||
let testSubscription: smartrx.rxjs.Subscription;
|
|
||||||
tap.test('should create a new instance', async () => {
|
|
||||||
testSmartwatch = new smartwatch.Smartwatch([]);
|
|
||||||
expect(testSmartwatch).toBeInstanceOf(smartwatch.Smartwatch);
|
|
||||||
});
|
|
||||||
|
|
||||||
tap.test('should add some files to watch and start', async () => {
|
|
||||||
testSmartwatch.add(['./test/**/*.txt']);
|
|
||||||
await testSmartwatch.start()
|
|
||||||
testSmartwatch.add(['./test/**/*.md']);
|
|
||||||
});
|
|
||||||
|
|
||||||
tap.test('should get an observable for a certain event', async () => {
|
|
||||||
await testSmartwatch.getObservableFor('add').then(async (observableArg) => {
|
|
||||||
testAddObservable = observableArg;
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
tap.test('should register an add operation', async () => {
|
|
||||||
let testDeferred = smartpromise.defer();
|
|
||||||
testSubscription = testAddObservable.subscribe(pathArg => {
|
|
||||||
const pathResult = pathArg[0];
|
|
||||||
console.log(pathResult);
|
|
||||||
testDeferred.resolve();
|
|
||||||
});
|
|
||||||
smartfile.memory.toFs('HI', './test/assets/hi.txt');
|
|
||||||
await testDeferred.promise;
|
|
||||||
});
|
|
||||||
|
|
||||||
tap.test('should stop the watch process', async (tools) => {
|
|
||||||
await tools.delayFor(10000);
|
|
||||||
testSmartwatch.stop();
|
|
||||||
});
|
|
||||||
|
|
||||||
export default tap.start();
|
|
||||||
@@ -3,6 +3,6 @@
|
|||||||
*/
|
*/
|
||||||
export const commitinfo = {
|
export const commitinfo = {
|
||||||
name: '@push.rocks/smartwatch',
|
name: '@push.rocks/smartwatch',
|
||||||
version: '5.0.0',
|
version: '6.2.2',
|
||||||
description: 'A cross-runtime file watcher with glob pattern support for Node.js, Deno, and Bun.'
|
description: 'A cross-runtime file watcher with glob pattern support for Node.js, Deno, and Bun.'
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -132,8 +132,7 @@ export class Smartwatch {
|
|||||||
basePaths: watchPaths,
|
basePaths: watchPaths,
|
||||||
depth: 4,
|
depth: 4,
|
||||||
followSymlinks: false,
|
followSymlinks: false,
|
||||||
stabilityThreshold: 300,
|
debounceMs: 100
|
||||||
pollInterval: 100
|
|
||||||
});
|
});
|
||||||
|
|
||||||
// Subscribe to watcher events and dispatch to appropriate subjects
|
// Subscribe to watcher events and dispatch to appropriate subjects
|
||||||
|
|||||||
@@ -1,97 +0,0 @@
|
|||||||
import * as fs from 'fs';
|
|
||||||
|
|
||||||
interface IPendingWrite {
|
|
||||||
lastSize: number;
|
|
||||||
lastChange: number;
|
|
||||||
timeoutId: ReturnType<typeof setTimeout>;
|
|
||||||
resolve: (stats: fs.Stats) => void;
|
|
||||||
reject: (error: Error) => void;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Implements awaitWriteFinish functionality by polling file size until stable.
|
|
||||||
* This replaces chokidar's built-in write stabilization.
|
|
||||||
*/
|
|
||||||
export class WriteStabilizer {
|
|
||||||
private pendingWrites = new Map<string, IPendingWrite>();
|
|
||||||
|
|
||||||
constructor(
|
|
||||||
private stabilityThreshold: number = 300,
|
|
||||||
private pollInterval: number = 100
|
|
||||||
) {}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Wait for a file write to complete by polling until size is stable
|
|
||||||
*/
|
|
||||||
async waitForWriteFinish(filePath: string): Promise<fs.Stats> {
|
|
||||||
// Cancel any existing pending check for this file
|
|
||||||
this.cancel(filePath);
|
|
||||||
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
const poll = async () => {
|
|
||||||
try {
|
|
||||||
const stats = await fs.promises.stat(filePath);
|
|
||||||
const pending = this.pendingWrites.get(filePath);
|
|
||||||
|
|
||||||
if (!pending) {
|
|
||||||
// Was cancelled
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const now = Date.now();
|
|
||||||
|
|
||||||
if (stats.size !== pending.lastSize) {
|
|
||||||
// Size changed - file is still being written, reset timer
|
|
||||||
pending.lastSize = stats.size;
|
|
||||||
pending.lastChange = now;
|
|
||||||
pending.timeoutId = setTimeout(poll, this.pollInterval);
|
|
||||||
} else if (now - pending.lastChange >= this.stabilityThreshold) {
|
|
||||||
// Size has been stable for the threshold duration
|
|
||||||
this.pendingWrites.delete(filePath);
|
|
||||||
resolve(stats);
|
|
||||||
} else {
|
|
||||||
// Size is the same but not yet past threshold
|
|
||||||
pending.timeoutId = setTimeout(poll, this.pollInterval);
|
|
||||||
}
|
|
||||||
} catch (error: any) {
|
|
||||||
this.pendingWrites.delete(filePath);
|
|
||||||
if (error.code === 'ENOENT') {
|
|
||||||
// File was deleted during polling
|
|
||||||
reject(new Error(`File was deleted: ${filePath}`));
|
|
||||||
} else {
|
|
||||||
reject(error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
this.pendingWrites.set(filePath, {
|
|
||||||
lastSize: -1,
|
|
||||||
lastChange: Date.now(),
|
|
||||||
timeoutId: setTimeout(poll, this.pollInterval),
|
|
||||||
resolve,
|
|
||||||
reject
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Cancel any pending write stabilization for a file
|
|
||||||
*/
|
|
||||||
cancel(filePath: string): void {
|
|
||||||
const pending = this.pendingWrites.get(filePath);
|
|
||||||
if (pending) {
|
|
||||||
clearTimeout(pending.timeoutId);
|
|
||||||
this.pendingWrites.delete(filePath);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Cancel all pending write stabilizations
|
|
||||||
*/
|
|
||||||
cancelAll(): void {
|
|
||||||
for (const [filePath, pending] of this.pendingWrites) {
|
|
||||||
clearTimeout(pending.timeoutId);
|
|
||||||
}
|
|
||||||
this.pendingWrites.clear();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -28,6 +28,5 @@ export const defaultWatcherOptions: IWatcherOptions = {
|
|||||||
basePaths: [],
|
basePaths: [],
|
||||||
depth: 4,
|
depth: 4,
|
||||||
followSymlinks: false,
|
followSymlinks: false,
|
||||||
stabilityThreshold: 300,
|
debounceMs: 100
|
||||||
pollInterval: 100
|
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -26,10 +26,8 @@ export interface IWatcherOptions {
|
|||||||
depth: number;
|
depth: number;
|
||||||
/** Whether to follow symbolic links */
|
/** Whether to follow symbolic links */
|
||||||
followSymlinks: boolean;
|
followSymlinks: boolean;
|
||||||
/** Stability threshold for write detection (ms) */
|
/** Debounce time in ms - events for the same file within this window are coalesced */
|
||||||
stabilityThreshold: number;
|
debounceMs: number;
|
||||||
/** Poll interval for write detection (ms) */
|
|
||||||
pollInterval: number;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
@@ -65,15 +65,27 @@ export class DenoWatcher implements IWatcher {
|
|||||||
private watcher: ReturnType<typeof Deno.watchFs> | null = null;
|
private watcher: ReturnType<typeof Deno.watchFs> | null = null;
|
||||||
private watchedFiles: Set<string> = new Set();
|
private watchedFiles: Set<string> = new Set();
|
||||||
private _isWatching = false;
|
private _isWatching = false;
|
||||||
private abortController: AbortController | null = null;
|
|
||||||
private recentEvents: Map<string, number> = new Map();
|
// Debounce: pending emits per file path
|
||||||
private throttleMs = 50;
|
private pendingEmits: Map<string, ReturnType<typeof setTimeout>> = new Map();
|
||||||
private pendingWrites: Map<string, ReturnType<typeof setTimeout>> = new Map();
|
|
||||||
|
|
||||||
public readonly events$ = new smartrx.rxjs.Subject<IWatchEvent>();
|
public readonly events$ = new smartrx.rxjs.Subject<IWatchEvent>();
|
||||||
|
|
||||||
constructor(private options: IWatcherOptions) {}
|
constructor(private options: IWatcherOptions) {}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if a file is a temporary file created by editors
|
||||||
|
*/
|
||||||
|
private isTemporaryFile(filePath: string): boolean {
|
||||||
|
const basename = filePath.split('/').pop() || '';
|
||||||
|
// Editor temp files: *.tmp.*, *.swp, *.swx, *~, .#*
|
||||||
|
if (basename.includes('.tmp.')) return true;
|
||||||
|
if (basename.endsWith('.swp') || basename.endsWith('.swx')) return true;
|
||||||
|
if (basename.endsWith('~')) return true;
|
||||||
|
if (basename.startsWith('.#')) return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
get isWatching(): boolean {
|
get isWatching(): boolean {
|
||||||
return this._isWatching;
|
return this._isWatching;
|
||||||
}
|
}
|
||||||
@@ -84,8 +96,6 @@ export class DenoWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
this.abortController = new AbortController();
|
|
||||||
|
|
||||||
// Start watching all base paths
|
// Start watching all base paths
|
||||||
this.watcher = Deno.watchFs(this.options.basePaths, { recursive: true });
|
this.watcher = Deno.watchFs(this.options.basePaths, { recursive: true });
|
||||||
this._isWatching = true;
|
this._isWatching = true;
|
||||||
@@ -109,11 +119,11 @@ export class DenoWatcher implements IWatcher {
|
|||||||
async stop(): Promise<void> {
|
async stop(): Promise<void> {
|
||||||
this._isWatching = false;
|
this._isWatching = false;
|
||||||
|
|
||||||
// Cancel all pending write stabilizations
|
// Cancel all pending debounced emits
|
||||||
for (const timeout of this.pendingWrites.values()) {
|
for (const timeout of this.pendingEmits.values()) {
|
||||||
clearTimeout(timeout);
|
clearTimeout(timeout);
|
||||||
}
|
}
|
||||||
this.pendingWrites.clear();
|
this.pendingEmits.clear();
|
||||||
|
|
||||||
// Close the watcher
|
// Close the watcher
|
||||||
if (this.watcher) {
|
if (this.watcher) {
|
||||||
@@ -122,7 +132,6 @@ export class DenoWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
|
|
||||||
this.watchedFiles.clear();
|
this.watchedFiles.clear();
|
||||||
this.recentEvents.clear();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -140,7 +149,7 @@ export class DenoWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
|
|
||||||
for (const filePath of event.paths) {
|
for (const filePath of event.paths) {
|
||||||
await this.handleDenoEvent(event.kind, filePath);
|
this.handleDenoEvent(event.kind, filePath);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
@@ -151,46 +160,56 @@ export class DenoWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Handle a Deno file system event
|
* Handle a Deno file system event - debounce and normalize
|
||||||
*/
|
*/
|
||||||
private async handleDenoEvent(
|
private handleDenoEvent(
|
||||||
kind: 'create' | 'modify' | 'remove' | 'access' | 'any' | 'other',
|
kind: 'create' | 'modify' | 'remove' | 'access' | 'any' | 'other',
|
||||||
filePath: string
|
filePath: string
|
||||||
): Promise<void> {
|
): void {
|
||||||
// Ignore 'access' events (just reading the file)
|
// Ignore 'access' events (just reading the file)
|
||||||
if (kind === 'access') {
|
if (kind === 'access') {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Throttle duplicate events
|
// Skip temporary files created by editors (atomic saves)
|
||||||
if (!this.shouldEmit(filePath, kind)) {
|
if (this.isTemporaryFile(filePath)) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Debounce: cancel any pending emit for this file
|
||||||
|
const existing = this.pendingEmits.get(filePath);
|
||||||
|
if (existing) {
|
||||||
|
clearTimeout(existing);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Schedule debounced emit
|
||||||
|
const timeout = setTimeout(() => {
|
||||||
|
this.pendingEmits.delete(filePath);
|
||||||
|
this.emitFileEvent(filePath, kind);
|
||||||
|
}, this.options.debounceMs);
|
||||||
|
|
||||||
|
this.pendingEmits.set(filePath, timeout);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Emit the actual file event after debounce
|
||||||
|
*/
|
||||||
|
private async emitFileEvent(
|
||||||
|
filePath: string,
|
||||||
|
kind: 'create' | 'modify' | 'remove' | 'access' | 'any' | 'other'
|
||||||
|
): Promise<void> {
|
||||||
try {
|
try {
|
||||||
if (kind === 'create') {
|
if (kind === 'create') {
|
||||||
const stats = await this.statSafe(filePath);
|
const stats = await this.statSafe(filePath);
|
||||||
if (stats) {
|
if (stats) {
|
||||||
// Wait for write to stabilize
|
|
||||||
await this.waitForWriteFinish(filePath);
|
|
||||||
const finalStats = await this.statSafe(filePath);
|
|
||||||
|
|
||||||
if (finalStats) {
|
|
||||||
this.watchedFiles.add(filePath);
|
this.watchedFiles.add(filePath);
|
||||||
const eventType: TWatchEventType = finalStats.isDirectory() ? 'addDir' : 'add';
|
const eventType: TWatchEventType = stats.isDirectory() ? 'addDir' : 'add';
|
||||||
this.events$.next({ type: eventType, path: filePath, stats: finalStats });
|
this.events$.next({ type: eventType, path: filePath, stats });
|
||||||
}
|
|
||||||
}
|
}
|
||||||
} else if (kind === 'modify') {
|
} else if (kind === 'modify') {
|
||||||
const stats = await this.statSafe(filePath);
|
const stats = await this.statSafe(filePath);
|
||||||
if (stats && !stats.isDirectory()) {
|
if (stats && !stats.isDirectory()) {
|
||||||
// Wait for write to stabilize
|
this.events$.next({ type: 'change', path: filePath, stats });
|
||||||
await this.waitForWriteFinish(filePath);
|
|
||||||
const finalStats = await this.statSafe(filePath);
|
|
||||||
|
|
||||||
if (finalStats) {
|
|
||||||
this.events$.next({ type: 'change', path: filePath, stats: finalStats });
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
} else if (kind === 'remove') {
|
} else if (kind === 'remove') {
|
||||||
const wasDirectory = this.isKnownDirectory(filePath);
|
const wasDirectory = this.isKnownDirectory(filePath);
|
||||||
@@ -205,43 +224,6 @@ export class DenoWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Wait for file write to complete (polling-based)
|
|
||||||
*/
|
|
||||||
private async waitForWriteFinish(filePath: string): Promise<void> {
|
|
||||||
return new Promise((resolve) => {
|
|
||||||
let lastSize = -1;
|
|
||||||
let lastChange = Date.now();
|
|
||||||
|
|
||||||
const poll = async () => {
|
|
||||||
try {
|
|
||||||
const stats = await this.statSafe(filePath);
|
|
||||||
if (!stats) {
|
|
||||||
resolve();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const now = Date.now();
|
|
||||||
if (stats.size !== lastSize) {
|
|
||||||
lastSize = stats.size;
|
|
||||||
lastChange = now;
|
|
||||||
this.pendingWrites.set(filePath, setTimeout(poll, this.options.pollInterval));
|
|
||||||
} else if (now - lastChange >= this.options.stabilityThreshold) {
|
|
||||||
this.pendingWrites.delete(filePath);
|
|
||||||
resolve();
|
|
||||||
} else {
|
|
||||||
this.pendingWrites.set(filePath, setTimeout(poll, this.options.pollInterval));
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
this.pendingWrites.delete(filePath);
|
|
||||||
resolve();
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
this.pendingWrites.set(filePath, setTimeout(poll, this.options.pollInterval));
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Scan directory and emit 'add' events for existing files
|
* Scan directory and emit 'add' events for existing files
|
||||||
*/
|
*/
|
||||||
@@ -253,6 +235,12 @@ export class DenoWatcher implements IWatcher {
|
|||||||
try {
|
try {
|
||||||
for await (const entry of Deno.readDir(dirPath)) {
|
for await (const entry of Deno.readDir(dirPath)) {
|
||||||
const fullPath = `${dirPath}/${entry.name}`;
|
const fullPath = `${dirPath}/${entry.name}`;
|
||||||
|
|
||||||
|
// Skip temp files during initial scan too
|
||||||
|
if (this.isTemporaryFile(fullPath)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
const stats = await this.statSafe(fullPath);
|
const stats = await this.statSafe(fullPath);
|
||||||
|
|
||||||
if (!stats) {
|
if (!stats) {
|
||||||
@@ -299,31 +287,4 @@ export class DenoWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Throttle duplicate events
|
|
||||||
*/
|
|
||||||
private shouldEmit(filePath: string, eventType: string): boolean {
|
|
||||||
const key = `${filePath}:${eventType}`;
|
|
||||||
const now = Date.now();
|
|
||||||
const lastEmit = this.recentEvents.get(key);
|
|
||||||
|
|
||||||
if (lastEmit && now - lastEmit < this.throttleMs) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
this.recentEvents.set(key, now);
|
|
||||||
|
|
||||||
// Clean up old entries periodically
|
|
||||||
if (this.recentEvents.size > 1000) {
|
|
||||||
const cutoff = now - this.throttleMs * 2;
|
|
||||||
for (const [k, time] of this.recentEvents) {
|
|
||||||
if (time < cutoff) {
|
|
||||||
this.recentEvents.delete(k);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ import * as fs from 'fs';
|
|||||||
import * as path from 'path';
|
import * as path from 'path';
|
||||||
import * as smartrx from '@push.rocks/smartrx';
|
import * as smartrx from '@push.rocks/smartrx';
|
||||||
import type { IWatcher, IWatcherOptions, IWatchEvent, TWatchEventType } from './interfaces.js';
|
import type { IWatcher, IWatcherOptions, IWatchEvent, TWatchEventType } from './interfaces.js';
|
||||||
import { WriteStabilizer } from '../utils/write-stabilizer.js';
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Node.js/Bun file watcher using native fs.watch API
|
* Node.js/Bun file watcher using native fs.watch API
|
||||||
@@ -11,17 +10,221 @@ export class NodeWatcher implements IWatcher {
|
|||||||
private watchers: Map<string, fs.FSWatcher> = new Map();
|
private watchers: Map<string, fs.FSWatcher> = new Map();
|
||||||
private watchedFiles: Set<string> = new Set();
|
private watchedFiles: Set<string> = new Set();
|
||||||
private _isWatching = false;
|
private _isWatching = false;
|
||||||
private writeStabilizer: WriteStabilizer;
|
|
||||||
private recentEvents: Map<string, number> = new Map();
|
// Debounce: pending emits per file path
|
||||||
private throttleMs = 50;
|
// Fix 2: Track event sequence instead of just last event type
|
||||||
|
// This prevents losing intermediate events (add→change→delete should not lose add)
|
||||||
|
private pendingEmits: Map<string, {
|
||||||
|
timeout: NodeJS.Timeout;
|
||||||
|
events: Array<'rename' | 'change'>;
|
||||||
|
}> = new Map();
|
||||||
|
|
||||||
|
// Restart tracking
|
||||||
|
private restartDelays: Map<string, number> = new Map();
|
||||||
|
private restartAttempts: Map<string, number> = new Map();
|
||||||
|
private healthCheckInterval: NodeJS.Timeout | null = null;
|
||||||
|
|
||||||
|
// Inode tracking - detect when directories are replaced (atomic saves, etc.)
|
||||||
|
// fs.watch watches the inode, not the path. If inode changes, we need to restart.
|
||||||
|
private watchedInodes: Map<string, bigint> = new Map();
|
||||||
|
|
||||||
|
// File inode tracking - detect when individual files are deleted and recreated
|
||||||
|
// This is critical: editors delete+recreate files, fs.watch watches OLD inode!
|
||||||
|
// See: https://github.com/paulmillr/chokidar/issues/972
|
||||||
|
private fileInodes: Map<string, bigint> = new Map();
|
||||||
|
|
||||||
|
// Abort controllers for pending restart delays - prevents orphan watchers on stop()
|
||||||
|
private restartAbortControllers: Map<string, AbortController> = new Map();
|
||||||
|
|
||||||
|
// Prevent concurrent restarts for the same path (health check + error can race)
|
||||||
|
private restartingPaths: Set<string> = new Set();
|
||||||
|
|
||||||
|
// Initial scan state - events are deferred until scan completes to avoid race conditions
|
||||||
|
// Without this, events can arrive before watchedFiles is populated, causing inconsistent state
|
||||||
|
private initialScanComplete: boolean = false;
|
||||||
|
private deferredEvents: Array<{basePath: string; filename: string; eventType: string}> = [];
|
||||||
|
|
||||||
|
// Configuration constants
|
||||||
|
private static readonly MAX_RETRIES = 3;
|
||||||
|
private static readonly INITIAL_RESTART_DELAY = 1000;
|
||||||
|
private static readonly MAX_RESTART_DELAY = 30000;
|
||||||
|
private static readonly HEALTH_CHECK_INTERVAL = 30000;
|
||||||
|
|
||||||
public readonly events$ = new smartrx.rxjs.Subject<IWatchEvent>();
|
public readonly events$ = new smartrx.rxjs.Subject<IWatchEvent>();
|
||||||
|
|
||||||
constructor(private options: IWatcherOptions) {
|
constructor(private options: IWatcherOptions) {}
|
||||||
this.writeStabilizer = new WriteStabilizer(
|
|
||||||
options.stabilityThreshold,
|
/**
|
||||||
options.pollInterval
|
* Safely emit an event, catching any subscriber errors
|
||||||
);
|
*/
|
||||||
|
private safeEmit(event: IWatchEvent): void {
|
||||||
|
try {
|
||||||
|
this.events$.next(event);
|
||||||
|
} catch (error) {
|
||||||
|
console.error('[smartwatch] Subscriber threw error (isolated):', error);
|
||||||
|
// Don't let subscriber errors kill the watcher
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Restart a watcher after an error with exponential backoff
|
||||||
|
* Includes guards against:
|
||||||
|
* - Dual restart race condition (health check + error handler calling simultaneously)
|
||||||
|
* - Orphan watchers when stop() is called during restart delay
|
||||||
|
*/
|
||||||
|
private async restartWatcher(basePath: string, error: Error): Promise<void> {
|
||||||
|
// Guard: Prevent concurrent restarts for the same path
|
||||||
|
if (this.restartingPaths.has(basePath)) {
|
||||||
|
console.log(`[smartwatch] Restart already in progress for ${basePath}, skipping`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.restartingPaths.add(basePath);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const attempts = (this.restartAttempts.get(basePath) || 0) + 1;
|
||||||
|
this.restartAttempts.set(basePath, attempts);
|
||||||
|
|
||||||
|
console.log(`[smartwatch] Watcher error for ${basePath}: ${error.message}`);
|
||||||
|
console.log(`[smartwatch] Restart attempt ${attempts}/${NodeWatcher.MAX_RETRIES}`);
|
||||||
|
|
||||||
|
if (attempts > NodeWatcher.MAX_RETRIES) {
|
||||||
|
console.error(`[smartwatch] Max retries exceeded for ${basePath}, giving up`);
|
||||||
|
this.safeEmit({
|
||||||
|
type: 'error',
|
||||||
|
path: basePath,
|
||||||
|
error: new Error(`Max restart retries (${NodeWatcher.MAX_RETRIES}) exceeded`)
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close failed watcher
|
||||||
|
const oldWatcher = this.watchers.get(basePath);
|
||||||
|
if (oldWatcher) {
|
||||||
|
try {
|
||||||
|
oldWatcher.close();
|
||||||
|
} catch {
|
||||||
|
// Ignore close errors
|
||||||
|
}
|
||||||
|
this.watchers.delete(basePath);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Exponential backoff with AbortController (so stop() can cancel)
|
||||||
|
const delay = this.restartDelays.get(basePath) || NodeWatcher.INITIAL_RESTART_DELAY;
|
||||||
|
console.log(`[smartwatch] Waiting ${delay}ms before restart...`);
|
||||||
|
|
||||||
|
const abortController = new AbortController();
|
||||||
|
this.restartAbortControllers.set(basePath, abortController);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await new Promise<void>((resolve, reject) => {
|
||||||
|
const timeout = setTimeout(resolve, delay);
|
||||||
|
abortController.signal.addEventListener('abort', () => {
|
||||||
|
clearTimeout(timeout);
|
||||||
|
reject(new Error('Restart aborted by stop()'));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
} catch (abortError) {
|
||||||
|
console.log(`[smartwatch] Restart aborted for ${basePath}`);
|
||||||
|
return; // stop() was called, don't continue
|
||||||
|
} finally {
|
||||||
|
this.restartAbortControllers.delete(basePath);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Double-check we're still watching after the delay
|
||||||
|
if (!this._isWatching) {
|
||||||
|
console.log(`[smartwatch] Watcher stopped during restart delay, aborting`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
this.restartDelays.set(basePath, Math.min(delay * 2, NodeWatcher.MAX_RESTART_DELAY));
|
||||||
|
|
||||||
|
try {
|
||||||
|
await this.watchPath(basePath, 0);
|
||||||
|
console.log(`[smartwatch] Successfully restarted watcher for ${basePath}`);
|
||||||
|
this.restartDelays.set(basePath, NodeWatcher.INITIAL_RESTART_DELAY);
|
||||||
|
this.restartAttempts.set(basePath, 0);
|
||||||
|
} catch (restartError) {
|
||||||
|
console.error(`[smartwatch] Restart failed for ${basePath}:`, restartError);
|
||||||
|
// Clear restartingPaths before recursive call
|
||||||
|
this.restartingPaths.delete(basePath);
|
||||||
|
this.restartWatcher(basePath, restartError as Error); // Recursive retry
|
||||||
|
return; // Don't delete from restartingPaths again in finally
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
this.restartingPaths.delete(basePath);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Start periodic health checks to detect silent failures
|
||||||
|
* Checks for:
|
||||||
|
* 1. Path no longer exists
|
||||||
|
* 2. Inode changed (directory was replaced - fs.watch watches inode, not path!)
|
||||||
|
*/
|
||||||
|
private startHealthCheck(): void {
|
||||||
|
console.log('[smartwatch] Starting health check (every 30s)');
|
||||||
|
this.healthCheckInterval = setInterval(async () => {
|
||||||
|
console.log(`[smartwatch] Health check: ${this.watchers.size} watchers active`);
|
||||||
|
for (const [basePath] of this.watchers) {
|
||||||
|
try {
|
||||||
|
const stats = await fs.promises.stat(basePath);
|
||||||
|
const currentInode = stats.ino;
|
||||||
|
const previousInode = this.watchedInodes.get(basePath);
|
||||||
|
|
||||||
|
if (!stats) {
|
||||||
|
console.error(`[smartwatch] Health check failed: ${basePath} no longer exists`);
|
||||||
|
this.safeEmit({
|
||||||
|
type: 'error',
|
||||||
|
path: basePath,
|
||||||
|
error: new Error('Watched path no longer exists')
|
||||||
|
});
|
||||||
|
this.restartWatcher(basePath, new Error('Watched path disappeared'));
|
||||||
|
} else if (previousInode !== undefined && BigInt(currentInode) !== previousInode) {
|
||||||
|
// CRITICAL: Inode changed! fs.watch is now watching a stale inode.
|
||||||
|
// This happens when the directory is replaced (atomic operations, git checkout, etc.)
|
||||||
|
console.warn(`[smartwatch] Inode changed for ${basePath}: ${previousInode} -> ${currentInode}`);
|
||||||
|
console.warn('[smartwatch] fs.watch watches inode, not path - restarting watcher');
|
||||||
|
this.restartWatcher(basePath, new Error('Inode changed - directory was replaced'));
|
||||||
|
}
|
||||||
|
} catch (error: any) {
|
||||||
|
if (error.code === 'ENOENT') {
|
||||||
|
console.error(`[smartwatch] Health check failed: ${basePath} no longer exists`);
|
||||||
|
this.restartWatcher(basePath, new Error('Watched path disappeared'));
|
||||||
|
} else if (error.code === 'ENOSPC') {
|
||||||
|
// inotify watch limit exceeded - critical system issue
|
||||||
|
console.error(`[smartwatch] ENOSPC: inotify watch limit exceeded!`);
|
||||||
|
console.error('[smartwatch] Fix: echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p');
|
||||||
|
this.safeEmit({ type: 'error', path: basePath, error });
|
||||||
|
} else {
|
||||||
|
console.error(`[smartwatch] Health check error for ${basePath}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, NodeWatcher.HEALTH_CHECK_INTERVAL);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stop health check interval
|
||||||
|
*/
|
||||||
|
private stopHealthCheck(): void {
|
||||||
|
if (this.healthCheckInterval) {
|
||||||
|
clearInterval(this.healthCheckInterval);
|
||||||
|
this.healthCheckInterval = null;
|
||||||
|
console.log('[smartwatch] Stopped health check');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if a file is a temporary file created by editors
|
||||||
|
*/
|
||||||
|
private isTemporaryFile(filePath: string): boolean {
|
||||||
|
const basename = path.basename(filePath);
|
||||||
|
// Editor temp files: *.tmp.*, *.swp, *.swx, *~, .#*
|
||||||
|
if (basename.includes('.tmp.')) return true;
|
||||||
|
if (basename.endsWith('.swp') || basename.endsWith('.swx')) return true;
|
||||||
|
if (basename.endsWith('~')) return true;
|
||||||
|
if (basename.startsWith('.#')) return true;
|
||||||
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
get isWatching(): boolean {
|
get isWatching(): boolean {
|
||||||
@@ -33,38 +236,93 @@ export class NodeWatcher implements IWatcher {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
console.log(`[smartwatch] Starting watcher for ${this.options.basePaths.length} base path(s)...`);
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
// Reset initial scan state
|
||||||
|
this.initialScanComplete = false;
|
||||||
|
this.deferredEvents = [];
|
||||||
|
|
||||||
// Start watching each base path
|
// Start watching each base path
|
||||||
|
// NOTE: Events may arrive immediately but will be deferred until scan completes
|
||||||
for (const basePath of this.options.basePaths) {
|
for (const basePath of this.options.basePaths) {
|
||||||
await this.watchPath(basePath, 0);
|
await this.watchPath(basePath, 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
this._isWatching = true;
|
this._isWatching = true;
|
||||||
|
|
||||||
|
// Start health check monitoring
|
||||||
|
this.startHealthCheck();
|
||||||
|
|
||||||
// Perform initial scan to emit 'add' events for existing files
|
// Perform initial scan to emit 'add' events for existing files
|
||||||
|
// This populates watchedFiles and fileInodes BEFORE we process events
|
||||||
for (const basePath of this.options.basePaths) {
|
for (const basePath of this.options.basePaths) {
|
||||||
await this.scanDirectory(basePath, 0);
|
await this.scanDirectory(basePath, 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Mark scan complete and process any events that arrived during scan
|
||||||
|
this.initialScanComplete = true;
|
||||||
|
if (this.deferredEvents.length > 0) {
|
||||||
|
console.log(`[smartwatch] Processing ${this.deferredEvents.length} deferred events from initial scan window`);
|
||||||
|
for (const event of this.deferredEvents) {
|
||||||
|
this.handleFsEvent(event.basePath, event.filename, event.eventType);
|
||||||
|
}
|
||||||
|
this.deferredEvents = [];
|
||||||
|
}
|
||||||
|
|
||||||
// Emit ready event
|
// Emit ready event
|
||||||
this.events$.next({ type: 'ready', path: '' });
|
this.safeEmit({ type: 'ready', path: '' });
|
||||||
|
console.log(`[smartwatch] Watcher started with ${this.watchers.size} active watcher(s)`);
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
this.events$.next({ type: 'error', path: '', error });
|
console.error('[smartwatch] Failed to start watcher:', error);
|
||||||
|
this.safeEmit({ type: 'error', path: '', error });
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async stop(): Promise<void> {
|
async stop(): Promise<void> {
|
||||||
|
console.log('[smartwatch] Stopping watcher...');
|
||||||
|
|
||||||
|
// Fix 4: Cancel pending debounced emits FIRST (before flag changes)
|
||||||
|
// This prevents handleFsEvent from creating new pendingEmits during shutdown
|
||||||
|
for (const pending of this.pendingEmits.values()) {
|
||||||
|
clearTimeout(pending.timeout);
|
||||||
|
}
|
||||||
|
this.pendingEmits.clear();
|
||||||
|
|
||||||
|
// NOW set the flag - handleFsEvent will return early after this
|
||||||
this._isWatching = false;
|
this._isWatching = false;
|
||||||
this.writeStabilizer.cancelAll();
|
|
||||||
|
// Stop health check monitoring
|
||||||
|
this.stopHealthCheck();
|
||||||
|
|
||||||
|
// Abort all pending restart delays (prevents orphan watchers)
|
||||||
|
for (const [path, controller] of this.restartAbortControllers) {
|
||||||
|
console.log(`[smartwatch] Aborting pending restart for: ${path}`);
|
||||||
|
controller.abort();
|
||||||
|
}
|
||||||
|
this.restartAbortControllers.clear();
|
||||||
|
|
||||||
// Close all watchers
|
// Close all watchers
|
||||||
for (const [watchPath, watcher] of this.watchers) {
|
for (const [watchPath, watcher] of this.watchers) {
|
||||||
|
console.log(`[smartwatch] Closing watcher for: ${watchPath}`);
|
||||||
watcher.close();
|
watcher.close();
|
||||||
}
|
}
|
||||||
this.watchers.clear();
|
this.watchers.clear();
|
||||||
this.watchedFiles.clear();
|
this.watchedFiles.clear();
|
||||||
this.recentEvents.clear();
|
|
||||||
|
// Clear all tracking state
|
||||||
|
this.restartDelays.clear();
|
||||||
|
this.restartAttempts.clear();
|
||||||
|
this.watchedInodes.clear();
|
||||||
|
this.fileInodes.clear();
|
||||||
|
this.restartingPaths.clear();
|
||||||
|
|
||||||
|
// Fix 5: Reset initial scan state
|
||||||
|
this.initialScanComplete = false;
|
||||||
|
this.deferredEvents = [];
|
||||||
|
|
||||||
|
console.log('[smartwatch] Watcher stopped');
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -82,6 +340,10 @@ export class NodeWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (stats.isDirectory()) {
|
if (stats.isDirectory()) {
|
||||||
|
// Store inode for health check - fs.watch watches inode, not path!
|
||||||
|
// If inode changes (directory replaced), watcher becomes stale
|
||||||
|
this.watchedInodes.set(watchPath, BigInt(stats.ino));
|
||||||
|
|
||||||
// Watch the directory with recursive option (Node.js 20+ supports this on all platforms)
|
// Watch the directory with recursive option (Node.js 20+ supports this on all platforms)
|
||||||
const watcher = fs.watch(
|
const watcher = fs.watch(
|
||||||
watchPath,
|
watchPath,
|
||||||
@@ -93,99 +355,217 @@ export class NodeWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
);
|
);
|
||||||
|
|
||||||
watcher.on('error', (error) => {
|
watcher.on('error', (error: NodeJS.ErrnoException) => {
|
||||||
this.events$.next({ type: 'error', path: watchPath, error });
|
console.error(`[smartwatch] FSWatcher error event on ${watchPath}:`, error);
|
||||||
|
|
||||||
|
// Detect inotify watch limit exceeded - common cause of "stops working"
|
||||||
|
if (error.code === 'ENOSPC') {
|
||||||
|
console.error('[smartwatch] CRITICAL: inotify watch limit exceeded!');
|
||||||
|
console.error('[smartwatch] Fix with: echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p');
|
||||||
|
}
|
||||||
|
|
||||||
|
this.safeEmit({ type: 'error', path: watchPath, error });
|
||||||
|
if (this._isWatching) {
|
||||||
|
this.restartWatcher(watchPath, error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Handle 'close' event - fs.watch can close without error
|
||||||
|
watcher.on('close', () => {
|
||||||
|
// Only log/restart if we didn't intentionally stop
|
||||||
|
if (this._isWatching) {
|
||||||
|
console.warn(`[smartwatch] FSWatcher closed unexpectedly for ${watchPath}`);
|
||||||
|
this.restartWatcher(watchPath, new Error('Watcher closed unexpectedly'));
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
this.watchers.set(watchPath, watcher);
|
this.watchers.set(watchPath, watcher);
|
||||||
|
console.log(`[smartwatch] Started watching: ${watchPath}`);
|
||||||
}
|
}
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
this.events$.next({ type: 'error', path: watchPath, error });
|
console.error(`[smartwatch] Failed to watch path ${watchPath}:`, error);
|
||||||
|
this.safeEmit({ type: 'error', path: watchPath, error });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Handle raw fs.watch events and normalize them
|
* Handle raw fs.watch events - debounce and normalize them
|
||||||
*/
|
*/
|
||||||
private async handleFsEvent(
|
private handleFsEvent(
|
||||||
basePath: string,
|
basePath: string,
|
||||||
filename: string,
|
filename: string,
|
||||||
eventType: 'rename' | 'change' | string
|
eventType: 'rename' | 'change' | string
|
||||||
): Promise<void> {
|
): void {
|
||||||
const fullPath = path.join(basePath, filename);
|
// Fix 3: Guard against post-stop events (events queued before watcher closed)
|
||||||
|
if (!this._isWatching) {
|
||||||
// Throttle duplicate events
|
|
||||||
if (!this.shouldEmit(fullPath, eventType)) {
|
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Fix 1: Defer events until initial scan completes
|
||||||
|
// This prevents race conditions where events arrive before watchedFiles is populated
|
||||||
|
if (!this.initialScanComplete) {
|
||||||
|
this.deferredEvents.push({ basePath, filename, eventType });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const fullPath = path.join(basePath, filename);
|
||||||
|
|
||||||
|
// Skip temporary files - but ONLY pure temp files, not the target of atomic writes
|
||||||
|
// Atomic writes: editor writes to file.tmp.xxx then renames to file
|
||||||
|
// We need to detect the final file, so only skip files that ARE temp files
|
||||||
|
// and haven't been renamed to the real file yet
|
||||||
|
if (this.isTemporaryFile(fullPath)) {
|
||||||
|
// For temp files, we still want to track if they get renamed TO a real file
|
||||||
|
// The 'rename' event fires for both source and target, so we'll catch the real file
|
||||||
|
console.log(`[smartwatch] Skipping temp file event: ${filename}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fix 2: Track event sequence in debounce instead of collapsing to last event
|
||||||
|
// This ensures we don't lose intermediate events (e.g., add→change→delete)
|
||||||
|
const existing = this.pendingEmits.get(fullPath);
|
||||||
|
if (existing) {
|
||||||
|
// Cancel existing timeout but KEEP the event sequence
|
||||||
|
clearTimeout(existing.timeout);
|
||||||
|
// Add this event to the sequence
|
||||||
|
existing.events.push(eventType as 'rename' | 'change');
|
||||||
|
// Reschedule the emit with the accumulated events
|
||||||
|
existing.timeout = setTimeout(() => {
|
||||||
|
const pending = this.pendingEmits.get(fullPath);
|
||||||
|
if (pending) {
|
||||||
|
this.pendingEmits.delete(fullPath);
|
||||||
|
this.emitFileEvent(fullPath, pending.events);
|
||||||
|
}
|
||||||
|
}, this.options.debounceMs);
|
||||||
|
} else {
|
||||||
|
// First event for this file - create new sequence
|
||||||
|
const timeout = setTimeout(() => {
|
||||||
|
const pending = this.pendingEmits.get(fullPath);
|
||||||
|
if (pending) {
|
||||||
|
this.pendingEmits.delete(fullPath);
|
||||||
|
this.emitFileEvent(fullPath, pending.events);
|
||||||
|
}
|
||||||
|
}, this.options.debounceMs);
|
||||||
|
|
||||||
|
this.pendingEmits.set(fullPath, {
|
||||||
|
timeout,
|
||||||
|
events: [eventType as 'rename' | 'change']
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Emit the actual file event after debounce
|
||||||
|
*
|
||||||
|
* Fix 2: Now receives event sequence instead of single event type
|
||||||
|
* This allows intelligent processing of rapid event sequences:
|
||||||
|
* - add→change→delete: File was created and deleted rapidly
|
||||||
|
* - rename→rename: File was deleted and recreated (or vice versa)
|
||||||
|
*
|
||||||
|
* Also handles file inode tracking to detect delete+recreate scenarios:
|
||||||
|
* - fs.watch watches the inode, not the path
|
||||||
|
* - When editors delete+recreate files, the inode changes
|
||||||
|
* - Without inode tracking, events for the new file would be missed
|
||||||
|
* - See: https://github.com/paulmillr/chokidar/issues/972
|
||||||
|
*/
|
||||||
|
private async emitFileEvent(
|
||||||
|
fullPath: string,
|
||||||
|
eventSequence: Array<'rename' | 'change'>
|
||||||
|
): Promise<void> {
|
||||||
try {
|
try {
|
||||||
const stats = await this.statSafe(fullPath);
|
const stats = await this.statSafe(fullPath);
|
||||||
|
|
||||||
if (eventType === 'rename') {
|
|
||||||
// 'rename' can mean add or unlink - check if file exists
|
|
||||||
if (stats) {
|
|
||||||
// File exists - it's either a new file or was renamed to this location
|
|
||||||
if (stats.isDirectory()) {
|
|
||||||
if (!this.watchedFiles.has(fullPath)) {
|
|
||||||
this.watchedFiles.add(fullPath);
|
|
||||||
this.events$.next({ type: 'addDir', path: fullPath, stats });
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// Wait for write to stabilize before emitting
|
|
||||||
try {
|
|
||||||
const stableStats = await this.writeStabilizer.waitForWriteFinish(fullPath);
|
|
||||||
const wasWatched = this.watchedFiles.has(fullPath);
|
const wasWatched = this.watchedFiles.has(fullPath);
|
||||||
this.watchedFiles.add(fullPath);
|
const previousInode = this.fileInodes.get(fullPath);
|
||||||
this.events$.next({
|
|
||||||
type: wasWatched ? 'change' : 'add',
|
// Analyze event sequence to understand what happened
|
||||||
path: fullPath,
|
const hasRename = eventSequence.includes('rename');
|
||||||
stats: stableStats
|
const hasChange = eventSequence.includes('change');
|
||||||
});
|
const renameCount = eventSequence.filter(e => e === 'rename').length;
|
||||||
} catch {
|
|
||||||
// File was deleted during stabilization
|
// Log sequence for debugging complex scenarios
|
||||||
if (this.watchedFiles.has(fullPath)) {
|
if (eventSequence.length > 1) {
|
||||||
this.watchedFiles.delete(fullPath);
|
console.log(`[smartwatch] Processing event sequence for ${fullPath}: [${eventSequence.join(', ')}]`);
|
||||||
this.events$.next({ type: 'unlink', path: fullPath });
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (stats) {
|
||||||
|
// File EXISTS now
|
||||||
|
const currentInode = BigInt(stats.ino);
|
||||||
|
const inodeChanged = previousInode !== undefined && previousInode !== currentInode;
|
||||||
|
|
||||||
|
if (stats.isDirectory()) {
|
||||||
|
if (!wasWatched) {
|
||||||
|
this.watchedFiles.add(fullPath);
|
||||||
|
this.safeEmit({ type: 'addDir', path: fullPath, stats });
|
||||||
|
}
|
||||||
|
// Directories don't track inodes at file level
|
||||||
|
} else {
|
||||||
|
// Update tracking
|
||||||
|
this.fileInodes.set(fullPath, currentInode);
|
||||||
|
this.watchedFiles.add(fullPath);
|
||||||
|
|
||||||
|
if (!wasWatched) {
|
||||||
|
// File wasn't tracked before - this is an add
|
||||||
|
// Even if there were multiple events, the end result is a new file
|
||||||
|
this.safeEmit({ type: 'add', path: fullPath, stats });
|
||||||
|
} else if (inodeChanged) {
|
||||||
|
// File was recreated with different inode (delete+recreate)
|
||||||
|
console.log(`[smartwatch] File inode changed (delete+recreate): ${fullPath}`);
|
||||||
|
console.log(`[smartwatch] Previous inode: ${previousInode}, current: ${currentInode}`);
|
||||||
|
// Multiple rename events with inode change = delete+recreate pattern
|
||||||
|
// Emit unlink for the old file, then add for the new one
|
||||||
|
if (renameCount >= 2) {
|
||||||
|
this.safeEmit({ type: 'unlink', path: fullPath });
|
||||||
|
this.safeEmit({ type: 'add', path: fullPath, stats });
|
||||||
|
} else {
|
||||||
|
// Single rename with inode change = atomic save (emit as change)
|
||||||
|
this.safeEmit({ type: 'change', path: fullPath, stats });
|
||||||
|
}
|
||||||
|
} else if (hasChange || hasRename) {
|
||||||
|
// File exists, was tracked, inode same - content changed
|
||||||
|
this.safeEmit({ type: 'change', path: fullPath, stats });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
// File doesn't exist - it was deleted
|
// File does NOT exist now - it was deleted
|
||||||
if (this.watchedFiles.has(fullPath)) {
|
|
||||||
const wasDir = this.isKnownDirectory(fullPath);
|
const wasDir = this.isKnownDirectory(fullPath);
|
||||||
|
|
||||||
|
if (wasWatched) {
|
||||||
|
// File was tracked and is now gone
|
||||||
this.watchedFiles.delete(fullPath);
|
this.watchedFiles.delete(fullPath);
|
||||||
this.events$.next({
|
this.fileInodes.delete(fullPath);
|
||||||
|
|
||||||
|
// If there were multiple events, file may have been created then deleted
|
||||||
|
if (renameCount >= 2 && !wasDir) {
|
||||||
|
// add→delete sequence - emit both events
|
||||||
|
console.log(`[smartwatch] File created and deleted rapidly: ${fullPath}`);
|
||||||
|
this.safeEmit({ type: 'add', path: fullPath });
|
||||||
|
this.safeEmit({ type: 'unlink', path: fullPath });
|
||||||
|
} else {
|
||||||
|
this.safeEmit({
|
||||||
type: wasDir ? 'unlinkDir' : 'unlink',
|
type: wasDir ? 'unlinkDir' : 'unlink',
|
||||||
path: fullPath
|
path: fullPath
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
|
||||||
} else if (eventType === 'change') {
|
|
||||||
// File was modified
|
|
||||||
if (stats && !stats.isDirectory()) {
|
|
||||||
try {
|
|
||||||
const stableStats = await this.writeStabilizer.waitForWriteFinish(fullPath);
|
|
||||||
// Check if this is a new file (not yet in watchedFiles)
|
|
||||||
const wasWatched = this.watchedFiles.has(fullPath);
|
|
||||||
if (!wasWatched) {
|
|
||||||
// This is actually an 'add' - file wasn't being watched before
|
|
||||||
this.watchedFiles.add(fullPath);
|
|
||||||
this.events$.next({ type: 'add', path: fullPath, stats: stableStats });
|
|
||||||
} else {
|
} else {
|
||||||
this.events$.next({ type: 'change', path: fullPath, stats: stableStats });
|
// File wasn't tracked - but events occurred for it
|
||||||
}
|
this.fileInodes.delete(fullPath);
|
||||||
} catch {
|
|
||||||
// File was deleted during write
|
if (renameCount >= 2) {
|
||||||
if (this.watchedFiles.has(fullPath)) {
|
// Multiple rename events for untracked file that doesn't exist
|
||||||
this.watchedFiles.delete(fullPath);
|
// Likely: created → deleted rapidly
|
||||||
this.events$.next({ type: 'unlink', path: fullPath });
|
console.log(`[smartwatch] Untracked file created and deleted: ${fullPath}`);
|
||||||
}
|
this.safeEmit({ type: 'add', path: fullPath });
|
||||||
|
this.safeEmit({ type: 'unlink', path: fullPath });
|
||||||
|
} else if (hasRename) {
|
||||||
|
// Single event for file that doesn't exist and wasn't tracked
|
||||||
|
console.log(`[smartwatch] Untracked file deleted: ${fullPath}`);
|
||||||
|
this.safeEmit({ type: 'unlink', path: fullPath });
|
||||||
}
|
}
|
||||||
|
// If only 'change' events for non-existent untracked file, ignore
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
this.events$.next({ type: 'error', path: fullPath, error });
|
this.safeEmit({ type: 'error', path: fullPath, error });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -202,6 +582,12 @@ export class NodeWatcher implements IWatcher {
|
|||||||
|
|
||||||
for (const entry of entries) {
|
for (const entry of entries) {
|
||||||
const fullPath = path.join(dirPath, entry.name);
|
const fullPath = path.join(dirPath, entry.name);
|
||||||
|
|
||||||
|
// Skip temp files during initial scan too
|
||||||
|
if (this.isTemporaryFile(fullPath)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
const stats = await this.statSafe(fullPath);
|
const stats = await this.statSafe(fullPath);
|
||||||
|
|
||||||
if (!stats) {
|
if (!stats) {
|
||||||
@@ -210,16 +596,18 @@ export class NodeWatcher implements IWatcher {
|
|||||||
|
|
||||||
if (entry.isDirectory()) {
|
if (entry.isDirectory()) {
|
||||||
this.watchedFiles.add(fullPath);
|
this.watchedFiles.add(fullPath);
|
||||||
this.events$.next({ type: 'addDir', path: fullPath, stats });
|
this.safeEmit({ type: 'addDir', path: fullPath, stats });
|
||||||
await this.scanDirectory(fullPath, depth + 1);
|
await this.scanDirectory(fullPath, depth + 1);
|
||||||
} else if (entry.isFile()) {
|
} else if (entry.isFile()) {
|
||||||
this.watchedFiles.add(fullPath);
|
this.watchedFiles.add(fullPath);
|
||||||
this.events$.next({ type: 'add', path: fullPath, stats });
|
// Track file inode for delete+recreate detection
|
||||||
|
this.fileInodes.set(fullPath, BigInt(stats.ino));
|
||||||
|
this.safeEmit({ type: 'add', path: fullPath, stats });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
if (error.code !== 'ENOENT' && error.code !== 'EACCES') {
|
if (error.code !== 'ENOENT' && error.code !== 'EACCES') {
|
||||||
this.events$.next({ type: 'error', path: dirPath, error });
|
this.safeEmit({ type: 'error', path: dirPath, error });
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -229,12 +617,16 @@ export class NodeWatcher implements IWatcher {
|
|||||||
*/
|
*/
|
||||||
private async statSafe(filePath: string): Promise<fs.Stats | null> {
|
private async statSafe(filePath: string): Promise<fs.Stats | null> {
|
||||||
try {
|
try {
|
||||||
if (this.options.followSymlinks) {
|
return await (this.options.followSymlinks
|
||||||
return await fs.promises.stat(filePath);
|
? fs.promises.stat(filePath)
|
||||||
} else {
|
: fs.promises.lstat(filePath));
|
||||||
return await fs.promises.lstat(filePath);
|
} catch (error: any) {
|
||||||
|
// Only silently return null for expected "file doesn't exist" errors
|
||||||
|
if (error.code === 'ENOENT' || error.code === 'ENOTDIR') {
|
||||||
|
return null;
|
||||||
}
|
}
|
||||||
} catch {
|
// Log other errors (permission, I/O) but still return null
|
||||||
|
console.warn(`[smartwatch] statSafe warning for ${filePath}: ${error.code} - ${error.message}`);
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -251,31 +643,4 @@ export class NodeWatcher implements IWatcher {
|
|||||||
}
|
}
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Throttle duplicate events
|
|
||||||
*/
|
|
||||||
private shouldEmit(filePath: string, eventType: string): boolean {
|
|
||||||
const key = `${filePath}:${eventType}`;
|
|
||||||
const now = Date.now();
|
|
||||||
const lastEmit = this.recentEvents.get(key);
|
|
||||||
|
|
||||||
if (lastEmit && now - lastEmit < this.throttleMs) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
this.recentEvents.set(key, now);
|
|
||||||
|
|
||||||
// Clean up old entries periodically
|
|
||||||
if (this.recentEvents.size > 1000) {
|
|
||||||
const cutoff = now - this.throttleMs * 2;
|
|
||||||
for (const [k, time] of this.recentEvents) {
|
|
||||||
if (time < cutoff) {
|
|
||||||
this.recentEvents.delete(k);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user