BREAKING CHANGE(taskbuffer): Introduce constraint-based concurrency with TaskConstraintGroup and TaskManager integration; remove legacy TaskRunner and several Task APIs (breaking); add typed Task.data and update exports and tests.

2026-02-15 12:20:01 +00:00
parent 9d78933a46
commit d3e8ff1a11
16 changed files with 924 additions and 296 deletions
--- a/readme.md
+++ b/readme.md
@@ -1,6 +1,6 @@
 # @push.rocks/taskbuffer 🚀

-> **Modern TypeScript task orchestration with smart buffering, scheduling, labels, and real-time event streaming**
+> **Modern TypeScript task orchestration with constraint-based concurrency, smart buffering, scheduling, labels, and real-time event streaming**

 [![npm version](https://img.shields.io/npm/v/@push.rocks/taskbuffer.svg)](https://www.npmjs.com/package/@push.rocks/taskbuffer)
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue.svg)](https://www.typescriptlang.org/)
@@ -13,6 +13,7 @@ For reporting bugs, issues, or security vulnerabilities, please visit [community
 ## 🌟 Features

 - **🎯 Type-Safe Task Management** — Full TypeScript support with generics and type inference
+- **🔒 Constraint-Based Concurrency** — Per-key mutual exclusion, group concurrency limits, and cooldown enforcement via `TaskConstraintGroup`
 - **📊 Real-Time Progress Tracking** — Step-based progress with percentage weights
 - **⚡ Smart Buffering** — Intelligent request debouncing and batching
 - **⏰ Cron Scheduling** — Schedule tasks with cron expressions
@@ -49,6 +50,24 @@ const result = await greetTask.trigger('World');
 console.log(result); // "Hello, World!"
 ```

+### Task with Typed Data 📦
+
+Every task can carry a typed data bag — perfect for constraint matching, routing, and metadata:
+
+```typescript
+const task = new Task<undefined, [], { domain: string; priority: number }>({
+  name: 'update-dns',
+  data: { domain: 'example.com', priority: 1 },
+  taskFunction: async () => {
+    // task.data is fully typed here
+    console.log(`Updating DNS for ${task.data.domain}`);
+  },
+});
+
+task.data.domain;   // string — fully typed
+task.data.priority; // number — fully typed
+```
+
 ### Task with Steps & Progress 📊

 ```typescript
@@ -84,6 +103,132 @@ console.log(deployTask.getStepsMetadata()); // Step details with status

 > **Note:** `notifyStep()` is fully type-safe — TypeScript only accepts step names you declared in the `steps` array when you use `as const`.

+## 🔒 Task Constraints — Concurrency, Mutual Exclusion & Cooldowns
+
+`TaskConstraintGroup` is the unified mechanism for controlling how tasks run relative to each other. It replaces older patterns like task runners, blocking tasks, and execution delays with a single, composable, key-based constraint system.
+
+### Per-Key Mutual Exclusion
+
+Ensure only one task runs at a time for a given key (e.g. per domain, per tenant, per resource):
+
+```typescript
+import { Task, TaskManager, TaskConstraintGroup } from '@push.rocks/taskbuffer';
+
+const manager = new TaskManager();
+
+// Only one DNS update per domain at a time
+const domainMutex = new TaskConstraintGroup<{ domain: string }>({
+  name: 'domain-mutex',
+  maxConcurrent: 1,
+  constraintKeyForTask: (task) => task.data.domain,
+});
+
+manager.addConstraintGroup(domainMutex);
+
+const task1 = new Task<undefined, [], { domain: string }>({
+  name: 'update-a.com',
+  data: { domain: 'a.com' },
+  taskFunction: async () => { /* update DNS for a.com */ },
+});
+
+const task2 = new Task<undefined, [], { domain: string }>({
+  name: 'update-a.com-2',
+  data: { domain: 'a.com' },
+  taskFunction: async () => { /* another update for a.com */ },
+});
+
+manager.addTask(task1);
+manager.addTask(task2);
+
+// task2 waits until task1 finishes (same domain key)
+await Promise.all([
+  manager.triggerTask(task1),
+  manager.triggerTask(task2),
+]);
+```
+
+### Group Concurrency Limits
+
+Cap how many tasks can run concurrently across a group:
+
+```typescript
+// Max 3 DNS updaters running globally at once
+const dnsLimit = new TaskConstraintGroup<{ group: string }>({
+  name: 'dns-concurrency',
+  maxConcurrent: 3,
+  constraintKeyForTask: (task) =>
+    task.data.group === 'dns' ? 'dns' : null, // null = skip constraint
+});
+
+manager.addConstraintGroup(dnsLimit);
+```
+
+### Cooldowns (Rate Limiting)
+
+Enforce a minimum time gap between consecutive executions for the same key:
+
+```typescript
+// No more than one API call per domain every 11 seconds
+const rateLimiter = new TaskConstraintGroup<{ domain: string }>({
+  name: 'api-rate-limit',
+  maxConcurrent: 1,
+  cooldownMs: 11000,
+  constraintKeyForTask: (task) => task.data.domain,
+});
+
+manager.addConstraintGroup(rateLimiter);
+```
+
+### Global Concurrency Cap
+
+Limit total concurrent tasks system-wide:
+
+```typescript
+const globalCap = new TaskConstraintGroup({
+  name: 'global-cap',
+  maxConcurrent: 10,
+  constraintKeyForTask: () => 'all', // same key = shared limit
+});
+
+manager.addConstraintGroup(globalCap);
+```
+
+### Composing Multiple Constraints
+
+Multiple constraint groups stack — a task only runs when **all** applicable constraints allow it:
+
+```typescript
+manager.addConstraintGroup(globalCap);     // max 10 globally
+manager.addConstraintGroup(domainMutex);   // max 1 per domain
+manager.addConstraintGroup(rateLimiter);   // 11s cooldown per domain
+
+// A task must satisfy ALL three constraints before it starts
+await manager.triggerTask(dnsTask);
+```
+
+### Selective Constraints
+
+Return `null` from `constraintKeyForTask` to exempt a task from a constraint group:
+
+```typescript
+const constraint = new TaskConstraintGroup<{ priority: string }>({
+  name: 'low-priority-limit',
+  maxConcurrent: 2,
+  constraintKeyForTask: (task) =>
+    task.data.priority === 'low' ? 'low-priority' : null, // high priority tasks skip this constraint
+});
+```
+
+### How It Works
+
+When you trigger a task through `TaskManager` (via `triggerTask`, `triggerTaskByName`, `addExecuteRemoveTask`, or cron), the manager:
+
+1. Evaluates all registered constraint groups against the task
+2. If no constraints apply (all matchers return `null`) → runs immediately
+3. If all applicable constraints have capacity → acquires slots and runs
+4. If any constraint blocks → enqueues the task; when a running task completes, the queue is drained
+5. Cooldown-blocked tasks auto-retry after the shortest remaining cooldown expires
+
 ## 🎯 Core Concepts

 ### Task Buffering — Intelligent Request Management
@@ -95,7 +240,6 @@ const apiTask = new Task({
  name: 'APIRequest',
  buffered: true,
  bufferMax: 5, // Maximum 5 concurrent executions
-  execDelay: 100, // Minimum 100ms between executions
  taskFunction: async (endpoint) => {
    return await fetch(endpoint).then((r) => r.json());
  },
@@ -156,9 +300,9 @@ console.log(`Saved ${savedCount} items`);
 Taskchain also supports dynamic mutation:

 ```typescript
-pipeline.addTask(newTask); // Append to chain
-pipeline.removeTask(oldTask); // Remove by reference (returns boolean)
-pipeline.shiftTask(); // Remove & return first task
+pipeline.addTask(newTask);      // Append to chain
+pipeline.removeTask(oldTask);   // Remove by reference (returns boolean)
+pipeline.shiftTask();           // Remove & return first task
 ```

 Error context is rich — a chain failure includes the chain name, failing task name, task index, and preserves the original error as `.cause`.
@@ -220,27 +364,6 @@ await initTask.trigger(); // No-op
 console.log(initTask.hasTriggered); // true
 ```

-### TaskRunner — Managed Queue with Concurrency Control
-
-Process a queue of tasks with a configurable parallelism limit:
-
-```typescript
-import { TaskRunner } from '@push.rocks/taskbuffer';
-
-const runner = new TaskRunner();
-runner.setMaxParallelJobs(3); // Run up to 3 tasks concurrently
-
-await runner.start();
-
-runner.addTask(taskA);
-runner.addTask(taskB);
-runner.addTask(taskC);
-runner.addTask(taskD); // Queued until a slot opens
-
-// When done:
-await runner.stop();
-```
-
 ## 🏷️ Labels — Multi-Tenant Task Filtering

 Attach arbitrary key-value labels to any task for filtering, grouping, or multi-tenant isolation:
@@ -411,6 +534,10 @@ manager.addTask(deployTask);
 manager.addAndScheduleTask(backupTask, '0 2 * * *'); // Daily at 2 AM
 manager.addAndScheduleTask(healthCheck, '*/5 * * * *'); // Every 5 minutes

+// Register constraint groups
+manager.addConstraintGroup(globalCap);
+manager.addConstraintGroup(perDomainMutex);
+
 // Query metadata
 const meta = manager.getTaskMetadata('Deploy');
 console.log(meta);
@@ -433,7 +560,7 @@ const allMeta = manager.getAllTasksMetadata();
 const scheduled = manager.getScheduledTasks();
 const nextRuns = manager.getNextScheduledRuns(5);

-// Trigger by name
+// Trigger by name (routes through constraints)
 await manager.triggerTaskByName('Deploy');

 // One-shot: add, execute, collect report, remove
@@ -462,6 +589,12 @@ manager.removeTask(task); // Removes from map and unsubscribes event forwarding
 manager.descheduleTaskByName('Deploy'); // Remove cron schedule only
 ```

+### Remove Constraint Groups
+
+```typescript
+manager.removeConstraintGroup('domain-mutex'); // By name
+```
+
 ## 🎨 Web Component Dashboard

 Visualize your tasks in real-time with the included Lit-based web component:
@@ -570,32 +703,6 @@ await task.trigger('SELECT * FROM users'); // Setup runs here
 await task.trigger('SELECT * FROM orders'); // Setup skipped, pool reused
 ```

-### Blocking Tasks
-
-Make one task wait for another to finish before executing:
-
-```typescript
-const initTask = new Task({
-  name: 'Init',
-  taskFunction: async () => {
-    await initializeSystem();
-  },
-});
-
-const workerTask = new Task({
-  name: 'Worker',
-  taskFunction: async () => {
-    await doWork();
-  },
-});
-
-workerTask.blockingTasks.push(initTask);
-
-// Triggering worker will automatically wait for init to complete
-initTask.trigger();
-workerTask.trigger(); // Waits until initTask.finished resolves
-```
-
 ### Database Migration Pipeline

 ```typescript
@@ -616,15 +723,24 @@ try {

 ### Multi-Tenant SaaS Monitoring

-Combine labels + events for a real-time multi-tenant dashboard:
+Combine labels + events + constraints for a real-time multi-tenant system:

 ```typescript
 const manager = new TaskManager();

+// Per-tenant concurrency limit
+const tenantLimit = new TaskConstraintGroup<{ tenantId: string }>({
+  name: 'tenant-concurrency',
+  maxConcurrent: 2,
+  constraintKeyForTask: (task) => task.data.tenantId,
+});
+manager.addConstraintGroup(tenantLimit);
+
 // Create tenant-scoped tasks
 function createTenantTask(tenantId: string, taskName: string, fn: () => Promise<any>) {
-  const task = new Task({
+  const task = new Task<undefined, [], { tenantId: string }>({
    name: `${tenantId}:${taskName}`,
+    data: { tenantId },
    labels: { tenantId },
    taskFunction: fn,
  });
@@ -653,15 +769,31 @@ const acmeTasks = manager.getTasksMetadataByLabel('tenantId', 'acme');

 | Class | Description |
 | --- | --- |
-| `Task<T, TSteps>` | Core task unit with optional step tracking, labels, and event streaming |
-| `TaskManager` | Centralized orchestrator with scheduling, label queries, and aggregated events |
+| `Task<T, TSteps, TData>` | Core task unit with typed data, optional step tracking, labels, and event streaming |
+| `TaskManager` | Centralized orchestrator with constraint groups, scheduling, label queries, and aggregated events |
+| `TaskConstraintGroup<TData>` | Concurrency, mutual exclusion, and cooldown constraints with key-based grouping |
 | `Taskchain` | Sequential task executor with data flow between tasks |
 | `Taskparallel` | Concurrent task executor via `Promise.all()` |
 | `TaskOnce` | Single-execution guard |
 | `TaskDebounced` | Debounced task using rxjs |
-| `TaskRunner` | Sequential queue with configurable parallelism |
 | `TaskStep` | Step tracking unit (internal, exposed via metadata) |

+### Task Constructor Options
+
+| Option | Type | Default | Description |
+| --- | --- | --- | --- |
+| `taskFunction` | `ITaskFunction<T>` | *required* | The async function to execute |
+| `name` | `string` | — | Task identifier (required for TaskManager) |
+| `data` | `TData` | `{}` | Typed data bag for constraint matching and routing |
+| `steps` | `ReadonlyArray<{name, description, percentage}>` | — | Step definitions for progress tracking |
+| `buffered` | `boolean` | — | Enable request buffering |
+| `bufferMax` | `number` | — | Max buffered calls |
+| `preTask` | `Task \| () => Task` | — | Task to run before |
+| `afterTask` | `Task \| () => Task` | — | Task to run after |
+| `taskSetup` | `() => Promise<T>` | — | One-time setup function |
+| `catchErrors` | `boolean` | `false` | Swallow errors instead of rejecting |
+| `labels` | `Record<string, string>` | `{}` | Initial labels |
+
 ### Task Methods

 | Method | Returns | Description |
@@ -682,6 +814,7 @@ const acmeTasks = manager.getTasksMetadataByLabel('tenantId', 'acme');
 | Property | Type | Description |
 | --- | --- | --- |
 | `name` | `string` | Task identifier |
+| `data` | `TData` | Typed data bag |
 | `running` | `boolean` | Whether the task is currently executing |
 | `idle` | `boolean` | Inverse of `running` |
 | `labels` | `Record<string, string>` | Attached labels |
@@ -690,7 +823,27 @@ const acmeTasks = manager.getTasksMetadataByLabel('tenantId', 'acme');
 | `errorCount` | `number` | Total error count across all runs |
 | `runCount` | `number` | Total execution count |
 | `lastRun` | `Date \| undefined` | Timestamp of last execution |
-| `blockingTasks` | `Task[]` | Tasks that must finish before this one starts |
+
+### TaskConstraintGroup Constructor Options
+
+| Option | Type | Default | Description |
+| --- | --- | --- | --- |
+| `name` | `string` | *required* | Constraint group identifier |
+| `constraintKeyForTask` | `(task) => string \| null` | *required* | Returns key for grouping, or `null` to skip |
+| `maxConcurrent` | `number` | `Infinity` | Max concurrent tasks per key |
+| `cooldownMs` | `number` | `0` | Minimum ms between completions per key |
+
+### TaskConstraintGroup Methods
+
+| Method | Returns | Description |
+| --- | --- | --- |
+| `getConstraintKey(task)` | `string \| null` | Get the constraint key for a task |
+| `canRun(key)` | `boolean` | Check if a slot is available |
+| `acquireSlot(key)` | `void` | Claim a running slot |
+| `releaseSlot(key)` | `void` | Release a slot and record completion time |
+| `getCooldownRemaining(key)` | `number` | Milliseconds until cooldown expires |
+| `getRunningCount(key)` | `number` | Current running count for key |
+| `reset()` | `void` | Clear all state |

 ### TaskManager Methods

@@ -699,7 +852,11 @@ const acmeTasks = manager.getTasksMetadataByLabel('tenantId', 'acme');
 | `addTask(task)` | `void` | Register a task (wires event forwarding) |
 | `removeTask(task)` | `void` | Remove task and unsubscribe events |
 | `getTaskByName(name)` | `Task \| undefined` | Look up by name |
-| `triggerTaskByName(name)` | `Promise<any>` | Trigger by name |
+| `triggerTaskByName(name)` | `Promise<any>` | Trigger by name (routes through constraints) |
+| `triggerTask(task)` | `Promise<any>` | Trigger directly (routes through constraints) |
+| `triggerTaskConstrained(task, input?)` | `Promise<any>` | Core constraint evaluation entry point |
+| `addConstraintGroup(group)` | `void` | Register a constraint group |
+| `removeConstraintGroup(name)` | `void` | Remove a constraint group by name |
 | `addAndScheduleTask(task, cron)` | `void` | Register + schedule |
 | `scheduleTaskByName(name, cron)` | `void` | Schedule existing task |
 | `descheduleTaskByName(name)` | `void` | Remove schedule |
@@ -719,6 +876,7 @@ const acmeTasks = manager.getTasksMetadataByLabel('tenantId', 'acme');
 | --- | --- | --- |
 | `taskSubject` | `Subject<ITaskEvent>` | Aggregated events from all added tasks |
 | `taskMap` | `ObjectMap<Task>` | Internal task registry |
+| `constraintGroups` | `TaskConstraintGroup[]` | Registered constraint groups |

 ### Exported Types

@@ -731,13 +889,14 @@ import type {
  TTaskEventType,
  ITaskStep,
  ITaskFunction,
+  ITaskConstraintGroupOptions,
  StepNames,
 } from '@push.rocks/taskbuffer';
 ```

 ## License and Legal Information

-This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./license.md) file.
+This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [LICENSE](./LICENSE) file.

 **Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.