fix(tests): Update test runner and imports, refresh README and package metadata, add project tooling/config files
This commit is contained in:
495
readme.md
495
readme.md
@@ -1,151 +1,478 @@
|
||||
# @push.rocks/taskbuffer
|
||||
flexible task management. TypeScript ready!
|
||||
# @push.rocks/taskbuffer 🚀
|
||||
|
||||
## Install
|
||||
To install `@push.rocks/taskbuffer`, use the following npm command:
|
||||
A **powerful**, **flexible**, and **TypeScript-first** task management library for orchestrating asynchronous operations with style. From simple task execution to complex distributed workflows, taskbuffer has got you covered.
|
||||
|
||||
## Install 📦
|
||||
|
||||
```bash
|
||||
npm install @push.rocks/taskbuffer --save
|
||||
```
|
||||
|
||||
This will add `@push.rocks/taskbuffer` to your project's dependencies, allowing you to use it within your TypeScript project.
|
||||
Or with **pnpm** (recommended):
|
||||
|
||||
## Usage
|
||||
`@push.rocks/taskbuffer` offers a versatile way to manage tasks in your application. Whether you need simple asynchronous task execution, buffered tasks to limit concurrent executions, or more complex scenarios involving task chaining and parallel execution, `@push.rocks/taskbuffer` has got you covered. Below, you'll find detailed examples showcasing the module's capabilities and how to use them in TypeScript.
|
||||
```bash
|
||||
pnpm add @push.rocks/taskbuffer
|
||||
```
|
||||
|
||||
## Why taskbuffer? 🤔
|
||||
|
||||
In the modern JavaScript ecosystem, managing asynchronous tasks efficiently is crucial. Whether you're building a data pipeline, managing API rate limits, or orchestrating complex workflows, **@push.rocks/taskbuffer** provides the tools you need:
|
||||
|
||||
- **🎯 TypeScript-first**: Built with TypeScript for TypeScript - enjoy complete type safety and excellent IDE support
|
||||
- **⚡ Flexible execution**: From simple tasks to complex parallel workflows with dependencies
|
||||
- **🔄 Smart buffering**: Control concurrent executions with intelligent buffer management
|
||||
- **⏰ Built-in scheduling**: Cron-based task scheduling without additional dependencies
|
||||
- **🎭 Multiple paradigms**: Support for debounced, throttled, and one-time execution patterns
|
||||
- **🔌 Extensible**: Clean architecture that's easy to extend and customize
|
||||
- **🏃 Zero dependencies on external schedulers**: Everything you need is included
|
||||
|
||||
## Core Concepts 🎓
|
||||
|
||||
### Task
|
||||
The fundamental unit of work. A task wraps an asynchronous function and provides powerful execution control.
|
||||
|
||||
### Taskchain
|
||||
Sequential task execution - tasks run one after another, with results passed along the chain.
|
||||
|
||||
### Taskparallel
|
||||
Parallel task execution - multiple tasks run simultaneously for maximum performance.
|
||||
|
||||
### TaskManager
|
||||
Centralized task scheduling and management using cron expressions.
|
||||
|
||||
### TaskDebounced
|
||||
Debounced task execution - prevents rapid repeated executions, only running after a quiet period.
|
||||
|
||||
### TaskOnce
|
||||
Singleton task execution - ensures a task runs exactly once, perfect for initialization routines.
|
||||
|
||||
## Quick Start 🏁
|
||||
|
||||
### Basic Task Execution
|
||||
To create and run a simple task:
|
||||
|
||||
```typescript
|
||||
import { Task } from '@push.rocks/taskbuffer';
|
||||
|
||||
const simpleTask = new Task({
|
||||
name: 'SimpleTask',
|
||||
// Create a simple task
|
||||
const myTask = new Task({
|
||||
name: 'DataProcessor',
|
||||
taskFunction: async () => {
|
||||
console.log('Executing a simple task');
|
||||
// Perform your task here (e.g., fetch data from an API)
|
||||
const data = await fetchData();
|
||||
return processData(data);
|
||||
}
|
||||
});
|
||||
|
||||
// Trigger the task
|
||||
simpleTask.trigger();
|
||||
// Execute the task
|
||||
const result = await myTask.trigger();
|
||||
```
|
||||
|
||||
This basic example creates a task that prints a message to the console when executed.
|
||||
### Buffered Execution (Rate Limiting)
|
||||
|
||||
### Buffered Task Execution
|
||||
Buffered tasks allow you to throttle task executions, for example, to avoid overwhelming a server with simultaneous requests:
|
||||
Perfect for API calls or database operations that need throttling:
|
||||
|
||||
```typescript
|
||||
import { Task } from '@push.rocks/taskbuffer';
|
||||
|
||||
const bufferedTask = new Task({
|
||||
name: 'BufferedTask',
|
||||
taskFunction: async () => {
|
||||
// Task implementation
|
||||
console.log('Buffered task execution');
|
||||
const apiTask = new Task({
|
||||
name: 'APICall',
|
||||
taskFunction: async (endpoint: string) => {
|
||||
return await fetch(endpoint);
|
||||
},
|
||||
buffered: true,
|
||||
bufferMax: 3 // Maximum number of concurrent executions
|
||||
bufferMax: 3, // Maximum 3 concurrent executions
|
||||
execDelay: 1000 // Wait 1 second between executions
|
||||
});
|
||||
|
||||
// Trigger the task multiple times
|
||||
bufferedTask.trigger();
|
||||
bufferedTask.trigger();
|
||||
bufferedTask.trigger();
|
||||
bufferedTask.trigger();
|
||||
// These will be automatically throttled
|
||||
for (let i = 0; i < 10; i++) {
|
||||
apiTask.trigger(`/api/data/${i}`);
|
||||
}
|
||||
```
|
||||
|
||||
In this scenario, only three instances of `bufferedTask` will run concurrently, thanks to the buffer settings.
|
||||
### Task Chains - Sequential Workflows
|
||||
|
||||
### Task Chains
|
||||
Task chains are a powerful feature that allows you to execute tasks in a specified order:
|
||||
Build complex workflows where each step depends on the previous:
|
||||
|
||||
```typescript
|
||||
import { Task, Taskchain } from '@push.rocks/taskbuffer';
|
||||
|
||||
const firstTask = new Task({
|
||||
name: 'FirstTask',
|
||||
const fetchTask = new Task({
|
||||
name: 'FetchData',
|
||||
taskFunction: async () => {
|
||||
console.log('First task execution');
|
||||
// First task logic
|
||||
const response = await fetch('/api/data');
|
||||
return response.json();
|
||||
}
|
||||
});
|
||||
|
||||
const secondTask = new Task({
|
||||
name: 'SecondTask',
|
||||
taskFunction: async () => {
|
||||
console.log('Second task execution');
|
||||
// Second task logic
|
||||
const transformTask = new Task({
|
||||
name: 'TransformData',
|
||||
taskFunction: async (data) => {
|
||||
return data.map(item => ({
|
||||
...item,
|
||||
processed: true,
|
||||
timestamp: Date.now()
|
||||
}));
|
||||
}
|
||||
});
|
||||
|
||||
const taskChain = new Taskchain({
|
||||
name: 'MyTaskChain',
|
||||
taskArray: [firstTask, secondTask] // Tasks will be executed in the order they are provided
|
||||
const saveTask = new Task({
|
||||
name: 'SaveData',
|
||||
taskFunction: async (transformedData) => {
|
||||
await database.bulkInsert(transformedData);
|
||||
return { saved: transformedData.length };
|
||||
}
|
||||
});
|
||||
|
||||
// Trigger the task chain
|
||||
taskChain.trigger();
|
||||
const workflow = new Taskchain({
|
||||
name: 'DataPipeline',
|
||||
taskArray: [fetchTask, transformTask, saveTask]
|
||||
});
|
||||
|
||||
// Execute the entire chain
|
||||
const result = await workflow.trigger();
|
||||
console.log(`Processed ${result.saved} items`);
|
||||
```
|
||||
|
||||
### Parallel Task Execution
|
||||
To run tasks in parallel, use `Taskparallel`:
|
||||
### Parallel Execution - Maximum Performance
|
||||
|
||||
Execute multiple independent tasks simultaneously:
|
||||
|
||||
```typescript
|
||||
import { Task, Taskparallel } from '@push.rocks/taskbuffer';
|
||||
|
||||
const taskOne = new Task({
|
||||
name: 'TaskOne',
|
||||
taskFunction: async () => {
|
||||
// Task one logic
|
||||
console.log('Task one execution');
|
||||
}
|
||||
const tasks = ['user', 'posts', 'comments'].map(resource =>
|
||||
new Task({
|
||||
name: `Fetch${resource}`,
|
||||
taskFunction: async () => {
|
||||
const data = await fetch(`/api/${resource}`);
|
||||
return data.json();
|
||||
}
|
||||
})
|
||||
);
|
||||
|
||||
const parallelFetch = new Taskparallel({
|
||||
taskArray: tasks
|
||||
});
|
||||
|
||||
const taskTwo = new Task({
|
||||
name: 'TaskTwo',
|
||||
taskFunction: async () => {
|
||||
// Task two logic
|
||||
console.log('Task two execution');
|
||||
}
|
||||
});
|
||||
|
||||
const parallelTask = new Taskparallel({
|
||||
taskArray: [taskOne, taskTwo] // These tasks will be executed in parallel
|
||||
});
|
||||
|
||||
// Trigger the parallel tasks
|
||||
parallelTask.trigger();
|
||||
// All tasks execute simultaneously
|
||||
const [users, posts, comments] = await parallelFetch.trigger();
|
||||
```
|
||||
|
||||
### Task Scheduling with Task Manager
|
||||
`@push.rocks/taskbuffer` also provides a `TaskManager` to schedule tasks using cron expressions:
|
||||
### Scheduled Tasks with TaskManager
|
||||
|
||||
Run tasks on a schedule using cron expressions:
|
||||
|
||||
```typescript
|
||||
import { Task, TaskManager } from '@push.rocks/taskbuffer';
|
||||
|
||||
const scheduledTask = new Task({
|
||||
name: 'ScheduledTask',
|
||||
const backupTask = new Task({
|
||||
name: 'DatabaseBackup',
|
||||
taskFunction: async () => {
|
||||
console.log('Scheduled task execution');
|
||||
// Task logic here
|
||||
await performBackup();
|
||||
console.log(`Backup completed at ${new Date().toISOString()}`);
|
||||
}
|
||||
});
|
||||
|
||||
const taskManager = new TaskManager();
|
||||
taskManager.addAndScheduleTask(scheduledTask, '0 * * * * *'); // Execute task at the start of every minute
|
||||
const manager = new TaskManager();
|
||||
|
||||
// Start the task manager to enable the scheduled execution
|
||||
taskManager.start();
|
||||
// Add and schedule tasks
|
||||
manager.addAndScheduleTask(backupTask, '0 0 * * *'); // Daily at midnight
|
||||
manager.addAndScheduleTask(healthCheck, '*/5 * * * *'); // Every 5 minutes
|
||||
|
||||
// Start the scheduler
|
||||
manager.start();
|
||||
|
||||
// Later... stop if needed
|
||||
manager.stop();
|
||||
```
|
||||
|
||||
### Advanced Use Cases
|
||||
The flexibility of `@push.rocks/taskbuffer` allows for more sophisticated task management scenarios. You can combine the features mentioned above to fit your specific needs, such as chaining parallel tasks, dynamically adding tasks to a manager or chain based on conditions, and much more.
|
||||
### Debounced Tasks - Smart Throttling
|
||||
|
||||
`@push.rocks/taskbuffer` leverages TypeScript's advanced typing features to provide you with clear and predictable APIs, making it easier to build and maintain complex task management logic within your application.
|
||||
Prevent task spam with intelligent debouncing:
|
||||
|
||||
For further information and advanced configurations, please refer to the [official documentation](https://gitlab.com/pushrocks/taskbuffer).
|
||||
```typescript
|
||||
import { TaskDebounced } from '@push.rocks/taskbuffer';
|
||||
|
||||
const saveTask = new TaskDebounced({
|
||||
name: 'AutoSave',
|
||||
taskFunction: async (content: string) => {
|
||||
await saveToDatabase(content);
|
||||
console.log('Content saved');
|
||||
},
|
||||
debounceTimeInMillis: 2000 // Wait 2 seconds of inactivity
|
||||
});
|
||||
|
||||
// Rapid calls will be debounced
|
||||
input.addEventListener('input', (e) => {
|
||||
saveTask.trigger(e.target.value);
|
||||
});
|
||||
```
|
||||
|
||||
### One-Time Tasks - Initialize Once
|
||||
|
||||
Ensure initialization code runs exactly once:
|
||||
|
||||
```typescript
|
||||
import { TaskOnce } from '@push.rocks/taskbuffer';
|
||||
|
||||
const initTask = new TaskOnce({
|
||||
name: 'SystemInitialization',
|
||||
taskFunction: async () => {
|
||||
await database.connect();
|
||||
await cache.initialize();
|
||||
await loadConfiguration();
|
||||
console.log('System initialized');
|
||||
}
|
||||
});
|
||||
|
||||
// Safe to call multiple times - only runs once
|
||||
await initTask.trigger();
|
||||
await initTask.trigger(); // This won't run again
|
||||
```
|
||||
|
||||
## Advanced Features 🔥
|
||||
|
||||
### Task Dependencies with Pre/Post Hooks
|
||||
|
||||
Create sophisticated task relationships:
|
||||
|
||||
```typescript
|
||||
const validationTask = new Task({
|
||||
name: 'ValidateInput',
|
||||
taskFunction: async (data) => {
|
||||
if (!isValid(data)) {
|
||||
throw new Error('Validation failed');
|
||||
}
|
||||
return data;
|
||||
}
|
||||
});
|
||||
|
||||
const mainTask = new Task({
|
||||
name: 'ProcessData',
|
||||
taskFunction: async (data) => {
|
||||
return await complexProcessing(data);
|
||||
},
|
||||
preTask: validationTask, // Runs before main task
|
||||
afterTask: cleanupTask // Runs after main task
|
||||
});
|
||||
```
|
||||
|
||||
### Task Runners - Distributed Execution
|
||||
|
||||
The TaskRunner system enables distributed task execution across multiple workers:
|
||||
|
||||
```typescript
|
||||
import { TaskRunner } from '@push.rocks/taskbuffer';
|
||||
|
||||
const runner = new TaskRunner({
|
||||
name: 'WorkerNode1',
|
||||
maxConcurrentTasks: 5
|
||||
});
|
||||
|
||||
// Register tasks this runner can handle
|
||||
runner.registerTask(dataProcessingTask);
|
||||
runner.registerTask(imageResizeTask);
|
||||
|
||||
// Start processing
|
||||
runner.start();
|
||||
```
|
||||
|
||||
### Buffer Management Strategies
|
||||
|
||||
Fine-tune concurrent execution behavior:
|
||||
|
||||
```typescript
|
||||
const task = new Task({
|
||||
name: 'ResourceIntensive',
|
||||
taskFunction: async () => { /* ... */ },
|
||||
buffered: true,
|
||||
bufferMax: 5, // Max 5 concurrent
|
||||
execDelay: 100, // 100ms between starts
|
||||
timeout: 30000 // 30 second timeout
|
||||
});
|
||||
```
|
||||
|
||||
### Cycle Detection and Prevention
|
||||
|
||||
TaskBuffer automatically detects and prevents circular dependencies:
|
||||
|
||||
```typescript
|
||||
const taskA = new Task({
|
||||
name: 'TaskA',
|
||||
taskFunction: async () => { /* ... */ },
|
||||
preTask: taskB // This would create a cycle
|
||||
});
|
||||
|
||||
const taskB = new Task({
|
||||
name: 'TaskB',
|
||||
taskFunction: async () => { /* ... */ },
|
||||
preTask: taskA // Circular dependency detected!
|
||||
});
|
||||
```
|
||||
|
||||
### Dynamic Task Creation
|
||||
|
||||
Create tasks on-the-fly based on runtime conditions:
|
||||
|
||||
```typescript
|
||||
const dynamicWorkflow = async (config: Config) => {
|
||||
const tasks = config.steps.map(step =>
|
||||
new Task({
|
||||
name: step.name,
|
||||
taskFunction: async (input) => {
|
||||
return await processStep(step, input);
|
||||
}
|
||||
})
|
||||
);
|
||||
|
||||
const chain = new Taskchain({
|
||||
name: 'DynamicWorkflow',
|
||||
taskArray: tasks
|
||||
});
|
||||
|
||||
return await chain.trigger();
|
||||
};
|
||||
```
|
||||
|
||||
## API Reference 📚
|
||||
|
||||
### Task Options
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `name` | `string` | Unique identifier for the task |
|
||||
| `taskFunction` | `Function` | Async function to execute |
|
||||
| `buffered` | `boolean` | Enable buffer management |
|
||||
| `bufferMax` | `number` | Maximum concurrent executions |
|
||||
| `execDelay` | `number` | Delay between executions (ms) |
|
||||
| `timeout` | `number` | Task timeout (ms) |
|
||||
| `preTask` | `Task` | Task to run before |
|
||||
| `afterTask` | `Task` | Task to run after |
|
||||
|
||||
### TaskManager Methods
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `addTask(task, cronExpression)` | Add and schedule a task |
|
||||
| `removeTask(taskName)` | Remove a scheduled task |
|
||||
| `start()` | Start the scheduler |
|
||||
| `stop()` | Stop the scheduler |
|
||||
| `getStats()` | Get execution statistics |
|
||||
|
||||
### Taskchain Methods
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `addTask(task)` | Add task to chain |
|
||||
| `removeTask(taskName)` | Remove task from chain |
|
||||
| `trigger(initialValue)` | Execute the chain |
|
||||
| `reset()` | Reset chain state |
|
||||
|
||||
## Performance Tips 🏎️
|
||||
|
||||
1. **Use buffering for I/O operations**: Prevents overwhelming external services
|
||||
2. **Leverage parallel execution**: When tasks are independent, run them simultaneously
|
||||
3. **Implement proper error handling**: Use try-catch in task functions
|
||||
4. **Monitor task execution**: Use the built-in stats and logging
|
||||
5. **Set appropriate timeouts**: Prevent hanging tasks from blocking your system
|
||||
|
||||
## Error Handling 🛡️
|
||||
|
||||
```typescript
|
||||
const robustTask = new Task({
|
||||
name: 'RobustOperation',
|
||||
taskFunction: async (input) => {
|
||||
try {
|
||||
return await riskyOperation(input);
|
||||
} catch (error) {
|
||||
// Log error
|
||||
console.error(`Task failed: ${error.message}`);
|
||||
|
||||
// Optionally retry
|
||||
if (error.retryable) {
|
||||
return await riskyOperation(input);
|
||||
}
|
||||
|
||||
// Or return default value
|
||||
return defaultValue;
|
||||
}
|
||||
},
|
||||
timeout: 5000 // Fail if takes longer than 5 seconds
|
||||
});
|
||||
```
|
||||
|
||||
## Real-World Examples 🌍
|
||||
|
||||
### API Rate Limiting
|
||||
|
||||
```typescript
|
||||
const apiClient = new Task({
|
||||
name: 'RateLimitedAPI',
|
||||
taskFunction: async (endpoint: string) => {
|
||||
return await fetch(`https://api.example.com${endpoint}`);
|
||||
},
|
||||
buffered: true,
|
||||
bufferMax: 10, // 10 requests
|
||||
execDelay: 100 // Per 100ms = 100 req/s max
|
||||
});
|
||||
```
|
||||
|
||||
### Database Migration Pipeline
|
||||
|
||||
```typescript
|
||||
const migrationChain = new Taskchain({
|
||||
name: 'DatabaseMigration',
|
||||
taskArray: [
|
||||
backupTask,
|
||||
schemaUpdateTask,
|
||||
dataTransformTask,
|
||||
validationTask,
|
||||
cleanupTask
|
||||
]
|
||||
});
|
||||
```
|
||||
|
||||
### Microservice Health Monitoring
|
||||
|
||||
```typescript
|
||||
const healthMonitor = new TaskManager();
|
||||
|
||||
services.forEach(service => {
|
||||
const healthCheck = new Task({
|
||||
name: `HealthCheck:${service.name}`,
|
||||
taskFunction: async () => {
|
||||
const healthy = await checkHealth(service.url);
|
||||
if (!healthy) {
|
||||
await alertOps(service);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
healthMonitor.addAndScheduleTask(healthCheck, '*/1 * * * *'); // Every minute
|
||||
});
|
||||
```
|
||||
|
||||
## Testing 🧪
|
||||
|
||||
```typescript
|
||||
import { expect, tap } from '@git.zone/tstest';
|
||||
import { Task } from '@push.rocks/taskbuffer';
|
||||
|
||||
tap.test('should execute task successfully', async () => {
|
||||
const result = await myTask.trigger();
|
||||
expect(result).toEqual(expectedValue);
|
||||
});
|
||||
|
||||
tap.start();
|
||||
```
|
||||
|
||||
## Contributing 🤝
|
||||
|
||||
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
|
||||
|
||||
## Support 💬
|
||||
|
||||
- 📧 Email: [hello@task.vc](mailto:hello@task.vc)
|
||||
- 🐛 Issues: [GitHub Issues](https://github.com/push-rocks/taskbuffer/issues)
|
||||
- 📖 Docs: [Documentation](https://code.foss.global/push.rocks/taskbuffer)
|
||||
|
||||
## License and Legal Information
|
||||
|
||||
@@ -164,4 +491,4 @@ Registered at District court Bremen HRB 35230 HB, Germany
|
||||
|
||||
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
|
||||
|
||||
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
|
||||
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
|
Reference in New Issue
Block a user