Juergen Kunz
820f84ee61
- Fix ES client v8+ API: use document/doc instead of body for index/update operations - Add type assertions (as any) for ES client ILM, template, and search APIs - Fix strict null checks with proper undefined handling (nullish coalescing) - Fix MetricsCollector interface to match required method signatures - Fix Logger.error signature compatibility in plugins - Resolve TermsQuery type index signature conflict - Remove sourceMap from tsconfig (handled by tsbuild with inlineSourceMap)
128 lines
4.9 KiB
Markdown
128 lines
4.9 KiB
Markdown
# Project Readme Hints
|
|
|
|
## v3.0 Transformation Progress
|
|
|
|
### Completed Components
|
|
|
|
**Phase 1: Foundation (100%)** - Complete
|
|
|
|
- Error handling with typed hierarchy
|
|
- Observability (logging, metrics, tracing)
|
|
- Configuration management with fluent builder
|
|
- Connection management with health checks and circuit breaker
|
|
|
|
**Phase 2: Domain APIs (100%)** - Complete! 🎉
|
|
|
|
- **Document API (100%)** - Complete fluent CRUD operations with sessions, iteration, snapshots
|
|
- **Query Builder (100%)** - Type-safe query construction with aggregations
|
|
- **Logging API (100%)** - Enterprise structured logging with:
|
|
- Batched bulk indexing with auto-flush
|
|
- Log enrichment pipeline (host, environment, service, sanitization)
|
|
- Sampling strategies (all, errors-only, percentage, rate-limit)
|
|
- ILM (Index Lifecycle Management) integration
|
|
- Metric extraction from logs
|
|
- Dynamic tagging based on content
|
|
- Queue overflow protection
|
|
- Example at `ts/examples/logging/logging-example.ts`
|
|
- **Bulk Indexer (100%)** - High-throughput document ingestion with:
|
|
- Fixed, adaptive, and size-based batching strategies
|
|
- Parallel workers for concurrent processing
|
|
- Backpressure handling with queue management
|
|
- Dead-letter queue for failed operations
|
|
- Progress callbacks with ETA estimation
|
|
- Automatic retries with exponential backoff
|
|
- Mixed operations (index, update, delete)
|
|
- Comprehensive statistics tracking
|
|
- Example at `ts/examples/bulk/bulk-indexer-example.ts`
|
|
- **KV Store (100%)** - Distributed key-value storage with:
|
|
- Time-to-live (TTL) support with automatic expiration
|
|
- In-memory caching layer with multiple eviction policies (LRU, LFU, FIFO, TTL)
|
|
- Batch operations (mget, mset, mdelete)
|
|
- Scan/cursor support for large keyspaces with wildcard patterns
|
|
- Automatic compression for large values
|
|
- Optimistic concurrency control
|
|
- Cache hit/miss statistics
|
|
- Example at `ts/examples/kv/kv-store-example.ts`
|
|
|
|
**Phase 3: Advanced Features (100%)** - Complete!
|
|
|
|
- **Plugin Architecture (100%)** - Extensible request/response middleware:
|
|
- Plugin lifecycle hooks (beforeRequest, afterResponse, onError)
|
|
- Plugin priority and execution ordering
|
|
- Plugin statistics and monitoring
|
|
- Built-in plugins:
|
|
- Logging plugin - automatic request/response logging
|
|
- Metrics plugin - automatic metrics collection
|
|
- Cache plugin - response caching for GET requests
|
|
- Retry plugin - automatic retry with exponential backoff
|
|
- Rate limit plugin - token bucket rate limiting
|
|
- Custom plugin creation with factories
|
|
- Dynamic plugin registration/unregistration
|
|
- Shared context between plugins
|
|
- Timeout protection for plugin hooks
|
|
- Example at `ts/examples/plugins/plugin-example.ts`
|
|
- **Transaction Support (100%)** - ACID-like distributed transactions:
|
|
- Optimistic concurrency control (seq_no/primary_term)
|
|
- Automatic rollback with compensation operations
|
|
- Savepoints for partial rollback
|
|
- Conflict detection and configurable resolution (retry, abort, skip, force)
|
|
- Multi-document atomic operations
|
|
- Isolation levels (read_uncommitted, read_committed, repeatable_read)
|
|
- Transaction callbacks (onBegin, onCommit, onRollback, onConflict)
|
|
- Transaction statistics and monitoring
|
|
- Timeout with automatic cleanup
|
|
- Example at `ts/examples/transactions/transaction-example.ts`
|
|
- **Schema Management (100%)** - Index mapping management and migrations:
|
|
- Index creation with mappings, settings, and aliases
|
|
- Schema validation before apply
|
|
- Versioned migrations with history tracking
|
|
- Schema diff and comparison (added/removed/modified fields)
|
|
- Breaking change detection
|
|
- Index templates (composable and legacy)
|
|
- Component templates
|
|
- Alias management with filters
|
|
- Dynamic settings updates
|
|
- Migration rollback support
|
|
- Dry run mode for testing
|
|
- Example at `ts/examples/schema/schema-example.ts`
|
|
|
|
### Query Builder Usage
|
|
|
|
```typescript
|
|
import { createQuery } from './ts';
|
|
|
|
// Simple query
|
|
const results = await createQuery<Product>('products')
|
|
.match('name', 'laptop')
|
|
.range('price', { gte: 100, lte: 1000 })
|
|
.sort('price', 'asc')
|
|
.execute();
|
|
|
|
// With aggregations
|
|
const stats = await createQuery<Product>('products')
|
|
.matchAll()
|
|
.aggregations((agg) => {
|
|
agg
|
|
.terms('brands', 'brand.keyword')
|
|
.subAggregation('avg_price', (sub) => sub.avg('avg_price', 'price'));
|
|
})
|
|
.execute();
|
|
```
|
|
|
|
### Next Priorities (Phase 4)
|
|
|
|
1. **Comprehensive Test Suite** - Full test coverage for all modules
|
|
2. **Migration Guide** - Guide from v2 to v3 API
|
|
3. **README Update** - Update main README with new v3 API documentation
|
|
|
|
### Structure
|
|
|
|
- Single `ts/` directory (no parallel structures)
|
|
- Single `tsconfig.json` with strict mode enabled
|
|
- All v3.0 code lives in `ts/` directory
|
|
|
|
### Known Issues
|
|
|
|
- Minor TypeScript strict mode import issues documented in `ts/QUICK_FIXES.md`
|
|
- Need to apply `import type` fixes for verbatimModuleSyntax compliance
|