apiclient.xyz/elasticsearch
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

chore: Bump to 3.1.2

Browse Source
This commit is contained in:
Jürgen Kunz
2025-11-29 21:23:18 +00:00
parent 175762c9a7
commit 9d18af91fe
2 changed files with 1 additions and 378 deletions
Download Patch File Download Diff File Expand all files Collapse all files

377
IMPLEMENTATION_STATUS.md
View File

@@ -1,377 +0,0 @@
# Enterprise Elasticsearch Client - Implementation Status
**Last Updated**: Current conversation
**Status**: Phase 1 Complete, Phase 2 In Progress (60% complete)
## 📋 Complete Plan Reference
The complete transformation plan is documented in this conversation under "Enterprise-Grade Elasticsearch Client - Complete Overhaul Plan".
## ✅ Phase 1: Foundation - COMPLETE
### Directory Structure
- ✅ `ts/core/` - Core infrastructure
- ✅ `ts/domain/` - Domain APIs
- ✅ `ts/plugins/` - Plugin architecture
- ✅ `ts/testing/` - Test utilities
- ✅ `ts/examples/` - Usage examples
- ✅ `tsconfig.json` - Strict TypeScript configuration
### Error Handling (ts/core/errors/)
- ✅ `types.ts` - Error codes, context types, retry configuration
- ✅ `elasticsearch-error.ts` - Complete error hierarchy:
- ElasticsearchError (base)
- ConnectionError
- TimeoutError
- IndexNotFoundError
- DocumentNotFoundError
- DocumentConflictError
- AuthenticationError
- AuthorizationError
- ConfigurationError
- QueryParseError
- BulkOperationError
- ClusterUnavailableError
- ✅ `retry-policy.ts` - Retry logic with exponential backoff, jitter, circuit breaking
- ✅ `index.ts` - Module exports
### Observability (ts/core/observability/)
- ✅ `logger.ts` - Structured logging:
- LogLevel enum (DEBUG, INFO, WARN, ERROR)
- Logger class with context and correlation
- ConsoleTransport and JsonTransport
- Child logger support
- Default logger instance
- ✅ `metrics.ts` - Prometheus-compatible metrics:
- Counter, Gauge, Histogram classes
- MetricsRegistry
- MetricsCollector with standard metrics
- Prometheus text format export
- Default metrics collector
- ✅ `tracing.ts` - Distributed tracing:
- Span interface and implementation
- InMemoryTracer
- TracingProvider
- W3C trace context propagation
- No-op tracer for performance
- ✅ `index.ts` - Module exports
### Configuration (ts/core/config/)
- ✅ `types.ts` - Configuration types:
- AuthConfig (basic, apiKey, bearer, cloud)
- TLSConfig
- ConnectionPoolConfig
- RequestConfig
- DiscoveryConfig
- ElasticsearchConfig (main)
- SecretProvider interface
- EnvironmentSecretProvider
- InMemorySecretProvider
- ✅ `configuration-builder.ts` - Fluent configuration:
- ConfigurationBuilder class
- Methods: nodes(), auth(), timeout(), retries(), poolSize(), etc.
- fromEnv() - Load from environment variables
- fromFile() - Load from JSON file
- fromObject() - Load from object
- withSecrets() - Secret provider integration
- Validation with detailed error messages
- ✅ `index.ts` - Module exports
### Connection Management (ts/core/connection/)
- ✅ `health-check.ts` - Health monitoring:
- HealthChecker class
- Periodic health checks
- Cluster health status (green/yellow/red)
- Consecutive failure/success thresholds
- Health change callbacks
- ✅ `circuit-breaker.ts` - Fault tolerance:
- CircuitBreaker class
- States: CLOSED, OPEN, HALF_OPEN
- Configurable failure thresholds
- Automatic recovery testing
- Rolling window for failure counting
- CircuitBreakerOpenError
- ✅ `connection-manager.ts` - Connection lifecycle:
- Singleton ElasticsearchConnectionManager
- Integration with health checker
- Integration with circuit breaker
- Automatic client creation from config
- Metrics integration
- Initialize/destroy lifecycle
- execute() method for circuit-breaker-protected operations
- ✅ `index.ts` - Module exports
### Core Index
- ✅ `ts/core/index.ts` - Exports all core modules
## ✅ Phase 2: Domain APIs - In Progress (85% of Phase 2)
### Document API (ts/domain/documents/) - COMPLETE ✅
- ✅ `types.ts` - Type definitions
- ✅ `document-session.ts` - Session management with efficient cleanup
- ✅ `document-manager.ts` - Main fluent API class:
- DocumentManager class with full CRUD
- Integration with ConnectionManager, Logger, Metrics, Tracing
- Static factory method
- Individual operations (create, update, upsert, delete, get)
- Session support
- Async iteration with search_after
- Snapshot functionality with analytics
- Auto index creation
- Health integration
- ✅ `index.ts` - Module exports
- ✅ **Example**: Complete working example in `examples/basic/complete-example.ts`
### Query Builder (ts/domain/query/) - COMPLETE ✅
- ✅ `types.ts` - Complete query DSL type definitions:
- All query types (match, term, range, bool, etc.)
- Aggregation types (terms, metrics, histogram, etc.)
- Search options and result types
- ✅ `query-builder.ts` - Fluent QueryBuilder class:
- All standard query methods (match, term, range, exists, etc.)
- Boolean queries (must, should, must_not, filter)
- Result shaping (sort, fields, pagination)
- Aggregation integration
- Execute and count methods
- Full observability integration
- ✅ `aggregation-builder.ts` - Fluent AggregationBuilder class:
- Bucket aggregations (terms, histogram, date_histogram, range)
- Metric aggregations (avg, sum, min, max, cardinality, stats)
- Nested sub-aggregations
- Custom aggregation support
- ✅ `index.ts` - Module exports
- ✅ **Example**: Comprehensive query example in `examples/query/query-builder-example.ts`
### Logging API (ts/domain/logging/) - NOT STARTED
- ⏳ TODO: Enhanced SmartLog destination
- ⏳ TODO: Log enrichment
- ⏳ TODO: Sampling and rate limiting
- ⏳ TODO: ILM integration
### Bulk Indexer (ts/domain/bulk/) - NOT STARTED
- ⏳ TODO: Adaptive batching
- ⏳ TODO: Parallel workers
- ⏳ TODO: Progress callbacks
- ⏳ TODO: Stream support
### KV Store (ts/domain/kv/) - NOT STARTED
- ⏳ TODO: TTL support
- ⏳ TODO: Caching layer
- ⏳ TODO: Batch operations
- ⏳ TODO: Scan/cursor support
## ⏸️ Phase 3: Advanced Features - NOT STARTED
### Plugin Architecture
- ⏳ TODO: Plugin interface
- ⏳ TODO: Request/response interceptors
- ⏳ TODO: Built-in plugins (compression, caching, rate-limiting)
### Transaction Support
- ⏳ TODO: Optimistic locking with versioning
- ⏳ TODO: Transaction manager
- ⏳ TODO: Rollback support
### Schema Management
- ⏳ TODO: Type-safe schema definition
- ⏳ TODO: Migration support
- ⏳ TODO: Index template management
## ⏸️ Phase 4: Testing & Documentation - NOT STARTED
### Test Suite
- ⏳ TODO: Unit tests for all core modules
- ⏳ TODO: Integration tests
- ⏳ TODO: Chaos tests
- ⏳ TODO: Performance benchmarks
### Documentation
- ⏳ TODO: API reference (TSDoc)
- ⏳ TODO: Migration guide (v2 → v3)
- ⏳ TODO: Usage examples
- ⏳ TODO: Architecture documentation
### README
- ⏳ TODO: Update with new API examples
- ⏳ TODO: Performance benchmarks
- ⏳ TODO: Migration guide link
## 📁 File Structure Created
```
ts/
├── core/
│ ├── errors/
│ │ ├── types.ts ✅
│ │ ├── elasticsearch-error.ts ✅
│ │ ├── retry-policy.ts ✅
│ │ └── index.ts ✅
│ ├── observability/
│ │ ├── logger.ts ✅
│ │ ├── metrics.ts ✅
│ │ ├── tracing.ts ✅
│ │ └── index.ts ✅
│ ├── config/
│ │ ├── types.ts ✅
│ │ ├── configuration-builder.ts ✅
│ │ └── index.ts ✅
│ ├── connection/
│ │ ├── health-check.ts ✅
│ │ ├── circuit-breaker.ts ✅
│ │ ├── connection-manager.ts ✅
│ │ └── index.ts ✅
│ └── index.ts ✅
├── domain/
│ ├── documents/
│ │ ├── types.ts ✅
│ │ ├── document-session.ts ✅
│ │ ├── document-manager.ts ⏳ NEXT
│ │ ├── document-iterator.ts ⏳ NEXT
│ │ └── snapshot-manager.ts ⏳ NEXT
│ ├── query/ ⏳
│ ├── logging/ ⏳
│ ├── bulk/ ⏳
│ └── kv/ ⏳
├── plugins/ ⏳
├── testing/ ⏳
└── examples/ ⏳
```
## 🎯 Next Steps for Continuation
1. **✅ Document API** - COMPLETE
- Full CRUD operations with fluent API
- Session management with efficient cleanup
- Iterator and snapshot support
2. **✅ Query Builder** - COMPLETE
- Type-safe query construction
- All standard query types
- Aggregations with sub-aggregations
- Full observability integration
3. **⏳ Complete remaining Phase 2 APIs** (Current Priority):
- Logging API
- Bulk Indexer
- KV Store
4. **Phase 3 & 4** as planned
## 💡 Key Design Decisions Made
1. **Connection Manager is Singleton** - Prevents connection proliferation
2. **Circuit Breaker Pattern** - Prevents cascading failures
3. **Health Checks are Periodic** - Automatic monitoring
4. **Fluent Builder Pattern** - Discoverable, chainable API
5. **Typed Error Hierarchy** - Better error handling
6. **Observability Built-in** - Metrics, logging, tracing from start
7. **Configuration from Multiple Sources** - Env vars, files, objects, secrets
8. **Strict TypeScript** - Maximum type safety
9. **deleteByQuery for Cleanup** - Much more efficient than old scroll approach
10. **Point-in-Time API** - Will use for iteration instead of scroll
## 🔧 How to Build Current Code
```bash
# Build with strict TypeScript
npx tsc --project tsconfig.json
# This will compile all ts/**/* files with strict mode enabled
```
## 📁 Additional Files Created
- ✅ `ts/index.ts` - Main entry point with re-exports
- ✅ `ts/README.md` - Complete documentation for v3.0
- ✅ `ts/examples/basic/complete-example.ts` - Comprehensive working example
- ✅ `IMPLEMENTATION_STATUS.md` - This file
## 📝 Usage Example (Working Now!)
```typescript
import { createConfig, ElasticsearchConnectionManager } from './ts/core';
import { DocumentManager } from './ts/domain/documents';
// Configure
const config = createConfig()
.fromEnv()
.nodes('http://localhost:9200')
.basicAuth('elastic', 'changeme')
.timeout(30000)
.enableMetrics()
.enableTracing()
.build();
// Initialize connection
const manager = ElasticsearchConnectionManager.getInstance(config);
await manager.initialize();
// Use Document API
const docs = new DocumentManager<Product>('products', manager);
await docs
.session()
.start()
.upsert('prod-1', { name: 'Widget', price: 99.99 })
.upsert('prod-2', { name: 'Gadget', price: 149.99 })
.commit();
```
## 📊 Progress Summary
- **Total Files Created**: 34
- **Total Lines of Code**: ~9,000
- **Phase 1 Completion**: 100% ✅
- **Phase 2 Completion**: 85% (Document API + Query Builder complete)
- **Overall Completion**: 60%
- **Remaining Work**: Logging API, Bulk Indexer, KV Store, Plugins, Transactions, Schema, Tests
## 🔄 Resumption Instructions
When resuming:
1. Read this status file
2. Review the complete plan in the original conversation
3. **Next Priority**: Enhanced Logging API in `ts/domain/logging/`
4. Then continue with remaining Phase 2 APIs (Bulk, KV)
5. Update todo list as you progress
6. Update this status file when completing major milestones
### Immediate Next Steps
**Phase 2 Remaining (Priority Order):**
1. **Logging API** (`ts/domain/logging/`)
- Enhanced SmartLogDestination
- Log enrichment and sampling
- ILM integration
- Metric extraction
2. **Bulk Indexer** (`ts/domain/bulk/`)
- Adaptive batching logic
- Parallel workers
- Progress callbacks
- Stream support
3. **KV Store** (`ts/domain/kv/`)
- TTL support
- Caching layer
- Batch operations
- Scan/cursor

2
package.json
View File

@@ -1,6 +1,6 @@
{ {
"name": "@apiclient.xyz/elasticsearch", "name": "@apiclient.xyz/elasticsearch",
"version": "3.1.1", "version": "3.1.2",
"private": false, "private": false,
"description": "log to elasticsearch in a kibana compatible format", "description": "log to elasticsearch in a kibana compatible format",
"main": "dist_ts/index.js", "main": "dist_ts/index.js",