apiclient.xyz/elasticsearch
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: Comprehensive v3 readme with full API documentation

Browse Source
This commit is contained in:
Jürgen Kunz
2025-11-29 21:23:07 +00:00
parent 72370b754b
commit 175762c9a7
4 changed files with 395 additions and 1247 deletions
Download Patch File Download Diff File Expand all files Collapse all files

108
ts/QUICK_FIXES.md
View File

@@ -1,108 +0,0 @@
# Quick Fixes Needed for TypeScript Strict Mode
## Import Fixes (Use `import type` for verbatimModuleSyntax)
### Files to fix:
1. **ts/core/connection/connection-manager.ts**
```typescript
// Change:
import { ElasticsearchConfig } from '../config/types.js';
import { HealthCheckResult, HealthStatus } from './health-check.js';
// To:
import type { ElasticsearchConfig } from '../config/types.js';
import type { HealthCheckResult } from './health-check.js';
import { HealthStatus } from './health-check.js';
```
2. **ts/core/errors/elasticsearch-error.ts**
```typescript
// Change:
import { ErrorCode, ErrorContext } from './types.js';
// To:
import { ErrorCode } from './types.js';
import type { ErrorContext } from './types.js';
```
3. **ts/core/errors/retry-policy.ts**
```typescript
// Change:
import { RetryConfig, RetryStrategy } from './types.js';
// To:
import type { RetryConfig, RetryStrategy } from './types.js';
```
4. **ts/domain/documents/document-manager.ts**
```typescript
// Change:
import {
DocumentWithMeta,
SessionConfig,
SnapshotProcessor,
SnapshotMeta,
IteratorOptions,
} from './types.js';
// To:
import type {
DocumentWithMeta,
SessionConfig,
SnapshotProcessor,
SnapshotMeta,
IteratorOptions,
} from './types.js';
```
## Tracing undefined issue (ts/core/observability/tracing.ts:315-317)
```typescript
// In TracingProvider.createSpan(), change:
const span = this.tracer.startSpan(name, {
...attributes,
'service.name': this.config.serviceName,
...(this.config.serviceVersion && { 'service.version': this.config.serviceVersion }),
});
// To:
const spanAttributes = {
...attributes,
'service.name': this.config.serviceName || 'elasticsearch-client',
};
if (this.config.serviceVersion) {
spanAttributes['service.version'] = this.config.serviceVersion;
}
const span = this.tracer.startSpan(name, spanAttributes);
```
## Generic Type Constraints for Elasticsearch Client
In **ts/domain/documents/document-manager.ts**, add constraint:
```typescript
// Change class definition:
export class DocumentManager<T = unknown> {
// To:
export class DocumentManager<T extends Record<string, any> = Record<string, any>> {
```
This ensures T is always an object type compatible with Elasticsearch operations.
## Alternative: Relax Strict Mode Temporarily
If immediate fixes are needed, you can temporarily relax some strict checks in tsconfig.json:
```json
{
"compilerOptions": {
// Comment out temporarily:
// "verbatimModuleSyntax": true,
// "noUncheckedIndexedAccess": true,
}
}
```
But the proper fix is to address the imports and type issues as outlined above.

386
ts/README.md
View File

@@ -1,386 +0,0 @@
# Enterprise Elasticsearch Client v3.0 (NEW Architecture)
> 🚧 **Status**: Phase 1 & Core Phase 2 Complete | 70% Implementation Complete
**Modern, type-safe, production-ready Elasticsearch client** with enterprise features built-in from the ground up.
## 🎯 What's New in v3.0
### Core Infrastructure
- ✅ **Connection Manager** - Singleton with pooling, health checks, circuit breaker
- ✅ **Configuration System** - Environment variables, files, secrets, validation
- ✅ **Error Handling** - Typed error hierarchy with retry policies
- ✅ **Observability** - Structured logging, Prometheus metrics, distributed tracing
- ✅ **Circuit Breaker** - Prevent cascading failures
- ✅ **Health Monitoring** - Automatic cluster health checks
### Domain APIs
- ✅ **Document Manager** - Fluent API for CRUD operations
- ✅ **Session Management** - Batch operations with automatic cleanup
- ✅ **Snapshot System** - Point-in-time analytics
- ⏳ **Query Builder** - Type-safe query DSL (Coming soon)
- ⏳ **Bulk Indexer** - Adaptive batching, parallel workers (Coming soon)
- ⏳ **KV Store** - TTL, caching, batch ops (Coming soon)
- ⏳ **Logging API** - Kibana integration, enrichment (Coming soon)
### Advanced Features
- ⏳ **Plugin System** - Extensible with middleware
- ⏳ **Transactions** - Optimistic locking, rollback
- ⏳ **Schema Management** - Type-safe schemas, migrations
## 🚀 Quick Start
### Installation
```bash
# Install dependencies
pnpm install
# Build the new implementation
npx tsc --project tsconfig.json
```
### Basic Usage
```typescript
import {
createConfig,
ElasticsearchConnectionManager,
DocumentManager,
LogLevel,
} from './ts';
// 1. Configure
const config = createConfig()
.fromEnv() // Load from ELASTICSEARCH_URL, etc.
.nodes('http://localhost:9200')
.basicAuth('elastic', 'changeme')
.timeout(30000)
.retries(3)
.logLevel(LogLevel.INFO)
.enableMetrics()
.enableTracing()
.build();
// 2. Initialize connection
const manager = ElasticsearchConnectionManager.getInstance(config);
await manager.initialize();
// 3. Create document manager
const docs = new DocumentManager<Product>({
index: 'products',
autoCreateIndex: true,
});
await docs.initialize();
// 4. Use fluent API
await docs.upsert('prod-1', {
name: 'Widget',
price: 99.99,
inStock: true,
});
// 5. Session-based batch operations
await docs
.session()
.start()
.upsert('prod-2', { name: 'Gadget', price: 149.99, inStock: true })
.upsert('prod-3', { name: 'Tool', price: 49.99, inStock: false })
.commit();
// 6. Iterate over documents
for await (const doc of docs.iterate()) {
console.log(doc._source);
}
// 7. Create snapshots
const snapshot = await docs.snapshot(async (iterator) => {
const items = [];
for await (const doc of iterator) {
items.push(doc._source);
}
return { count: items.length, items };
});
```
## 📚 Complete Example
See [`examples/basic/complete-example.ts`](./examples/basic/complete-example.ts) for a comprehensive demonstration including:
- Configuration from environment
- Connection management with health checks
- Individual and batch operations
- Document iteration
- Snapshot analytics
- Metrics and observability
- Error handling
Run it with:
```bash
npx tsx ts/examples/basic/complete-example.ts
```
## 🏗️ Architecture
```
ts/
├── core/ # Foundation layer
│ ├── config/ # Configuration management ✅
│ ├── connection/ # Connection pooling, health ✅
│ ├── errors/ # Error hierarchy, retry ✅
│ └── observability/ # Logging, metrics, tracing ✅
├── domain/ # Business logic layer
│ ├── documents/ # Document API ✅
│ ├── query/ # Query builder ⏳
│ ├── logging/ # Log destination ⏳
│ ├── bulk/ # Bulk indexer ⏳
│ └── kv/ # Key-value store ⏳
├── plugins/ # Extension points ⏳
├── testing/ # Test utilities ⏳
└── examples/ # Usage examples ✅
```
## ⚡ Key Improvements Over v2.x
| Feature | v2.x | v3.0 |
|---------|------|------|
| Connection Pooling | ❌ Each class creates own client | ✅ Singleton connection manager |
| Health Checks | ❌ None | ✅ Automatic periodic checks |
| Circuit Breaker | ❌ None | ✅ Fault tolerance built-in |
| Error Handling | ⚠️ Inconsistent | ✅ Typed error hierarchy |
| Retry Logic | ⚠️ Basic scheduler | ✅ Exponential backoff, jitter |
| Configuration | ⚠️ Constructor only | ✅ Env vars, files, secrets |
| Logging | ⚠️ console.log scattered | ✅ Structured logging with context |
| Metrics | ❌ None | ✅ Prometheus-compatible |
| Tracing | ❌ None | ✅ OpenTelemetry-compatible |
| Type Safety | ⚠️ Partial, uses `any` | ✅ Strict TypeScript, no `any` |
| API Design | ⚠️ Inconsistent constructors | ✅ Fluent, discoverable |
| Bulk Operations | ⚠️ Sequential, inefficient | ✅ Batched with error handling |
| Document Cleanup | ⚠️ O(n) scroll all docs | ✅ deleteByQuery (efficient) |
| Observability | ❌ None | ✅ Full observability stack |
## 📖 API Documentation
### Configuration
```typescript
import { createConfig, LogLevel } from './ts';
const config = createConfig()
// Data sources
.fromEnv() // Load from environment variables
.fromFile('config.json') // Load from JSON file
.fromObject({ ... }) // Load from object
// Connection
.nodes(['http://es1:9200', 'http://es2:9200'])
.auth({ type: 'basic', username: 'user', password: 'pass' })
.apiKeyAuth('api-key')
.timeout(30000)
.retries(3)
.compression(true)
.poolSize(10, 2) // max, min idle
// Discovery
.discovery(true, { interval: 60000 })
// Observability
.logLevel(LogLevel.INFO)
.enableRequestLogging(true)
.enableMetrics(true, 'my_app')
.enableTracing(true, { serviceName: 'api', serviceVersion: '1.0.0' })
// Secrets
.withSecrets(secretProvider)
.build();
```
### Connection Management
```typescript
import { ElasticsearchConnectionManager } from './ts';
const manager = ElasticsearchConnectionManager.getInstance(config);
await manager.initialize();
// Health check
const health = await manager.healthCheck();
console.log(health.status, health.clusterHealth, health.activeNodes);
// Circuit breaker
const result = await manager.execute(async () => {
return await someOperation();
});
// Stats
const stats = manager.getStats();
console.log(stats.healthStatus, stats.circuitState);
// Cleanup
await manager.destroy();
```
### Document Operations
```typescript
import { DocumentManager } from './ts';
const docs = new DocumentManager<MyType>({ index: 'my-index', autoCreateIndex: true });
await docs.initialize();
// CRUD
await docs.create('id', doc);
await docs.update('id', { field: 'value' });
await docs.upsert('id', doc);
await docs.delete('id');
const doc = await docs.get('id');
// Optimistic locking
await docs.update('id', doc, { seqNo: 123, primaryTerm: 1 });
// Batch operations
const result = await docs
.session({ cleanupStale: true })
.start()
.upsert('id1', doc1)
.upsert('id2', doc2)
.delete('id3')
.commit();
// Iteration
for await (const doc of docs.iterate({ batchSize: 500 })) {
console.log(doc._source);
}
// Snapshots
const snapshot = await docs.snapshot(async (iterator, prev) => {
// Custom analytics
return computedData;
});
// Utilities
const count = await docs.count();
const exists = await docs.exists();
await docs.deleteIndex();
```
### Error Handling
```typescript
import {
ElasticsearchError,
ConnectionError,
DocumentNotFoundError,
BulkOperationError,
ErrorCode,
} from './ts';
try {
await docs.get('id');
} catch (error) {
if (error instanceof DocumentNotFoundError) {
// Handle not found
} else if (error instanceof ConnectionError) {
// Handle connection error
} else if (error instanceof ElasticsearchError) {
console.log(error.code, error.retryable, error.context);
}
}
```
### Observability
```typescript
import { defaultLogger, defaultMetricsCollector, defaultTracingProvider } from './ts';
// Logging
const logger = defaultLogger.child('my-component');
logger.info('Message', { key: 'value' });
logger.error('Error', error, { context: 'data' });
// Correlation
const correlatedLogger = logger.withCorrelation(requestId);
// Metrics
defaultMetricsCollector.requestsTotal.inc({ operation: 'search', index: 'products' });
defaultMetricsCollector.requestDuration.observe(0.234, { operation: 'search' });
// Export metrics
const prometheus = defaultMetricsCollector.export();
// Tracing
await defaultTracingProvider.withSpan('operation', async (span) => {
span.setAttribute('key', 'value');
return await doWork();
});
```
## 🔒 Security
- ✅ Support for basic, API key, bearer token, cloud ID authentication
- ✅ TLS/SSL configuration
- ✅ Secret provider integration (environment, AWS Secrets Manager, Vault, etc.)
- ✅ Credential validation
- ✅ No credentials in logs or error messages
## 🧪 Testing
```bash
# Run tests (when implemented)
pnpm test
# Type check
npx tsc --project tsconfig.json --noEmit
# Lint
npx eslint ts/**/*.ts
```
## 📊 Performance
- ✅ Connection pooling reduces overhead
- ✅ Batch operations use bulk API
- ✅ deleteByQuery for efficient cleanup (vs old scroll approach)
- ✅ Point-in-Time API for iteration (vs scroll)
- ✅ Circuit breaker prevents wasted requests
- ⏳ Adaptive batching (coming soon)
- ⏳ Parallel bulk workers (coming soon)
## 🗺️ Roadmap
### Phase 2 Remaining (In Progress)
- [ ] Type-safe Query Builder
- [ ] Enhanced Logging API with Kibana integration
- [ ] Bulk Indexer with adaptive batching
- [ ] KV Store with TTL and caching
### Phase 3 (Planned)
- [ ] Plugin architecture with middleware
- [ ] Transaction support with optimistic locking
- [ ] Schema management and migrations
### Phase 4 (Planned)
- [ ] Comprehensive test suite (unit, integration, chaos)
- [ ] Migration guide from v2.x to v3.0
- [ ] Performance benchmarks
- [ ] Full API documentation
## 📄 License and Legal Information
This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](../license) file within this repository.
**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
### Trademarks
This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.
### Company Information
Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany
For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.
By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.