docs: Comprehensive v3 readme with full API documentation
This commit is contained in:
654
readme.md
654
readme.md
@@ -1,306 +1,442 @@
|
||||
# @apiclient.xyz/elasticsearch
|
||||
|
||||
> 🔍 **Modern TypeScript client for Elasticsearch with built-in Kibana compatibility and advanced logging features**
|
||||
> 🚀 **Enterprise-grade TypeScript client for Elasticsearch** — Type-safe, observable, and production-ready
|
||||
|
||||
A powerful, type-safe wrapper around the official Elasticsearch client that provides intelligent log management, document handling, key-value storage, and fast data ingestion - all optimized for production use.
|
||||
A modern, fully-typed Elasticsearch client built for scale. Features fluent configuration, distributed transactions, intelligent bulk operations, advanced logging with Kibana compatibility, and comprehensive observability out of the box.
|
||||
|
||||
## Issue Reporting and Security
|
||||
|
||||
For reporting bugs, issues, or security vulnerabilities, please visit [community.foss.global/](https://community.foss.global/). This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull Requests directly.
|
||||
|
||||
## Features ✨
|
||||
## ✨ Features
|
||||
|
||||
- **🎯 SmartLog Destination** - Full-featured logging destination compatible with Kibana, automatic index rotation, and retention management
|
||||
- **📦 ElasticDoc** - Advanced document management with piping sessions, snapshots, and automatic cleanup
|
||||
- **🚀 FastPush** - High-performance bulk document insertion with automatic index management
|
||||
- **💾 KVStore** - Simple key-value storage interface backed by Elasticsearch
|
||||
- **🔧 TypeScript First** - Complete type safety with full TypeScript support
|
||||
- **🌊 Data Streams** - Built-in support for Elasticsearch data streams
|
||||
- **⚡ Production Ready** - Designed for high-throughput production environments
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| 🔧 **Fluent Configuration** | Builder pattern for type-safe, environment-aware configuration |
|
||||
| 📦 **Document Management** | Full CRUD with sessions, snapshots, and lifecycle management |
|
||||
| 🔍 **Query Builder** | Type-safe DSL for complex queries and aggregations |
|
||||
| 📊 **Bulk Operations** | High-throughput indexing with backpressure and adaptive batching |
|
||||
| 💾 **KV Store** | Distributed key-value storage with TTL, caching, and compression |
|
||||
| 🔄 **Transactions** | ACID-like semantics with optimistic concurrency control |
|
||||
| 📋 **Schema Management** | Index templates, migrations, and schema validation |
|
||||
| 📝 **Logging** | Kibana-compatible log destination with ILM policies |
|
||||
| 🔌 **Plugin System** | Extensible architecture with built-in retry, caching, rate limiting |
|
||||
| 📈 **Observability** | Prometheus metrics, distributed tracing, structured logging |
|
||||
| ⚡ **Circuit Breaker** | Automatic failure detection and recovery |
|
||||
|
||||
## Installation 📦
|
||||
## 📦 Installation
|
||||
|
||||
```bash
|
||||
npm install @apiclient.xyz/elasticsearch
|
||||
# or
|
||||
pnpm install @apiclient.xyz/elasticsearch
|
||||
pnpm add @apiclient.xyz/elasticsearch
|
||||
```
|
||||
|
||||
## Quick Start 🚀
|
||||
## 🚀 Quick Start
|
||||
|
||||
### SmartLog Destination
|
||||
|
||||
Perfect for application logging with automatic index rotation and Kibana compatibility:
|
||||
### Configuration
|
||||
|
||||
```typescript
|
||||
import { ElsSmartlogDestination } from '@apiclient.xyz/elasticsearch';
|
||||
import { createConfig, ElasticsearchConnectionManager, LogLevel } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const logger = new ElsSmartlogDestination({
|
||||
indexPrefix: 'app-logs',
|
||||
indexRetention: 7, // Keep logs for 7 days
|
||||
node: 'http://localhost:9200',
|
||||
auth: {
|
||||
username: 'elastic',
|
||||
password: 'your-password',
|
||||
},
|
||||
});
|
||||
// Fluent configuration builder
|
||||
const config = createConfig()
|
||||
.fromEnv() // Load from ELASTICSEARCH_URL, ELASTICSEARCH_USERNAME, etc.
|
||||
.nodes('http://localhost:9200')
|
||||
.basicAuth('elastic', 'changeme')
|
||||
.timeout(30000)
|
||||
.retries(3)
|
||||
.compression(true)
|
||||
.poolSize(10, 2)
|
||||
.logLevel(LogLevel.INFO)
|
||||
.enableMetrics(true)
|
||||
.enableTracing(true, {
|
||||
serviceName: 'my-service',
|
||||
serviceVersion: '1.0.0',
|
||||
})
|
||||
.build();
|
||||
|
||||
// Log messages that automatically appear in Kibana
|
||||
await logger.log({
|
||||
timestamp: Date.now(),
|
||||
type: 'increment',
|
||||
level: 'info',
|
||||
context: {
|
||||
company: 'YourCompany',
|
||||
companyunit: 'api-service',
|
||||
containerName: 'web-server',
|
||||
environment: 'production',
|
||||
runtime: 'node',
|
||||
zone: 'us-east-1',
|
||||
},
|
||||
message: 'User authentication successful',
|
||||
correlation: null,
|
||||
});
|
||||
// Initialize connection with health monitoring
|
||||
const connection = ElasticsearchConnectionManager.getInstance(config);
|
||||
await connection.initialize();
|
||||
|
||||
console.log(`Health: ${connection.getHealthStatus()}`);
|
||||
console.log(`Circuit: ${connection.getCircuitState()}`);
|
||||
```
|
||||
|
||||
### ElasticDoc - Document Management
|
||||
|
||||
Handle documents with advanced features like piping sessions and snapshots:
|
||||
### Document Management
|
||||
|
||||
```typescript
|
||||
import { ElasticDoc } from '@apiclient.xyz/elasticsearch';
|
||||
import { DocumentManager } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const docManager = new ElasticDoc({
|
||||
index: 'products',
|
||||
node: 'http://localhost:9200',
|
||||
auth: {
|
||||
username: 'elastic',
|
||||
password: 'your-password',
|
||||
},
|
||||
});
|
||||
|
||||
// Start a piping session to manage document lifecycle
|
||||
await docManager.startPipingSession({});
|
||||
|
||||
// Add or update documents
|
||||
await docManager.pipeDocument({
|
||||
docId: 'product-001',
|
||||
timestamp: new Date().toISOString(),
|
||||
doc: {
|
||||
name: 'Premium Widget',
|
||||
price: 99.99,
|
||||
inStock: true,
|
||||
},
|
||||
});
|
||||
|
||||
await docManager.pipeDocument({
|
||||
docId: 'product-002',
|
||||
timestamp: new Date().toISOString(),
|
||||
doc: {
|
||||
name: 'Deluxe Gadget',
|
||||
price: 149.99,
|
||||
inStock: false,
|
||||
},
|
||||
});
|
||||
|
||||
// End session - automatically removes documents not in this session
|
||||
await docManager.endPipingSession();
|
||||
|
||||
// Take and store snapshots with custom aggregations
|
||||
await docManager.takeSnapshot(async (iterator, prevSnapshot) => {
|
||||
const aggregationData = [];
|
||||
for await (const doc of iterator) {
|
||||
aggregationData.push(doc);
|
||||
}
|
||||
return {
|
||||
date: new Date().toISOString(),
|
||||
aggregationData,
|
||||
};
|
||||
});
|
||||
```
|
||||
|
||||
### FastPush - Bulk Data Ingestion
|
||||
|
||||
Efficiently push large datasets with automatic index management:
|
||||
|
||||
```typescript
|
||||
import { FastPush } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const fastPush = new FastPush({
|
||||
node: 'http://localhost:9200',
|
||||
auth: {
|
||||
username: 'elastic',
|
||||
password: 'your-password',
|
||||
},
|
||||
});
|
||||
|
||||
const documents = [
|
||||
{ id: 1, name: 'Document 1', data: 'Some data' },
|
||||
{ id: 2, name: 'Document 2', data: 'More data' },
|
||||
// ... thousands more documents
|
||||
];
|
||||
|
||||
// Push all documents with automatic batching
|
||||
await fastPush.pushDocuments('bulk-data', documents, {
|
||||
deleteOldData: true, // Clear old data before inserting
|
||||
});
|
||||
```
|
||||
|
||||
### KVStore - Key-Value Storage
|
||||
|
||||
Simple key-value storage backed by the power of Elasticsearch:
|
||||
|
||||
```typescript
|
||||
import { KVStore } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const kvStore = new KVStore({
|
||||
index: 'app-config',
|
||||
node: 'http://localhost:9200',
|
||||
auth: {
|
||||
username: 'elastic',
|
||||
password: 'your-password',
|
||||
},
|
||||
});
|
||||
|
||||
// Set values
|
||||
await kvStore.set('api-key', 'sk-1234567890');
|
||||
await kvStore.set('feature-flags', JSON.stringify({ newUI: true }));
|
||||
|
||||
// Get values
|
||||
const apiKey = await kvStore.get('api-key');
|
||||
console.log(apiKey); // 'sk-1234567890'
|
||||
|
||||
// Clear all data
|
||||
await kvStore.clear();
|
||||
```
|
||||
|
||||
## Core Classes 🏗️
|
||||
|
||||
### ElsSmartlogDestination
|
||||
|
||||
The main logging destination class that provides:
|
||||
|
||||
- Automatic index rotation based on date
|
||||
- Configurable retention policies
|
||||
- Kibana-compatible log format
|
||||
- Data stream support
|
||||
- Built-in scheduler for maintenance tasks
|
||||
|
||||
### ElasticDoc
|
||||
|
||||
Advanced document management with:
|
||||
|
||||
- Piping sessions for tracking document lifecycles
|
||||
- Automatic cleanup of stale documents
|
||||
- Snapshot functionality with custom processors
|
||||
- Iterator-based document access
|
||||
- Fast-forward mode for incremental processing
|
||||
|
||||
### FastPush
|
||||
|
||||
High-performance bulk operations:
|
||||
|
||||
- Automatic batching for optimal performance
|
||||
- Index management (create, delete, clear)
|
||||
- Dynamic mapping support
|
||||
- Efficient bulk API usage
|
||||
|
||||
### KVStore
|
||||
|
||||
Simple key-value interface:
|
||||
|
||||
- Elasticsearch-backed storage
|
||||
- Async/await API
|
||||
- Automatic index initialization
|
||||
- Clear and get operations
|
||||
|
||||
## Advanced Usage 🎓
|
||||
|
||||
### Index Rotation and Retention
|
||||
|
||||
```typescript
|
||||
const logger = new ElsSmartlogDestination({
|
||||
indexPrefix: 'myapp',
|
||||
indexRetention: 30, // Keep 30 days of logs
|
||||
node: 'http://localhost:9200',
|
||||
});
|
||||
|
||||
// Indices are automatically created as: myapp-2025-01-22
|
||||
// Old indices are automatically deleted after 30 days
|
||||
```
|
||||
|
||||
### Document Iteration
|
||||
|
||||
```typescript
|
||||
// Iterate over all documents in an index
|
||||
const iterator = docManager.getDocumentIterator();
|
||||
for await (const doc of iterator) {
|
||||
console.log(doc);
|
||||
interface Product {
|
||||
name: string;
|
||||
price: number;
|
||||
category: string;
|
||||
inStock: boolean;
|
||||
}
|
||||
|
||||
// Only process new documents since last run
|
||||
docManager.fastForward = true;
|
||||
await docManager.startPipingSession({ onlyNew: true });
|
||||
```
|
||||
const products = new DocumentManager<Product>({
|
||||
index: 'products',
|
||||
connectionManager: connection,
|
||||
autoCreateIndex: true,
|
||||
});
|
||||
|
||||
### Custom Snapshots
|
||||
await products.initialize();
|
||||
|
||||
```typescript
|
||||
await docManager.takeSnapshot(async (iterator, prevSnapshot) => {
|
||||
let totalValue = 0;
|
||||
let count = 0;
|
||||
// CRUD operations
|
||||
await products.create('prod-001', {
|
||||
name: 'Premium Widget',
|
||||
price: 99.99,
|
||||
category: 'widgets',
|
||||
inStock: true,
|
||||
});
|
||||
|
||||
const product = await products.get('prod-001');
|
||||
await products.update('prod-001', { price: 89.99 });
|
||||
await products.delete('prod-001');
|
||||
|
||||
// Session-based batch operations with auto-cleanup
|
||||
const result = await products.session({ cleanupStale: true })
|
||||
.start()
|
||||
.upsert('prod-001', { name: 'Widget A', price: 10, category: 'widgets', inStock: true })
|
||||
.upsert('prod-002', { name: 'Widget B', price: 20, category: 'widgets', inStock: false })
|
||||
.commit();
|
||||
|
||||
console.log(`Processed ${result.successful} documents in ${result.took}ms`);
|
||||
|
||||
// Iterate over all documents
|
||||
for await (const doc of products.iterate()) {
|
||||
console.log(doc._source.name);
|
||||
}
|
||||
|
||||
// Create analytics snapshot
|
||||
const snapshot = await products.snapshot(async (iterator, previous) => {
|
||||
let total = 0, count = 0;
|
||||
for await (const doc of iterator) {
|
||||
totalValue += doc._source.price;
|
||||
total += doc._source.price;
|
||||
count++;
|
||||
}
|
||||
|
||||
return {
|
||||
date: new Date().toISOString(),
|
||||
aggregationData: {
|
||||
totalValue,
|
||||
averagePrice: totalValue / count,
|
||||
count,
|
||||
previousSnapshot: prevSnapshot,
|
||||
},
|
||||
};
|
||||
return { averagePrice: total / count, productCount: count };
|
||||
});
|
||||
```
|
||||
|
||||
## API Compatibility 🔄
|
||||
|
||||
This module is built on top of `@elastic/elasticsearch` v9.x and is compatible with:
|
||||
|
||||
- Elasticsearch 8.x and 9.x clusters
|
||||
- Kibana 8.x and 9.x for log visualization
|
||||
- OpenSearch (with some limitations)
|
||||
|
||||
## TypeScript Support 💙
|
||||
|
||||
Full TypeScript support with comprehensive type definitions:
|
||||
### Query Builder
|
||||
|
||||
```typescript
|
||||
import type {
|
||||
IElasticDocConstructorOptions,
|
||||
ISnapshot,
|
||||
SnapshotProcessor,
|
||||
} from '@apiclient.xyz/elasticsearch';
|
||||
import { createQuery } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
// Fluent query building
|
||||
const results = await createQuery<Product>('products')
|
||||
.match('name', 'widget', { fuzziness: 'AUTO' })
|
||||
.range('price', { gte: 10, lte: 100 })
|
||||
.term('inStock', true)
|
||||
.sort('price', 'asc')
|
||||
.size(20)
|
||||
.from(0)
|
||||
.highlight({ fields: { name: {} } })
|
||||
.aggregations(agg => agg
|
||||
.terms('categories', 'category')
|
||||
.avg('avgPrice', 'price')
|
||||
)
|
||||
.execute();
|
||||
|
||||
console.log(`Found ${results.hits.total} products`);
|
||||
console.log('Categories:', results.aggregations?.categories);
|
||||
```
|
||||
|
||||
## Performance Considerations ⚡
|
||||
### Bulk Operations
|
||||
|
||||
- **Bulk Operations**: FastPush uses 1000-document batches by default
|
||||
- **Connection Pooling**: Reuses Elasticsearch client connections
|
||||
- **Index Management**: Automatic index creation and deletion
|
||||
- **Data Streams**: Built-in support for efficient log ingestion
|
||||
```typescript
|
||||
import { createBulkIndexer } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
## Best Practices 💡
|
||||
const indexer = createBulkIndexer({
|
||||
flushThreshold: 1000,
|
||||
flushIntervalMs: 5000,
|
||||
maxConcurrent: 3,
|
||||
enableBackpressure: true,
|
||||
onProgress: (progress) => {
|
||||
console.log(`Progress: ${progress.processed}/${progress.total} (${progress.successRate}%)`);
|
||||
},
|
||||
});
|
||||
|
||||
1. **Always use authentication** in production environments
|
||||
2. **Set appropriate retention policies** to manage storage costs
|
||||
3. **Use piping sessions** to automatically clean up stale documents
|
||||
4. **Leverage snapshots** for point-in-time analytics
|
||||
5. **Configure index templates** for consistent mappings
|
||||
await indexer.initialize();
|
||||
|
||||
// Queue operations
|
||||
for (const item of largeDataset) {
|
||||
await indexer.index('products', item.id, item);
|
||||
}
|
||||
|
||||
// Wait for completion
|
||||
const stats = await indexer.flush();
|
||||
console.log(`Indexed ${stats.successful} documents, ${stats.failed} failures`);
|
||||
```
|
||||
|
||||
### Key-Value Store
|
||||
|
||||
```typescript
|
||||
import { createKVStore } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const kv = createKVStore<string>({
|
||||
index: 'app-config',
|
||||
enableCache: true,
|
||||
cacheMaxSize: 10000,
|
||||
defaultTTL: 3600, // 1 hour
|
||||
enableCompression: true,
|
||||
});
|
||||
|
||||
await kv.initialize();
|
||||
|
||||
// Basic operations
|
||||
await kv.set('api-key', 'sk-secret-key', { ttl: 86400 });
|
||||
const key = await kv.get('api-key');
|
||||
|
||||
// Batch operations
|
||||
await kv.mset([
|
||||
{ key: 'config:a', value: 'value-a' },
|
||||
{ key: 'config:b', value: 'value-b' },
|
||||
]);
|
||||
|
||||
const values = await kv.mget(['config:a', 'config:b']);
|
||||
|
||||
// Scan with pattern matching
|
||||
const scan = await kv.scan({ pattern: 'config:*', limit: 100 });
|
||||
```
|
||||
|
||||
### Transactions
|
||||
|
||||
```typescript
|
||||
import { createTransactionManager } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const txManager = createTransactionManager({
|
||||
defaultIsolationLevel: 'read_committed',
|
||||
defaultLockingStrategy: 'optimistic',
|
||||
conflictResolution: 'retry',
|
||||
maxConcurrentTransactions: 100,
|
||||
});
|
||||
|
||||
await txManager.initialize();
|
||||
|
||||
// ACID-like transaction
|
||||
const tx = await txManager.begin({ autoRollback: true });
|
||||
|
||||
try {
|
||||
const account1 = await tx.read('accounts', 'acc-001');
|
||||
const account2 = await tx.read('accounts', 'acc-002');
|
||||
|
||||
await tx.update('accounts', 'acc-001', { balance: account1.balance - 100 });
|
||||
await tx.update('accounts', 'acc-002', { balance: account2.balance + 100 });
|
||||
|
||||
tx.savepoint('after-transfer');
|
||||
|
||||
await tx.commit();
|
||||
} catch (error) {
|
||||
await tx.rollback();
|
||||
}
|
||||
```
|
||||
|
||||
### Logging Destination
|
||||
|
||||
```typescript
|
||||
import { createLogDestination, chainEnrichers, addHostInfo, addTimestamp } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const logger = createLogDestination({
|
||||
indexPrefix: 'app-logs',
|
||||
dataStream: true,
|
||||
ilmPolicy: {
|
||||
name: 'logs-policy',
|
||||
phases: {
|
||||
hot: { maxAge: '7d', maxSize: '50gb' },
|
||||
warm: { minAge: '7d' },
|
||||
delete: { minAge: '30d' },
|
||||
},
|
||||
},
|
||||
enricher: chainEnrichers(addHostInfo(), addTimestamp()),
|
||||
sampling: { strategy: 'rate', rate: 0.1 }, // 10% sampling
|
||||
});
|
||||
|
||||
await logger.initialize();
|
||||
|
||||
await logger.log({
|
||||
level: 'info',
|
||||
message: 'User authenticated',
|
||||
context: { userId: '12345', service: 'auth' },
|
||||
});
|
||||
|
||||
// Batch logging
|
||||
await logger.logBatch([
|
||||
{ level: 'debug', message: 'Request received' },
|
||||
{ level: 'info', message: 'Processing complete' },
|
||||
]);
|
||||
```
|
||||
|
||||
### Schema Management
|
||||
|
||||
```typescript
|
||||
import { createSchemaManager } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const schema = createSchemaManager({ enableMigrationHistory: true });
|
||||
|
||||
await schema.initialize();
|
||||
|
||||
// Create index with schema
|
||||
await schema.createIndex('products', {
|
||||
mappings: {
|
||||
properties: {
|
||||
name: { type: 'text', analyzer: 'standard' },
|
||||
price: { type: 'float' },
|
||||
category: { type: 'keyword' },
|
||||
createdAt: { type: 'date' },
|
||||
},
|
||||
},
|
||||
settings: {
|
||||
numberOfShards: 3,
|
||||
numberOfReplicas: 1,
|
||||
},
|
||||
});
|
||||
|
||||
// Run migrations
|
||||
await schema.migrate('products', [
|
||||
{
|
||||
version: '1.0.1',
|
||||
description: 'Add tags field',
|
||||
up: async (client, index) => {
|
||||
await client.indices.putMapping({
|
||||
index,
|
||||
properties: { tags: { type: 'keyword' } },
|
||||
});
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
// Create index template
|
||||
await schema.createIndexTemplate('products-template', {
|
||||
indexPatterns: ['products-*'],
|
||||
mappings: { /* ... */ },
|
||||
});
|
||||
```
|
||||
|
||||
### Plugin System
|
||||
|
||||
```typescript
|
||||
import {
|
||||
createPluginManager,
|
||||
createRetryPlugin,
|
||||
createCachePlugin,
|
||||
createRateLimitPlugin,
|
||||
} from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
const plugins = createPluginManager({ enableMetrics: true });
|
||||
|
||||
// Built-in plugins
|
||||
plugins.register(createRetryPlugin({ maxRetries: 3, backoffMs: 1000 }));
|
||||
plugins.register(createCachePlugin({ maxSize: 1000, ttlMs: 60000 }));
|
||||
plugins.register(createRateLimitPlugin({ maxRequests: 100, windowMs: 1000 }));
|
||||
|
||||
// Custom plugin
|
||||
plugins.register({
|
||||
name: 'custom-logger',
|
||||
version: '1.0.0',
|
||||
hooks: {
|
||||
beforeRequest: async (ctx) => {
|
||||
console.log(`Request: ${ctx.operation} on ${ctx.index}`);
|
||||
},
|
||||
afterResponse: async (ctx, response) => {
|
||||
console.log(`Response: ${response.statusCode}`);
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
```
|
||||
@apiclient.xyz/elasticsearch
|
||||
├── core/
|
||||
│ ├── config/ # Fluent configuration builder
|
||||
│ ├── connection/ # Connection manager, circuit breaker, health checks
|
||||
│ ├── errors/ # Typed errors and retry policies
|
||||
│ ├── observability/ # Logger, metrics (Prometheus), tracing (OpenTelemetry)
|
||||
│ └── plugins/ # Plugin manager and built-in plugins
|
||||
└── domain/
|
||||
├── documents/ # DocumentManager, sessions, snapshots
|
||||
├── query/ # QueryBuilder, AggregationBuilder
|
||||
├── bulk/ # BulkIndexer with backpressure
|
||||
├── kv/ # Distributed KV store
|
||||
├── transactions/ # ACID-like transaction support
|
||||
├── logging/ # Log destination with ILM
|
||||
└── schema/ # Schema management and migrations
|
||||
```
|
||||
|
||||
## 📊 Observability
|
||||
|
||||
### Prometheus Metrics
|
||||
|
||||
```typescript
|
||||
import { defaultMetricsCollector } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
// Export metrics in Prometheus format
|
||||
const metrics = defaultMetricsCollector.export();
|
||||
|
||||
// Available metrics:
|
||||
// - elasticsearch_requests_total{operation, index, status}
|
||||
// - elasticsearch_request_duration_seconds{operation, index}
|
||||
// - elasticsearch_errors_total{operation, index, error_type}
|
||||
// - elasticsearch_circuit_breaker_state{state}
|
||||
// - elasticsearch_connection_pool_size
|
||||
// - elasticsearch_bulk_operations_total{type, status}
|
||||
```
|
||||
|
||||
### Distributed Tracing
|
||||
|
||||
```typescript
|
||||
import { defaultTracingProvider } from '@apiclient.xyz/elasticsearch';
|
||||
|
||||
// OpenTelemetry-compatible tracing
|
||||
const span = defaultTracingProvider.startSpan('custom-operation');
|
||||
span.setAttributes({ 'custom.attribute': 'value' });
|
||||
// ... operation
|
||||
span.end();
|
||||
```
|
||||
|
||||
## 🔒 Authentication Methods
|
||||
|
||||
```typescript
|
||||
// Basic auth
|
||||
createConfig().basicAuth('username', 'password')
|
||||
|
||||
// API key
|
||||
createConfig().apiKeyAuth('api-key-value')
|
||||
|
||||
// Bearer token
|
||||
createConfig().bearerAuth('jwt-token')
|
||||
|
||||
// Elastic Cloud
|
||||
createConfig().cloudAuth('cloud-id', { apiKey: 'key' })
|
||||
|
||||
// Certificate
|
||||
createConfig().auth({
|
||||
type: 'certificate',
|
||||
certPath: '/path/to/cert.pem',
|
||||
keyPath: '/path/to/key.pem',
|
||||
})
|
||||
```
|
||||
|
||||
## ⚡ Performance Tips
|
||||
|
||||
1. **Use bulk operations** for batch inserts — `BulkIndexer` handles batching, retries, and backpressure
|
||||
2. **Enable compression** for large payloads — reduces network overhead
|
||||
3. **Configure connection pooling** — `poolSize(max, min)` for optimal concurrency
|
||||
4. **Leverage caching** — `KVStore` has built-in LRU/LFU caching
|
||||
5. **Use sessions** for document sync — automatic cleanup of stale documents
|
||||
6. **Enable circuit breaker** — prevents cascade failures during outages
|
||||
|
||||
## 🔄 Compatibility
|
||||
|
||||
- **Elasticsearch**: 8.x, 9.x
|
||||
- **Kibana**: 8.x, 9.x (for log visualization)
|
||||
- **Node.js**: 18+
|
||||
- **TypeScript**: 5.0+
|
||||
|
||||
## License and Legal Information
|
||||
|
||||
|
||||
Reference in New Issue
Block a user