feat(stocks/CoinGeckoProvider): Add CoinGecko provider for cryptocurrency prices, export and tests, and update documentation

This commit is contained in:
2025-11-02 16:34:24 +00:00
parent fdea1bb149
commit 47fd770e48
7 changed files with 1137 additions and 5 deletions

View File

@@ -1,5 +1,14 @@
# Changelog # Changelog
## 2025-11-02 - 3.3.0 - feat(stocks/CoinGeckoProvider)
Add CoinGecko provider for cryptocurrency prices, export and tests, and update documentation
- Implemented CoinGeckoProvider with rate limiting, coin-id resolution, and support for current, batch, historical and intraday price endpoints
- Added unit/integration tests for CoinGecko: test/test.coingecko.node+bun+deno.ts
- Exported CoinGeckoProvider from ts/stocks/index.ts
- Updated README and readme.hints.md with CoinGecko usage, provider notes and examples
- Added .claude/settings.local.json with webfetch and bash permissions required for testing and CI
## 2025-11-01 - 3.2.2 - fix(handelsregister) ## 2025-11-01 - 3.2.2 - fix(handelsregister)
Correct screenshot path handling in HandelsRegister and add local tool permissions Correct screenshot path handling in HandelsRegister and add local tool permissions

View File

@@ -3,7 +3,7 @@
## Stocks Module ## Stocks Module
### Overview ### Overview
The stocks module provides real-time stock price data through various provider implementations. Currently supports Yahoo Finance with an extensible architecture for additional providers. The stocks module provides real-time stock price data and cryptocurrency prices through various provider implementations. Currently supports Yahoo Finance, Marketstack, and CoinGecko with an extensible architecture for additional providers.
### Architecture ### Architecture
- **Provider Pattern**: Each stock data source implements the `IStockProvider` interface - **Provider Pattern**: Each stock data source implements the `IStockProvider` interface
@@ -31,11 +31,55 @@ const price = await stockService.getPrice({ ticker: 'AAPL' });
console.log(`${price.ticker}: $${price.price}`); console.log(`${price.ticker}: $${price.price}`);
``` ```
### CoinGecko Provider Notes
- Cryptocurrency price provider supporting 13M+ tokens
- Three main endpoints:
- `/simple/price` - Current prices with market data (batch supported)
- `/coins/{id}/market_chart` - Historical and intraday prices with OHLCV
- `/coins/list` - Complete coin list for ticker-to-ID mapping
- **Rate Limiting**:
- Free tier: 5-15 calls/minute (no registration)
- Demo plan: 30 calls/minute, 10,000/month (free with registration)
- Custom rate limiter tracks requests per minute
- **Ticker Resolution**:
- Accepts both ticker symbols (BTC, ETH) and CoinGecko IDs (bitcoin, ethereum)
- Lazy-loads coin list on first ticker resolution
- Caches coin mappings for 24 hours
- IDs with hyphens (wrapped-bitcoin) assumed to be CoinGecko IDs
- **24/7 Markets**: Crypto markets always return `marketState: 'REGULAR'`
- **Optional API Key**: Pass key to constructor for higher rate limits
- Demo plan: `x-cg-demo-api-key` header
- Paid plans: `x-cg-pro-api-key` header
- **Data Granularity**:
- Historical: Daily data for date ranges
- Intraday: Hourly data only (1-90 days based on `days` param)
- Current: Real-time prices with 24h change and volume
### Usage Example (Crypto)
```typescript
import { StockPriceService, CoinGeckoProvider } from '@fin.cx/opendata';
const stockService = new StockPriceService({ ttl: 30000 });
const coingeckoProvider = new CoinGeckoProvider(); // or new CoinGeckoProvider('api-key')
stockService.register(coingeckoProvider);
// Using ticker symbol
const btc = await stockService.getPrice({ ticker: 'BTC' });
console.log(`${btc.ticker}: $${btc.price}`);
// Using CoinGecko ID
const ethereum = await stockService.getPrice({ ticker: 'ethereum' });
// Batch fetch
const cryptos = await stockService.getPrices({ tickers: ['BTC', 'ETH', 'USDT'] });
```
### Testing ### Testing
- Tests use real API calls (be mindful of rate limits) - Tests use real API calls (be mindful of rate limits)
- Mock invalid ticker 'INVALID_TICKER_XYZ' for error testing - Mock invalid ticker 'INVALID_TICKER_XYZ' for error testing
- Clear cache between tests to ensure fresh data - Clear cache between tests to ensure fresh data
- The spark endpoint may return fewer results than requested - The spark endpoint may return fewer results than requested
- CoinGecko tests may take longer due to rate limiting (wait between requests)
### Future Providers ### Future Providers
To add a new provider: To add a new provider:

View File

@@ -87,6 +87,56 @@ const apple = await stockService.getData({ type: 'current', ticker: 'AAPL' });
console.log(`${apple.companyFullName}: $${apple.price}`); console.log(`${apple.companyFullName}: $${apple.price}`);
``` ```
### 🪙 Cryptocurrency Prices
Get real-time crypto prices with the CoinGecko provider:
```typescript
import { StockPriceService, CoinGeckoProvider } from '@fin.cx/opendata';
const stockService = new StockPriceService({ ttl: 30000 });
// Optional: Pass API key for higher rate limits
// const provider = new CoinGeckoProvider('your-api-key');
const provider = new CoinGeckoProvider();
stockService.register(provider);
// Fetch single crypto price (using ticker symbol)
const btc = await stockService.getPrice({ ticker: 'BTC' });
console.log(`${btc.ticker}: $${btc.price.toLocaleString()}`);
console.log(`24h Change: ${btc.changePercent.toFixed(2)}%`);
console.log(`24h Volume: $${btc.volume?.toLocaleString()}`);
// Or use CoinGecko ID directly
const ethereum = await stockService.getPrice({ ticker: 'ethereum' });
// Batch fetch multiple cryptos
const cryptos = await stockService.getPrices({
tickers: ['BTC', 'ETH', 'USDT', 'BNB', 'SOL']
});
cryptos.forEach(crypto => {
console.log(`${crypto.ticker}: $${crypto.price.toFixed(2)} (${crypto.changePercent >= 0 ? '+' : ''}${crypto.changePercent.toFixed(2)}%)`);
});
// Historical crypto prices
const history = await stockService.getData({
type: 'historical',
ticker: 'BTC',
from: new Date('2025-01-01'),
to: new Date('2025-01-31')
});
// Intraday crypto prices (hourly)
const intraday = await stockService.getData({
type: 'intraday',
ticker: 'ETH',
interval: '1hour',
limit: 24 // Last 24 hours
});
```
### 💰 Fundamentals-Only Data (Alternative) ### 💰 Fundamentals-Only Data (Alternative)
If you only need fundamentals without prices: If you only need fundamentals without prices:
@@ -149,9 +199,10 @@ const details = await openData.handelsregister.getSpecificCompany({
## Features ## Features
### 📊 Stock Market Module ### 📊 Stock & Crypto Market Module
- **Real-Time Prices** - Live and EOD stock prices from Yahoo Finance and Marketstack - **Real-Time Prices** - Live and EOD prices from Yahoo Finance, Marketstack, and CoinGecko
- **Cryptocurrency Support** - 13M+ crypto tokens with 24/7 market data via CoinGecko
- **Company Names** - Automatic company name extraction (e.g., "Apple Inc (NASDAQ:AAPL)") - **Company Names** - Automatic company name extraction (e.g., "Apple Inc (NASDAQ:AAPL)")
- **Historical Data** - Up to 15 years of daily EOD prices with pagination - **Historical Data** - Up to 15 years of daily EOD prices with pagination
- **OHLCV Data** - Open, High, Low, Close, Volume for technical analysis - **OHLCV Data** - Open, High, Low, Close, Volume for technical analysis
@@ -622,6 +673,15 @@ interface IStockFundamentals {
- ✅ Company names included - ✅ Company names included
- ⚠️ Rate limits may apply - ⚠️ Rate limits may apply
**CoinGeckoProvider**
- ✅ Cryptocurrency prices (Bitcoin, Ethereum, 13M+ tokens)
- ✅ Current, historical, and intraday data
- ✅ 24/7 market data (crypto never closes)
- ✅ OHLCV data with market cap and volume
- ✅ Supports both ticker symbols (BTC, ETH) and CoinGecko IDs (bitcoin, ethereum)
- ✅ 240+ networks, 1600+ exchanges
- Optional API key (free tier: 30 requests/min, 10K/month)
**OpenData** **OpenData**
- `start()` - Initialize MongoDB connection - `start()` - Initialize MongoDB connection
- `buildInitialDb()` - Import bulk data - `buildInitialDb()` - Import bulk data

View File

@@ -0,0 +1,261 @@
import { expect, tap } from '@git.zone/tstest/tapbundle';
import * as opendata from '../ts/index.js';
// Test data
const testCryptos = ['BTC', 'ETH', 'USDT'];
const testCryptoIds = ['bitcoin', 'ethereum', 'tether'];
const invalidCrypto = 'INVALID_CRYPTO_XYZ_12345';
let stockService: opendata.StockPriceService;
let coingeckoProvider: opendata.CoinGeckoProvider;
tap.test('should create StockPriceService instance', async () => {
stockService = new opendata.StockPriceService({
ttl: 30000, // 30 seconds cache
maxEntries: 100
});
expect(stockService).toBeInstanceOf(opendata.StockPriceService);
});
tap.test('should create CoinGeckoProvider instance without API key', async () => {
coingeckoProvider = new opendata.CoinGeckoProvider();
expect(coingeckoProvider).toBeInstanceOf(opendata.CoinGeckoProvider);
expect(coingeckoProvider.name).toEqual('CoinGecko');
expect(coingeckoProvider.requiresAuth).toEqual(false);
expect(coingeckoProvider.priority).toEqual(90);
});
tap.test('should register CoinGecko provider with the service', async () => {
stockService.register(coingeckoProvider);
const providers = stockService.getAllProviders();
expect(providers).toContainEqual(coingeckoProvider);
expect(stockService.getProvider('CoinGecko')).toEqual(coingeckoProvider);
});
tap.test('should check CoinGecko provider health', async () => {
const health = await stockService.checkProvidersHealth();
expect(health.get('CoinGecko')).toEqual(true);
});
tap.test('should fetch single crypto price using ticker symbol (BTC)', async () => {
const price = await stockService.getPrice({ ticker: 'BTC' });
expect(price).toHaveProperty('ticker');
expect(price).toHaveProperty('price');
expect(price).toHaveProperty('currency');
expect(price).toHaveProperty('change');
expect(price).toHaveProperty('changePercent');
expect(price).toHaveProperty('previousClose');
expect(price).toHaveProperty('timestamp');
expect(price).toHaveProperty('provider');
expect(price).toHaveProperty('marketState');
expect(price.ticker).toEqual('BTC');
expect(price.price).toBeGreaterThan(0);
expect(price.currency).toEqual('USD');
expect(price.provider).toEqual('CoinGecko');
expect(price.marketState).toEqual('REGULAR'); // Crypto is 24/7
expect(price.timestamp).toBeInstanceOf(Date);
expect(price.dataType).toEqual('live');
console.log(`\n📊 BTC Price: $${price.price.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 })}`);
console.log(` Change: ${price.changePercent >= 0 ? '+' : ''}${price.changePercent.toFixed(2)}%`);
});
tap.test('should fetch single crypto price using CoinGecko ID (bitcoin)', async () => {
// Clear cache to ensure fresh fetch
stockService.clearCache();
const price = await stockService.getPrice({ ticker: 'bitcoin' });
expect(price.ticker).toEqual('BITCOIN');
expect(price.price).toBeGreaterThan(0);
expect(price.provider).toEqual('CoinGecko');
expect(price.companyName).toInclude('Bitcoin');
});
tap.test('should fetch multiple crypto prices (batch)', async () => {
stockService.clearCache();
const prices = await stockService.getPrices({
tickers: testCryptos
});
expect(prices).toBeArray();
expect(prices.length).toEqual(testCryptos.length);
for (const price of prices) {
expect(testCryptos).toContain(price.ticker);
expect(price.price).toBeGreaterThan(0);
expect(price.provider).toEqual('CoinGecko');
expect(price.marketState).toEqual('REGULAR');
console.log(`\n💰 ${price.ticker}: $${price.price.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 8 })}`);
console.log(` Change 24h: ${price.changePercent >= 0 ? '+' : ''}${price.changePercent.toFixed(2)}%`);
if (price.volume) {
console.log(` Volume 24h: $${price.volume.toLocaleString('en-US')}`);
}
}
});
tap.test('should fetch historical crypto prices', async () => {
// Add delay to avoid rate limiting
await new Promise(resolve => setTimeout(resolve, 3000));
const to = new Date();
const from = new Date(to.getTime() - 7 * 24 * 60 * 60 * 1000); // 7 days ago
const prices = await stockService.getData({
type: 'historical',
ticker: 'BTC',
from: from,
to: to
});
expect(prices).toBeArray();
expect((prices as opendata.IStockPrice[]).length).toBeGreaterThan(0);
const pricesArray = prices as opendata.IStockPrice[];
console.log(`\n📈 Historical BTC Prices (${pricesArray.length} days):`);
// Show first few and last few
const toShow = Math.min(3, pricesArray.length);
for (let i = 0; i < toShow; i++) {
const price = pricesArray[i];
const date = price.timestamp.toISOString().split('T')[0];
console.log(` ${date}: $${price.price.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 })}`);
}
if (pricesArray.length > toShow * 2) {
console.log(' ...');
}
for (let i = Math.max(toShow, pricesArray.length - toShow); i < pricesArray.length; i++) {
const price = pricesArray[i];
const date = price.timestamp.toISOString().split('T')[0];
console.log(` ${date}: $${price.price.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 })}`);
}
// Validate first entry
const firstPrice = pricesArray[0];
expect(firstPrice.ticker).toEqual('BTC');
expect(firstPrice.dataType).toEqual('eod');
expect(firstPrice.provider).toEqual('CoinGecko');
});
tap.test('should fetch intraday crypto prices (hourly)', async () => {
// Add delay to avoid rate limiting
await new Promise(resolve => setTimeout(resolve, 3000));
const prices = await stockService.getData({
type: 'intraday',
ticker: 'ETH',
interval: '1hour',
limit: 12 // Last 12 hours
});
expect(prices).toBeArray();
expect((prices as opendata.IStockPrice[]).length).toBeGreaterThan(0);
const pricesArray = prices as opendata.IStockPrice[];
console.log(`\n⏰ Intraday ETH Prices (hourly, last ${pricesArray.length} hours):`);
// Show first few entries
const toShow = Math.min(5, pricesArray.length);
for (let i = 0; i < toShow; i++) {
const price = pricesArray[i];
const time = price.timestamp.toISOString().replace('T', ' ').substring(0, 16);
console.log(` ${time}: $${price.price.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 })}`);
}
// Validate first entry
const firstPrice = pricesArray[0];
expect(firstPrice.ticker).toEqual('ETH');
expect(firstPrice.dataType).toEqual('intraday');
expect(firstPrice.provider).toEqual('CoinGecko');
});
tap.test('should serve cached prices on subsequent requests', async () => {
// First request - should hit the API
const firstRequest = await stockService.getPrice({ ticker: 'BTC' });
// Second request - should be served from cache
const secondRequest = await stockService.getPrice({ ticker: 'BTC' });
expect(secondRequest.ticker).toEqual(firstRequest.ticker);
expect(secondRequest.price).toEqual(firstRequest.price);
expect(secondRequest.timestamp).toEqual(firstRequest.timestamp);
expect(secondRequest.fetchedAt).toEqual(firstRequest.fetchedAt);
});
tap.test('should handle invalid crypto ticker gracefully', async () => {
try {
await stockService.getPrice({ ticker: invalidCrypto });
throw new Error('Should have thrown an error for invalid ticker');
} catch (error) {
expect(error.message).toInclude('Failed to fetch');
}
});
tap.test('should support market checking', async () => {
expect(coingeckoProvider.supportsMarket('CRYPTO')).toEqual(true);
expect(coingeckoProvider.supportsMarket('BTC')).toEqual(true);
expect(coingeckoProvider.supportsMarket('ETH')).toEqual(true);
expect(coingeckoProvider.supportsMarket('NASDAQ')).toEqual(false);
});
tap.test('should support ticker validation', async () => {
expect(coingeckoProvider.supportsTicker('BTC')).toEqual(true);
expect(coingeckoProvider.supportsTicker('bitcoin')).toEqual(true);
expect(coingeckoProvider.supportsTicker('wrapped-bitcoin')).toEqual(true);
expect(coingeckoProvider.supportsTicker('BTC!')).toEqual(false);
expect(coingeckoProvider.supportsTicker('BTC@USD')).toEqual(false);
});
tap.test('should display provider statistics', async () => {
const stats = stockService.getProviderStats();
const coingeckoStats = stats.get('CoinGecko');
expect(coingeckoStats).toBeTruthy();
expect(coingeckoStats.successCount).toBeGreaterThan(0);
console.log('\n📊 CoinGecko Provider Statistics:');
console.log(` Success Count: ${coingeckoStats.successCount}`);
console.log(` Error Count: ${coingeckoStats.errorCount}`);
if (coingeckoStats.lastError) {
console.log(` Last Error: ${coingeckoStats.lastError}`);
}
});
tap.test('should display crypto price dashboard', async () => {
// Add delay to avoid rate limiting
await new Promise(resolve => setTimeout(resolve, 3000));
stockService.clearCache();
const cryptos = ['BTC', 'ETH', 'BNB', 'SOL', 'ADA'];
const prices = await stockService.getPrices({ tickers: cryptos });
console.log('\n╔═══════════════════════════════════════════════════════════╗');
console.log('║ 🌐 CRYPTOCURRENCY PRICE DASHBOARD ║');
console.log('╠═══════════════════════════════════════════════════════════╣');
for (const price of prices) {
const priceStr = `$${price.price.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 8 })}`;
const changeStr = `${price.changePercent >= 0 ? '+' : ''}${price.changePercent.toFixed(2)}%`;
const changeIcon = price.changePercent >= 0 ? '📈' : '📉';
console.log(`${price.ticker.padEnd(6)} ${changeIcon} ${priceStr.padStart(20)}${changeStr.padStart(10)}`);
}
console.log('╚═══════════════════════════════════════════════════════════╝');
console.log(`Provider: ${prices[0].provider} | Market State: ${prices[0].marketState} (24/7)`);
console.log(`Fetched at: ${prices[0].fetchedAt.toISOString()}`);
});
tap.test('should clear cache', async () => {
stockService.clearCache();
// Cache is cleared, no assertions needed
});
export default tap.start();

View File

@@ -3,6 +3,6 @@
*/ */
export const commitinfo = { export const commitinfo = {
name: '@fin.cx/opendata', name: '@fin.cx/opendata',
version: '3.2.2', version: '3.3.0',
description: 'A comprehensive TypeScript library for accessing business data and real-time financial information. Features include German company data management with MongoDB integration, JSONL bulk processing, automated Handelsregister interactions, and real-time stock market data from multiple providers.' description: 'A comprehensive TypeScript library for accessing business data and real-time financial information. Features include German company data management with MongoDB integration, JSONL bulk processing, automated Handelsregister interactions, and real-time stock market data from multiple providers.'
} }

View File

@@ -15,4 +15,5 @@ export * from './classes.baseproviderservice.js';
// Export providers // Export providers
export * from './providers/provider.yahoo.js'; export * from './providers/provider.yahoo.js';
export * from './providers/provider.marketstack.js'; export * from './providers/provider.marketstack.js';
export * from './providers/provider.secedgar.js'; export * from './providers/provider.secedgar.js';
export * from './providers/provider.coingecko.js';

View File

@@ -0,0 +1,757 @@
import * as plugins from '../../plugins.js';
import type { IStockProvider, IProviderConfig } from '../interfaces/provider.js';
import type {
IStockPrice,
IStockDataRequest,
IStockCurrentRequest,
IStockHistoricalRequest,
IStockIntradayRequest,
IStockBatchCurrentRequest
} from '../interfaces/stockprice.js';
/**
* Custom error for rate limit exceeded responses
*/
class RateLimitError extends Error {
constructor(
message: string,
public waitTime: number,
public retryAfter?: number
) {
super(message);
this.name = 'RateLimitError';
}
}
/**
* Rate limiter for CoinGecko API
* Free tier (Demo): 30 requests per minute
* Without registration: 5-15 requests per minute
*/
class RateLimiter {
private requestTimes: number[] = [];
private maxRequestsPerMinute: number;
private consecutiveRateLimitErrors: number = 0;
constructor(maxRequestsPerMinute: number = 30) {
this.maxRequestsPerMinute = maxRequestsPerMinute;
}
public async waitForSlot(): Promise<void> {
const now = Date.now();
const oneMinuteAgo = now - 60000;
// Remove requests older than 1 minute
this.requestTimes = this.requestTimes.filter(time => time > oneMinuteAgo);
// If we've hit the limit, wait
if (this.requestTimes.length >= this.maxRequestsPerMinute) {
const oldestRequest = this.requestTimes[0];
const waitTime = 60000 - (now - oldestRequest) + 100; // +100ms buffer
await plugins.smartdelay.delayFor(waitTime);
return this.waitForSlot(); // Recursively check again
}
// Record this request
this.requestTimes.push(now);
}
/**
* Get time in milliseconds until next request slot is available
*/
public getTimeUntilNextSlot(): number {
const now = Date.now();
const oneMinuteAgo = now - 60000;
// Clean old requests
const recentRequests = this.requestTimes.filter(time => time > oneMinuteAgo);
if (recentRequests.length < this.maxRequestsPerMinute) {
return 0; // Slot available now
}
// Calculate wait time until oldest request expires
const oldestRequest = recentRequests[0];
return Math.max(0, 60000 - (now - oldestRequest) + 100);
}
/**
* Handle rate limit error with exponential backoff
* Returns wait time in milliseconds
*/
public handleRateLimitError(): number {
this.consecutiveRateLimitErrors++;
// Exponential backoff: 1s, 2s, 4s, 8s, 16s, 32s, 60s (max)
const baseWait = 1000; // 1 second
const exponent = this.consecutiveRateLimitErrors - 1;
const backoff = Math.min(
baseWait * Math.pow(2, exponent),
60000 // max 60 seconds
);
// After 3 consecutive 429s, reduce rate limit to 80% as safety measure
if (this.consecutiveRateLimitErrors >= 3) {
const newLimit = Math.floor(this.maxRequestsPerMinute * 0.8);
if (newLimit < this.maxRequestsPerMinute) {
console.warn(
`Adjusting rate limit from ${this.maxRequestsPerMinute} to ${newLimit} requests/min due to repeated 429 errors`
);
this.maxRequestsPerMinute = newLimit;
}
}
return backoff;
}
/**
* Reset consecutive error count on successful request
*/
public resetErrors(): void {
if (this.consecutiveRateLimitErrors > 0) {
this.consecutiveRateLimitErrors = 0;
}
}
}
/**
* Interface for coin list response
*/
interface ICoinListItem {
id: string;
symbol: string;
name: string;
}
/**
* CoinGecko Crypto Price Provider
*
* Documentation: https://docs.coingecko.com/v3.0.1/reference/endpoint-overview
*
* Features:
* - Current crypto prices (single and batch)
* - Historical price data with OHLCV
* - 13M+ tokens, 240+ networks, 1600+ exchanges
* - Accepts both ticker symbols (BTC, ETH) and CoinGecko IDs (bitcoin, ethereum)
* - 24/7 market data (crypto never closes)
*
* Rate Limits:
* - Free tier (no key): 5-15 requests/minute
* - Demo plan (free with registration): ~30 requests/minute, 10,000/month
* - Paid plans: Higher limits
*
* API Authentication:
* - Optional API key for Demo/paid plans
* - Header: x-cg-demo-api-key (Demo) or x-cg-pro-api-key (paid)
*/
export class CoinGeckoProvider implements IStockProvider {
public name = 'CoinGecko';
public priority = 90; // High priority for crypto, between Yahoo (100) and Marketstack (80)
public readonly requiresAuth = false; // API key is optional
public readonly rateLimit = {
requestsPerMinute: 30, // Demo plan default
requestsPerDay: 10000 // Demo plan monthly quota / 30
};
private logger = console;
private baseUrl = 'https://api.coingecko.com/api/v3';
private apiKey?: string;
private rateLimiter: RateLimiter;
// Coin mapping cache
private coinMapCache = new Map<string, string>(); // ticker/id -> coingecko id
private coinListLoadedAt: Date | null = null;
private readonly coinListCacheTTL = 24 * 60 * 60 * 1000; // 24 hours
// Priority map for common crypto tickers (to avoid conflicts)
private readonly priorityTickerMap = new Map<string, string>([
['btc', 'bitcoin'],
['eth', 'ethereum'],
['usdt', 'tether'],
['bnb', 'binancecoin'],
['sol', 'solana'],
['usdc', 'usd-coin'],
['xrp', 'ripple'],
['ada', 'cardano'],
['doge', 'dogecoin'],
['trx', 'tron'],
['dot', 'polkadot'],
['matic', 'matic-network'],
['ltc', 'litecoin'],
['shib', 'shiba-inu'],
['avax', 'avalanche-2'],
['link', 'chainlink'],
['atom', 'cosmos'],
['uni', 'uniswap'],
['etc', 'ethereum-classic'],
['xlm', 'stellar']
]);
constructor(apiKey?: string, private config?: IProviderConfig) {
this.apiKey = apiKey;
this.rateLimiter = new RateLimiter(this.rateLimit.requestsPerMinute);
}
/**
* Unified data fetching method supporting all request types
*/
public async fetchData(request: IStockDataRequest): Promise<IStockPrice | IStockPrice[]> {
switch (request.type) {
case 'current':
return this.fetchCurrentPrice(request);
case 'batch':
return this.fetchBatchCurrentPrices(request);
case 'historical':
return this.fetchHistoricalPrices(request);
case 'intraday':
return this.fetchIntradayPrices(request);
default:
throw new Error(`Unsupported request type: ${(request as any).type}`);
}
}
/**
* Fetch current price for a single crypto
*/
private async fetchCurrentPrice(request: IStockCurrentRequest): Promise<IStockPrice> {
return this.fetchWithRateLimitRetry(async () => {
// Resolve ticker to CoinGecko ID
const coinId = await this.resolveCoinId(request.ticker);
// Build URL
const params = new URLSearchParams({
ids: coinId,
vs_currencies: 'usd',
include_market_cap: 'true',
include_24hr_vol: 'true',
include_24hr_change: 'true',
include_last_updated_at: 'true'
});
const url = `${this.baseUrl}/simple/price?${params}`;
// Wait for rate limit slot
await this.rateLimiter.waitForSlot();
// Make request
const response = await plugins.smartrequest.SmartRequest.create()
.url(url)
.headers(this.buildHeaders())
.timeout(this.config?.timeout || 10000)
.get();
const responseData = await response.json() as any;
// Check for rate limit error
if (this.isRateLimitError(responseData)) {
const waitTime = this.rateLimiter.handleRateLimitError();
throw new RateLimitError(
`Rate limit exceeded for ${request.ticker}`,
waitTime
);
}
if (!responseData[coinId]) {
throw new Error(`No data found for ${request.ticker} (${coinId})`);
}
return this.mapToStockPrice(request.ticker, coinId, responseData[coinId], 'live');
}, `current price for ${request.ticker}`);
}
/**
* Fetch batch current prices for multiple cryptos
*/
private async fetchBatchCurrentPrices(request: IStockBatchCurrentRequest): Promise<IStockPrice[]> {
return this.fetchWithRateLimitRetry(async () => {
// Resolve all tickers to CoinGecko IDs
const coinIds = await Promise.all(
request.tickers.map(ticker => this.resolveCoinId(ticker))
);
// Build URL with comma-separated IDs
const params = new URLSearchParams({
ids: coinIds.join(','),
vs_currencies: 'usd',
include_market_cap: 'true',
include_24hr_vol: 'true',
include_24hr_change: 'true',
include_last_updated_at: 'true'
});
const url = `${this.baseUrl}/simple/price?${params}`;
// Wait for rate limit slot
await this.rateLimiter.waitForSlot();
// Make request
const response = await plugins.smartrequest.SmartRequest.create()
.url(url)
.headers(this.buildHeaders())
.timeout(this.config?.timeout || 15000)
.get();
const responseData = await response.json() as any;
// Check for rate limit error
if (this.isRateLimitError(responseData)) {
const waitTime = this.rateLimiter.handleRateLimitError();
throw new RateLimitError(
`Rate limit exceeded for batch request`,
waitTime
);
}
const prices: IStockPrice[] = [];
// Map responses back to original tickers
for (let i = 0; i < request.tickers.length; i++) {
const ticker = request.tickers[i];
const coinId = coinIds[i];
if (responseData[coinId]) {
try {
prices.push(this.mapToStockPrice(ticker, coinId, responseData[coinId], 'live'));
} catch (error) {
this.logger.warn(`Failed to parse data for ${ticker}:`, error);
}
} else {
this.logger.warn(`No data returned for ${ticker} (${coinId})`);
}
}
if (prices.length === 0) {
throw new Error('No valid price data received from batch request');
}
return prices;
}, `batch prices for ${request.tickers.length} tickers`);
}
/**
* Fetch historical prices with OHLCV data
*/
private async fetchHistoricalPrices(request: IStockHistoricalRequest): Promise<IStockPrice[]> {
return this.fetchWithRateLimitRetry(async () => {
const coinId = await this.resolveCoinId(request.ticker);
// Calculate days between dates
const days = Math.ceil((request.to.getTime() - request.from.getTime()) / (1000 * 60 * 60 * 24));
// Build URL
const params = new URLSearchParams({
vs_currency: 'usd',
days: days.toString(),
interval: 'daily' // Explicit daily granularity for historical data
});
const url = `${this.baseUrl}/coins/${coinId}/market_chart?${params}`;
// Wait for rate limit slot
await this.rateLimiter.waitForSlot();
// Make request
const response = await plugins.smartrequest.SmartRequest.create()
.url(url)
.headers(this.buildHeaders())
.timeout(this.config?.timeout || 20000)
.get();
const responseData = await response.json() as any;
// Check for rate limit error
if (this.isRateLimitError(responseData)) {
const waitTime = this.rateLimiter.handleRateLimitError();
throw new RateLimitError(
`Rate limit exceeded for historical ${request.ticker}`,
waitTime
);
}
if (!responseData.prices || !Array.isArray(responseData.prices)) {
this.logger.error(`Invalid API response for ${request.ticker}:`, JSON.stringify(responseData).substring(0, 500));
throw new Error(`Invalid response format for ${request.ticker}: ${JSON.stringify(responseData).substring(0, 200)}`);
}
const prices: IStockPrice[] = [];
const priceData = responseData.prices;
const marketCapData = responseData.market_caps || [];
const volumeData = responseData.total_volumes || [];
// Process each data point
for (let i = 0; i < priceData.length; i++) {
const [timestamp, price] = priceData[i];
const date = new Date(timestamp);
// Filter by date range
if (date < request.from || date > request.to) continue;
const marketCap = marketCapData[i]?.[1];
const volume = volumeData[i]?.[1];
// Calculate previous close for change calculation
const previousClose = i > 0 ? priceData[i - 1][1] : price;
const change = price - previousClose;
const changePercent = previousClose !== 0 ? (change / previousClose) * 100 : 0;
prices.push({
ticker: request.ticker.toUpperCase(),
price: price,
currency: 'USD',
change: change,
changePercent: changePercent,
previousClose: previousClose,
timestamp: date,
provider: this.name,
marketState: 'REGULAR', // Crypto markets are always open
// OHLCV data (note: market_chart doesn't provide OHLC, only close prices)
volume: volume,
dataType: 'eod',
fetchedAt: new Date(),
companyName: coinId.charAt(0).toUpperCase() + coinId.slice(1)
});
}
return prices;
}, `historical prices for ${request.ticker}`);
}
/**
* Fetch intraday prices with hourly intervals
*/
private async fetchIntradayPrices(request: IStockIntradayRequest): Promise<IStockPrice[]> {
return this.fetchWithRateLimitRetry(async () => {
const coinId = await this.resolveCoinId(request.ticker);
// Map interval to days parameter (CoinGecko auto-granularity)
// For hourly data, request 1-7 days
let days = 1;
switch (request.interval) {
case '1min':
case '5min':
case '10min':
case '15min':
case '30min':
throw new Error('CoinGecko only supports hourly intervals in market_chart. Use interval: "1hour"');
case '1hour':
days = 1; // Last 24 hours with hourly granularity
break;
}
// Build URL (omit interval param for automatic granularity based on days)
const params = new URLSearchParams({
vs_currency: 'usd',
days: days.toString()
});
const url = `${this.baseUrl}/coins/${coinId}/market_chart?${params}`;
// Wait for rate limit slot
await this.rateLimiter.waitForSlot();
// Make request
const response = await plugins.smartrequest.SmartRequest.create()
.url(url)
.headers(this.buildHeaders())
.timeout(this.config?.timeout || 15000)
.get();
const responseData = await response.json() as any;
// Check for rate limit error
if (this.isRateLimitError(responseData)) {
const waitTime = this.rateLimiter.handleRateLimitError();
throw new RateLimitError(
`Rate limit exceeded for intraday ${request.ticker}`,
waitTime
);
}
if (!responseData.prices || !Array.isArray(responseData.prices)) {
this.logger.error(`Invalid API response for ${request.ticker}:`, JSON.stringify(responseData).substring(0, 500));
throw new Error(`Invalid response format for ${request.ticker}: ${JSON.stringify(responseData).substring(0, 200)}`);
}
const prices: IStockPrice[] = [];
const priceData = responseData.prices;
const marketCapData = responseData.market_caps || [];
const volumeData = responseData.total_volumes || [];
// Apply limit if specified
const limit = request.limit || priceData.length;
const dataToProcess = priceData.slice(-limit);
for (let i = 0; i < dataToProcess.length; i++) {
const actualIndex = priceData.length - limit + i;
const [timestamp, price] = dataToProcess[i];
const date = new Date(timestamp);
const marketCap = marketCapData[actualIndex]?.[1];
const volume = volumeData[actualIndex]?.[1];
const previousClose = i > 0 ? dataToProcess[i - 1][1] : price;
const change = price - previousClose;
const changePercent = previousClose !== 0 ? (change / previousClose) * 100 : 0;
prices.push({
ticker: request.ticker.toUpperCase(),
price: price,
currency: 'USD',
change: change,
changePercent: changePercent,
previousClose: previousClose,
timestamp: date,
provider: this.name,
marketState: 'REGULAR',
volume: volume,
dataType: 'intraday',
fetchedAt: new Date(),
companyName: coinId.charAt(0).toUpperCase() + coinId.slice(1)
});
}
return prices;
}, `intraday prices for ${request.ticker}`);
}
/**
* Check if CoinGecko API is available
*/
public async isAvailable(): Promise<boolean> {
try {
const url = `${this.baseUrl}/simple/price?ids=bitcoin&vs_currencies=usd`;
await this.rateLimiter.waitForSlot();
const response = await plugins.smartrequest.SmartRequest.create()
.url(url)
.headers(this.buildHeaders())
.timeout(5000)
.get();
const responseData = await response.json() as any;
return responseData.bitcoin?.usd !== undefined;
} catch (error) {
this.logger.warn('CoinGecko provider is not available:', error);
return false;
}
}
/**
* Check if a market/network is supported
* CoinGecko supports 240+ networks
*/
public supportsMarket(market: string): boolean {
// CoinGecko has extensive crypto network coverage
const supportedNetworks = [
'CRYPTO', 'BTC', 'ETH', 'BSC', 'POLYGON', 'AVALANCHE',
'SOLANA', 'ARBITRUM', 'OPTIMISM', 'BASE'
];
return supportedNetworks.includes(market.toUpperCase());
}
/**
* Check if a ticker format is supported
* Supports both ticker symbols (BTC) and CoinGecko IDs (bitcoin)
*/
public supportsTicker(ticker: string): boolean {
// Accept alphanumeric with hyphens (for coin IDs like 'wrapped-bitcoin')
return /^[A-Za-z0-9\-]{1,50}$/.test(ticker);
}
/**
* Resolve ticker symbol or CoinGecko ID to canonical CoinGecko ID
* Supports both formats: "BTC" -> "bitcoin", "bitcoin" -> "bitcoin"
*/
private async resolveCoinId(tickerOrId: string): Promise<string> {
const normalized = tickerOrId.toLowerCase();
// Check priority map first (for common cryptos)
if (this.priorityTickerMap.has(normalized)) {
const coinId = this.priorityTickerMap.get(normalized)!;
this.coinMapCache.set(normalized, coinId);
return coinId;
}
// Check cache
if (this.coinMapCache.has(normalized)) {
return this.coinMapCache.get(normalized)!;
}
// Check if it's already a valid CoinGecko ID (contains hyphens or is all lowercase with original case)
if (normalized.includes('-') || normalized === tickerOrId) {
// Assume it's a CoinGecko ID, cache it
this.coinMapCache.set(normalized, normalized);
return normalized;
}
// Load coin list if needed
if (!this.coinListLoadedAt ||
Date.now() - this.coinListLoadedAt.getTime() > this.coinListCacheTTL) {
await this.loadCoinList();
}
// Try to find in cache after loading
if (this.coinMapCache.has(normalized)) {
return this.coinMapCache.get(normalized)!;
}
// Not found - return as-is and let API handle the error
this.logger.warn(`Could not resolve ticker ${tickerOrId} to CoinGecko ID, using as-is`);
return normalized;
}
/**
* Load complete coin list from CoinGecko API
*/
private async loadCoinList(): Promise<void> {
try {
const url = `${this.baseUrl}/coins/list`;
await this.rateLimiter.waitForSlot();
const response = await plugins.smartrequest.SmartRequest.create()
.url(url)
.headers(this.buildHeaders())
.timeout(10000)
.get();
const coinList = await response.json() as ICoinListItem[];
// Build mapping: symbol -> id
for (const coin of coinList) {
const symbol = coin.symbol.toLowerCase();
const id = coin.id.toLowerCase();
// Don't overwrite priority mappings or existing cache entries
if (!this.priorityTickerMap.has(symbol) && !this.coinMapCache.has(symbol)) {
this.coinMapCache.set(symbol, id);
}
// Always cache the ID mapping
this.coinMapCache.set(id, id);
}
this.coinListLoadedAt = new Date();
this.logger.info(`Loaded ${coinList.length} coins from CoinGecko`);
} catch (error) {
this.logger.error('Failed to load coin list from CoinGecko:', error);
// Don't throw - we can still work with direct IDs
}
}
/**
* Map CoinGecko simple/price response to IStockPrice
*/
private mapToStockPrice(
ticker: string,
coinId: string,
data: any,
dataType: 'live' | 'eod' | 'intraday'
): IStockPrice {
const price = data.usd;
const change24h = data.usd_24h_change || 0;
// Calculate previous close from 24h change
const changePercent = change24h;
const change = (price * changePercent) / 100;
const previousClose = price - change;
// Parse last updated timestamp
const timestamp = data.last_updated_at
? new Date(data.last_updated_at * 1000)
: new Date();
return {
ticker: ticker.toUpperCase(),
price: price,
currency: 'USD',
change: change,
changePercent: changePercent,
previousClose: previousClose,
timestamp: timestamp,
provider: this.name,
marketState: 'REGULAR', // Crypto markets are 24/7
// Volume and market cap
volume: data.usd_24h_vol,
dataType: dataType,
fetchedAt: new Date(),
// Company identification (use coin name)
companyName: coinId.charAt(0).toUpperCase() + coinId.slice(1),
companyFullName: `${coinId.charAt(0).toUpperCase() + coinId.slice(1)} (${ticker.toUpperCase()})`
};
}
/**
* Build HTTP headers with optional API key
*/
private buildHeaders(): Record<string, string> {
const headers: Record<string, string> = {
'Accept': 'application/json'
};
if (this.apiKey) {
// Use Demo or Pro API key header
// CoinGecko accepts both x-cg-demo-api-key and x-cg-pro-api-key
headers['x-cg-demo-api-key'] = this.apiKey;
}
return headers;
}
/**
* Check if response indicates a rate limit error (429)
*/
private isRateLimitError(responseData: any): boolean {
return responseData?.status?.error_code === 429;
}
/**
* Wrapper for fetch operations with automatic rate limit retry and exponential backoff
*/
private async fetchWithRateLimitRetry<T>(
fetchFn: () => Promise<T>,
operationName: string,
maxRetries: number = 3
): Promise<T> {
let lastError: Error | undefined;
for (let attempt = 0; attempt < maxRetries; attempt++) {
try {
const result = await fetchFn();
this.rateLimiter.resetErrors();
return result;
} catch (error) {
lastError = error as Error;
if (error instanceof RateLimitError) {
const attemptInfo = `${attempt + 1}/${maxRetries}`;
this.logger.warn(
`Rate limit hit for ${operationName}, waiting ${error.waitTime}ms before retry ${attemptInfo}`
);
if (attempt < maxRetries - 1) {
await plugins.smartdelay.delayFor(error.waitTime);
continue;
} else {
this.logger.error(`Max retries (${maxRetries}) exceeded for ${operationName} due to rate limiting`);
throw error;
}
}
// Non-rate-limit errors: throw immediately
throw error;
}
}
throw lastError!;
}
}