feat(stocks): Add Marketstack provider (EOD) with tests, exports and documentation updates

2025-10-11 15:21:57 +00:00
parent 3b76de0831
commit 4716ef03ba
10 changed files with 1388 additions and 269 deletions
--- a/readme.md
+++ b/readme.md
@@ -1,4 +1,5 @@
 # @fin.cx/opendata
+
 🚀 **Real-time financial data and German business intelligence in one powerful TypeScript library**

 Access live stock prices, cryptocurrencies, forex, commodities AND comprehensive German company data - all through a single, unified API.
@@ -8,34 +9,42 @@ Access live stock prices, cryptocurrencies, forex, commodities AND comprehensive
 ```bash
 npm install @fin.cx/opendata
 # or
-yarn add @fin.cx/opendata
+pnpm add @fin.cx/opendata
 ```

 ## Quick Start

-### 📈 Real-Time Stock Data
+### 📈 Stock Market Data

-Get live market data in seconds:
+Get market data with EOD (End-of-Day) pricing:

 ```typescript
-import { StockPriceService, YahooFinanceProvider } from '@fin.cx/opendata';
+import { StockPriceService, MarketstackProvider } from '@fin.cx/opendata';

-// Initialize the service
-const stockService = new StockPriceService();
-stockService.register(new YahooFinanceProvider());
+// Initialize the service with caching
+const stockService = new StockPriceService({
+  ttl: 60000,      // Cache for 1 minute
+  maxEntries: 1000 // Max cached symbols
+});
+
+// Register Marketstack provider with API key
+stockService.register(new MarketstackProvider('YOUR_API_KEY'), {
+  priority: 100,
+  retryAttempts: 3
+});

 // Get single stock price
 const apple = await stockService.getPrice({ ticker: 'AAPL' });
 console.log(`Apple: $${apple.price} (${apple.changePercent.toFixed(2)}%)`);

-// Get multiple prices at once
-const prices = await stockService.getPrices({ 
-  tickers: ['AAPL', 'MSFT', 'GOOGL', 'BTC-USD', 'ETH-USD'] 
+// Get multiple prices at once (batch fetching)
+const prices = await stockService.getPrices({
+  tickers: ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'TSLA']
 });

-// Market indices, crypto, forex, commodities - all supported!
-const marketData = await stockService.getPrices({
-  tickers: ['^GSPC', '^DJI', 'BTC-USD', 'EURUSD=X', 'GC=F']
+// 125,000+ tickers across 72+ exchanges worldwide
+const internationalStocks = await stockService.getPrices({
+  tickers: ['AAPL', 'VOD.LON', 'SAP.DEX', 'TM', 'BABA']
 });
 ```

@@ -73,11 +82,12 @@ await openData.buildInitialDb();

 ### 🎯 Stock Market Module

- **Real-time prices** for stocks, ETFs, indices, crypto, forex, and commodities
+- **Marketstack API** - End-of-Day (EOD) data for 125,000+ tickers across 72+ exchanges
+- **Stock prices** for stocks, ETFs, indices, and more
 - **Batch operations** - fetch 100+ symbols in one request
 - **Smart caching** - configurable TTL, automatic invalidation
- **Provider system** - easily extensible for new data sources
- **Automatic retries** and fallback mechanisms
+- **Extensible provider system** - easily add new data sources
+- **Retry logic** - configurable retry attempts and delays
 - **Type-safe** - full TypeScript support with detailed interfaces

 ### 🇩🇪 German Business Intelligence
@@ -92,29 +102,24 @@ await openData.buildInitialDb();

 ### Market Dashboard

-Create a real-time market overview:
+Create an EOD market overview:

 ```typescript
 const indicators = [
  // Indices
  { ticker: '^GSPC', name: 'S&P 500' },
-  { ticker: '^IXIC', name: 'NASDAQ' },
-  
+  { ticker: '^DJI', name: 'DOW Jones' },
+
  // Tech Giants
  { ticker: 'AAPL', name: 'Apple' },
  { ticker: 'MSFT', name: 'Microsoft' },
-  
-  // Crypto
-  { ticker: 'BTC-USD', name: 'Bitcoin' },
-  { ticker: 'ETH-USD', name: 'Ethereum' },
-  
-  // Commodities
-  { ticker: 'GC=F', name: 'Gold' },
-  { ticker: 'CL=F', name: 'Oil' }
+  { ticker: 'GOOGL', name: 'Alphabet' },
+  { ticker: 'AMZN', name: 'Amazon' },
+  { ticker: 'TSLA', name: 'Tesla' }
 ];

-const prices = await stockService.getPrices({ 
-  tickers: indicators.map(i => i.ticker) 
+const prices = await stockService.getPrices({
+  tickers: indicators.map(i => i.ticker)
 });

 // Display with color-coded changes
@@ -122,7 +127,7 @@ prices.forEach(price => {
  const indicator = indicators.find(i => i.ticker === price.ticker);
  const arrow = price.change >= 0 ? '↑' : '↓';
  const color = price.change >= 0 ? '\x1b[32m' : '\x1b[31m';
-  
+
  console.log(
    `${indicator.name.padEnd(15)} ${price.price.toFixed(2).padStart(10)} ` +
    `${color}${arrow} ${price.changePercent.toFixed(2)}%\x1b[0m`
@@ -130,6 +135,25 @@ prices.forEach(price => {
 });
 ```

+### Provider Health and Statistics
+
+Monitor your provider health and track usage:
+
+```typescript
+// Check provider health
+const health = await stockService.checkProvidersHealth();
+console.log(`Marketstack: ${health.get('Marketstack') ? '✅' : '❌'}`);
+
+// Get provider statistics
+const stats = stockService.getProviderStats();
+const marketstackStats = stats.get('Marketstack');
+console.log('Marketstack Stats:', {
+  successCount: marketstackStats.successCount,
+  errorCount: marketstackStats.errorCount,
+  lastError: marketstackStats.lastError
+});
+```
+
 ### Handelsregister Integration

 Automate German company data retrieval:
@@ -169,15 +193,15 @@ for (const company of publicCompanies) {
  try {
    // Map company to ticker (custom logic needed)
    const ticker = mapCompanyToTicker(company.data.name);
-    
+
    if (ticker) {
      const stock = await stockService.getPrice({ ticker });
-      
+
      // Add financial metrics
      company.data.stockPrice = stock.price;
      company.data.marketCap = stock.price * getSharesOutstanding(ticker);
      company.data.priceChange = stock.changePercent;
-      
+
      await company.save();
    }
  } catch (error) {
@@ -196,8 +220,8 @@ const stockService = new StockPriceService({
  maxEntries: 1000    // Max cached symbols
 });

-// Provider configuration
-stockService.register(new YahooFinanceProvider(), {
+// Marketstack - EOD data, requires API key
+stockService.register(new MarketstackProvider('YOUR_API_KEY'), {
  enabled: true,
  priority: 100,
  timeout: 10000,
@@ -208,7 +232,7 @@ stockService.register(new YahooFinanceProvider(), {

 ### MongoDB Setup

-Set environment variables:
+Set environment variables for German business data:

 ```env
 MONGODB_URL=mongodb://localhost:27017
@@ -217,6 +241,14 @@ MONGODB_USER=myuser
 MONGODB_PASS=mypass
 ```

+### Marketstack API Key
+
+Get your free API key at [marketstack.com](https://marketstack.com) and set it in your environment:
+
+```env
+MARKETSTACK_COM_TOKEN=your_api_key_here
+```
+
 ## API Reference

 ### Stock Types
@@ -240,10 +272,21 @@ interface IStockPrice {
 ### Key Methods

 **StockPriceService**
- `getPrice(request)` - Single stock price
- `getPrices(request)` - Batch prices
- `register(provider)` - Add data provider
+- `getPrice(request)` - Single stock price with automatic provider selection
+- `getPrices(request)` - Batch prices (100+ symbols in one request)
+- `register(provider, config)` - Add data provider with priority and retry config
+- `checkProvidersHealth()` - Test all providers and return health status
+- `getProviderStats()` - Get success/error statistics for each provider
 - `clearCache()` - Clear price cache
+- `setCacheTTL(ttl)` - Update cache TTL dynamically
+
+**MarketstackProvider**
+- ✅ End-of-Day (EOD) data
+- ✅ 125,000+ tickers across 72+ exchanges worldwide
+- ✅ Batch fetching support (multiple symbols in one request)
+- ✅ Comprehensive data: open, high, low, close, volume, splits, dividends
+- ⚠️ Requires API key (free tier: 100 requests/month)
+- ⚠️ EOD data only (not real-time)

 **OpenData**
 - `start()` - Initialize MongoDB connection
@@ -251,31 +294,48 @@ interface IStockPrice {
 - `CBusinessRecord` - Business record class
 - `handelsregister` - Registry automation

-## Performance
+## Provider Architecture

- **Batch fetching**: Get 100+ prices in <500ms
- **Caching**: Instant repeated queries
- **Concurrent processing**: Handle 1000+ records/second
- **Streaming**: Process GB-sized datasets without memory issues
-
-## Extensibility
-
-The provider architecture makes it easy to add new data sources:
+The library uses a flexible provider system that makes it easy to add new data sources:

 ```typescript
 class MyCustomProvider implements IStockProvider {
  name = 'My Provider';
-  
+  priority = 50;
+  requiresAuth = true;
+
  async fetchPrice(request: IStockQuoteRequest): Promise<IStockPrice> {
    // Your implementation
  }
-  
-  // ... other required methods
+
+  async fetchPrices(request: IStockBatchQuoteRequest): Promise<IStockPrice[]> {
+    // Batch implementation
+  }
+
+  async isAvailable(): Promise<boolean> {
+    // Health check
+  }
+
+  supportsMarket(market: string): boolean {
+    // Market validation
+  }
+
+  supportsTicker(ticker: string): boolean {
+    // Ticker validation
+  }
 }

 stockService.register(new MyCustomProvider());
 ```

+## Performance
+
+- **Batch fetching**: Get 100+ EOD prices in one API request
+- **Smart caching**: Instant repeated queries with configurable TTL
+- **Rate limit aware**: Automatic retry logic for API limits
+- **Concurrent processing**: Handle 1000+ business records/second
+- **Streaming**: Process GB-sized datasets without memory issues
+
 ## Testing

 Run the comprehensive test suite:
@@ -284,19 +344,21 @@ Run the comprehensive test suite:
 npm test
 ```

-View live market data:
+Test stock provider:

 ```bash
-npm test -- --grep "market indicators"
+npx tstest test/test.marketstack.node.ts --verbose
 ```

-## Contributing
+Test German business data:

-We welcome contributions! Please see our contributing guidelines for details.
+```bash
+npx tstest test/test.handelsregister.ts --verbose
+```

 ## License and Legal Information

-This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository. 
+This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository.

 **Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

@@ -306,9 +368,9 @@ This project is owned and maintained by Task Venture Capital GmbH. The names and

 ### Company Information

-Task Venture Capital GmbH  
+Task Venture Capital GmbH
 Registered at District court Bremen HRB 35230 HB, Germany

 For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.

-By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.
+By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.