Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| d49a738880 | |||
| 6273faa2f9 | |||
| d33c7e0f52 | |||
| 79930c40ac | |||
| 448278243e | |||
| ec3e4dde75 | |||
| 596be63554 | |||
| 8632f0e94b |
46
changelog.md
46
changelog.md
@@ -1,5 +1,51 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 2025-11-01 - 3.2.0 - feat(StockDataService)
|
||||||
|
Add unified StockDataService and BaseProviderService with new stockdata interfaces, provider integrations, tests and README updates
|
||||||
|
|
||||||
|
- Introduce StockDataService: unified API to fetch prices and fundamentals with automatic enrichment and caching
|
||||||
|
- Add IStockData and IStockDataServiceConfig interfaces to define combined price+fundamentals payloads and configuration
|
||||||
|
- Implement BaseProviderService abstraction to share provider registration, health, stats and caching logic
|
||||||
|
- Add classes.stockdataservice.ts implementing batch/single fetch, enrichment, caching, health checks and provider stats
|
||||||
|
- Export new stockdata module and classes from ts/stocks/index.ts
|
||||||
|
- Add comprehensive tests: test/test.stockdata.service.node.ts to cover setup, provider registration, fetching, caching, enrichment, health and error handling
|
||||||
|
- Update README with Unified Stock Data API examples, usage, and documentation reflecting new unified service
|
||||||
|
- Minor infra: add .claude/settings.local.json permissions for local tooling and web fetch domains
|
||||||
|
|
||||||
|
## 2025-11-01 - 3.1.0 - feat(fundamentals)
|
||||||
|
Add FundamentalsService and SEC EDGAR provider with caching, rate-limiting, tests, and docs updates
|
||||||
|
|
||||||
|
- Introduce FundamentalsService to manage fundamentals providers, caching, retry logic and provider statistics
|
||||||
|
- Add SecEdgarProvider to fetch SEC EDGAR company facts (CIK lookup, company facts parsing) with rate limiting and local caches
|
||||||
|
- Expose fundamentals interfaces and services from ts/stocks (exports updated)
|
||||||
|
- Add comprehensive tests for FundamentalsService and SecEdgarProvider (new test files)
|
||||||
|
- Update README with new Fundamentals module documentation, usage examples, and configuration guidance
|
||||||
|
- Implement caching and TTL handling for fundamentals data and provider-specific cache TTL support
|
||||||
|
- Add .claude/settings.local.json (local permissions) and various test improvements
|
||||||
|
|
||||||
|
## 2025-10-31 - 3.0.0 - BREAKING CHANGE(stocks)
|
||||||
|
Unify stock provider API to discriminated IStockDataRequest and add company name/fullname enrichment
|
||||||
|
|
||||||
|
- Replace legacy provider methods (fetchPrice/fetchPrices) with a single fetchData(request: IStockDataRequest) on IStockProvider — providers must be migrated to the new signature.
|
||||||
|
- Migrate StockPriceService to the unified getData(request: IStockDataRequest) API. Convenience helpers getPrice/getPrices now wrap getData.
|
||||||
|
- Add companyName and companyFullName fields to IStockPrice and populate them in provider mappings (Marketstack mapping updated; Yahoo provider updated to support the unified API).
|
||||||
|
- MarketstackProvider: added buildCompanyFullName helper and improved mapping to include company identification fields and full name formatting.
|
||||||
|
- YahooFinanceProvider: updated to implement fetchData and to route current/batch requests through the new unified request types; historical/intraday throw explicit errors.
|
||||||
|
- Updated tests to exercise the new unified API, company-name enrichment, caching behavior, and provider direct methods.
|
||||||
|
- Note: This is a breaking change for external providers and integrations that implemented the old fetchPrice/fetchPrices API. Bump major version.
|
||||||
|
|
||||||
|
## 2025-10-31 - 2.1.0 - feat(stocks)
|
||||||
|
Add unified stock data API (getData) with historical/OHLCV support, smart caching and provider enhancements
|
||||||
|
|
||||||
|
- Introduce discriminated union request types (IStockDataRequest) and a unified getData() method (replaces legacy getPrice/getPrices for new use cases)
|
||||||
|
- Add OHLCV fields (open, high, low, volume, adjusted) and metadata (dataType, fetchedAt) to IStockPrice
|
||||||
|
- Implement data-type aware smart caching with TTLs (historical = never expire, EOD = 24h, live = 30s, intraday matches interval)
|
||||||
|
- Extend StockPriceService: new getData(), data-specific cache keys, cache maxEntries increased (default 10000), and TTL-aware add/get cache logic
|
||||||
|
- Enhance Marketstack provider: unified fetchData(), historical date-range retrieval with pagination, exchange filtering, batch current fetch, OHLCV mapping, and intraday placeholder
|
||||||
|
- Update Yahoo provider to include dataType and fetchedAt (live data) and maintain legacy fetchPrice/fetchPrices compatibility
|
||||||
|
- Add/adjust tests to cover unified API, historical retrieval, OHLCV presence and smart caching behavior; test setup updated to require explicit OpenData directory paths
|
||||||
|
- Update README to document v2.1 changes, migration examples, and new stock provider capabilities
|
||||||
|
|
||||||
## 2025-10-31 - 2.0.0 - BREAKING CHANGE(OpenData)
|
## 2025-10-31 - 2.0.0 - BREAKING CHANGE(OpenData)
|
||||||
Require explicit directory paths for OpenData (nogit/download/germanBusinessData); remove automatic .nogit creation; update HandelsRegister, JsonlDataProcessor, tests and README.
|
Require explicit directory paths for OpenData (nogit/download/germanBusinessData); remove automatic .nogit creation; update HandelsRegister, JsonlDataProcessor, tests and README.
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@fin.cx/opendata",
|
"name": "@fin.cx/opendata",
|
||||||
"version": "2.0.0",
|
"version": "3.2.0",
|
||||||
"private": false,
|
"private": false,
|
||||||
"description": "A comprehensive TypeScript library for accessing business data and real-time financial information. Features include German company data management with MongoDB integration, JSONL bulk processing, automated Handelsregister interactions, and real-time stock market data from multiple providers.",
|
"description": "A comprehensive TypeScript library for accessing business data and real-time financial information. Features include German company data management with MongoDB integration, JSONL bulk processing, automated Handelsregister interactions, and real-time stock market data from multiple providers.",
|
||||||
"main": "dist_ts/index.js",
|
"main": "dist_ts/index.js",
|
||||||
|
|||||||
785
readme.md
785
readme.md
@@ -1,22 +1,8 @@
|
|||||||
# @fin.cx/opendata
|
# @fin.cx/opendata
|
||||||
|
|
||||||
🚀 **Real-time financial data and German business intelligence in one powerful TypeScript library**
|
🚀 **Complete financial intelligence toolkit for TypeScript**
|
||||||
|
|
||||||
Access live stock prices, cryptocurrencies, forex, commodities AND comprehensive German company data - all through a single, unified API.
|
Access real-time stock prices, fundamental financial data, and comprehensive German company information - all through a single, unified API.
|
||||||
|
|
||||||
## ⚠️ Breaking Change in v2.0
|
|
||||||
|
|
||||||
**Directory paths are now MANDATORY when using German business data features.** The package no longer creates `.nogit/` directories automatically. You must explicitly configure all directory paths when instantiating `OpenData`:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
const openData = new OpenData({
|
|
||||||
nogitDir: '/path/to/your/data',
|
|
||||||
downloadDir: '/path/to/your/data/downloads',
|
|
||||||
germanBusinessDataDir: '/path/to/your/data/germanbusinessdata'
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
This change enables the package to work in read-only filesystems (like Deno compiled binaries) and gives you full control over where data is stored.
|
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@@ -28,49 +14,121 @@ pnpm add @fin.cx/opendata
|
|||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
### 📈 Stock Market Data
|
### ✨ Unified Stock Data API (Recommended)
|
||||||
|
|
||||||
Get market data with EOD (End-of-Day) pricing:
|
Get complete stock data with automatic enrichment - the elegant way:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
import { StockPriceService, MarketstackProvider } from '@fin.cx/opendata';
|
import { StockDataService, YahooFinanceProvider, SecEdgarProvider } from '@fin.cx/opendata';
|
||||||
|
|
||||||
// Initialize the service with caching
|
// Initialize unified service
|
||||||
const stockService = new StockPriceService({
|
const stockData = new StockDataService();
|
||||||
ttl: 60000, // Cache for 1 minute
|
|
||||||
maxEntries: 1000 // Max cached symbols
|
// Register providers
|
||||||
|
stockData.registerPriceProvider(new YahooFinanceProvider());
|
||||||
|
stockData.registerFundamentalsProvider(new SecEdgarProvider({
|
||||||
|
userAgent: 'YourCompany youremail@example.com'
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✨ Get complete stock data with ONE method call
|
||||||
|
const apple = await stockData.getStockData('AAPL');
|
||||||
|
|
||||||
|
console.log({
|
||||||
|
company: apple.fundamentals.companyName, // "Apple Inc."
|
||||||
|
price: apple.price.price, // $270.37
|
||||||
|
marketCap: apple.fundamentals.marketCap, // $4.13T (auto-calculated!)
|
||||||
|
peRatio: apple.fundamentals.priceToEarnings, // 28.42 (auto-calculated!)
|
||||||
|
pbRatio: apple.fundamentals.priceToBook, // 45.12 (auto-calculated!)
|
||||||
|
eps: apple.fundamentals.earningsPerShareDiluted, // $6.13
|
||||||
|
revenue: apple.fundamentals.revenue // $385.6B
|
||||||
});
|
});
|
||||||
|
|
||||||
// Register Marketstack provider with API key
|
// Batch fetch with automatic enrichment
|
||||||
stockService.register(new MarketstackProvider('YOUR_API_KEY'), {
|
const stocks = await stockData.getBatchStockData(['AAPL', 'MSFT', 'GOOGL']);
|
||||||
priority: 100,
|
|
||||||
retryAttempts: 3
|
|
||||||
});
|
|
||||||
|
|
||||||
// Get single stock price
|
stocks.forEach(stock => {
|
||||||
const apple = await stockService.getPrice({ ticker: 'AAPL' });
|
console.log(`${stock.ticker}: $${stock.price.price.toFixed(2)}, ` +
|
||||||
console.log(`Apple: $${apple.price} (${apple.changePercent.toFixed(2)}%)`);
|
`P/E ${stock.fundamentals?.priceToEarnings?.toFixed(2)}`);
|
||||||
|
|
||||||
// Get multiple prices at once (batch fetching)
|
|
||||||
const prices = await stockService.getPrices({
|
|
||||||
tickers: ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'TSLA']
|
|
||||||
});
|
|
||||||
|
|
||||||
// 125,000+ tickers across 72+ exchanges worldwide
|
|
||||||
const internationalStocks = await stockService.getPrices({
|
|
||||||
tickers: ['AAPL', 'VOD.LON', 'SAP.DEX', 'TM', 'BABA']
|
|
||||||
});
|
});
|
||||||
|
// Output:
|
||||||
|
// AAPL: $270.37, P/E 28.42
|
||||||
|
// MSFT: $425.50, P/E 34.21
|
||||||
|
// GOOGL: $142.15, P/E 25.63
|
||||||
```
|
```
|
||||||
|
|
||||||
### 🏢 German Business Data
|
**Why use the unified API?**
|
||||||
|
- ✅ **Single service** for both prices and fundamentals
|
||||||
|
- ✅ **Automatic enrichment** - Market cap, P/E, P/B calculated automatically
|
||||||
|
- ✅ **One method call** - No manual `enrichWithPrice()` calls
|
||||||
|
- ✅ **Simplified code** - Less boilerplate, more readable
|
||||||
|
- ✅ **Type-safe** - Full TypeScript support
|
||||||
|
|
||||||
Access comprehensive data on German companies:
|
### 📈 Price-Only Data (Alternative)
|
||||||
|
|
||||||
|
If you only need prices without fundamentals:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { StockDataService, YahooFinanceProvider } from '@fin.cx/opendata';
|
||||||
|
|
||||||
|
const stockData = new StockDataService();
|
||||||
|
stockData.registerPriceProvider(new YahooFinanceProvider());
|
||||||
|
|
||||||
|
// Get just the price
|
||||||
|
const price = await stockData.getPrice('AAPL');
|
||||||
|
console.log(`${price.ticker}: $${price.price}`);
|
||||||
|
|
||||||
|
// Or use StockPriceService directly for more control
|
||||||
|
import { StockPriceService } from '@fin.cx/opendata';
|
||||||
|
|
||||||
|
const stockService = new StockPriceService({ ttl: 60000 });
|
||||||
|
stockService.register(new YahooFinanceProvider());
|
||||||
|
|
||||||
|
const apple = await stockService.getData({ type: 'current', ticker: 'AAPL' });
|
||||||
|
console.log(`${apple.companyFullName}: $${apple.price}`);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 💰 Fundamentals-Only Data (Alternative)
|
||||||
|
|
||||||
|
If you only need fundamentals without prices:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { StockDataService, SecEdgarProvider } from '@fin.cx/opendata';
|
||||||
|
|
||||||
|
const stockData = new StockDataService();
|
||||||
|
stockData.registerFundamentalsProvider(new SecEdgarProvider({
|
||||||
|
userAgent: 'YourCompany youremail@example.com'
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Get just fundamentals
|
||||||
|
const fundamentals = await stockData.getFundamentals('AAPL');
|
||||||
|
|
||||||
|
console.log({
|
||||||
|
company: fundamentals.companyName,
|
||||||
|
eps: fundamentals.earningsPerShareDiluted,
|
||||||
|
revenue: fundamentals.revenue,
|
||||||
|
sharesOutstanding: fundamentals.sharesOutstanding
|
||||||
|
});
|
||||||
|
|
||||||
|
// Or use FundamentalsService directly
|
||||||
|
import { FundamentalsService } from '@fin.cx/opendata';
|
||||||
|
|
||||||
|
const fundamentalsService = new FundamentalsService();
|
||||||
|
fundamentalsService.register(new SecEdgarProvider({
|
||||||
|
userAgent: 'YourCompany youremail@example.com'
|
||||||
|
}));
|
||||||
|
|
||||||
|
const data = await fundamentalsService.getFundamentals('AAPL');
|
||||||
|
```
|
||||||
|
|
||||||
|
### 🏢 German Business Intelligence
|
||||||
|
|
||||||
|
Access comprehensive German company data:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
import { OpenData } from '@fin.cx/opendata';
|
import { OpenData } from '@fin.cx/opendata';
|
||||||
import * as path from 'path';
|
import * as path from 'path';
|
||||||
|
|
||||||
// REQUIRED: Configure directory paths
|
// Configure directory paths
|
||||||
const openData = new OpenData({
|
const openData = new OpenData({
|
||||||
nogitDir: path.join(process.cwd(), '.nogit'),
|
nogitDir: path.join(process.cwd(), '.nogit'),
|
||||||
downloadDir: path.join(process.cwd(), '.nogit', 'downloads'),
|
downloadDir: path.join(process.cwd(), '.nogit', 'downloads'),
|
||||||
@@ -78,59 +136,234 @@ const openData = new OpenData({
|
|||||||
});
|
});
|
||||||
await openData.start();
|
await openData.start();
|
||||||
|
|
||||||
// Create a business record
|
// Search for companies
|
||||||
const company = new openData.CBusinessRecord();
|
const results = await openData.handelsregister.searchCompany("Siemens AG");
|
||||||
company.data = {
|
|
||||||
name: "TechStart GmbH",
|
|
||||||
city: "Berlin",
|
|
||||||
registrationId: "HRB 123456",
|
|
||||||
// ... more fields
|
|
||||||
};
|
|
||||||
await company.save();
|
|
||||||
|
|
||||||
// Search companies by city
|
// Get detailed information with documents
|
||||||
const berlinCompanies = await openData.db
|
const details = await openData.handelsregister.getSpecificCompany({
|
||||||
.collection('businessrecords')
|
court: "Munich",
|
||||||
.find({ city: "Berlin" })
|
type: "HRB",
|
||||||
.toArray();
|
number: "6684"
|
||||||
|
});
|
||||||
// Import bulk data from official sources
|
|
||||||
await openData.buildInitialDb();
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
### 🎯 Stock Market Module
|
### 📊 Stock Market Module
|
||||||
|
|
||||||
- **Marketstack API** - End-of-Day (EOD) data for 125,000+ tickers across 72+ exchanges
|
- **Real-Time Prices** - Live and EOD stock prices from Yahoo Finance and Marketstack
|
||||||
- **Stock prices** for stocks, ETFs, indices, and more
|
- **Company Names** - Automatic company name extraction (e.g., "Apple Inc (NASDAQ:AAPL)")
|
||||||
- **Batch operations** - fetch 100+ symbols in one request
|
- **Historical Data** - Up to 15 years of daily EOD prices with pagination
|
||||||
- **Smart caching** - configurable TTL, automatic invalidation
|
- **OHLCV Data** - Open, High, Low, Close, Volume for technical analysis
|
||||||
- **Extensible provider system** - easily add new data sources
|
- **Exchange Filtering** - Query specific exchanges via MIC codes (XNAS, XLON, XNYS)
|
||||||
- **Retry logic** - configurable retry attempts and delays
|
- **Smart Caching** - Data-type aware TTL (historical cached forever, EOD 24h, live 30s)
|
||||||
- **Type-safe** - full TypeScript support with detailed interfaces
|
- **Batch Operations** - Fetch 100+ symbols in one request
|
||||||
|
- **Type-Safe API** - Full TypeScript support with discriminated unions
|
||||||
|
- **Multi-Provider** - Automatic fallback between providers
|
||||||
|
|
||||||
|
### 💰 Fundamental Data Module
|
||||||
|
|
||||||
|
- **SEC EDGAR Integration** - FREE fundamental data directly from SEC filings
|
||||||
|
- **Comprehensive Metrics** - EPS, Revenue, Assets, Liabilities, Cash Flow, and more
|
||||||
|
- **All US Public Companies** - Complete coverage of SEC-registered companies
|
||||||
|
- **Historical Filings** - Data back to ~2009
|
||||||
|
- **CIK Lookup** - Automatic ticker-to-CIK mapping with smart caching
|
||||||
|
- **Calculated Ratios** - Market Cap, P/E, P/B ratios when combined with prices
|
||||||
|
- **No API Key Required** - Direct access to SEC's public API
|
||||||
|
- **Rate Limit Management** - Built-in 10 req/sec rate limiting
|
||||||
|
|
||||||
### 🇩🇪 German Business Intelligence
|
### 🇩🇪 German Business Intelligence
|
||||||
|
|
||||||
- **MongoDB integration** for scalable data storage
|
- **MongoDB Integration** - Scalable data storage for millions of records
|
||||||
- **Bulk JSONL import** from official German data sources
|
- **Bulk JSONL Import** - Process multi-GB datasets efficiently
|
||||||
- **Handelsregister automation** - automated document retrieval
|
- **Handelsregister Automation** - Automated document retrieval
|
||||||
- **CRUD operations** with validation
|
- **CRUD Operations** - Full database management with validation
|
||||||
- **Streaming processing** for multi-GB datasets
|
- **Streaming Processing** - Handle large datasets without memory issues
|
||||||
|
|
||||||
## Advanced Examples
|
## Advanced Examples
|
||||||
|
|
||||||
|
### Combined Market Analysis (Unified API)
|
||||||
|
|
||||||
|
Analyze multiple companies with automatic enrichment:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { StockDataService, YahooFinanceProvider, SecEdgarProvider } from '@fin.cx/opendata';
|
||||||
|
|
||||||
|
// Setup unified service
|
||||||
|
const stockData = new StockDataService();
|
||||||
|
stockData.registerPriceProvider(new YahooFinanceProvider());
|
||||||
|
stockData.registerFundamentalsProvider(new SecEdgarProvider({
|
||||||
|
userAgent: 'YourCompany youremail@example.com'
|
||||||
|
}));
|
||||||
|
|
||||||
|
// Analyze multiple companies with ONE call
|
||||||
|
const tickers = ['AAPL', 'MSFT', 'GOOGL', 'AMZN'];
|
||||||
|
const stocks = await stockData.getBatchStockData(tickers);
|
||||||
|
|
||||||
|
// All metrics are automatically calculated!
|
||||||
|
stocks.forEach(stock => {
|
||||||
|
if (stock.fundamentals) {
|
||||||
|
console.log(`\n${stock.fundamentals.companyName} (${stock.ticker})`);
|
||||||
|
console.log(` Price: $${stock.price.price.toFixed(2)}`);
|
||||||
|
console.log(` Market Cap: $${(stock.fundamentals.marketCap! / 1e9).toFixed(2)}B`);
|
||||||
|
console.log(` P/E Ratio: ${stock.fundamentals.priceToEarnings!.toFixed(2)}`);
|
||||||
|
console.log(` Revenue: $${(stock.fundamentals.revenue! / 1e9).toFixed(2)}B`);
|
||||||
|
console.log(` EPS: $${stock.fundamentals.earningsPerShareDiluted!.toFixed(2)}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Or analyze one-by-one with automatic enrichment
|
||||||
|
for (const ticker of tickers) {
|
||||||
|
const stock = await stockData.getStockData(ticker);
|
||||||
|
|
||||||
|
// Everything is already enriched - no manual calculations needed!
|
||||||
|
console.log(`${ticker}: P/E ${stock.fundamentals?.priceToEarnings?.toFixed(2)}`);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Fundamental Data Screening
|
||||||
|
|
||||||
|
Screen stocks by financial metrics:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Fetch data for multiple tickers
|
||||||
|
const tickers = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'META', 'NVDA', 'TSLA'];
|
||||||
|
const stocks = await stockData.getBatchStockData(tickers);
|
||||||
|
|
||||||
|
// Filter by criteria (all metrics auto-calculated!)
|
||||||
|
const valueStocks = stocks.filter(stock => {
|
||||||
|
const f = stock.fundamentals;
|
||||||
|
return f &&
|
||||||
|
f.priceToEarnings! < 30 && // P/E under 30
|
||||||
|
f.priceToBook! < 10 && // P/B under 10
|
||||||
|
f.revenue! > 100_000_000_000; // Revenue > $100B
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log('\n💎 Value Stocks:');
|
||||||
|
valueStocks.forEach(stock => {
|
||||||
|
console.log(`${stock.ticker}: P/E ${stock.fundamentals!.priceToEarnings!.toFixed(2)}, ` +
|
||||||
|
`P/B ${stock.fundamentals!.priceToBook!.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Historical Data Analysis
|
||||||
|
|
||||||
|
Fetch and analyze historical price trends:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Get 1 year of historical data
|
||||||
|
const history = await stockService.getData({
|
||||||
|
type: 'historical',
|
||||||
|
ticker: 'AAPL',
|
||||||
|
from: new Date('2024-01-01'),
|
||||||
|
to: new Date('2024-12-31'),
|
||||||
|
sort: 'DESC' // Newest first
|
||||||
|
});
|
||||||
|
|
||||||
|
// Calculate statistics
|
||||||
|
const prices = history.map(p => p.price);
|
||||||
|
const high52Week = Math.max(...prices);
|
||||||
|
const low52Week = Math.min(...prices);
|
||||||
|
const avgPrice = prices.reduce((a, b) => a + b) / prices.length;
|
||||||
|
|
||||||
|
console.log(`52-Week Analysis for AAPL:`);
|
||||||
|
console.log(` High: $${high52Week.toFixed(2)}`);
|
||||||
|
console.log(` Low: $${low52Week.toFixed(2)}`);
|
||||||
|
console.log(` Average: $${avgPrice.toFixed(2)}`);
|
||||||
|
console.log(` Days: ${history.length}`);
|
||||||
|
|
||||||
|
// Calculate Simple Moving Average
|
||||||
|
const calculateSMA = (data: IStockPrice[], period: number) => {
|
||||||
|
const sma: number[] = [];
|
||||||
|
for (let i = period - 1; i < data.length; i++) {
|
||||||
|
const sum = data.slice(i - period + 1, i + 1)
|
||||||
|
.reduce((acc, p) => acc + p.price, 0);
|
||||||
|
sma.push(sum / period);
|
||||||
|
}
|
||||||
|
return sma;
|
||||||
|
};
|
||||||
|
|
||||||
|
const sma20 = calculateSMA(history, 20);
|
||||||
|
const sma50 = calculateSMA(history, 50);
|
||||||
|
|
||||||
|
console.log(`\nMoving Averages:`);
|
||||||
|
console.log(` 20-day SMA: $${sma20[0].toFixed(2)}`);
|
||||||
|
console.log(` 50-day SMA: $${sma50[0].toFixed(2)}`);
|
||||||
|
```
|
||||||
|
|
||||||
|
### OHLCV Technical Analysis
|
||||||
|
|
||||||
|
Use OHLCV data for technical indicators:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const history = await stockService.getData({
|
||||||
|
type: 'historical',
|
||||||
|
ticker: 'TSLA',
|
||||||
|
from: new Date('2024-11-01'),
|
||||||
|
to: new Date('2024-11-30')
|
||||||
|
});
|
||||||
|
|
||||||
|
// Calculate daily trading range
|
||||||
|
for (const day of history) {
|
||||||
|
const range = day.high! - day.low!;
|
||||||
|
const rangePercent = (range / day.low!) * 100;
|
||||||
|
|
||||||
|
console.log(`${day.timestamp.toISOString().split('T')[0]}:`);
|
||||||
|
console.log(` Open: $${day.open}`);
|
||||||
|
console.log(` High: $${day.high}`);
|
||||||
|
console.log(` Low: $${day.low}`);
|
||||||
|
console.log(` Close: $${day.price}`);
|
||||||
|
console.log(` Volume: ${day.volume?.toLocaleString()}`);
|
||||||
|
console.log(` Range: $${range.toFixed(2)} (${rangePercent.toFixed(2)}%)`);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Fundamental Data Screening
|
||||||
|
|
||||||
|
Screen stocks based on fundamental metrics:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const tickers = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'TSLA', 'META', 'NVDA'];
|
||||||
|
|
||||||
|
// Fetch fundamentals for all tickers
|
||||||
|
const allFundamentals = await fundamentalsService.getBatchFundamentals(tickers);
|
||||||
|
|
||||||
|
// Get current prices
|
||||||
|
const prices = await stockService.getData({
|
||||||
|
type: 'batch',
|
||||||
|
tickers: tickers
|
||||||
|
});
|
||||||
|
|
||||||
|
// Create price map
|
||||||
|
const priceMap = new Map(prices.map(p => [p.ticker, p.price]));
|
||||||
|
|
||||||
|
// Enrich with prices
|
||||||
|
const enriched = await fundamentalsService.enrichBatchWithPrices(
|
||||||
|
allFundamentals,
|
||||||
|
priceMap
|
||||||
|
);
|
||||||
|
|
||||||
|
// Screen for value stocks (P/E < 25, P/B < 5)
|
||||||
|
const valueStocks = enriched.filter(f =>
|
||||||
|
f.priceToEarnings && f.priceToEarnings < 25 &&
|
||||||
|
f.priceToBook && f.priceToBook < 5
|
||||||
|
);
|
||||||
|
|
||||||
|
console.log('Value Stocks:');
|
||||||
|
valueStocks.forEach(stock => {
|
||||||
|
console.log(`\n${stock.companyName} (${stock.ticker})`);
|
||||||
|
console.log(` P/E Ratio: ${stock.priceToEarnings!.toFixed(2)}`);
|
||||||
|
console.log(` P/B Ratio: ${stock.priceToBook!.toFixed(2)}`);
|
||||||
|
console.log(` Market Cap: $${(stock.marketCap! / 1e9).toFixed(2)}B`);
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
### Market Dashboard
|
### Market Dashboard
|
||||||
|
|
||||||
Create an EOD market overview:
|
Create a comprehensive market overview:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
const indicators = [
|
const indicators = [
|
||||||
// Indices
|
|
||||||
{ ticker: '^GSPC', name: 'S&P 500' },
|
|
||||||
{ ticker: '^DJI', name: 'DOW Jones' },
|
|
||||||
|
|
||||||
// Tech Giants
|
|
||||||
{ ticker: 'AAPL', name: 'Apple' },
|
{ ticker: 'AAPL', name: 'Apple' },
|
||||||
{ ticker: 'MSFT', name: 'Microsoft' },
|
{ ticker: 'MSFT', name: 'Microsoft' },
|
||||||
{ ticker: 'GOOGL', name: 'Alphabet' },
|
{ ticker: 'GOOGL', name: 'Alphabet' },
|
||||||
@@ -138,7 +371,8 @@ const indicators = [
|
|||||||
{ ticker: 'TSLA', name: 'Tesla' }
|
{ ticker: 'TSLA', name: 'Tesla' }
|
||||||
];
|
];
|
||||||
|
|
||||||
const prices = await stockService.getPrices({
|
const prices = await stockService.getData({
|
||||||
|
type: 'batch',
|
||||||
tickers: indicators.map(i => i.ticker)
|
tickers: indicators.map(i => i.ticker)
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -149,166 +383,52 @@ prices.forEach(price => {
|
|||||||
const color = price.change >= 0 ? '\x1b[32m' : '\x1b[31m';
|
const color = price.change >= 0 ? '\x1b[32m' : '\x1b[31m';
|
||||||
|
|
||||||
console.log(
|
console.log(
|
||||||
`${indicator.name.padEnd(15)} ${price.price.toFixed(2).padStart(10)} ` +
|
`${price.companyName!.padEnd(25)} $${price.price.toFixed(2).padStart(8)} ` +
|
||||||
`${color}${arrow} ${price.changePercent.toFixed(2)}%\x1b[0m`
|
`${color}${arrow} ${price.changePercent.toFixed(2)}%\x1b[0m`
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
### Provider Health and Statistics
|
### Exchange-Specific Trading
|
||||||
|
|
||||||
Monitor your provider health and track usage:
|
Compare prices across different exchanges:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
// Check provider health
|
// Vodafone trades on both London and NYSE
|
||||||
const health = await stockService.checkProvidersHealth();
|
const exchanges = [
|
||||||
console.log(`Marketstack: ${health.get('Marketstack') ? '✅' : '❌'}`);
|
{ mic: 'XLON', name: 'London Stock Exchange' },
|
||||||
|
{ mic: 'XNYS', name: 'New York Stock Exchange' }
|
||||||
|
];
|
||||||
|
|
||||||
// Get provider statistics
|
for (const exchange of exchanges) {
|
||||||
const stats = stockService.getProviderStats();
|
|
||||||
const marketstackStats = stats.get('Marketstack');
|
|
||||||
console.log('Marketstack Stats:', {
|
|
||||||
successCount: marketstackStats.successCount,
|
|
||||||
errorCount: marketstackStats.errorCount,
|
|
||||||
lastError: marketstackStats.lastError
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### Handelsregister Integration
|
|
||||||
|
|
||||||
Automate German company data retrieval:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { OpenData } from '@fin.cx/opendata';
|
|
||||||
import * as path from 'path';
|
|
||||||
|
|
||||||
// Configure paths first
|
|
||||||
const openData = new OpenData({
|
|
||||||
nogitDir: path.join(process.cwd(), '.nogit'),
|
|
||||||
downloadDir: path.join(process.cwd(), '.nogit', 'downloads'),
|
|
||||||
germanBusinessDataDir: path.join(process.cwd(), '.nogit', 'germanbusinessdata')
|
|
||||||
});
|
|
||||||
await openData.start();
|
|
||||||
|
|
||||||
// Search for a company
|
|
||||||
const results = await openData.handelsregister.searchCompany("Siemens AG");
|
|
||||||
|
|
||||||
// Get detailed information and documents
|
|
||||||
const details = await openData.handelsregister.getSpecificCompany({
|
|
||||||
court: "Munich",
|
|
||||||
type: "HRB",
|
|
||||||
number: "6684"
|
|
||||||
});
|
|
||||||
|
|
||||||
// Downloaded files include:
|
|
||||||
// - XML data (SI files)
|
|
||||||
// - PDF documents (AD files)
|
|
||||||
for (const file of details.files) {
|
|
||||||
await file.writeToDir('./downloads');
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Combined Data Analysis
|
|
||||||
|
|
||||||
Merge financial and business data:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
import { OpenData, StockPriceService, MarketstackProvider } from '@fin.cx/opendata';
|
|
||||||
import * as path from 'path';
|
|
||||||
|
|
||||||
// Configure OpenData with paths
|
|
||||||
const openData = new OpenData({
|
|
||||||
nogitDir: path.join(process.cwd(), '.nogit'),
|
|
||||||
downloadDir: path.join(process.cwd(), '.nogit', 'downloads'),
|
|
||||||
germanBusinessDataDir: path.join(process.cwd(), '.nogit', 'germanbusinessdata')
|
|
||||||
});
|
|
||||||
await openData.start();
|
|
||||||
|
|
||||||
// Setup stock service
|
|
||||||
const stockService = new StockPriceService({ ttl: 60000, maxEntries: 1000 });
|
|
||||||
stockService.register(new MarketstackProvider('YOUR_API_KEY'));
|
|
||||||
|
|
||||||
// Find all public German companies (AG)
|
|
||||||
const publicCompanies = await openData.db
|
|
||||||
.collection('businessrecords')
|
|
||||||
.find({ legalForm: 'AG' })
|
|
||||||
.toArray();
|
|
||||||
|
|
||||||
// Enrich with stock data
|
|
||||||
for (const company of publicCompanies) {
|
|
||||||
try {
|
try {
|
||||||
// Map company to ticker (custom logic needed)
|
const price = await stockService.getData({
|
||||||
const ticker = mapCompanyToTicker(company.data.name);
|
type: 'current',
|
||||||
|
ticker: 'VOD',
|
||||||
|
exchange: exchange.mic
|
||||||
|
});
|
||||||
|
|
||||||
if (ticker) {
|
console.log(`${exchange.name}:`);
|
||||||
const stock = await stockService.getPrice({ ticker });
|
console.log(` Price: ${price.price} ${price.currency}`);
|
||||||
|
console.log(` Volume: ${price.volume?.toLocaleString()}`);
|
||||||
// Add financial metrics
|
console.log(` Exchange: ${price.exchangeName}`);
|
||||||
company.data.stockPrice = stock.price;
|
|
||||||
company.data.marketCap = stock.price * getSharesOutstanding(ticker);
|
|
||||||
company.data.priceChange = stock.changePercent;
|
|
||||||
|
|
||||||
await company.save();
|
|
||||||
}
|
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
// Handle missing tickers gracefully
|
console.log(`${exchange.name}: Not available`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
### Directory Configuration (Required for German Business Data)
|
|
||||||
|
|
||||||
**All directory paths are mandatory when using `OpenData`.** Here are examples for different environments:
|
|
||||||
|
|
||||||
#### Development Environment
|
|
||||||
```typescript
|
|
||||||
import { OpenData } from '@fin.cx/opendata';
|
|
||||||
import * as path from 'path';
|
|
||||||
|
|
||||||
const openData = new OpenData({
|
|
||||||
nogitDir: path.join(process.cwd(), '.nogit'),
|
|
||||||
downloadDir: path.join(process.cwd(), '.nogit', 'downloads'),
|
|
||||||
germanBusinessDataDir: path.join(process.cwd(), '.nogit', 'germanbusinessdata')
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Production Environment
|
|
||||||
```typescript
|
|
||||||
import { OpenData } from '@fin.cx/opendata';
|
|
||||||
import * as path from 'path';
|
|
||||||
|
|
||||||
const openData = new OpenData({
|
|
||||||
nogitDir: '/var/lib/myapp/data',
|
|
||||||
downloadDir: '/var/lib/myapp/data/downloads',
|
|
||||||
germanBusinessDataDir: '/var/lib/myapp/data/germanbusinessdata'
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Deno Compiled Binaries (or other read-only filesystems)
|
|
||||||
```typescript
|
|
||||||
import { OpenData } from '@fin.cx/opendata';
|
|
||||||
|
|
||||||
// Use OS temp directory or user data directory
|
|
||||||
const dataDir = Deno.env.get('HOME') + '/.myapp/data';
|
|
||||||
|
|
||||||
const openData = new OpenData({
|
|
||||||
nogitDir: dataDir,
|
|
||||||
downloadDir: dataDir + '/downloads',
|
|
||||||
germanBusinessDataDir: dataDir + '/germanbusinessdata'
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### Stock Service Options
|
### Stock Service Options
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
const stockService = new StockPriceService({
|
const stockService = new StockPriceService({
|
||||||
ttl: 60000, // Cache for 1 minute
|
ttl: 60000, // Default cache TTL in ms
|
||||||
maxEntries: 1000 // Max cached symbols
|
maxEntries: 10000 // Max cached entries
|
||||||
});
|
});
|
||||||
|
|
||||||
// Marketstack - EOD data, requires API key
|
// Marketstack - EOD data (requires API key)
|
||||||
stockService.register(new MarketstackProvider('YOUR_API_KEY'), {
|
stockService.register(new MarketstackProvider('YOUR_API_KEY'), {
|
||||||
enabled: true,
|
enabled: true,
|
||||||
priority: 100,
|
priority: 100,
|
||||||
@@ -316,30 +436,72 @@ stockService.register(new MarketstackProvider('YOUR_API_KEY'), {
|
|||||||
retryAttempts: 3,
|
retryAttempts: 3,
|
||||||
retryDelay: 1000
|
retryDelay: 1000
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Yahoo Finance - Real-time data (no API key)
|
||||||
|
stockService.register(new YahooFinanceProvider(), {
|
||||||
|
enabled: true,
|
||||||
|
priority: 50
|
||||||
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
### MongoDB Setup
|
### Fundamentals Service Options
|
||||||
|
|
||||||
Set environment variables for German business data:
|
```typescript
|
||||||
|
const fundamentalsService = new FundamentalsService({
|
||||||
|
ttl: 90 * 24 * 60 * 60 * 1000, // 90 days (quarterly refresh)
|
||||||
|
maxEntries: 10000
|
||||||
|
});
|
||||||
|
|
||||||
|
// SEC EDGAR provider (FREE - no API key!)
|
||||||
|
fundamentalsService.register(new SecEdgarProvider({
|
||||||
|
userAgent: 'YourCompany youremail@example.com',
|
||||||
|
cikCacheTTL: 30 * 24 * 60 * 60 * 1000, // 30 days
|
||||||
|
fundamentalsCacheTTL: 90 * 24 * 60 * 60 * 1000, // 90 days
|
||||||
|
timeout: 30000
|
||||||
|
}));
|
||||||
|
```
|
||||||
|
|
||||||
|
### Directory Configuration (German Business Data)
|
||||||
|
|
||||||
|
All directory paths are mandatory when using German business data features:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { OpenData } from '@fin.cx/opendata';
|
||||||
|
import * as path from 'path';
|
||||||
|
|
||||||
|
// Development
|
||||||
|
const openData = new OpenData({
|
||||||
|
nogitDir: path.join(process.cwd(), '.nogit'),
|
||||||
|
downloadDir: path.join(process.cwd(), '.nogit', 'downloads'),
|
||||||
|
germanBusinessDataDir: path.join(process.cwd(), '.nogit', 'germanbusinessdata')
|
||||||
|
});
|
||||||
|
|
||||||
|
// Production
|
||||||
|
const openDataProd = new OpenData({
|
||||||
|
nogitDir: '/var/lib/myapp/data',
|
||||||
|
downloadDir: '/var/lib/myapp/data/downloads',
|
||||||
|
germanBusinessDataDir: '/var/lib/myapp/data/germanbusinessdata'
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
Set environment variables for API keys and database:
|
||||||
|
|
||||||
```env
|
```env
|
||||||
|
# Marketstack API (for EOD stock data)
|
||||||
|
MARKETSTACK_COM_TOKEN=your_api_key_here
|
||||||
|
|
||||||
|
# MongoDB (for German business data)
|
||||||
MONGODB_URL=mongodb://localhost:27017
|
MONGODB_URL=mongodb://localhost:27017
|
||||||
MONGODB_NAME=opendata
|
MONGODB_NAME=opendata
|
||||||
MONGODB_USER=myuser
|
MONGODB_USER=myuser
|
||||||
MONGODB_PASS=mypass
|
MONGODB_PASS=mypass
|
||||||
```
|
```
|
||||||
|
|
||||||
### Marketstack API Key
|
|
||||||
|
|
||||||
Get your free API key at [marketstack.com](https://marketstack.com) and set it in your environment:
|
|
||||||
|
|
||||||
```env
|
|
||||||
MARKETSTACK_COM_TOKEN=your_api_key_here
|
|
||||||
```
|
|
||||||
|
|
||||||
## API Reference
|
## API Reference
|
||||||
|
|
||||||
### Stock Types
|
### Stock Price Interfaces
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
interface IStockPrice {
|
interface IStockPrice {
|
||||||
@@ -354,62 +516,154 @@ interface IStockPrice {
|
|||||||
marketState: 'PRE' | 'REGULAR' | 'POST' | 'CLOSED';
|
marketState: 'PRE' | 'REGULAR' | 'POST' | 'CLOSED';
|
||||||
exchange?: string;
|
exchange?: string;
|
||||||
exchangeName?: string;
|
exchangeName?: string;
|
||||||
|
|
||||||
|
// OHLCV data
|
||||||
|
volume?: number;
|
||||||
|
open?: number;
|
||||||
|
high?: number;
|
||||||
|
low?: number;
|
||||||
|
adjusted?: boolean;
|
||||||
|
dataType: 'eod' | 'intraday' | 'live';
|
||||||
|
fetchedAt: Date;
|
||||||
|
|
||||||
|
// Company identification
|
||||||
|
companyName?: string; // "Apple Inc"
|
||||||
|
companyFullName?: string; // "Apple Inc (NASDAQ:AAPL)"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Fundamental Data Interfaces
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface IStockFundamentals {
|
||||||
|
ticker: string;
|
||||||
|
cik: string;
|
||||||
|
companyName: string;
|
||||||
|
provider: string;
|
||||||
|
timestamp: Date;
|
||||||
|
fetchedAt: Date;
|
||||||
|
|
||||||
|
// Per-share metrics
|
||||||
|
earningsPerShareBasic?: number;
|
||||||
|
earningsPerShareDiluted?: number;
|
||||||
|
sharesOutstanding?: number;
|
||||||
|
|
||||||
|
// Income statement (annual USD)
|
||||||
|
revenue?: number;
|
||||||
|
netIncome?: number;
|
||||||
|
operatingIncome?: number;
|
||||||
|
grossProfit?: number;
|
||||||
|
|
||||||
|
// Balance sheet (annual USD)
|
||||||
|
assets?: number;
|
||||||
|
liabilities?: number;
|
||||||
|
stockholdersEquity?: number;
|
||||||
|
cash?: number;
|
||||||
|
propertyPlantEquipment?: number;
|
||||||
|
|
||||||
|
// Calculated metrics (requires price)
|
||||||
|
marketCap?: number; // price × sharesOutstanding
|
||||||
|
priceToEarnings?: number; // price / EPS
|
||||||
|
priceToBook?: number; // marketCap / stockholdersEquity
|
||||||
|
|
||||||
|
// Metadata
|
||||||
|
fiscalYear?: string;
|
||||||
|
fiscalQuarter?: string;
|
||||||
|
filingDate?: Date;
|
||||||
|
form?: '10-K' | '10-Q' | string;
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
### Key Methods
|
### Key Methods
|
||||||
|
|
||||||
**StockPriceService**
|
**StockPriceService**
|
||||||
- `getPrice(request)` - Single stock price with automatic provider selection
|
- `getData(request)` - Unified method for all stock data (current, historical, batch)
|
||||||
- `getPrices(request)` - Batch prices (100+ symbols in one request)
|
- `getPrice(request)` - Convenience method for single current price
|
||||||
|
- `getPrices(request)` - Convenience method for batch current prices
|
||||||
- `register(provider, config)` - Add data provider with priority and retry config
|
- `register(provider, config)` - Add data provider with priority and retry config
|
||||||
- `checkProvidersHealth()` - Test all providers and return health status
|
- `checkProvidersHealth()` - Test all providers and return health status
|
||||||
- `getProviderStats()` - Get success/error statistics for each provider
|
- `getProviderStats()` - Get success/error statistics for each provider
|
||||||
- `clearCache()` - Clear price cache
|
- `clearCache()` - Clear price cache
|
||||||
- `setCacheTTL(ttl)` - Update cache TTL dynamically
|
- `setCacheTTL(ttl)` - Update cache TTL dynamically
|
||||||
|
|
||||||
|
**FundamentalsService**
|
||||||
|
- `getFundamentals(ticker)` - Get fundamental data for single ticker
|
||||||
|
- `getBatchFundamentals(tickers)` - Get fundamentals for multiple tickers
|
||||||
|
- `enrichWithPrice(fundamentals, price)` - Calculate market cap, P/E, P/B ratios
|
||||||
|
- `enrichBatchWithPrices(fundamentals, priceMap)` - Batch enrich with prices
|
||||||
|
- `register(provider, config)` - Add fundamentals provider
|
||||||
|
- `checkProvidersHealth()` - Test all providers
|
||||||
|
- `getProviderStats()` - Get success/error statistics
|
||||||
|
- `clearCache()` - Clear fundamentals cache
|
||||||
|
|
||||||
|
**SecEdgarProvider**
|
||||||
|
- ✅ FREE - No API key required
|
||||||
|
- ✅ All US public companies
|
||||||
|
- ✅ Comprehensive US GAAP financial metrics
|
||||||
|
- ✅ Historical data back to ~2009
|
||||||
|
- ✅ Direct access to SEC filings (10-K, 10-Q)
|
||||||
|
- ✅ Smart caching (CIK: 30 days, Fundamentals: 90 days)
|
||||||
|
- ✅ Rate limiting (10 requests/second)
|
||||||
|
- ℹ️ Requires User-Agent header in format: "Company Name Email"
|
||||||
|
|
||||||
**MarketstackProvider**
|
**MarketstackProvider**
|
||||||
- ✅ End-of-Day (EOD) data
|
- ✅ End-of-Day (EOD) stock prices
|
||||||
- ✅ 125,000+ tickers across 72+ exchanges worldwide
|
- ✅ 500,000+ tickers across 72+ exchanges worldwide
|
||||||
- ✅ Batch fetching support (multiple symbols in one request)
|
- ✅ Historical data with pagination
|
||||||
- ✅ Comprehensive data: open, high, low, close, volume, splits, dividends
|
- ✅ Batch fetching support
|
||||||
|
- ✅ OHLCV data (Open, High, Low, Close, Volume)
|
||||||
|
- ✅ Company names included automatically
|
||||||
- ⚠️ Requires API key (free tier: 100 requests/month)
|
- ⚠️ Requires API key (free tier: 100 requests/month)
|
||||||
- ⚠️ EOD data only (not real-time)
|
|
||||||
|
**YahooFinanceProvider**
|
||||||
|
- ✅ Real-time stock prices
|
||||||
|
- ✅ No API key required
|
||||||
|
- ✅ Global coverage
|
||||||
|
- ✅ Company names included
|
||||||
|
- ⚠️ Rate limits may apply
|
||||||
|
|
||||||
**OpenData**
|
**OpenData**
|
||||||
- `start()` - Initialize MongoDB connection
|
- `start()` - Initialize MongoDB connection
|
||||||
- `buildInitialDb()` - Import bulk data
|
- `buildInitialDb()` - Import bulk data
|
||||||
- `CBusinessRecord` - Business record class
|
- `CBusinessRecord` - Business record class
|
||||||
- `handelsregister` - Registry automation
|
- `handelsregister` - German registry automation
|
||||||
|
|
||||||
## Provider Architecture
|
## Provider Architecture
|
||||||
|
|
||||||
The library uses a flexible provider system that makes it easy to add new data sources:
|
Add custom data providers easily:
|
||||||
|
|
||||||
```typescript
|
```typescript
|
||||||
class MyCustomProvider implements IStockProvider {
|
class MyCustomProvider implements IStockProvider {
|
||||||
name = 'My Provider';
|
name = 'My Provider';
|
||||||
priority = 50;
|
priority = 50;
|
||||||
requiresAuth = true;
|
requiresAuth = true;
|
||||||
|
rateLimit = { requestsPerMinute: 60 };
|
||||||
|
|
||||||
async fetchPrice(request: IStockQuoteRequest): Promise<IStockPrice> {
|
async fetchData(request: IStockDataRequest): Promise<IStockPrice | IStockPrice[]> {
|
||||||
// Your implementation
|
// Implement unified data fetching
|
||||||
}
|
switch (request.type) {
|
||||||
|
case 'current':
|
||||||
async fetchPrices(request: IStockBatchQuoteRequest): Promise<IStockPrice[]> {
|
return this.fetchCurrentPrice(request);
|
||||||
// Batch implementation
|
case 'batch':
|
||||||
|
return this.fetchBatchPrices(request);
|
||||||
|
case 'historical':
|
||||||
|
return this.fetchHistoricalPrices(request);
|
||||||
|
default:
|
||||||
|
throw new Error(`Unsupported request type`);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async isAvailable(): Promise<boolean> {
|
async isAvailable(): Promise<boolean> {
|
||||||
// Health check
|
// Health check
|
||||||
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
supportsMarket(market: string): boolean {
|
supportsMarket(market: string): boolean {
|
||||||
// Market validation
|
return ['US', 'UK', 'DE'].includes(market);
|
||||||
}
|
}
|
||||||
|
|
||||||
supportsTicker(ticker: string): boolean {
|
supportsTicker(ticker: string): boolean {
|
||||||
// Ticker validation
|
return /^[A-Z]{1,5}$/.test(ticker);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -418,30 +672,53 @@ stockService.register(new MyCustomProvider());
|
|||||||
|
|
||||||
## Performance
|
## Performance
|
||||||
|
|
||||||
- **Batch fetching**: Get 100+ EOD prices in one API request
|
- **Batch Fetching**: Get 100+ prices in one API request
|
||||||
- **Smart caching**: Instant repeated queries with configurable TTL
|
- **Smart Caching**: Data-type aware TTL (historical cached forever, EOD 24h, live 30s)
|
||||||
- **Rate limit aware**: Automatic retry logic for API limits
|
- **Rate Limit Management**: Automatic retry logic for API limits
|
||||||
- **Concurrent processing**: Handle 1000+ business records/second
|
- **Concurrent Processing**: Handle 1000+ records/second
|
||||||
- **Streaming**: Process GB-sized datasets without memory issues
|
- **Streaming**: Process GB-sized datasets without memory issues
|
||||||
|
- **Provider Fallback**: Automatic failover between data sources
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
Run the comprehensive test suite:
|
Run the comprehensive test suite:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm test
|
pnpm test
|
||||||
```
|
```
|
||||||
|
|
||||||
Test stock provider:
|
Test specific modules:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npx tstest test/test.marketstack.node.ts --verbose
|
# Stock price providers
|
||||||
|
pnpm tstest test/test.marketstack.node.ts --verbose
|
||||||
|
pnpm tstest test/test.stocks.ts --verbose
|
||||||
|
|
||||||
|
# Fundamental data
|
||||||
|
pnpm tstest test/test.secedgar.provider.node.ts --verbose
|
||||||
|
pnpm tstest test/test.fundamentals.service.node.ts --verbose
|
||||||
|
|
||||||
|
# German business data
|
||||||
|
pnpm tstest test/test.handelsregister.ts --verbose
|
||||||
```
|
```
|
||||||
|
|
||||||
Test German business data:
|
## Getting API Keys
|
||||||
|
|
||||||
```bash
|
### Marketstack (EOD Stock Data)
|
||||||
npx tstest test/test.handelsregister.ts --verbose
|
|
||||||
|
1. Visit [marketstack.com](https://marketstack.com)
|
||||||
|
2. Sign up for a free account (100 requests/month)
|
||||||
|
3. Get your API key from the dashboard
|
||||||
|
4. Set environment variable: `MARKETSTACK_COM_TOKEN=your_key_here`
|
||||||
|
|
||||||
|
### SEC EDGAR (Fundamental Data)
|
||||||
|
|
||||||
|
**No API key required!** SEC EDGAR is completely free and public. Just provide your company name and email in the User-Agent:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
new SecEdgarProvider({
|
||||||
|
userAgent: 'YourCompany youremail@example.com'
|
||||||
|
});
|
||||||
```
|
```
|
||||||
|
|
||||||
## License and Legal Information
|
## License and Legal Information
|
||||||
|
|||||||
287
test/test.fundamentals.service.node.ts
Normal file
287
test/test.fundamentals.service.node.ts
Normal file
@@ -0,0 +1,287 @@
|
|||||||
|
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||||
|
import * as opendata from '../ts/index.js';
|
||||||
|
|
||||||
|
const TEST_USER_AGENT = 'fin.cx test@fin.cx';
|
||||||
|
|
||||||
|
tap.test('FundamentalsService - Provider Registration', async () => {
|
||||||
|
const service = new opendata.FundamentalsService();
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should register provider', async () => {
|
||||||
|
service.register(provider);
|
||||||
|
|
||||||
|
const registered = service.getProvider('SEC EDGAR');
|
||||||
|
expect(registered).toBeDefined();
|
||||||
|
expect(registered?.name).toEqual('SEC EDGAR');
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should get all providers', async () => {
|
||||||
|
const providers = service.getAllProviders();
|
||||||
|
expect(providers.length).toBeGreaterThan(0);
|
||||||
|
expect(providers[0].name).toEqual('SEC EDGAR');
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should get enabled providers', async () => {
|
||||||
|
const providers = service.getEnabledProviders();
|
||||||
|
expect(providers.length).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should unregister provider', async () => {
|
||||||
|
service.unregister('SEC EDGAR');
|
||||||
|
|
||||||
|
const registered = service.getProvider('SEC EDGAR');
|
||||||
|
expect(registered).toBeUndefined();
|
||||||
|
|
||||||
|
// Re-register for other tests
|
||||||
|
service.register(provider);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('FundamentalsService - Fetch Fundamentals', async () => {
|
||||||
|
const service = new opendata.FundamentalsService();
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.register(provider);
|
||||||
|
|
||||||
|
await tap.test('should fetch fundamentals for single ticker', async () => {
|
||||||
|
const fundamentals = await service.getFundamentals('AAPL');
|
||||||
|
|
||||||
|
expect(fundamentals).toBeDefined();
|
||||||
|
expect(fundamentals.ticker).toEqual('AAPL');
|
||||||
|
expect(fundamentals.companyName).toEqual('Apple Inc.');
|
||||||
|
expect(fundamentals.provider).toEqual('SEC EDGAR');
|
||||||
|
expect(fundamentals.earningsPerShareDiluted).toBeGreaterThan(0);
|
||||||
|
expect(fundamentals.sharesOutstanding).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n📊 Fetched via Service:');
|
||||||
|
console.log(` ${fundamentals.ticker}: ${fundamentals.companyName}`);
|
||||||
|
console.log(` EPS: $${fundamentals.earningsPerShareDiluted?.toFixed(2)}`);
|
||||||
|
console.log(` Shares: ${(fundamentals.sharesOutstanding! / 1_000_000_000).toFixed(2)}B`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch fundamentals for multiple tickers', async () => {
|
||||||
|
const fundamentalsList = await service.getBatchFundamentals(['AAPL', 'MSFT']);
|
||||||
|
|
||||||
|
expect(fundamentalsList).toBeInstanceOf(Array);
|
||||||
|
expect(fundamentalsList.length).toEqual(2);
|
||||||
|
|
||||||
|
const apple = fundamentalsList.find(f => f.ticker === 'AAPL');
|
||||||
|
const msft = fundamentalsList.find(f => f.ticker === 'MSFT');
|
||||||
|
|
||||||
|
expect(apple).toBeDefined();
|
||||||
|
expect(msft).toBeDefined();
|
||||||
|
expect(apple!.companyName).toEqual('Apple Inc.');
|
||||||
|
expect(msft!.companyName).toContain('Microsoft');
|
||||||
|
|
||||||
|
console.log('\n📊 Batch Fetch via Service:');
|
||||||
|
fundamentalsList.forEach(f => {
|
||||||
|
console.log(` ${f.ticker}: ${f.companyName} - EPS: $${f.earningsPerShareDiluted?.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('FundamentalsService - Caching', async () => {
|
||||||
|
const service = new opendata.FundamentalsService({
|
||||||
|
ttl: 60000, // 60 seconds for testing
|
||||||
|
maxEntries: 100
|
||||||
|
});
|
||||||
|
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.register(provider);
|
||||||
|
|
||||||
|
await tap.test('should cache fundamentals data', async () => {
|
||||||
|
// Clear cache first
|
||||||
|
service.clearCache();
|
||||||
|
|
||||||
|
let stats = service.getCacheStats();
|
||||||
|
expect(stats.size).toEqual(0);
|
||||||
|
|
||||||
|
// First fetch (should hit API)
|
||||||
|
const start1 = Date.now();
|
||||||
|
await service.getFundamentals('AAPL');
|
||||||
|
const duration1 = Date.now() - start1;
|
||||||
|
|
||||||
|
stats = service.getCacheStats();
|
||||||
|
expect(stats.size).toEqual(1);
|
||||||
|
|
||||||
|
// Second fetch (should hit cache - much faster)
|
||||||
|
const start2 = Date.now();
|
||||||
|
await service.getFundamentals('AAPL');
|
||||||
|
const duration2 = Date.now() - start2;
|
||||||
|
|
||||||
|
expect(duration2).toBeLessThan(duration1);
|
||||||
|
|
||||||
|
console.log('\n⚡ Cache Performance:');
|
||||||
|
console.log(` First fetch: ${duration1}ms`);
|
||||||
|
console.log(` Cached fetch: ${duration2}ms`);
|
||||||
|
console.log(` Speedup: ${Math.round(duration1 / duration2)}x`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should respect cache TTL', async () => {
|
||||||
|
// Set very short TTL
|
||||||
|
service.setCacheTTL(100); // 100ms
|
||||||
|
|
||||||
|
// Fetch and cache
|
||||||
|
await service.getFundamentals('MSFT');
|
||||||
|
|
||||||
|
// Wait for TTL to expire
|
||||||
|
await new Promise(resolve => setTimeout(resolve, 150));
|
||||||
|
|
||||||
|
// This should fetch again (cache expired)
|
||||||
|
const stats = service.getCacheStats();
|
||||||
|
console.log(`\n⏱️ Cache TTL: ${stats.ttl}ms`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should clear cache', async () => {
|
||||||
|
service.clearCache();
|
||||||
|
|
||||||
|
const stats = service.getCacheStats();
|
||||||
|
expect(stats.size).toEqual(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('FundamentalsService - Price Enrichment', async () => {
|
||||||
|
const service = new opendata.FundamentalsService();
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.register(provider);
|
||||||
|
|
||||||
|
await tap.test('should enrich fundamentals with price to calculate market cap', async () => {
|
||||||
|
const fundamentals = await service.getFundamentals('AAPL');
|
||||||
|
|
||||||
|
// Simulate current price
|
||||||
|
const currentPrice = 270.37;
|
||||||
|
|
||||||
|
const enriched = await service.enrichWithPrice(fundamentals, currentPrice);
|
||||||
|
|
||||||
|
expect(enriched.marketCap).toBeDefined();
|
||||||
|
expect(enriched.priceToEarnings).toBeDefined();
|
||||||
|
expect(enriched.priceToBook).toBeDefined();
|
||||||
|
|
||||||
|
expect(enriched.marketCap).toBeGreaterThan(0);
|
||||||
|
expect(enriched.priceToEarnings).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n💰 Enriched with Price ($270.37):');
|
||||||
|
console.log(` Market Cap: $${(enriched.marketCap! / 1_000_000_000_000).toFixed(2)}T`);
|
||||||
|
console.log(` P/E Ratio: ${enriched.priceToEarnings!.toFixed(2)}`);
|
||||||
|
console.log(` P/B Ratio: ${enriched.priceToBook?.toFixed(2) || 'N/A'}`);
|
||||||
|
|
||||||
|
// Verify calculations
|
||||||
|
const expectedMarketCap = fundamentals.sharesOutstanding! * currentPrice;
|
||||||
|
expect(Math.abs(enriched.marketCap! - expectedMarketCap)).toBeLessThan(1); // Allow for rounding
|
||||||
|
|
||||||
|
const expectedPE = currentPrice / fundamentals.earningsPerShareDiluted!;
|
||||||
|
expect(Math.abs(enriched.priceToEarnings! - expectedPE)).toBeLessThan(0.01);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should enrich batch fundamentals with prices', async () => {
|
||||||
|
const fundamentalsList = await service.getBatchFundamentals(['AAPL', 'MSFT']);
|
||||||
|
|
||||||
|
const priceMap = new Map<string, number>([
|
||||||
|
['AAPL', 270.37],
|
||||||
|
['MSFT', 425.50]
|
||||||
|
]);
|
||||||
|
|
||||||
|
const enriched = await service.enrichBatchWithPrices(fundamentalsList, priceMap);
|
||||||
|
|
||||||
|
expect(enriched.length).toEqual(2);
|
||||||
|
|
||||||
|
const apple = enriched.find(f => f.ticker === 'AAPL')!;
|
||||||
|
const msft = enriched.find(f => f.ticker === 'MSFT')!;
|
||||||
|
|
||||||
|
expect(apple.marketCap).toBeGreaterThan(0);
|
||||||
|
expect(msft.marketCap).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n💰 Batch Enrichment:');
|
||||||
|
console.log(` AAPL: Market Cap $${(apple.marketCap! / 1_000_000_000_000).toFixed(2)}T, P/E ${apple.priceToEarnings!.toFixed(2)}`);
|
||||||
|
console.log(` MSFT: Market Cap $${(msft.marketCap! / 1_000_000_000_000).toFixed(2)}T, P/E ${msft.priceToEarnings!.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('FundamentalsService - Provider Health', async () => {
|
||||||
|
const service = new opendata.FundamentalsService();
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.register(provider);
|
||||||
|
|
||||||
|
await tap.test('should check provider health', async () => {
|
||||||
|
const health = await service.checkProvidersHealth();
|
||||||
|
|
||||||
|
expect(health.size).toEqual(1);
|
||||||
|
expect(health.get('SEC EDGAR')).toBe(true);
|
||||||
|
|
||||||
|
console.log('\n💚 Provider Health:');
|
||||||
|
health.forEach((isHealthy, name) => {
|
||||||
|
console.log(` ${name}: ${isHealthy ? '✅ Healthy' : '❌ Unhealthy'}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('FundamentalsService - Provider Statistics', async () => {
|
||||||
|
const service = new opendata.FundamentalsService();
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.register(provider);
|
||||||
|
|
||||||
|
await tap.test('should track provider statistics', async () => {
|
||||||
|
// Make some requests
|
||||||
|
await service.getFundamentals('AAPL');
|
||||||
|
await service.getFundamentals('MSFT');
|
||||||
|
|
||||||
|
const stats = service.getProviderStats();
|
||||||
|
|
||||||
|
expect(stats.size).toEqual(1);
|
||||||
|
|
||||||
|
const secStats = stats.get('SEC EDGAR');
|
||||||
|
expect(secStats).toBeDefined();
|
||||||
|
expect(secStats!.successCount).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n📈 Provider Stats:');
|
||||||
|
console.log(` Success Count: ${secStats!.successCount}`);
|
||||||
|
console.log(` Error Count: ${secStats!.errorCount}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('FundamentalsService - Error Handling', async () => {
|
||||||
|
const service = new opendata.FundamentalsService();
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.register(provider);
|
||||||
|
|
||||||
|
await tap.test('should throw error for invalid ticker', async () => {
|
||||||
|
try {
|
||||||
|
await service.getFundamentals('INVALIDTICKER123456');
|
||||||
|
throw new Error('Should have thrown error');
|
||||||
|
} catch (error) {
|
||||||
|
expect(error.message).toContain('CIK not found');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should throw error when no providers available', async () => {
|
||||||
|
const emptyService = new opendata.FundamentalsService();
|
||||||
|
|
||||||
|
try {
|
||||||
|
await emptyService.getFundamentals('AAPL');
|
||||||
|
throw new Error('Should have thrown error');
|
||||||
|
} catch (error) {
|
||||||
|
expect(error.message).toContain('No fundamentals providers available');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
export default tap.start();
|
||||||
@@ -151,7 +151,7 @@ tap.test('should handle invalid ticker gracefully', async () => {
|
|||||||
await stockService.getPrice({ ticker: invalidTicker });
|
await stockService.getPrice({ ticker: invalidTicker });
|
||||||
throw new Error('Should have thrown an error for invalid ticker');
|
throw new Error('Should have thrown an error for invalid ticker');
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
expect(error.message).toInclude('Failed to fetch price');
|
expect(error.message).toInclude('Failed to fetch');
|
||||||
console.log('✓ Invalid ticker handled correctly');
|
console.log('✓ Invalid ticker handled correctly');
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -215,19 +215,20 @@ tap.test('should test direct provider methods', async () => {
|
|||||||
expect(available).toEqual(true);
|
expect(available).toEqual(true);
|
||||||
console.log(' ✓ isAvailable() returned true');
|
console.log(' ✓ isAvailable() returned true');
|
||||||
|
|
||||||
// Test fetchPrice directly
|
// Test fetchData for single ticker
|
||||||
const price = await marketstackProvider.fetchPrice({ ticker: 'MSFT' });
|
const price = await marketstackProvider.fetchData({ type: 'current', ticker: 'MSFT' }) as opendata.IStockPrice;
|
||||||
expect(price.ticker).toEqual('MSFT');
|
expect(price.ticker).toEqual('MSFT');
|
||||||
expect(price.provider).toEqual('Marketstack');
|
expect(price.provider).toEqual('Marketstack');
|
||||||
expect(price.price).toBeGreaterThan(0);
|
expect(price.price).toBeGreaterThan(0);
|
||||||
console.log(` ✓ fetchPrice() for MSFT: $${price.price}`);
|
console.log(` ✓ fetchData (current) for MSFT: $${price.price}`);
|
||||||
|
|
||||||
// Test fetchPrices directly
|
// Test fetchData for batch
|
||||||
const prices = await marketstackProvider.fetchPrices({
|
const prices = await marketstackProvider.fetchData({
|
||||||
|
type: 'batch',
|
||||||
tickers: ['AAPL', 'GOOGL']
|
tickers: ['AAPL', 'GOOGL']
|
||||||
});
|
}) as opendata.IStockPrice[];
|
||||||
expect(prices.length).toBeGreaterThan(0);
|
expect(prices.length).toBeGreaterThan(0);
|
||||||
console.log(` ✓ fetchPrices() returned ${prices.length} prices`);
|
console.log(` ✓ fetchData (batch) returned ${prices.length} prices`);
|
||||||
|
|
||||||
for (const p of prices) {
|
for (const p of prices) {
|
||||||
console.log(` ${p.ticker}: $${p.price}`);
|
console.log(` ${p.ticker}: $${p.price}`);
|
||||||
@@ -252,9 +253,10 @@ tap.test('should fetch sample EOD data', async () => {
|
|||||||
];
|
];
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const prices = await marketstackProvider.fetchPrices({
|
const prices = await marketstackProvider.fetchData({
|
||||||
|
type: 'batch',
|
||||||
tickers: sampleTickers.map(t => t.ticker)
|
tickers: sampleTickers.map(t => t.ticker)
|
||||||
});
|
}) as opendata.IStockPrice[];
|
||||||
|
|
||||||
const priceMap = new Map(prices.map(p => [p.ticker, p]));
|
const priceMap = new Map(prices.map(p => [p.ticker, p]));
|
||||||
|
|
||||||
@@ -302,4 +304,269 @@ tap.test('should clear cache', async () => {
|
|||||||
expect(price).not.toEqual(undefined);
|
expect(price).not.toEqual(undefined);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Phase 1 Feature Tests
|
||||||
|
|
||||||
|
tap.test('should fetch data using new unified API (current price)', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n🎯 Testing Phase 1: Unified getData API');
|
||||||
|
|
||||||
|
const price = await stockService.getData({
|
||||||
|
type: 'current',
|
||||||
|
ticker: 'MSFT'
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(price).not.toEqual(undefined);
|
||||||
|
expect((price as opendata.IStockPrice).ticker).toEqual('MSFT');
|
||||||
|
expect((price as opendata.IStockPrice).dataType).toEqual('eod');
|
||||||
|
expect((price as opendata.IStockPrice).fetchedAt).toBeInstanceOf(Date);
|
||||||
|
|
||||||
|
console.log(`✓ Fetched current price: $${(price as opendata.IStockPrice).price}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should fetch historical data with date range', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n📅 Testing Phase 1: Historical Data Retrieval');
|
||||||
|
|
||||||
|
const fromDate = new Date('2024-12-01');
|
||||||
|
const toDate = new Date('2024-12-31');
|
||||||
|
|
||||||
|
const prices = await stockService.getData({
|
||||||
|
type: 'historical',
|
||||||
|
ticker: 'AAPL',
|
||||||
|
from: fromDate,
|
||||||
|
to: toDate,
|
||||||
|
sort: 'DESC'
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(prices).toBeArray();
|
||||||
|
expect((prices as opendata.IStockPrice[]).length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log(`✓ Fetched ${(prices as opendata.IStockPrice[]).length} historical prices`);
|
||||||
|
|
||||||
|
// Verify all data types are 'eod'
|
||||||
|
for (const price of (prices as opendata.IStockPrice[])) {
|
||||||
|
expect(price.dataType).toEqual('eod');
|
||||||
|
expect(price.ticker).toEqual('AAPL');
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('✓ All prices have correct dataType');
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should include OHLCV data in responses', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n📊 Testing Phase 1: OHLCV Data');
|
||||||
|
|
||||||
|
const price = await stockService.getData({
|
||||||
|
type: 'current',
|
||||||
|
ticker: 'GOOGL'
|
||||||
|
});
|
||||||
|
|
||||||
|
const stockPrice = price as opendata.IStockPrice;
|
||||||
|
|
||||||
|
// Verify OHLCV fields are present
|
||||||
|
expect(stockPrice.open).not.toEqual(undefined);
|
||||||
|
expect(stockPrice.high).not.toEqual(undefined);
|
||||||
|
expect(stockPrice.low).not.toEqual(undefined);
|
||||||
|
expect(stockPrice.price).not.toEqual(undefined); // close
|
||||||
|
expect(stockPrice.volume).not.toEqual(undefined);
|
||||||
|
|
||||||
|
console.log(`✓ OHLCV Data:`);
|
||||||
|
console.log(` Open: $${stockPrice.open}`);
|
||||||
|
console.log(` High: $${stockPrice.high}`);
|
||||||
|
console.log(` Low: $${stockPrice.low}`);
|
||||||
|
console.log(` Close: $${stockPrice.price}`);
|
||||||
|
console.log(` Volume: ${stockPrice.volume?.toLocaleString()}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should support exchange filtering', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n🌍 Testing Phase 1: Exchange Filtering');
|
||||||
|
|
||||||
|
// Note: This test may fail if the exchange doesn't have data for the ticker
|
||||||
|
// In production, you'd test with tickers known to exist on specific exchanges
|
||||||
|
try {
|
||||||
|
const price = await stockService.getData({
|
||||||
|
type: 'current',
|
||||||
|
ticker: 'AAPL',
|
||||||
|
exchange: 'XNAS' // NASDAQ
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(price).not.toEqual(undefined);
|
||||||
|
console.log(`✓ Successfully filtered by exchange: ${(price as opendata.IStockPrice).exchange}`);
|
||||||
|
} catch (error) {
|
||||||
|
console.log('⚠️ Exchange filtering test inconclusive (may need tier upgrade)');
|
||||||
|
expect(true).toEqual(true); // Don't fail test
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should verify smart caching with historical data', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n💾 Testing Phase 1: Smart Caching');
|
||||||
|
|
||||||
|
const fromDate = new Date('2024-11-01');
|
||||||
|
const toDate = new Date('2024-11-30');
|
||||||
|
|
||||||
|
// First request - should hit API
|
||||||
|
const start1 = Date.now();
|
||||||
|
const prices1 = await stockService.getData({
|
||||||
|
type: 'historical',
|
||||||
|
ticker: 'TSLA',
|
||||||
|
from: fromDate,
|
||||||
|
to: toDate
|
||||||
|
});
|
||||||
|
const duration1 = Date.now() - start1;
|
||||||
|
|
||||||
|
// Second request - should be cached (historical data cached forever)
|
||||||
|
const start2 = Date.now();
|
||||||
|
const prices2 = await stockService.getData({
|
||||||
|
type: 'historical',
|
||||||
|
ticker: 'TSLA',
|
||||||
|
from: fromDate,
|
||||||
|
to: toDate
|
||||||
|
});
|
||||||
|
const duration2 = Date.now() - start2;
|
||||||
|
|
||||||
|
expect((prices1 as opendata.IStockPrice[]).length).toEqual((prices2 as opendata.IStockPrice[]).length);
|
||||||
|
expect(duration2).toBeLessThan(duration1); // Cached should be much faster
|
||||||
|
|
||||||
|
console.log(`✓ First request: ${duration1}ms (API call)`);
|
||||||
|
console.log(`✓ Second request: ${duration2}ms (cached)`);
|
||||||
|
console.log(`✓ Speed improvement: ${Math.round((duration1 / duration2) * 10) / 10}x faster`);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Company Name Feature Tests
|
||||||
|
|
||||||
|
tap.test('should include company name in single price request', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n🏢 Testing Company Name Feature: Single Request');
|
||||||
|
|
||||||
|
const price = await stockService.getPrice({ ticker: 'AAPL' });
|
||||||
|
|
||||||
|
expect(price.companyName).not.toEqual(undefined);
|
||||||
|
expect(typeof price.companyName).toEqual('string');
|
||||||
|
expect(price.companyName).toInclude('Apple');
|
||||||
|
|
||||||
|
console.log(`✓ Company name retrieved: "${price.companyName}"`);
|
||||||
|
console.log(` Ticker: ${price.ticker}`);
|
||||||
|
console.log(` Price: $${price.price}`);
|
||||||
|
console.log(` Company: ${price.companyName}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should include company names in batch price request', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n🏢 Testing Company Name Feature: Batch Request');
|
||||||
|
|
||||||
|
const prices = await stockService.getPrices({
|
||||||
|
tickers: ['AAPL', 'MSFT', 'GOOGL']
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(prices).toBeArray();
|
||||||
|
expect(prices.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log(`✓ Fetched ${prices.length} prices with company names:`);
|
||||||
|
|
||||||
|
for (const price of prices) {
|
||||||
|
expect(price.companyName).not.toEqual(undefined);
|
||||||
|
expect(typeof price.companyName).toEqual('string');
|
||||||
|
console.log(` ${price.ticker.padEnd(6)} - ${price.companyName}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should include company name in historical data', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n🏢 Testing Company Name Feature: Historical Data');
|
||||||
|
|
||||||
|
const prices = await stockService.getData({
|
||||||
|
type: 'historical',
|
||||||
|
ticker: 'TSLA',
|
||||||
|
from: new Date('2025-10-01'),
|
||||||
|
to: new Date('2025-10-05')
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(prices).toBeArray();
|
||||||
|
const historicalPrices = prices as opendata.IStockPrice[];
|
||||||
|
expect(historicalPrices.length).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// All historical records should have the same company name
|
||||||
|
for (const price of historicalPrices) {
|
||||||
|
expect(price.companyName).not.toEqual(undefined);
|
||||||
|
expect(typeof price.companyName).toEqual('string');
|
||||||
|
}
|
||||||
|
|
||||||
|
const firstPrice = historicalPrices[0];
|
||||||
|
console.log(`✓ Historical records include company name: "${firstPrice.companyName}"`);
|
||||||
|
console.log(` Ticker: ${firstPrice.ticker}`);
|
||||||
|
console.log(` Records: ${historicalPrices.length}`);
|
||||||
|
console.log(` Date range: ${historicalPrices[historicalPrices.length - 1].timestamp.toISOString().split('T')[0]} to ${firstPrice.timestamp.toISOString().split('T')[0]}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('should verify company name is included with zero extra API calls', async () => {
|
||||||
|
if (!marketstackProvider) {
|
||||||
|
console.log('⚠️ Skipping - Marketstack provider not initialized');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log('\n⚡ Testing Company Name Efficiency: Zero Extra API Calls');
|
||||||
|
|
||||||
|
// Clear cache to ensure we're making fresh API calls
|
||||||
|
stockService.clearCache();
|
||||||
|
|
||||||
|
// Single request timing
|
||||||
|
const start1 = Date.now();
|
||||||
|
const singlePrice = await stockService.getPrice({ ticker: 'AMZN' });
|
||||||
|
const duration1 = Date.now() - start1;
|
||||||
|
|
||||||
|
expect(singlePrice.companyName).not.toEqual(undefined);
|
||||||
|
|
||||||
|
// Batch request timing
|
||||||
|
stockService.clearCache();
|
||||||
|
const start2 = Date.now();
|
||||||
|
const batchPrices = await stockService.getPrices({ tickers: ['NVDA', 'AMD', 'INTC'] });
|
||||||
|
const duration2 = Date.now() - start2;
|
||||||
|
|
||||||
|
for (const price of batchPrices) {
|
||||||
|
expect(price.companyName).not.toEqual(undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`✓ Single request (with company name): ${duration1}ms`);
|
||||||
|
console.log(`✓ Batch request (with company names): ${duration2}ms`);
|
||||||
|
console.log(`✓ Company names included in standard EOD response - zero extra calls!`);
|
||||||
|
console.log(` Single: ${singlePrice.ticker} - "${singlePrice.companyName}"`);
|
||||||
|
for (const price of batchPrices) {
|
||||||
|
console.log(` Batch: ${price.ticker} - "${price.companyName}"`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
export default tap.start();
|
export default tap.start();
|
||||||
|
|||||||
261
test/test.secedgar.provider.node.ts
Normal file
261
test/test.secedgar.provider.node.ts
Normal file
@@ -0,0 +1,261 @@
|
|||||||
|
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||||
|
import * as opendata from '../ts/index.js';
|
||||||
|
|
||||||
|
// Test configuration
|
||||||
|
const TEST_USER_AGENT = 'fin.cx test@fin.cx';
|
||||||
|
const TEST_TICKER = 'AAPL'; // Apple Inc - well-known test case
|
||||||
|
const RATE_LIMIT_DELAY = 150; // 150ms between requests (< 10 req/sec)
|
||||||
|
|
||||||
|
tap.test('SEC EDGAR Provider - Constructor', async () => {
|
||||||
|
await tap.test('should create provider with valid User-Agent', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(provider.name).toEqual('SEC EDGAR');
|
||||||
|
expect(provider.priority).toEqual(100);
|
||||||
|
expect(provider.requiresAuth).toBe(false);
|
||||||
|
expect(provider.rateLimit?.requestsPerMinute).toEqual(600);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should throw error if User-Agent is missing', async () => {
|
||||||
|
expect(() => {
|
||||||
|
new opendata.SecEdgarProvider({
|
||||||
|
userAgent: ''
|
||||||
|
});
|
||||||
|
}).toThrow('User-Agent is required');
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should throw error if User-Agent format is invalid', async () => {
|
||||||
|
expect(() => {
|
||||||
|
new opendata.SecEdgarProvider({
|
||||||
|
userAgent: 'InvalidFormat'
|
||||||
|
});
|
||||||
|
}).toThrow('Invalid User-Agent format');
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should accept valid User-Agent with space and email', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: 'MyCompany contact@example.com'
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(provider).toBeInstanceOf(opendata.SecEdgarProvider);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('SEC EDGAR Provider - Availability', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should report as available', async () => {
|
||||||
|
const isAvailable = await provider.isAvailable();
|
||||||
|
expect(isAvailable).toBe(true);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('SEC EDGAR Provider - Fetch Fundamentals', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT,
|
||||||
|
timeout: 30000
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch fundamentals for Apple (AAPL)', async () => {
|
||||||
|
const fundamentals = await provider.fetchData({
|
||||||
|
type: 'fundamentals-current',
|
||||||
|
ticker: TEST_TICKER
|
||||||
|
});
|
||||||
|
|
||||||
|
// Verify structure
|
||||||
|
expect(fundamentals).toBeDefined();
|
||||||
|
expect(fundamentals).not.toBeInstanceOf(Array);
|
||||||
|
|
||||||
|
const data = fundamentals as opendata.IStockFundamentals;
|
||||||
|
|
||||||
|
// Basic fields
|
||||||
|
expect(data.ticker).toEqual('AAPL');
|
||||||
|
expect(data.cik).toBeDefined();
|
||||||
|
expect(data.companyName).toEqual('Apple Inc.');
|
||||||
|
expect(data.provider).toEqual('SEC EDGAR');
|
||||||
|
expect(data.timestamp).toBeInstanceOf(Date);
|
||||||
|
expect(data.fetchedAt).toBeInstanceOf(Date);
|
||||||
|
|
||||||
|
// Financial metrics (Apple should have all of these)
|
||||||
|
expect(data.earningsPerShareDiluted).toBeDefined();
|
||||||
|
expect(data.earningsPerShareDiluted).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
expect(data.sharesOutstanding).toBeDefined();
|
||||||
|
expect(data.sharesOutstanding).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
expect(data.revenue).toBeDefined();
|
||||||
|
expect(data.revenue).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
expect(data.netIncome).toBeDefined();
|
||||||
|
expect(data.assets).toBeDefined();
|
||||||
|
expect(data.assets).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
expect(data.liabilities).toBeDefined();
|
||||||
|
expect(data.stockholdersEquity).toBeDefined();
|
||||||
|
|
||||||
|
// Metadata
|
||||||
|
expect(data.fiscalYear).toBeDefined();
|
||||||
|
|
||||||
|
console.log('\n📊 Sample Apple Fundamentals:');
|
||||||
|
console.log(` Company: ${data.companyName} (CIK: ${data.cik})`);
|
||||||
|
console.log(` EPS (Diluted): $${data.earningsPerShareDiluted?.toFixed(2)}`);
|
||||||
|
console.log(` Shares Outstanding: ${(data.sharesOutstanding! / 1_000_000_000).toFixed(2)}B`);
|
||||||
|
console.log(` Revenue: $${(data.revenue! / 1_000_000_000).toFixed(2)}B`);
|
||||||
|
console.log(` Net Income: $${(data.netIncome! / 1_000_000_000).toFixed(2)}B`);
|
||||||
|
console.log(` Assets: $${(data.assets! / 1_000_000_000).toFixed(2)}B`);
|
||||||
|
console.log(` Fiscal Year: ${data.fiscalYear}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should throw error for invalid ticker', async () => {
|
||||||
|
try {
|
||||||
|
await provider.fetchData({
|
||||||
|
type: 'fundamentals-current',
|
||||||
|
ticker: 'INVALIDTICKER123456'
|
||||||
|
});
|
||||||
|
throw new Error('Should have thrown error');
|
||||||
|
} catch (error) {
|
||||||
|
expect(error.message).toContain('CIK not found');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('SEC EDGAR Provider - Batch Fetch', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT,
|
||||||
|
timeout: 30000
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch fundamentals for multiple tickers', async () => {
|
||||||
|
const result = await provider.fetchData({
|
||||||
|
type: 'fundamentals-batch',
|
||||||
|
tickers: ['AAPL', 'MSFT']
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result).toBeInstanceOf(Array);
|
||||||
|
|
||||||
|
const fundamentalsList = result as opendata.IStockFundamentals[];
|
||||||
|
expect(fundamentalsList.length).toEqual(2);
|
||||||
|
|
||||||
|
// Check Apple
|
||||||
|
const apple = fundamentalsList.find(f => f.ticker === 'AAPL');
|
||||||
|
expect(apple).toBeDefined();
|
||||||
|
expect(apple!.companyName).toEqual('Apple Inc.');
|
||||||
|
expect(apple!.earningsPerShareDiluted).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
// Check Microsoft
|
||||||
|
const microsoft = fundamentalsList.find(f => f.ticker === 'MSFT');
|
||||||
|
expect(microsoft).toBeDefined();
|
||||||
|
expect(microsoft!.companyName).toContain('Microsoft');
|
||||||
|
expect(microsoft!.earningsPerShareDiluted).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n📊 Batch Fetch Results:');
|
||||||
|
fundamentalsList.forEach(f => {
|
||||||
|
console.log(` ${f.ticker}: ${f.companyName} - EPS: $${f.earningsPerShareDiluted?.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('SEC EDGAR Provider - CIK Caching', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should cache CIK lookups', async () => {
|
||||||
|
// Clear cache first
|
||||||
|
provider.clearCache();
|
||||||
|
|
||||||
|
let stats = provider.getCacheStats();
|
||||||
|
expect(stats.cikCacheSize).toEqual(0);
|
||||||
|
|
||||||
|
// First fetch (should populate cache)
|
||||||
|
await provider.fetchData({
|
||||||
|
type: 'fundamentals-current',
|
||||||
|
ticker: 'AAPL'
|
||||||
|
});
|
||||||
|
|
||||||
|
stats = provider.getCacheStats();
|
||||||
|
expect(stats.cikCacheSize).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log(`\n💾 CIK Cache: ${stats.cikCacheSize} entries`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should clear cache', async () => {
|
||||||
|
provider.clearCache();
|
||||||
|
|
||||||
|
const stats = provider.getCacheStats();
|
||||||
|
expect(stats.cikCacheSize).toEqual(0);
|
||||||
|
expect(stats.hasTickerList).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('SEC EDGAR Provider - Rate Limiting', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should handle multiple rapid requests without exceeding rate limit', async () => {
|
||||||
|
// Make 5 requests in succession
|
||||||
|
// Rate limiter should ensure we don't exceed 10 req/sec
|
||||||
|
const tickers = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'META'];
|
||||||
|
const startTime = Date.now();
|
||||||
|
|
||||||
|
const promises = tickers.map(ticker =>
|
||||||
|
provider.fetchData({
|
||||||
|
type: 'fundamentals-current',
|
||||||
|
ticker
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
const results = await Promise.all(promises);
|
||||||
|
const duration = Date.now() - startTime;
|
||||||
|
|
||||||
|
expect(results.length).toEqual(5);
|
||||||
|
console.log(`\n⏱️ 5 requests completed in ${duration}ms (avg: ${Math.round(duration / 5)}ms/request)`);
|
||||||
|
|
||||||
|
// Verify all results are valid
|
||||||
|
results.forEach((result, index) => {
|
||||||
|
expect(result).toBeDefined();
|
||||||
|
const data = result as opendata.IStockFundamentals;
|
||||||
|
expect(data.ticker).toEqual(tickers[index]);
|
||||||
|
expect(data.earningsPerShareDiluted).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('SEC EDGAR Provider - Market Cap Calculation', async () => {
|
||||||
|
const provider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should provide data needed for market cap calculation', async () => {
|
||||||
|
const fundamentals = await provider.fetchData({
|
||||||
|
type: 'fundamentals-current',
|
||||||
|
ticker: 'AAPL'
|
||||||
|
}) as opendata.IStockFundamentals;
|
||||||
|
|
||||||
|
expect(fundamentals.sharesOutstanding).toBeDefined();
|
||||||
|
expect(fundamentals.earningsPerShareDiluted).toBeDefined();
|
||||||
|
|
||||||
|
// Simulate current price (in real usage, this comes from price provider)
|
||||||
|
const simulatedPrice = 270.37;
|
||||||
|
|
||||||
|
// Calculate market cap
|
||||||
|
const marketCap = fundamentals.sharesOutstanding! * simulatedPrice;
|
||||||
|
const pe = simulatedPrice / fundamentals.earningsPerShareDiluted!;
|
||||||
|
|
||||||
|
console.log('\n💰 Calculated Metrics (with simulated price $270.37):');
|
||||||
|
console.log(` Shares Outstanding: ${(fundamentals.sharesOutstanding! / 1_000_000_000).toFixed(2)}B`);
|
||||||
|
console.log(` Market Cap: $${(marketCap / 1_000_000_000_000).toFixed(2)}T`);
|
||||||
|
console.log(` EPS: $${fundamentals.earningsPerShareDiluted!.toFixed(2)}`);
|
||||||
|
console.log(` P/E Ratio: ${pe.toFixed(2)}`);
|
||||||
|
|
||||||
|
expect(marketCap).toBeGreaterThan(0);
|
||||||
|
expect(pe).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
export default tap.start();
|
||||||
418
test/test.stockdata.service.node.ts
Normal file
418
test/test.stockdata.service.node.ts
Normal file
@@ -0,0 +1,418 @@
|
|||||||
|
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||||
|
import * as opendata from '../ts/index.js';
|
||||||
|
|
||||||
|
const TEST_USER_AGENT = 'fin.cx test@fin.cx';
|
||||||
|
|
||||||
|
tap.test('StockDataService - Basic Setup', async () => {
|
||||||
|
await tap.test('should create StockDataService instance', async () => {
|
||||||
|
const service = new opendata.StockDataService({
|
||||||
|
cache: {
|
||||||
|
priceTTL: 60000, // 1 minute for testing
|
||||||
|
fundamentalsTTL: 120000, // 2 minutes for testing
|
||||||
|
maxEntries: 100
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(service).toBeInstanceOf(opendata.StockDataService);
|
||||||
|
|
||||||
|
const stats = service.getCacheStats();
|
||||||
|
expect(stats.priceCache.ttl).toEqual(60000);
|
||||||
|
expect(stats.fundamentalsCache.ttl).toEqual(120000);
|
||||||
|
expect(stats.maxEntries).toEqual(100);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('StockDataService - Provider Registration', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
|
||||||
|
await tap.test('should register price provider', async () => {
|
||||||
|
const yahooProvider = new opendata.YahooFinanceProvider();
|
||||||
|
service.registerPriceProvider(yahooProvider);
|
||||||
|
|
||||||
|
const providers = service.getPriceProviders();
|
||||||
|
expect(providers.length).toEqual(1);
|
||||||
|
expect(providers[0].name).toEqual('Yahoo Finance');
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should register fundamentals provider', async () => {
|
||||||
|
const secProvider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
service.registerFundamentalsProvider(secProvider);
|
||||||
|
|
||||||
|
const providers = service.getFundamentalsProviders();
|
||||||
|
expect(providers.length).toEqual(1);
|
||||||
|
expect(providers[0].name).toEqual('SEC EDGAR');
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should unregister providers', async () => {
|
||||||
|
service.unregisterPriceProvider('Yahoo Finance');
|
||||||
|
service.unregisterFundamentalsProvider('SEC EDGAR');
|
||||||
|
|
||||||
|
expect(service.getPriceProviders().length).toEqual(0);
|
||||||
|
expect(service.getFundamentalsProviders().length).toEqual(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('StockDataService - Price Fetching', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
const yahooProvider = new opendata.YahooFinanceProvider();
|
||||||
|
service.registerPriceProvider(yahooProvider);
|
||||||
|
|
||||||
|
await tap.test('should fetch single price', async () => {
|
||||||
|
const price = await service.getPrice('AAPL');
|
||||||
|
|
||||||
|
expect(price).toBeDefined();
|
||||||
|
expect(price.ticker).toEqual('AAPL');
|
||||||
|
expect(price.price).toBeGreaterThan(0);
|
||||||
|
expect(price.provider).toEqual('Yahoo Finance');
|
||||||
|
expect(price.timestamp).toBeInstanceOf(Date);
|
||||||
|
|
||||||
|
console.log(`\n💵 Single Price: ${price.ticker} = $${price.price.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch batch prices', async () => {
|
||||||
|
const prices = await service.getPrices(['AAPL', 'MSFT', 'GOOGL']);
|
||||||
|
|
||||||
|
expect(prices).toBeInstanceOf(Array);
|
||||||
|
expect(prices.length).toBeGreaterThan(0);
|
||||||
|
expect(prices.length).toBeLessThanOrEqual(3);
|
||||||
|
|
||||||
|
console.log('\n💵 Batch Prices:');
|
||||||
|
prices.forEach(p => {
|
||||||
|
console.log(` ${p.ticker}: $${p.price.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should cache prices', async () => {
|
||||||
|
// Clear cache
|
||||||
|
service.clearCache();
|
||||||
|
|
||||||
|
const stats1 = service.getCacheStats();
|
||||||
|
expect(stats1.priceCache.size).toEqual(0);
|
||||||
|
|
||||||
|
// Fetch price (should hit API)
|
||||||
|
const start1 = Date.now();
|
||||||
|
await service.getPrice('AAPL');
|
||||||
|
const duration1 = Date.now() - start1;
|
||||||
|
|
||||||
|
const stats2 = service.getCacheStats();
|
||||||
|
expect(stats2.priceCache.size).toEqual(1);
|
||||||
|
|
||||||
|
// Fetch again (should hit cache - much faster)
|
||||||
|
const start2 = Date.now();
|
||||||
|
await service.getPrice('AAPL');
|
||||||
|
const duration2 = Date.now() - start2;
|
||||||
|
|
||||||
|
expect(duration2).toBeLessThan(duration1);
|
||||||
|
|
||||||
|
console.log('\n⚡ Cache Performance:');
|
||||||
|
console.log(` First fetch: ${duration1}ms`);
|
||||||
|
console.log(` Cached fetch: ${duration2}ms`);
|
||||||
|
console.log(` Speedup: ${Math.round(duration1 / duration2)}x`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('StockDataService - Fundamentals Fetching', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
const secProvider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
service.registerFundamentalsProvider(secProvider);
|
||||||
|
|
||||||
|
await tap.test('should fetch single fundamentals', async () => {
|
||||||
|
const fundamentals = await service.getFundamentals('AAPL');
|
||||||
|
|
||||||
|
expect(fundamentals).toBeDefined();
|
||||||
|
expect(fundamentals.ticker).toEqual('AAPL');
|
||||||
|
expect(fundamentals.companyName).toEqual('Apple Inc.');
|
||||||
|
expect(fundamentals.provider).toEqual('SEC EDGAR');
|
||||||
|
expect(fundamentals.earningsPerShareDiluted).toBeGreaterThan(0);
|
||||||
|
expect(fundamentals.sharesOutstanding).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n📊 Single Fundamentals:');
|
||||||
|
console.log(` ${fundamentals.ticker}: ${fundamentals.companyName}`);
|
||||||
|
console.log(` EPS: $${fundamentals.earningsPerShareDiluted?.toFixed(2)}`);
|
||||||
|
console.log(` Shares: ${(fundamentals.sharesOutstanding! / 1_000_000_000).toFixed(2)}B`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch batch fundamentals', async () => {
|
||||||
|
const fundamentals = await service.getBatchFundamentals(['AAPL', 'MSFT']);
|
||||||
|
|
||||||
|
expect(fundamentals).toBeInstanceOf(Array);
|
||||||
|
expect(fundamentals.length).toEqual(2);
|
||||||
|
|
||||||
|
console.log('\n📊 Batch Fundamentals:');
|
||||||
|
fundamentals.forEach(f => {
|
||||||
|
console.log(` ${f.ticker}: ${f.companyName} - EPS: $${f.earningsPerShareDiluted?.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should cache fundamentals', async () => {
|
||||||
|
// Clear cache
|
||||||
|
service.clearCache();
|
||||||
|
|
||||||
|
const stats1 = service.getCacheStats();
|
||||||
|
expect(stats1.fundamentalsCache.size).toEqual(0);
|
||||||
|
|
||||||
|
// Fetch fundamentals (should hit API)
|
||||||
|
await service.getFundamentals('AAPL');
|
||||||
|
|
||||||
|
const stats2 = service.getCacheStats();
|
||||||
|
expect(stats2.fundamentalsCache.size).toEqual(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('StockDataService - Complete Stock Data', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
|
||||||
|
// Register both providers
|
||||||
|
const yahooProvider = new opendata.YahooFinanceProvider();
|
||||||
|
const secProvider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.registerPriceProvider(yahooProvider);
|
||||||
|
service.registerFundamentalsProvider(secProvider);
|
||||||
|
|
||||||
|
await tap.test('should fetch complete stock data with string', async () => {
|
||||||
|
const data = await service.getStockData('AAPL');
|
||||||
|
|
||||||
|
expect(data).toBeDefined();
|
||||||
|
expect(data.ticker).toEqual('AAPL');
|
||||||
|
expect(data.price).toBeDefined();
|
||||||
|
expect(data.price.ticker).toEqual('AAPL');
|
||||||
|
expect(data.fundamentals).toBeDefined();
|
||||||
|
expect(data.fundamentals?.ticker).toEqual('AAPL');
|
||||||
|
expect(data.fetchedAt).toBeInstanceOf(Date);
|
||||||
|
|
||||||
|
// Check automatic enrichment
|
||||||
|
expect(data.fundamentals?.marketCap).toBeDefined();
|
||||||
|
expect(data.fundamentals?.priceToEarnings).toBeDefined();
|
||||||
|
expect(data.fundamentals?.marketCap).toBeGreaterThan(0);
|
||||||
|
expect(data.fundamentals?.priceToEarnings).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n✨ Complete Stock Data (Auto-Enriched):');
|
||||||
|
console.log(` ${data.ticker}: ${data.fundamentals?.companyName}`);
|
||||||
|
console.log(` Price: $${data.price.price.toFixed(2)}`);
|
||||||
|
console.log(` Market Cap: $${(data.fundamentals!.marketCap! / 1_000_000_000_000).toFixed(2)}T`);
|
||||||
|
console.log(` P/E Ratio: ${data.fundamentals!.priceToEarnings!.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch complete stock data with request object', async () => {
|
||||||
|
const data = await service.getStockData({
|
||||||
|
ticker: 'MSFT',
|
||||||
|
includeFundamentals: true,
|
||||||
|
enrichFundamentals: true
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(data).toBeDefined();
|
||||||
|
expect(data.ticker).toEqual('MSFT');
|
||||||
|
expect(data.price).toBeDefined();
|
||||||
|
expect(data.fundamentals).toBeDefined();
|
||||||
|
expect(data.fundamentals?.marketCap).toBeDefined();
|
||||||
|
expect(data.fundamentals?.priceToEarnings).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch complete stock data without fundamentals', async () => {
|
||||||
|
const data = await service.getStockData({
|
||||||
|
ticker: 'GOOGL',
|
||||||
|
includeFundamentals: false
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(data).toBeDefined();
|
||||||
|
expect(data.ticker).toEqual('GOOGL');
|
||||||
|
expect(data.price).toBeDefined();
|
||||||
|
expect(data.fundamentals).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should handle fundamentals fetch failure gracefully', async () => {
|
||||||
|
// Try a ticker that might not have fundamentals
|
||||||
|
const data = await service.getStockData({
|
||||||
|
ticker: 'BTC-USD', // Crypto - no SEC filings
|
||||||
|
includeFundamentals: true
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(data).toBeDefined();
|
||||||
|
expect(data.price).toBeDefined();
|
||||||
|
// Fundamentals might be undefined due to error
|
||||||
|
console.log(`\n⚠️ ${data.ticker} - Price available, Fundamentals: ${data.fundamentals ? 'Yes' : 'No'}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('StockDataService - Batch Complete Stock Data', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
|
||||||
|
const yahooProvider = new opendata.YahooFinanceProvider();
|
||||||
|
const secProvider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.registerPriceProvider(yahooProvider);
|
||||||
|
service.registerFundamentalsProvider(secProvider);
|
||||||
|
|
||||||
|
await tap.test('should fetch batch complete data with array', async () => {
|
||||||
|
const data = await service.getBatchStockData(['AAPL', 'MSFT']);
|
||||||
|
|
||||||
|
expect(data).toBeInstanceOf(Array);
|
||||||
|
expect(data.length).toEqual(2);
|
||||||
|
|
||||||
|
data.forEach(stock => {
|
||||||
|
expect(stock.ticker).toBeDefined();
|
||||||
|
expect(stock.price).toBeDefined();
|
||||||
|
expect(stock.fundamentals).toBeDefined();
|
||||||
|
expect(stock.fundamentals?.marketCap).toBeGreaterThan(0);
|
||||||
|
expect(stock.fundamentals?.priceToEarnings).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log('\n✨ Batch Complete Data:');
|
||||||
|
data.forEach(stock => {
|
||||||
|
console.log(` ${stock.ticker}: Price $${stock.price.price.toFixed(2)}, P/E ${stock.fundamentals!.priceToEarnings!.toFixed(2)}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch batch complete data with request object', async () => {
|
||||||
|
const data = await service.getBatchStockData({
|
||||||
|
tickers: ['AAPL', 'GOOGL'],
|
||||||
|
includeFundamentals: true,
|
||||||
|
enrichFundamentals: true
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(data).toBeInstanceOf(Array);
|
||||||
|
expect(data.length).toEqual(2);
|
||||||
|
|
||||||
|
data.forEach(stock => {
|
||||||
|
expect(stock.fundamentals?.marketCap).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should fetch batch without enrichment', async () => {
|
||||||
|
const data = await service.getBatchStockData({
|
||||||
|
tickers: ['AAPL', 'MSFT'],
|
||||||
|
includeFundamentals: true,
|
||||||
|
enrichFundamentals: false
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(data).toBeInstanceOf(Array);
|
||||||
|
|
||||||
|
// Check that fundamentals exist but enrichment might not be complete
|
||||||
|
data.forEach(stock => {
|
||||||
|
if (stock.fundamentals) {
|
||||||
|
expect(stock.fundamentals.ticker).toBeDefined();
|
||||||
|
expect(stock.fundamentals.companyName).toBeDefined();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('StockDataService - Health & Statistics', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
|
||||||
|
const yahooProvider = new opendata.YahooFinanceProvider();
|
||||||
|
const secProvider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
|
||||||
|
service.registerPriceProvider(yahooProvider);
|
||||||
|
service.registerFundamentalsProvider(secProvider);
|
||||||
|
|
||||||
|
await tap.test('should check providers health', async () => {
|
||||||
|
const health = await service.checkProvidersHealth();
|
||||||
|
|
||||||
|
expect(health.size).toEqual(2);
|
||||||
|
expect(health.get('Yahoo Finance (price)')).toBe(true);
|
||||||
|
expect(health.get('SEC EDGAR (fundamentals)')).toBe(true);
|
||||||
|
|
||||||
|
console.log('\n💚 Provider Health:');
|
||||||
|
health.forEach((isHealthy, name) => {
|
||||||
|
console.log(` ${name}: ${isHealthy ? '✅ Healthy' : '❌ Unhealthy'}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should track provider statistics', async () => {
|
||||||
|
// Make some requests to generate stats
|
||||||
|
await service.getPrice('AAPL');
|
||||||
|
await service.getFundamentals('AAPL');
|
||||||
|
|
||||||
|
const stats = service.getProviderStats();
|
||||||
|
|
||||||
|
expect(stats.size).toEqual(2);
|
||||||
|
|
||||||
|
const yahooStats = stats.get('Yahoo Finance');
|
||||||
|
expect(yahooStats).toBeDefined();
|
||||||
|
expect(yahooStats!.type).toEqual('price');
|
||||||
|
expect(yahooStats!.successCount).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
const secStats = stats.get('SEC EDGAR');
|
||||||
|
expect(secStats).toBeDefined();
|
||||||
|
expect(secStats!.type).toEqual('fundamentals');
|
||||||
|
expect(secStats!.successCount).toBeGreaterThan(0);
|
||||||
|
|
||||||
|
console.log('\n📈 Provider Statistics:');
|
||||||
|
stats.forEach((stat, name) => {
|
||||||
|
console.log(` ${name} (${stat.type}): Success=${stat.successCount}, Errors=${stat.errorCount}`);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should clear all caches', async () => {
|
||||||
|
service.clearCache();
|
||||||
|
|
||||||
|
const stats = service.getCacheStats();
|
||||||
|
expect(stats.priceCache.size).toEqual(0);
|
||||||
|
expect(stats.fundamentalsCache.size).toEqual(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
tap.test('StockDataService - Error Handling', async () => {
|
||||||
|
await tap.test('should throw error when no price provider available', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
|
||||||
|
try {
|
||||||
|
await service.getPrice('AAPL');
|
||||||
|
throw new Error('Should have thrown error');
|
||||||
|
} catch (error) {
|
||||||
|
expect(error.message).toContain('No price providers available');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should throw error when no fundamentals provider available', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
|
||||||
|
try {
|
||||||
|
await service.getFundamentals('AAPL');
|
||||||
|
throw new Error('Should have thrown error');
|
||||||
|
} catch (error) {
|
||||||
|
expect(error.message).toContain('No fundamentals providers available');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should handle invalid ticker for price', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
const yahooProvider = new opendata.YahooFinanceProvider();
|
||||||
|
service.registerPriceProvider(yahooProvider);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await service.getPrice('INVALIDTICKER123456');
|
||||||
|
throw new Error('Should have thrown error');
|
||||||
|
} catch (error) {
|
||||||
|
expect(error.message).toContain('Failed to fetch price');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
await tap.test('should handle invalid ticker for fundamentals', async () => {
|
||||||
|
const service = new opendata.StockDataService();
|
||||||
|
const secProvider = new opendata.SecEdgarProvider({
|
||||||
|
userAgent: TEST_USER_AGENT
|
||||||
|
});
|
||||||
|
service.registerFundamentalsProvider(secProvider);
|
||||||
|
|
||||||
|
try {
|
||||||
|
await service.getFundamentals('INVALIDTICKER123456');
|
||||||
|
throw new Error('Should have thrown error');
|
||||||
|
} catch (error) {
|
||||||
|
expect(error.message).toContain('CIK not found');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
export default tap.start();
|
||||||
13
test/test.ts
13
test/test.ts
@@ -1,12 +1,23 @@
|
|||||||
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
import { expect, tap } from '@git.zone/tstest/tapbundle';
|
||||||
import * as opendata from '../ts/index.js'
|
import * as opendata from '../ts/index.js'
|
||||||
|
import * as paths from '../ts/paths.js';
|
||||||
|
import * as plugins from '../ts/plugins.js';
|
||||||
|
|
||||||
import { BusinessRecord } from '../ts/classes.businessrecord.js';
|
import { BusinessRecord } from '../ts/classes.businessrecord.js';
|
||||||
|
|
||||||
|
// Test configuration - explicit paths required
|
||||||
|
const testNogitDir = plugins.path.join(paths.packageDir, '.nogit');
|
||||||
|
const testDownloadDir = plugins.path.join(testNogitDir, 'downloads');
|
||||||
|
const testGermanBusinessDataDir = plugins.path.join(testNogitDir, 'germanbusinessdata');
|
||||||
|
|
||||||
let testOpenDataInstance: opendata.OpenData;
|
let testOpenDataInstance: opendata.OpenData;
|
||||||
|
|
||||||
tap.test('first test', async () => {
|
tap.test('first test', async () => {
|
||||||
testOpenDataInstance = new opendata.OpenData();
|
testOpenDataInstance = new opendata.OpenData({
|
||||||
|
nogitDir: testNogitDir,
|
||||||
|
downloadDir: testDownloadDir,
|
||||||
|
germanBusinessDataDir: testGermanBusinessDataDir
|
||||||
|
});
|
||||||
expect(testOpenDataInstance).toBeInstanceOf(opendata.OpenData);
|
expect(testOpenDataInstance).toBeInstanceOf(opendata.OpenData);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -3,6 +3,6 @@
|
|||||||
*/
|
*/
|
||||||
export const commitinfo = {
|
export const commitinfo = {
|
||||||
name: '@fin.cx/opendata',
|
name: '@fin.cx/opendata',
|
||||||
version: '2.0.0',
|
version: '3.2.0',
|
||||||
description: 'A comprehensive TypeScript library for accessing business data and real-time financial information. Features include German company data management with MongoDB integration, JSONL bulk processing, automated Handelsregister interactions, and real-time stock market data from multiple providers.'
|
description: 'A comprehensive TypeScript library for accessing business data and real-time financial information. Features include German company data management with MongoDB integration, JSONL bulk processing, automated Handelsregister interactions, and real-time stock market data from multiple providers.'
|
||||||
}
|
}
|
||||||
|
|||||||
296
ts/stocks/classes.baseproviderservice.ts
Normal file
296
ts/stocks/classes.baseproviderservice.ts
Normal file
@@ -0,0 +1,296 @@
|
|||||||
|
import * as plugins from '../plugins.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Base provider entry for tracking provider state
|
||||||
|
*/
|
||||||
|
export interface IBaseProviderEntry<TProvider> {
|
||||||
|
provider: TProvider;
|
||||||
|
config: IBaseProviderConfig;
|
||||||
|
lastError?: Error;
|
||||||
|
lastErrorTime?: Date;
|
||||||
|
successCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Base provider configuration
|
||||||
|
*/
|
||||||
|
export interface IBaseProviderConfig {
|
||||||
|
enabled: boolean;
|
||||||
|
priority: number;
|
||||||
|
timeout?: number;
|
||||||
|
retryAttempts?: number;
|
||||||
|
retryDelay?: number;
|
||||||
|
cacheTTL?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Base provider interface
|
||||||
|
*/
|
||||||
|
export interface IBaseProvider {
|
||||||
|
name: string;
|
||||||
|
priority: number;
|
||||||
|
isAvailable(): Promise<boolean>;
|
||||||
|
readonly requiresAuth: boolean;
|
||||||
|
readonly rateLimit?: {
|
||||||
|
requestsPerMinute: number;
|
||||||
|
requestsPerDay?: number;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Cache entry for any data type
|
||||||
|
*/
|
||||||
|
export interface IBaseCacheEntry<TData> {
|
||||||
|
data: TData;
|
||||||
|
timestamp: Date;
|
||||||
|
ttl: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Base service for managing data providers with caching
|
||||||
|
* Shared logic extracted from StockPriceService and FundamentalsService
|
||||||
|
*/
|
||||||
|
export abstract class BaseProviderService<TProvider extends IBaseProvider, TData> {
|
||||||
|
protected providers = new Map<string, IBaseProviderEntry<TProvider>>();
|
||||||
|
protected cache = new Map<string, IBaseCacheEntry<TData>>();
|
||||||
|
protected logger = console;
|
||||||
|
|
||||||
|
protected cacheConfig = {
|
||||||
|
ttl: 60000, // Default 60 seconds
|
||||||
|
maxEntries: 10000
|
||||||
|
};
|
||||||
|
|
||||||
|
constructor(cacheConfig?: { ttl?: number; maxEntries?: number }) {
|
||||||
|
if (cacheConfig) {
|
||||||
|
this.cacheConfig = { ...this.cacheConfig, ...cacheConfig };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Register a provider
|
||||||
|
*/
|
||||||
|
public register(provider: TProvider, config?: Partial<IBaseProviderConfig>): void {
|
||||||
|
const defaultConfig: IBaseProviderConfig = {
|
||||||
|
enabled: true,
|
||||||
|
priority: provider.priority,
|
||||||
|
timeout: 30000,
|
||||||
|
retryAttempts: 2,
|
||||||
|
retryDelay: 1000,
|
||||||
|
cacheTTL: this.cacheConfig.ttl
|
||||||
|
};
|
||||||
|
|
||||||
|
const mergedConfig = { ...defaultConfig, ...config };
|
||||||
|
|
||||||
|
this.providers.set(provider.name, {
|
||||||
|
provider,
|
||||||
|
config: mergedConfig,
|
||||||
|
successCount: 0,
|
||||||
|
errorCount: 0
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log(`Registered provider: ${provider.name}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unregister a provider
|
||||||
|
*/
|
||||||
|
public unregister(providerName: string): void {
|
||||||
|
this.providers.delete(providerName);
|
||||||
|
console.log(`Unregistered provider: ${providerName}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get a specific provider by name
|
||||||
|
*/
|
||||||
|
public getProvider(name: string): TProvider | undefined {
|
||||||
|
return this.providers.get(name)?.provider;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get all registered providers
|
||||||
|
*/
|
||||||
|
public getAllProviders(): TProvider[] {
|
||||||
|
return Array.from(this.providers.values()).map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get enabled providers sorted by priority
|
||||||
|
*/
|
||||||
|
public getEnabledProviders(): TProvider[] {
|
||||||
|
return Array.from(this.providers.values())
|
||||||
|
.filter(entry => entry.config.enabled)
|
||||||
|
.sort((a, b) => (b.config.priority || 0) - (a.config.priority || 0))
|
||||||
|
.map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check health of all providers
|
||||||
|
*/
|
||||||
|
public async checkProvidersHealth(): Promise<Map<string, boolean>> {
|
||||||
|
const health = new Map<string, boolean>();
|
||||||
|
|
||||||
|
for (const [name, entry] of this.providers) {
|
||||||
|
if (!entry.config.enabled) {
|
||||||
|
health.set(name, false);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const isAvailable = await entry.provider.isAvailable();
|
||||||
|
health.set(name, isAvailable);
|
||||||
|
} catch (error) {
|
||||||
|
health.set(name, false);
|
||||||
|
console.error(`Health check failed for ${name}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return health;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get provider statistics
|
||||||
|
*/
|
||||||
|
public getProviderStats(): Map<
|
||||||
|
string,
|
||||||
|
{
|
||||||
|
successCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
lastError?: string;
|
||||||
|
lastErrorTime?: Date;
|
||||||
|
}
|
||||||
|
> {
|
||||||
|
const stats = new Map();
|
||||||
|
|
||||||
|
for (const [name, entry] of this.providers) {
|
||||||
|
stats.set(name, {
|
||||||
|
successCount: entry.successCount,
|
||||||
|
errorCount: entry.errorCount,
|
||||||
|
lastError: entry.lastError?.message,
|
||||||
|
lastErrorTime: entry.lastErrorTime
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return stats;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clear all cached data
|
||||||
|
*/
|
||||||
|
public clearCache(): void {
|
||||||
|
this.cache.clear();
|
||||||
|
console.log('Cache cleared');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Set cache TTL
|
||||||
|
*/
|
||||||
|
public setCacheTTL(ttl: number): void {
|
||||||
|
this.cacheConfig.ttl = ttl;
|
||||||
|
console.log(`Cache TTL set to ${ttl}ms`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
*/
|
||||||
|
public getCacheStats(): {
|
||||||
|
size: number;
|
||||||
|
maxEntries: number;
|
||||||
|
ttl: number;
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
size: this.cache.size,
|
||||||
|
maxEntries: this.cacheConfig.maxEntries,
|
||||||
|
ttl: this.cacheConfig.ttl
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch with retry logic
|
||||||
|
*/
|
||||||
|
protected async fetchWithRetry<T>(
|
||||||
|
fetchFn: () => Promise<T>,
|
||||||
|
config: IBaseProviderConfig
|
||||||
|
): Promise<T> {
|
||||||
|
const maxAttempts = config.retryAttempts || 1;
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||||
|
try {
|
||||||
|
return await fetchFn();
|
||||||
|
} catch (error) {
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
if (attempt < maxAttempts) {
|
||||||
|
const delay = (config.retryDelay || 1000) * attempt;
|
||||||
|
console.log(`Retry attempt ${attempt} after ${delay}ms`);
|
||||||
|
await plugins.smartdelay.delayFor(delay);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw lastError || new Error('Unknown error during fetch');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get from cache if not expired
|
||||||
|
*/
|
||||||
|
protected getFromCache(key: string): TData | null {
|
||||||
|
const entry = this.cache.get(key);
|
||||||
|
|
||||||
|
if (!entry) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if cache entry has expired
|
||||||
|
const age = Date.now() - entry.timestamp.getTime();
|
||||||
|
if (entry.ttl !== Infinity && age > entry.ttl) {
|
||||||
|
this.cache.delete(key);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return entry.data;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Add to cache with TTL
|
||||||
|
*/
|
||||||
|
protected addToCache(key: string, data: TData, ttl?: number): void {
|
||||||
|
// Enforce max entries limit
|
||||||
|
if (this.cache.size >= this.cacheConfig.maxEntries) {
|
||||||
|
// Remove oldest entry
|
||||||
|
const oldestKey = this.cache.keys().next().value;
|
||||||
|
if (oldestKey) {
|
||||||
|
this.cache.delete(oldestKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
this.cache.set(key, {
|
||||||
|
data,
|
||||||
|
timestamp: new Date(),
|
||||||
|
ttl: ttl || this.cacheConfig.ttl
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Track successful fetch for provider
|
||||||
|
*/
|
||||||
|
protected trackSuccess(providerName: string): void {
|
||||||
|
const entry = this.providers.get(providerName);
|
||||||
|
if (entry) {
|
||||||
|
entry.successCount++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Track failed fetch for provider
|
||||||
|
*/
|
||||||
|
protected trackError(providerName: string, error: Error): void {
|
||||||
|
const entry = this.providers.get(providerName);
|
||||||
|
if (entry) {
|
||||||
|
entry.errorCount++;
|
||||||
|
entry.lastError = error;
|
||||||
|
entry.lastErrorTime = new Date();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
404
ts/stocks/classes.fundamentalsservice.ts
Normal file
404
ts/stocks/classes.fundamentalsservice.ts
Normal file
@@ -0,0 +1,404 @@
|
|||||||
|
import * as plugins from '../plugins.js';
|
||||||
|
import type {
|
||||||
|
IFundamentalsProvider,
|
||||||
|
IFundamentalsProviderConfig,
|
||||||
|
IFundamentalsProviderRegistry,
|
||||||
|
IStockFundamentals,
|
||||||
|
IFundamentalsRequest
|
||||||
|
} from './interfaces/fundamentals.js';
|
||||||
|
|
||||||
|
interface IProviderEntry {
|
||||||
|
provider: IFundamentalsProvider;
|
||||||
|
config: IFundamentalsProviderConfig;
|
||||||
|
lastError?: Error;
|
||||||
|
lastErrorTime?: Date;
|
||||||
|
successCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ICacheEntry {
|
||||||
|
fundamentals: IStockFundamentals | IStockFundamentals[];
|
||||||
|
timestamp: Date;
|
||||||
|
ttl: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Service for managing fundamental data providers and caching
|
||||||
|
* Parallel to StockPriceService but for fundamental data instead of prices
|
||||||
|
*/
|
||||||
|
export class FundamentalsService implements IFundamentalsProviderRegistry {
|
||||||
|
private providers = new Map<string, IProviderEntry>();
|
||||||
|
private cache = new Map<string, ICacheEntry>();
|
||||||
|
private logger = console;
|
||||||
|
|
||||||
|
private cacheConfig = {
|
||||||
|
ttl: 90 * 24 * 60 * 60 * 1000, // 90 days default (fundamentals change quarterly)
|
||||||
|
maxEntries: 10000
|
||||||
|
};
|
||||||
|
|
||||||
|
constructor(cacheConfig?: { ttl?: number; maxEntries?: number }) {
|
||||||
|
if (cacheConfig) {
|
||||||
|
this.cacheConfig = { ...this.cacheConfig, ...cacheConfig };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Register a fundamentals provider
|
||||||
|
*/
|
||||||
|
public register(provider: IFundamentalsProvider, config?: IFundamentalsProviderConfig): void {
|
||||||
|
const defaultConfig: IFundamentalsProviderConfig = {
|
||||||
|
enabled: true,
|
||||||
|
priority: provider.priority,
|
||||||
|
timeout: 30000, // Longer timeout for fundamental data
|
||||||
|
retryAttempts: 2,
|
||||||
|
retryDelay: 1000,
|
||||||
|
cacheTTL: this.cacheConfig.ttl
|
||||||
|
};
|
||||||
|
|
||||||
|
const mergedConfig = { ...defaultConfig, ...config };
|
||||||
|
|
||||||
|
this.providers.set(provider.name, {
|
||||||
|
provider,
|
||||||
|
config: mergedConfig,
|
||||||
|
successCount: 0,
|
||||||
|
errorCount: 0
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log(`Registered fundamentals provider: ${provider.name}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unregister a provider
|
||||||
|
*/
|
||||||
|
public unregister(providerName: string): void {
|
||||||
|
this.providers.delete(providerName);
|
||||||
|
console.log(`Unregistered fundamentals provider: ${providerName}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get a specific provider by name
|
||||||
|
*/
|
||||||
|
public getProvider(name: string): IFundamentalsProvider | undefined {
|
||||||
|
return this.providers.get(name)?.provider;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get all registered providers
|
||||||
|
*/
|
||||||
|
public getAllProviders(): IFundamentalsProvider[] {
|
||||||
|
return Array.from(this.providers.values()).map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get enabled providers sorted by priority
|
||||||
|
*/
|
||||||
|
public getEnabledProviders(): IFundamentalsProvider[] {
|
||||||
|
return Array.from(this.providers.values())
|
||||||
|
.filter(entry => entry.config.enabled)
|
||||||
|
.sort((a, b) => (b.config.priority || 0) - (a.config.priority || 0))
|
||||||
|
.map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get fundamental data for a single ticker
|
||||||
|
*/
|
||||||
|
public async getFundamentals(ticker: string): Promise<IStockFundamentals> {
|
||||||
|
const result = await this.getData({
|
||||||
|
type: 'fundamentals-current',
|
||||||
|
ticker
|
||||||
|
});
|
||||||
|
return result as IStockFundamentals;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get fundamental data for multiple tickers
|
||||||
|
*/
|
||||||
|
public async getBatchFundamentals(tickers: string[]): Promise<IStockFundamentals[]> {
|
||||||
|
const result = await this.getData({
|
||||||
|
type: 'fundamentals-batch',
|
||||||
|
tickers
|
||||||
|
});
|
||||||
|
return result as IStockFundamentals[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unified data fetching method
|
||||||
|
*/
|
||||||
|
public async getData(
|
||||||
|
request: IFundamentalsRequest
|
||||||
|
): Promise<IStockFundamentals | IStockFundamentals[]> {
|
||||||
|
const cacheKey = this.getCacheKey(request);
|
||||||
|
const cached = this.getFromCache(cacheKey);
|
||||||
|
|
||||||
|
if (cached) {
|
||||||
|
console.log(`Cache hit for ${this.getRequestDescription(request)}`);
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
const providers = this.getEnabledProviders();
|
||||||
|
if (providers.length === 0) {
|
||||||
|
throw new Error('No fundamentals providers available');
|
||||||
|
}
|
||||||
|
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (const provider of providers) {
|
||||||
|
const entry = this.providers.get(provider.name)!;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await this.fetchWithRetry(
|
||||||
|
() => provider.fetchData(request),
|
||||||
|
entry.config
|
||||||
|
);
|
||||||
|
|
||||||
|
entry.successCount++;
|
||||||
|
|
||||||
|
// Use provider-specific cache TTL or default
|
||||||
|
const ttl = entry.config.cacheTTL || this.cacheConfig.ttl;
|
||||||
|
this.addToCache(cacheKey, result, ttl);
|
||||||
|
|
||||||
|
console.log(`Successfully fetched ${this.getRequestDescription(request)} from ${provider.name}`);
|
||||||
|
return result;
|
||||||
|
} catch (error) {
|
||||||
|
entry.errorCount++;
|
||||||
|
entry.lastError = error as Error;
|
||||||
|
entry.lastErrorTime = new Date();
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
console.warn(
|
||||||
|
`Provider ${provider.name} failed for ${this.getRequestDescription(request)}: ${error.message}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error(
|
||||||
|
`Failed to fetch ${this.getRequestDescription(request)} from all providers. Last error: ${lastError?.message}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Enrich fundamentals with calculated metrics using current price
|
||||||
|
*/
|
||||||
|
public async enrichWithPrice(
|
||||||
|
fundamentals: IStockFundamentals,
|
||||||
|
price: number
|
||||||
|
): Promise<IStockFundamentals> {
|
||||||
|
const enriched = { ...fundamentals };
|
||||||
|
|
||||||
|
// Calculate market cap: price × shares outstanding
|
||||||
|
if (fundamentals.sharesOutstanding) {
|
||||||
|
enriched.marketCap = price * fundamentals.sharesOutstanding;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Calculate P/E ratio: price / EPS
|
||||||
|
if (fundamentals.earningsPerShareDiluted && fundamentals.earningsPerShareDiluted > 0) {
|
||||||
|
enriched.priceToEarnings = price / fundamentals.earningsPerShareDiluted;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Calculate price-to-book: market cap / stockholders equity
|
||||||
|
if (enriched.marketCap && fundamentals.stockholdersEquity && fundamentals.stockholdersEquity > 0) {
|
||||||
|
enriched.priceToBook = enriched.marketCap / fundamentals.stockholdersEquity;
|
||||||
|
}
|
||||||
|
|
||||||
|
return enriched;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Enrich batch fundamentals with prices
|
||||||
|
*/
|
||||||
|
public async enrichBatchWithPrices(
|
||||||
|
fundamentalsList: IStockFundamentals[],
|
||||||
|
priceMap: Map<string, number>
|
||||||
|
): Promise<IStockFundamentals[]> {
|
||||||
|
return Promise.all(
|
||||||
|
fundamentalsList.map(fundamentals => {
|
||||||
|
const price = priceMap.get(fundamentals.ticker);
|
||||||
|
if (price) {
|
||||||
|
return this.enrichWithPrice(fundamentals, price);
|
||||||
|
}
|
||||||
|
return Promise.resolve(fundamentals);
|
||||||
|
})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check health of all providers
|
||||||
|
*/
|
||||||
|
public async checkProvidersHealth(): Promise<Map<string, boolean>> {
|
||||||
|
const health = new Map<string, boolean>();
|
||||||
|
|
||||||
|
for (const [name, entry] of this.providers) {
|
||||||
|
if (!entry.config.enabled) {
|
||||||
|
health.set(name, false);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const isAvailable = await entry.provider.isAvailable();
|
||||||
|
health.set(name, isAvailable);
|
||||||
|
} catch (error) {
|
||||||
|
health.set(name, false);
|
||||||
|
console.error(`Health check failed for ${name}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return health;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get provider statistics
|
||||||
|
*/
|
||||||
|
public getProviderStats(): Map<
|
||||||
|
string,
|
||||||
|
{
|
||||||
|
successCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
lastError?: string;
|
||||||
|
lastErrorTime?: Date;
|
||||||
|
}
|
||||||
|
> {
|
||||||
|
const stats = new Map();
|
||||||
|
|
||||||
|
for (const [name, entry] of this.providers) {
|
||||||
|
stats.set(name, {
|
||||||
|
successCount: entry.successCount,
|
||||||
|
errorCount: entry.errorCount,
|
||||||
|
lastError: entry.lastError?.message,
|
||||||
|
lastErrorTime: entry.lastErrorTime
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return stats;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clear all cached data
|
||||||
|
*/
|
||||||
|
public clearCache(): void {
|
||||||
|
this.cache.clear();
|
||||||
|
console.log('Fundamentals cache cleared');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Set cache TTL
|
||||||
|
*/
|
||||||
|
public setCacheTTL(ttl: number): void {
|
||||||
|
this.cacheConfig.ttl = ttl;
|
||||||
|
console.log(`Fundamentals cache TTL set to ${ttl}ms`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
*/
|
||||||
|
public getCacheStats(): {
|
||||||
|
size: number;
|
||||||
|
maxEntries: number;
|
||||||
|
ttl: number;
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
size: this.cache.size,
|
||||||
|
maxEntries: this.cacheConfig.maxEntries,
|
||||||
|
ttl: this.cacheConfig.ttl
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch with retry logic
|
||||||
|
*/
|
||||||
|
private async fetchWithRetry<T>(
|
||||||
|
fetchFn: () => Promise<T>,
|
||||||
|
config: IFundamentalsProviderConfig
|
||||||
|
): Promise<T> {
|
||||||
|
const maxAttempts = config.retryAttempts || 1;
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||||
|
try {
|
||||||
|
return await fetchFn();
|
||||||
|
} catch (error) {
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
if (attempt < maxAttempts) {
|
||||||
|
const delay = (config.retryDelay || 1000) * attempt;
|
||||||
|
console.log(`Retry attempt ${attempt} after ${delay}ms`);
|
||||||
|
await plugins.smartdelay.delayFor(delay);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw lastError || new Error('Unknown error during fetch');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate cache key for request
|
||||||
|
*/
|
||||||
|
private getCacheKey(request: IFundamentalsRequest): string {
|
||||||
|
switch (request.type) {
|
||||||
|
case 'fundamentals-current':
|
||||||
|
return `fundamentals:${request.ticker}`;
|
||||||
|
case 'fundamentals-batch':
|
||||||
|
const tickers = request.tickers.sort().join(',');
|
||||||
|
return `fundamentals-batch:${tickers}`;
|
||||||
|
default:
|
||||||
|
return `unknown:${JSON.stringify(request)}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get from cache if not expired
|
||||||
|
*/
|
||||||
|
private getFromCache(key: string): IStockFundamentals | IStockFundamentals[] | null {
|
||||||
|
const entry = this.cache.get(key);
|
||||||
|
|
||||||
|
if (!entry) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if cache entry has expired
|
||||||
|
const age = Date.now() - entry.timestamp.getTime();
|
||||||
|
if (entry.ttl !== Infinity && age > entry.ttl) {
|
||||||
|
this.cache.delete(key);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return entry.fundamentals;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Add to cache with TTL
|
||||||
|
*/
|
||||||
|
private addToCache(
|
||||||
|
key: string,
|
||||||
|
fundamentals: IStockFundamentals | IStockFundamentals[],
|
||||||
|
ttl?: number
|
||||||
|
): void {
|
||||||
|
// Enforce max entries limit
|
||||||
|
if (this.cache.size >= this.cacheConfig.maxEntries) {
|
||||||
|
// Remove oldest entry
|
||||||
|
const oldestKey = this.cache.keys().next().value;
|
||||||
|
if (oldestKey) {
|
||||||
|
this.cache.delete(oldestKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
this.cache.set(key, {
|
||||||
|
fundamentals,
|
||||||
|
timestamp: new Date(),
|
||||||
|
ttl: ttl || this.cacheConfig.ttl
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get human-readable request description
|
||||||
|
*/
|
||||||
|
private getRequestDescription(request: IFundamentalsRequest): string {
|
||||||
|
switch (request.type) {
|
||||||
|
case 'fundamentals-current':
|
||||||
|
return `fundamentals for ${request.ticker}`;
|
||||||
|
case 'fundamentals-batch':
|
||||||
|
return `fundamentals for ${request.tickers.length} tickers`;
|
||||||
|
default:
|
||||||
|
return 'fundamentals data';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
647
ts/stocks/classes.stockdataservice.ts
Normal file
647
ts/stocks/classes.stockdataservice.ts
Normal file
@@ -0,0 +1,647 @@
|
|||||||
|
import * as plugins from '../plugins.js';
|
||||||
|
import type { IStockProvider, IProviderConfig } from './interfaces/provider.js';
|
||||||
|
import type { IFundamentalsProvider, IFundamentalsProviderConfig, IStockFundamentals } from './interfaces/fundamentals.js';
|
||||||
|
import type { IStockPrice, IStockDataRequest as IPriceRequest } from './interfaces/stockprice.js';
|
||||||
|
import type { IStockData, IStockDataServiceConfig, ICompleteStockDataRequest, ICompleteStockDataBatchRequest } from './interfaces/stockdata.js';
|
||||||
|
|
||||||
|
interface IProviderEntry<T> {
|
||||||
|
provider: T;
|
||||||
|
config: IProviderConfig | IFundamentalsProviderConfig;
|
||||||
|
lastError?: Error;
|
||||||
|
lastErrorTime?: Date;
|
||||||
|
successCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ICacheEntry<T> {
|
||||||
|
data: T;
|
||||||
|
timestamp: Date;
|
||||||
|
ttl: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unified service for managing both stock prices and fundamentals
|
||||||
|
* Provides automatic enrichment and convenient combined data access
|
||||||
|
*/
|
||||||
|
export class StockDataService {
|
||||||
|
private priceProviders = new Map<string, IProviderEntry<IStockProvider>>();
|
||||||
|
private fundamentalsProviders = new Map<string, IProviderEntry<IFundamentalsProvider>>();
|
||||||
|
|
||||||
|
private priceCache = new Map<string, ICacheEntry<IStockPrice | IStockPrice[]>>();
|
||||||
|
private fundamentalsCache = new Map<string, ICacheEntry<IStockFundamentals | IStockFundamentals[]>>();
|
||||||
|
|
||||||
|
private logger = console;
|
||||||
|
|
||||||
|
private config: Required<IStockDataServiceConfig> = {
|
||||||
|
cache: {
|
||||||
|
priceTTL: 24 * 60 * 60 * 1000, // 24 hours
|
||||||
|
fundamentalsTTL: 90 * 24 * 60 * 60 * 1000, // 90 days
|
||||||
|
maxEntries: 10000
|
||||||
|
},
|
||||||
|
timeout: {
|
||||||
|
price: 10000, // 10 seconds
|
||||||
|
fundamentals: 30000 // 30 seconds
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
constructor(config?: IStockDataServiceConfig) {
|
||||||
|
if (config) {
|
||||||
|
this.config = {
|
||||||
|
cache: { ...this.config.cache, ...config.cache },
|
||||||
|
timeout: { ...this.config.timeout, ...config.timeout }
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ========== Provider Management ==========
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Register a price provider
|
||||||
|
*/
|
||||||
|
public registerPriceProvider(provider: IStockProvider, config?: IProviderConfig): void {
|
||||||
|
const defaultConfig: IProviderConfig = {
|
||||||
|
enabled: true,
|
||||||
|
priority: provider.priority,
|
||||||
|
timeout: this.config.timeout.price,
|
||||||
|
retryAttempts: 2,
|
||||||
|
retryDelay: 1000
|
||||||
|
};
|
||||||
|
|
||||||
|
const mergedConfig = { ...defaultConfig, ...config };
|
||||||
|
|
||||||
|
this.priceProviders.set(provider.name, {
|
||||||
|
provider,
|
||||||
|
config: mergedConfig,
|
||||||
|
successCount: 0,
|
||||||
|
errorCount: 0
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log(`Registered price provider: ${provider.name}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Register a fundamentals provider
|
||||||
|
*/
|
||||||
|
public registerFundamentalsProvider(
|
||||||
|
provider: IFundamentalsProvider,
|
||||||
|
config?: IFundamentalsProviderConfig
|
||||||
|
): void {
|
||||||
|
const defaultConfig: IFundamentalsProviderConfig = {
|
||||||
|
enabled: true,
|
||||||
|
priority: provider.priority,
|
||||||
|
timeout: this.config.timeout.fundamentals,
|
||||||
|
retryAttempts: 2,
|
||||||
|
retryDelay: 1000,
|
||||||
|
cacheTTL: this.config.cache.fundamentalsTTL
|
||||||
|
};
|
||||||
|
|
||||||
|
const mergedConfig = { ...defaultConfig, ...config };
|
||||||
|
|
||||||
|
this.fundamentalsProviders.set(provider.name, {
|
||||||
|
provider,
|
||||||
|
config: mergedConfig,
|
||||||
|
successCount: 0,
|
||||||
|
errorCount: 0
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log(`Registered fundamentals provider: ${provider.name}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unregister a price provider
|
||||||
|
*/
|
||||||
|
public unregisterPriceProvider(providerName: string): void {
|
||||||
|
this.priceProviders.delete(providerName);
|
||||||
|
console.log(`Unregistered price provider: ${providerName}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unregister a fundamentals provider
|
||||||
|
*/
|
||||||
|
public unregisterFundamentalsProvider(providerName: string): void {
|
||||||
|
this.fundamentalsProviders.delete(providerName);
|
||||||
|
console.log(`Unregistered fundamentals provider: ${providerName}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get all registered price providers
|
||||||
|
*/
|
||||||
|
public getPriceProviders(): IStockProvider[] {
|
||||||
|
return Array.from(this.priceProviders.values()).map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get all registered fundamentals providers
|
||||||
|
*/
|
||||||
|
public getFundamentalsProviders(): IFundamentalsProvider[] {
|
||||||
|
return Array.from(this.fundamentalsProviders.values()).map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get enabled price providers sorted by priority
|
||||||
|
*/
|
||||||
|
private getEnabledPriceProviders(): IStockProvider[] {
|
||||||
|
return Array.from(this.priceProviders.values())
|
||||||
|
.filter(entry => entry.config.enabled)
|
||||||
|
.sort((a, b) => (b.config.priority || 0) - (a.config.priority || 0))
|
||||||
|
.map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get enabled fundamentals providers sorted by priority
|
||||||
|
*/
|
||||||
|
private getEnabledFundamentalsProviders(): IFundamentalsProvider[] {
|
||||||
|
return Array.from(this.fundamentalsProviders.values())
|
||||||
|
.filter(entry => entry.config.enabled)
|
||||||
|
.sort((a, b) => (b.config.priority || 0) - (a.config.priority || 0))
|
||||||
|
.map(entry => entry.provider);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ========== Data Fetching Methods ==========
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get current price for a single ticker
|
||||||
|
*/
|
||||||
|
public async getPrice(ticker: string): Promise<IStockPrice> {
|
||||||
|
const cacheKey = `price:${ticker}`;
|
||||||
|
const cached = this.getFromCache(this.priceCache, cacheKey);
|
||||||
|
|
||||||
|
if (cached) {
|
||||||
|
console.log(`Cache hit for price: ${ticker}`);
|
||||||
|
return cached as IStockPrice;
|
||||||
|
}
|
||||||
|
|
||||||
|
const providers = this.getEnabledPriceProviders();
|
||||||
|
if (providers.length === 0) {
|
||||||
|
throw new Error('No price providers available');
|
||||||
|
}
|
||||||
|
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (const provider of providers) {
|
||||||
|
const entry = this.priceProviders.get(provider.name)!;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await this.fetchWithRetry(
|
||||||
|
() => provider.fetchData({ type: 'current', ticker }),
|
||||||
|
entry.config
|
||||||
|
);
|
||||||
|
|
||||||
|
entry.successCount++;
|
||||||
|
|
||||||
|
const price = result as IStockPrice;
|
||||||
|
this.addToCache(this.priceCache, cacheKey, price, this.config.cache.priceTTL);
|
||||||
|
|
||||||
|
console.log(`Successfully fetched price for ${ticker} from ${provider.name}`);
|
||||||
|
return price;
|
||||||
|
} catch (error) {
|
||||||
|
entry.errorCount++;
|
||||||
|
entry.lastError = error as Error;
|
||||||
|
entry.lastErrorTime = new Date();
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
console.warn(`Provider ${provider.name} failed for ${ticker}: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error(
|
||||||
|
`Failed to fetch price for ${ticker} from all providers. Last error: ${lastError?.message}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get current prices for multiple tickers
|
||||||
|
*/
|
||||||
|
public async getPrices(tickers: string[]): Promise<IStockPrice[]> {
|
||||||
|
const cacheKey = `prices:${tickers.sort().join(',')}`;
|
||||||
|
const cached = this.getFromCache(this.priceCache, cacheKey);
|
||||||
|
|
||||||
|
if (cached) {
|
||||||
|
console.log(`Cache hit for prices: ${tickers.length} tickers`);
|
||||||
|
return cached as IStockPrice[];
|
||||||
|
}
|
||||||
|
|
||||||
|
const providers = this.getEnabledPriceProviders();
|
||||||
|
if (providers.length === 0) {
|
||||||
|
throw new Error('No price providers available');
|
||||||
|
}
|
||||||
|
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (const provider of providers) {
|
||||||
|
const entry = this.priceProviders.get(provider.name)!;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await this.fetchWithRetry(
|
||||||
|
() => provider.fetchData({ type: 'batch', tickers }),
|
||||||
|
entry.config
|
||||||
|
);
|
||||||
|
|
||||||
|
entry.successCount++;
|
||||||
|
|
||||||
|
const prices = result as IStockPrice[];
|
||||||
|
this.addToCache(this.priceCache, cacheKey, prices, this.config.cache.priceTTL);
|
||||||
|
|
||||||
|
console.log(`Successfully fetched ${prices.length} prices from ${provider.name}`);
|
||||||
|
return prices;
|
||||||
|
} catch (error) {
|
||||||
|
entry.errorCount++;
|
||||||
|
entry.lastError = error as Error;
|
||||||
|
entry.lastErrorTime = new Date();
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
console.warn(`Provider ${provider.name} failed for batch prices: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error(
|
||||||
|
`Failed to fetch prices for ${tickers.length} tickers from all providers. Last error: ${lastError?.message}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get fundamentals for a single ticker
|
||||||
|
*/
|
||||||
|
public async getFundamentals(ticker: string): Promise<IStockFundamentals> {
|
||||||
|
const cacheKey = `fundamentals:${ticker}`;
|
||||||
|
const cached = this.getFromCache(this.fundamentalsCache, cacheKey);
|
||||||
|
|
||||||
|
if (cached) {
|
||||||
|
console.log(`Cache hit for fundamentals: ${ticker}`);
|
||||||
|
return cached as IStockFundamentals;
|
||||||
|
}
|
||||||
|
|
||||||
|
const providers = this.getEnabledFundamentalsProviders();
|
||||||
|
if (providers.length === 0) {
|
||||||
|
throw new Error('No fundamentals providers available');
|
||||||
|
}
|
||||||
|
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (const provider of providers) {
|
||||||
|
const entry = this.fundamentalsProviders.get(provider.name)!;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await this.fetchWithRetry(
|
||||||
|
() => provider.fetchData({ type: 'fundamentals-current', ticker }),
|
||||||
|
entry.config
|
||||||
|
);
|
||||||
|
|
||||||
|
entry.successCount++;
|
||||||
|
|
||||||
|
const fundamentals = result as IStockFundamentals;
|
||||||
|
const ttl = (entry.config as IFundamentalsProviderConfig).cacheTTL || this.config.cache.fundamentalsTTL;
|
||||||
|
this.addToCache(this.fundamentalsCache, cacheKey, fundamentals, ttl);
|
||||||
|
|
||||||
|
console.log(`Successfully fetched fundamentals for ${ticker} from ${provider.name}`);
|
||||||
|
return fundamentals;
|
||||||
|
} catch (error) {
|
||||||
|
entry.errorCount++;
|
||||||
|
entry.lastError = error as Error;
|
||||||
|
entry.lastErrorTime = new Date();
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
console.warn(`Provider ${provider.name} failed for ${ticker} fundamentals: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error(
|
||||||
|
`Failed to fetch fundamentals for ${ticker} from all providers. Last error: ${lastError?.message}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get fundamentals for multiple tickers
|
||||||
|
*/
|
||||||
|
public async getBatchFundamentals(tickers: string[]): Promise<IStockFundamentals[]> {
|
||||||
|
const cacheKey = `fundamentals-batch:${tickers.sort().join(',')}`;
|
||||||
|
const cached = this.getFromCache(this.fundamentalsCache, cacheKey);
|
||||||
|
|
||||||
|
if (cached) {
|
||||||
|
console.log(`Cache hit for batch fundamentals: ${tickers.length} tickers`);
|
||||||
|
return cached as IStockFundamentals[];
|
||||||
|
}
|
||||||
|
|
||||||
|
const providers = this.getEnabledFundamentalsProviders();
|
||||||
|
if (providers.length === 0) {
|
||||||
|
throw new Error('No fundamentals providers available');
|
||||||
|
}
|
||||||
|
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (const provider of providers) {
|
||||||
|
const entry = this.fundamentalsProviders.get(provider.name)!;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const result = await this.fetchWithRetry(
|
||||||
|
() => provider.fetchData({ type: 'fundamentals-batch', tickers }),
|
||||||
|
entry.config
|
||||||
|
);
|
||||||
|
|
||||||
|
entry.successCount++;
|
||||||
|
|
||||||
|
const fundamentals = result as IStockFundamentals[];
|
||||||
|
const ttl = (entry.config as IFundamentalsProviderConfig).cacheTTL || this.config.cache.fundamentalsTTL;
|
||||||
|
this.addToCache(this.fundamentalsCache, cacheKey, fundamentals, ttl);
|
||||||
|
|
||||||
|
console.log(`Successfully fetched ${fundamentals.length} fundamentals from ${provider.name}`);
|
||||||
|
return fundamentals;
|
||||||
|
} catch (error) {
|
||||||
|
entry.errorCount++;
|
||||||
|
entry.lastError = error as Error;
|
||||||
|
entry.lastErrorTime = new Date();
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
console.warn(`Provider ${provider.name} failed for batch fundamentals: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Error(
|
||||||
|
`Failed to fetch fundamentals for ${tickers.length} tickers from all providers. Last error: ${lastError?.message}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ✨ Get complete stock data (price + fundamentals) with automatic enrichment
|
||||||
|
*/
|
||||||
|
public async getStockData(request: string | ICompleteStockDataRequest): Promise<IStockData> {
|
||||||
|
const normalizedRequest = typeof request === 'string'
|
||||||
|
? { ticker: request, includeFundamentals: true, enrichFundamentals: true }
|
||||||
|
: { includeFundamentals: true, enrichFundamentals: true, ...request };
|
||||||
|
|
||||||
|
const price = await this.getPrice(normalizedRequest.ticker);
|
||||||
|
|
||||||
|
let fundamentals: IStockFundamentals | undefined;
|
||||||
|
|
||||||
|
if (normalizedRequest.includeFundamentals) {
|
||||||
|
try {
|
||||||
|
fundamentals = await this.getFundamentals(normalizedRequest.ticker);
|
||||||
|
|
||||||
|
// Enrich fundamentals with price calculations
|
||||||
|
if (normalizedRequest.enrichFundamentals && fundamentals) {
|
||||||
|
fundamentals = this.enrichWithPrice(fundamentals, price.price);
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
console.warn(`Failed to fetch fundamentals for ${normalizedRequest.ticker}: ${error.message}`);
|
||||||
|
// Continue without fundamentals
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
ticker: normalizedRequest.ticker,
|
||||||
|
price,
|
||||||
|
fundamentals,
|
||||||
|
fetchedAt: new Date()
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ✨ Get complete stock data for multiple tickers with automatic enrichment
|
||||||
|
*/
|
||||||
|
public async getBatchStockData(request: string[] | ICompleteStockDataBatchRequest): Promise<IStockData[]> {
|
||||||
|
const normalizedRequest = Array.isArray(request)
|
||||||
|
? { tickers: request, includeFundamentals: true, enrichFundamentals: true }
|
||||||
|
: { includeFundamentals: true, enrichFundamentals: true, ...request };
|
||||||
|
|
||||||
|
const prices = await this.getPrices(normalizedRequest.tickers);
|
||||||
|
const priceMap = new Map(prices.map(p => [p.ticker, p]));
|
||||||
|
|
||||||
|
let fundamentalsMap = new Map<string, IStockFundamentals>();
|
||||||
|
|
||||||
|
if (normalizedRequest.includeFundamentals) {
|
||||||
|
try {
|
||||||
|
const fundamentals = await this.getBatchFundamentals(normalizedRequest.tickers);
|
||||||
|
|
||||||
|
// Enrich with prices if requested
|
||||||
|
if (normalizedRequest.enrichFundamentals) {
|
||||||
|
for (const fund of fundamentals) {
|
||||||
|
const price = priceMap.get(fund.ticker);
|
||||||
|
if (price) {
|
||||||
|
fundamentalsMap.set(fund.ticker, this.enrichWithPrice(fund, price.price));
|
||||||
|
} else {
|
||||||
|
fundamentalsMap.set(fund.ticker, fund);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
fundamentalsMap = new Map(fundamentals.map(f => [f.ticker, f]));
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
console.warn(`Failed to fetch batch fundamentals: ${error.message}`);
|
||||||
|
// Continue without fundamentals
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return normalizedRequest.tickers.map(ticker => ({
|
||||||
|
ticker,
|
||||||
|
price: priceMap.get(ticker)!,
|
||||||
|
fundamentals: fundamentalsMap.get(ticker),
|
||||||
|
fetchedAt: new Date()
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
// ========== Helper Methods ==========
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Enrich fundamentals with calculated metrics using current price
|
||||||
|
*/
|
||||||
|
private enrichWithPrice(fundamentals: IStockFundamentals, price: number): IStockFundamentals {
|
||||||
|
const enriched = { ...fundamentals };
|
||||||
|
|
||||||
|
// Calculate market cap: price × shares outstanding
|
||||||
|
if (fundamentals.sharesOutstanding) {
|
||||||
|
enriched.marketCap = price * fundamentals.sharesOutstanding;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Calculate P/E ratio: price / EPS
|
||||||
|
if (fundamentals.earningsPerShareDiluted && fundamentals.earningsPerShareDiluted > 0) {
|
||||||
|
enriched.priceToEarnings = price / fundamentals.earningsPerShareDiluted;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Calculate price-to-book: market cap / stockholders equity
|
||||||
|
if (enriched.marketCap && fundamentals.stockholdersEquity && fundamentals.stockholdersEquity > 0) {
|
||||||
|
enriched.priceToBook = enriched.marketCap / fundamentals.stockholdersEquity;
|
||||||
|
}
|
||||||
|
|
||||||
|
return enriched;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch with retry logic
|
||||||
|
*/
|
||||||
|
private async fetchWithRetry<T>(
|
||||||
|
fetchFn: () => Promise<T>,
|
||||||
|
config: IProviderConfig | IFundamentalsProviderConfig
|
||||||
|
): Promise<T> {
|
||||||
|
const maxAttempts = config.retryAttempts || 1;
|
||||||
|
let lastError: Error | undefined;
|
||||||
|
|
||||||
|
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||||
|
try {
|
||||||
|
return await fetchFn();
|
||||||
|
} catch (error) {
|
||||||
|
lastError = error as Error;
|
||||||
|
|
||||||
|
if (attempt < maxAttempts) {
|
||||||
|
const delay = (config.retryDelay || 1000) * attempt;
|
||||||
|
console.log(`Retry attempt ${attempt} after ${delay}ms`);
|
||||||
|
await plugins.smartdelay.delayFor(delay);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw lastError || new Error('Unknown error during fetch');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get from cache if not expired
|
||||||
|
*/
|
||||||
|
private getFromCache<T>(cache: Map<string, ICacheEntry<T>>, key: string): T | null {
|
||||||
|
const entry = cache.get(key);
|
||||||
|
|
||||||
|
if (!entry) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if cache entry has expired
|
||||||
|
const age = Date.now() - entry.timestamp.getTime();
|
||||||
|
if (entry.ttl !== Infinity && age > entry.ttl) {
|
||||||
|
cache.delete(key);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return entry.data;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Add to cache with TTL
|
||||||
|
*/
|
||||||
|
private addToCache<T>(cache: Map<string, ICacheEntry<T>>, key: string, data: T, ttl: number): void {
|
||||||
|
// Enforce max entries limit
|
||||||
|
if (cache.size >= this.config.cache.maxEntries) {
|
||||||
|
// Remove oldest entry
|
||||||
|
const oldestKey = cache.keys().next().value;
|
||||||
|
if (oldestKey) {
|
||||||
|
cache.delete(oldestKey);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
cache.set(key, {
|
||||||
|
data,
|
||||||
|
timestamp: new Date(),
|
||||||
|
ttl
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ========== Health & Statistics ==========
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check health of all providers (both price and fundamentals)
|
||||||
|
*/
|
||||||
|
public async checkProvidersHealth(): Promise<Map<string, boolean>> {
|
||||||
|
const health = new Map<string, boolean>();
|
||||||
|
|
||||||
|
// Check price providers
|
||||||
|
for (const [name, entry] of this.priceProviders) {
|
||||||
|
if (!entry.config.enabled) {
|
||||||
|
health.set(`${name} (price)`, false);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const isAvailable = await entry.provider.isAvailable();
|
||||||
|
health.set(`${name} (price)`, isAvailable);
|
||||||
|
} catch (error) {
|
||||||
|
health.set(`${name} (price)`, false);
|
||||||
|
console.error(`Health check failed for ${name}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check fundamentals providers
|
||||||
|
for (const [name, entry] of this.fundamentalsProviders) {
|
||||||
|
if (!entry.config.enabled) {
|
||||||
|
health.set(`${name} (fundamentals)`, false);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const isAvailable = await entry.provider.isAvailable();
|
||||||
|
health.set(`${name} (fundamentals)`, isAvailable);
|
||||||
|
} catch (error) {
|
||||||
|
health.set(`${name} (fundamentals)`, false);
|
||||||
|
console.error(`Health check failed for ${name}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return health;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get statistics for all providers
|
||||||
|
*/
|
||||||
|
public getProviderStats(): Map<
|
||||||
|
string,
|
||||||
|
{
|
||||||
|
type: 'price' | 'fundamentals';
|
||||||
|
successCount: number;
|
||||||
|
errorCount: number;
|
||||||
|
lastError?: string;
|
||||||
|
lastErrorTime?: Date;
|
||||||
|
}
|
||||||
|
> {
|
||||||
|
const stats = new Map();
|
||||||
|
|
||||||
|
// Price provider stats
|
||||||
|
for (const [name, entry] of this.priceProviders) {
|
||||||
|
stats.set(name, {
|
||||||
|
type: 'price',
|
||||||
|
successCount: entry.successCount,
|
||||||
|
errorCount: entry.errorCount,
|
||||||
|
lastError: entry.lastError?.message,
|
||||||
|
lastErrorTime: entry.lastErrorTime
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fundamentals provider stats
|
||||||
|
for (const [name, entry] of this.fundamentalsProviders) {
|
||||||
|
stats.set(name, {
|
||||||
|
type: 'fundamentals',
|
||||||
|
successCount: entry.successCount,
|
||||||
|
errorCount: entry.errorCount,
|
||||||
|
lastError: entry.lastError?.message,
|
||||||
|
lastErrorTime: entry.lastErrorTime
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return stats;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clear all caches
|
||||||
|
*/
|
||||||
|
public clearCache(): void {
|
||||||
|
this.priceCache.clear();
|
||||||
|
this.fundamentalsCache.clear();
|
||||||
|
console.log('All caches cleared');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
*/
|
||||||
|
public getCacheStats(): {
|
||||||
|
priceCache: { size: number; ttl: number };
|
||||||
|
fundamentalsCache: { size: number; ttl: number };
|
||||||
|
maxEntries: number;
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
priceCache: {
|
||||||
|
size: this.priceCache.size,
|
||||||
|
ttl: this.config.cache.priceTTL
|
||||||
|
},
|
||||||
|
fundamentalsCache: {
|
||||||
|
size: this.fundamentalsCache.size,
|
||||||
|
ttl: this.config.cache.fundamentalsTTL
|
||||||
|
},
|
||||||
|
maxEntries: this.config.cache.maxEntries
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,24 @@
|
|||||||
import * as plugins from '../plugins.js';
|
import * as plugins from '../plugins.js';
|
||||||
import type { IStockProvider, IProviderConfig, IProviderRegistry } from './interfaces/provider.js';
|
import type { IStockProvider, IProviderConfig, IProviderRegistry } from './interfaces/provider.js';
|
||||||
import type { IStockPrice, IStockQuoteRequest, IStockBatchQuoteRequest, IStockPriceError } from './interfaces/stockprice.js';
|
import type {
|
||||||
|
IStockPrice,
|
||||||
|
IStockPriceError,
|
||||||
|
IStockDataRequest,
|
||||||
|
IStockCurrentRequest,
|
||||||
|
IStockHistoricalRequest,
|
||||||
|
IStockIntradayRequest,
|
||||||
|
IStockBatchCurrentRequest,
|
||||||
|
TIntervalType
|
||||||
|
} from './interfaces/stockprice.js';
|
||||||
|
|
||||||
|
// Simple request interfaces for convenience methods
|
||||||
|
interface ISimpleQuoteRequest {
|
||||||
|
ticker: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ISimpleBatchRequest {
|
||||||
|
tickers: string[];
|
||||||
|
}
|
||||||
|
|
||||||
interface IProviderEntry {
|
interface IProviderEntry {
|
||||||
provider: IStockProvider;
|
provider: IStockProvider;
|
||||||
@@ -12,18 +30,19 @@ interface IProviderEntry {
|
|||||||
}
|
}
|
||||||
|
|
||||||
interface ICacheEntry {
|
interface ICacheEntry {
|
||||||
price: IStockPrice;
|
price: IStockPrice | IStockPrice[];
|
||||||
timestamp: Date;
|
timestamp: Date;
|
||||||
|
ttl: number; // Specific TTL for this entry
|
||||||
}
|
}
|
||||||
|
|
||||||
export class StockPriceService implements IProviderRegistry {
|
export class StockPriceService implements IProviderRegistry {
|
||||||
private providers = new Map<string, IProviderEntry>();
|
private providers = new Map<string, IProviderEntry>();
|
||||||
private cache = new Map<string, ICacheEntry>();
|
private cache = new Map<string, ICacheEntry>();
|
||||||
private logger = console;
|
private logger = console;
|
||||||
|
|
||||||
private cacheConfig = {
|
private cacheConfig = {
|
||||||
ttl: 60000, // 60 seconds default
|
ttl: 60000, // 60 seconds default (for backward compatibility)
|
||||||
maxEntries: 1000
|
maxEntries: 10000 // Increased for historical data
|
||||||
};
|
};
|
||||||
|
|
||||||
constructor(cacheConfig?: { ttl?: number; maxEntries?: number }) {
|
constructor(cacheConfig?: { ttl?: number; maxEntries?: number }) {
|
||||||
@@ -32,6 +51,43 @@ export class StockPriceService implements IProviderRegistry {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get data-type aware TTL for smart caching
|
||||||
|
*/
|
||||||
|
private getCacheTTL(dataType: 'eod' | 'historical' | 'intraday' | 'live', interval?: TIntervalType): number {
|
||||||
|
switch (dataType) {
|
||||||
|
case 'historical':
|
||||||
|
return Infinity; // Historical data never changes
|
||||||
|
case 'eod':
|
||||||
|
return 24 * 60 * 60 * 1000; // 24 hours (EOD is static after market close)
|
||||||
|
case 'intraday':
|
||||||
|
// Match cache TTL to interval
|
||||||
|
return this.getIntervalMs(interval);
|
||||||
|
case 'live':
|
||||||
|
return 30 * 1000; // 30 seconds for live data
|
||||||
|
default:
|
||||||
|
return this.cacheConfig.ttl; // Fallback to default
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Convert interval to milliseconds
|
||||||
|
*/
|
||||||
|
private getIntervalMs(interval?: TIntervalType): number {
|
||||||
|
if (!interval) return 60 * 1000; // Default 1 minute
|
||||||
|
|
||||||
|
const intervalMap: Record<TIntervalType, number> = {
|
||||||
|
'1min': 60 * 1000,
|
||||||
|
'5min': 5 * 60 * 1000,
|
||||||
|
'10min': 10 * 60 * 1000,
|
||||||
|
'15min': 15 * 60 * 1000,
|
||||||
|
'30min': 30 * 60 * 1000,
|
||||||
|
'1hour': 60 * 60 * 1000
|
||||||
|
};
|
||||||
|
|
||||||
|
return intervalMap[interval] || 60 * 1000;
|
||||||
|
}
|
||||||
|
|
||||||
public register(provider: IStockProvider, config?: IProviderConfig): void {
|
public register(provider: IStockProvider, config?: IProviderConfig): void {
|
||||||
const defaultConfig: IProviderConfig = {
|
const defaultConfig: IProviderConfig = {
|
||||||
enabled: true,
|
enabled: true,
|
||||||
@@ -73,12 +129,37 @@ export class StockPriceService implements IProviderRegistry {
|
|||||||
.map(entry => entry.provider);
|
.map(entry => entry.provider);
|
||||||
}
|
}
|
||||||
|
|
||||||
public async getPrice(request: IStockQuoteRequest): Promise<IStockPrice> {
|
/**
|
||||||
const cacheKey = this.getCacheKey(request);
|
* Convenience method: Get current price for a single ticker
|
||||||
|
*/
|
||||||
|
public async getPrice(request: ISimpleQuoteRequest): Promise<IStockPrice> {
|
||||||
|
const result = await this.getData({
|
||||||
|
type: 'current',
|
||||||
|
ticker: request.ticker
|
||||||
|
});
|
||||||
|
return result as IStockPrice;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Convenience method: Get current prices for multiple tickers
|
||||||
|
*/
|
||||||
|
public async getPrices(request: ISimpleBatchRequest): Promise<IStockPrice[]> {
|
||||||
|
const result = await this.getData({
|
||||||
|
type: 'batch',
|
||||||
|
tickers: request.tickers
|
||||||
|
});
|
||||||
|
return result as IStockPrice[];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* New unified data fetching method supporting all request types
|
||||||
|
*/
|
||||||
|
public async getData(request: IStockDataRequest): Promise<IStockPrice | IStockPrice[]> {
|
||||||
|
const cacheKey = this.getDataCacheKey(request);
|
||||||
const cached = this.getFromCache(cacheKey);
|
const cached = this.getFromCache(cacheKey);
|
||||||
|
|
||||||
if (cached) {
|
if (cached) {
|
||||||
console.log(`Cache hit for ${request.ticker}`);
|
console.log(`Cache hit for ${this.getRequestDescription(request)}`);
|
||||||
return cached;
|
return cached;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -91,109 +172,72 @@ export class StockPriceService implements IProviderRegistry {
|
|||||||
|
|
||||||
for (const provider of providers) {
|
for (const provider of providers) {
|
||||||
const entry = this.providers.get(provider.name)!;
|
const entry = this.providers.get(provider.name)!;
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const price = await this.fetchWithRetry(
|
const result = await this.fetchWithRetry(
|
||||||
() => provider.fetchPrice(request),
|
() => provider.fetchData(request),
|
||||||
entry.config
|
entry.config
|
||||||
);
|
) as IStockPrice | IStockPrice[];
|
||||||
|
|
||||||
entry.successCount++;
|
entry.successCount++;
|
||||||
this.addToCache(cacheKey, price);
|
|
||||||
console.log(`Successfully fetched ${request.ticker} from ${provider.name}`);
|
// Determine TTL based on request type
|
||||||
return price;
|
const ttl = this.getRequestTTL(request, result);
|
||||||
|
this.addToCache(cacheKey, result, ttl);
|
||||||
|
|
||||||
|
console.log(`Successfully fetched ${this.getRequestDescription(request)} from ${provider.name}`);
|
||||||
|
return result;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
entry.errorCount++;
|
entry.errorCount++;
|
||||||
entry.lastError = error as Error;
|
entry.lastError = error as Error;
|
||||||
entry.lastErrorTime = new Date();
|
entry.lastErrorTime = new Date();
|
||||||
lastError = error as Error;
|
lastError = error as Error;
|
||||||
|
|
||||||
console.warn(
|
console.warn(
|
||||||
`Provider ${provider.name} failed for ${request.ticker}: ${error.message}`
|
`Provider ${provider.name} failed for ${this.getRequestDescription(request)}: ${error.message}`
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
throw new Error(
|
throw new Error(
|
||||||
`Failed to fetch price for ${request.ticker} from all providers. Last error: ${lastError?.message}`
|
`Failed to fetch ${this.getRequestDescription(request)} from all providers. Last error: ${lastError?.message}`
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
public async getPrices(request: IStockBatchQuoteRequest): Promise<IStockPrice[]> {
|
/**
|
||||||
const cachedPrices: IStockPrice[] = [];
|
* Get TTL based on request type and result
|
||||||
const tickersToFetch: string[] = [];
|
*/
|
||||||
|
private getRequestTTL(request: IStockDataRequest, result: IStockPrice | IStockPrice[]): number {
|
||||||
// Check cache for each ticker
|
switch (request.type) {
|
||||||
for (const ticker of request.tickers) {
|
case 'historical':
|
||||||
const cacheKey = this.getCacheKey({ ticker, includeExtendedHours: request.includeExtendedHours });
|
return Infinity; // Historical data never changes
|
||||||
const cached = this.getFromCache(cacheKey);
|
case 'current':
|
||||||
|
return this.getCacheTTL('eod');
|
||||||
if (cached) {
|
case 'batch':
|
||||||
cachedPrices.push(cached);
|
return this.getCacheTTL('eod');
|
||||||
} else {
|
case 'intraday':
|
||||||
tickersToFetch.push(ticker);
|
return this.getCacheTTL('intraday', request.interval);
|
||||||
}
|
default:
|
||||||
|
return this.cacheConfig.ttl;
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if (tickersToFetch.length === 0) {
|
/**
|
||||||
console.log(`All ${request.tickers.length} tickers served from cache`);
|
* Get human-readable description of request
|
||||||
return cachedPrices;
|
*/
|
||||||
|
private getRequestDescription(request: IStockDataRequest): string {
|
||||||
|
switch (request.type) {
|
||||||
|
case 'current':
|
||||||
|
return `current price for ${request.ticker}${request.exchange ? ` on ${request.exchange}` : ''}`;
|
||||||
|
case 'historical':
|
||||||
|
return `historical prices for ${request.ticker} from ${request.from.toISOString().split('T')[0]} to ${request.to.toISOString().split('T')[0]}`;
|
||||||
|
case 'intraday':
|
||||||
|
return `intraday ${request.interval} prices for ${request.ticker}`;
|
||||||
|
case 'batch':
|
||||||
|
return `batch prices for ${request.tickers.length} tickers`;
|
||||||
|
default:
|
||||||
|
return 'data';
|
||||||
}
|
}
|
||||||
|
|
||||||
const providers = this.getEnabledProviders();
|
|
||||||
if (providers.length === 0) {
|
|
||||||
throw new Error('No stock price providers available');
|
|
||||||
}
|
|
||||||
|
|
||||||
let lastError: Error | undefined;
|
|
||||||
let fetchedPrices: IStockPrice[] = [];
|
|
||||||
|
|
||||||
for (const provider of providers) {
|
|
||||||
const entry = this.providers.get(provider.name)!;
|
|
||||||
|
|
||||||
try {
|
|
||||||
fetchedPrices = await this.fetchWithRetry(
|
|
||||||
() => provider.fetchPrices({
|
|
||||||
tickers: tickersToFetch,
|
|
||||||
includeExtendedHours: request.includeExtendedHours
|
|
||||||
}),
|
|
||||||
entry.config
|
|
||||||
);
|
|
||||||
|
|
||||||
entry.successCount++;
|
|
||||||
|
|
||||||
// Cache the fetched prices
|
|
||||||
for (const price of fetchedPrices) {
|
|
||||||
const cacheKey = this.getCacheKey({
|
|
||||||
ticker: price.ticker,
|
|
||||||
includeExtendedHours: request.includeExtendedHours
|
|
||||||
});
|
|
||||||
this.addToCache(cacheKey, price);
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log(
|
|
||||||
`Successfully fetched ${fetchedPrices.length} prices from ${provider.name}`
|
|
||||||
);
|
|
||||||
break;
|
|
||||||
} catch (error) {
|
|
||||||
entry.errorCount++;
|
|
||||||
entry.lastError = error as Error;
|
|
||||||
entry.lastErrorTime = new Date();
|
|
||||||
lastError = error as Error;
|
|
||||||
|
|
||||||
console.warn(
|
|
||||||
`Provider ${provider.name} failed for batch request: ${error.message}`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (fetchedPrices.length === 0 && lastError) {
|
|
||||||
throw new Error(
|
|
||||||
`Failed to fetch prices from all providers. Last error: ${lastError.message}`
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
return [...cachedPrices, ...fetchedPrices];
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public async checkProvidersHealth(): Promise<Map<string, boolean>> {
|
public async checkProvidersHealth(): Promise<Map<string, boolean>> {
|
||||||
@@ -271,19 +315,38 @@ export class StockPriceService implements IProviderRegistry {
|
|||||||
throw lastError || new Error('Unknown error during fetch');
|
throw lastError || new Error('Unknown error during fetch');
|
||||||
}
|
}
|
||||||
|
|
||||||
private getCacheKey(request: IStockQuoteRequest): string {
|
/**
|
||||||
return `${request.ticker}:${request.includeExtendedHours || false}`;
|
* New cache key generation for discriminated union requests
|
||||||
|
*/
|
||||||
|
private getDataCacheKey(request: IStockDataRequest): string {
|
||||||
|
switch (request.type) {
|
||||||
|
case 'current':
|
||||||
|
return `current:${request.ticker}${request.exchange ? `:${request.exchange}` : ''}`;
|
||||||
|
case 'historical':
|
||||||
|
const fromStr = request.from.toISOString().split('T')[0];
|
||||||
|
const toStr = request.to.toISOString().split('T')[0];
|
||||||
|
return `historical:${request.ticker}:${fromStr}:${toStr}${request.exchange ? `:${request.exchange}` : ''}`;
|
||||||
|
case 'intraday':
|
||||||
|
const dateStr = request.date ? request.date.toISOString().split('T')[0] : 'latest';
|
||||||
|
return `intraday:${request.ticker}:${request.interval}:${dateStr}${request.exchange ? `:${request.exchange}` : ''}`;
|
||||||
|
case 'batch':
|
||||||
|
const tickers = request.tickers.sort().join(',');
|
||||||
|
return `batch:${tickers}${request.exchange ? `:${request.exchange}` : ''}`;
|
||||||
|
default:
|
||||||
|
return `unknown:${JSON.stringify(request)}`;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private getFromCache(key: string): IStockPrice | null {
|
private getFromCache(key: string): IStockPrice | IStockPrice[] | null {
|
||||||
const entry = this.cache.get(key);
|
const entry = this.cache.get(key);
|
||||||
|
|
||||||
if (!entry) {
|
if (!entry) {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Check if cache entry has expired
|
||||||
const age = Date.now() - entry.timestamp.getTime();
|
const age = Date.now() - entry.timestamp.getTime();
|
||||||
if (age > this.cacheConfig.ttl) {
|
if (entry.ttl !== Infinity && age > entry.ttl) {
|
||||||
this.cache.delete(key);
|
this.cache.delete(key);
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
@@ -291,7 +354,7 @@ export class StockPriceService implements IProviderRegistry {
|
|||||||
return entry.price;
|
return entry.price;
|
||||||
}
|
}
|
||||||
|
|
||||||
private addToCache(key: string, price: IStockPrice): void {
|
private addToCache(key: string, price: IStockPrice | IStockPrice[], ttl?: number): void {
|
||||||
// Enforce max entries limit
|
// Enforce max entries limit
|
||||||
if (this.cache.size >= this.cacheConfig.maxEntries) {
|
if (this.cache.size >= this.cacheConfig.maxEntries) {
|
||||||
// Remove oldest entry
|
// Remove oldest entry
|
||||||
@@ -303,7 +366,8 @@ export class StockPriceService implements IProviderRegistry {
|
|||||||
|
|
||||||
this.cache.set(key, {
|
this.cache.set(key, {
|
||||||
price,
|
price,
|
||||||
timestamp: new Date()
|
timestamp: new Date(),
|
||||||
|
ttl: ttl || this.cacheConfig.ttl
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,10 +1,18 @@
|
|||||||
// Export all interfaces
|
// Export all interfaces
|
||||||
export * from './interfaces/stockprice.js';
|
export * from './interfaces/stockprice.js';
|
||||||
export * from './interfaces/provider.js';
|
export * from './interfaces/provider.js';
|
||||||
|
export * from './interfaces/fundamentals.js';
|
||||||
|
export * from './interfaces/stockdata.js';
|
||||||
|
|
||||||
// Export main service
|
// Export main services
|
||||||
export * from './classes.stockservice.js';
|
export * from './classes.stockservice.js';
|
||||||
|
export * from './classes.fundamentalsservice.js';
|
||||||
|
export * from './classes.stockdataservice.js'; // ✨ New unified service
|
||||||
|
|
||||||
|
// Export base service (for advanced use cases)
|
||||||
|
export * from './classes.baseproviderservice.js';
|
||||||
|
|
||||||
// Export providers
|
// Export providers
|
||||||
export * from './providers/provider.yahoo.js';
|
export * from './providers/provider.yahoo.js';
|
||||||
export * from './providers/provider.marketstack.js';
|
export * from './providers/provider.marketstack.js';
|
||||||
|
export * from './providers/provider.secedgar.js';
|
||||||
102
ts/stocks/interfaces/fundamentals.ts
Normal file
102
ts/stocks/interfaces/fundamentals.ts
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
/**
|
||||||
|
* Interfaces for stock fundamental data (financials from SEC filings)
|
||||||
|
* Separate from stock price data (OHLCV) to maintain clean architecture
|
||||||
|
*/
|
||||||
|
|
||||||
|
// Request types for fundamental data
|
||||||
|
export interface IFundamentalsCurrentRequest {
|
||||||
|
type: 'fundamentals-current';
|
||||||
|
ticker: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface IFundamentalsBatchRequest {
|
||||||
|
type: 'fundamentals-batch';
|
||||||
|
tickers: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export type IFundamentalsRequest =
|
||||||
|
| IFundamentalsCurrentRequest
|
||||||
|
| IFundamentalsBatchRequest;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stock fundamental data from SEC filings (10-K, 10-Q)
|
||||||
|
* Contains financial metrics like EPS, Revenue, Assets, etc.
|
||||||
|
*/
|
||||||
|
export interface IStockFundamentals {
|
||||||
|
ticker: string;
|
||||||
|
cik: string;
|
||||||
|
companyName: string;
|
||||||
|
provider: string;
|
||||||
|
timestamp: Date;
|
||||||
|
fetchedAt: Date;
|
||||||
|
|
||||||
|
// Per-share metrics
|
||||||
|
earningsPerShareBasic?: number;
|
||||||
|
earningsPerShareDiluted?: number;
|
||||||
|
sharesOutstanding?: number;
|
||||||
|
weightedAverageSharesOutstanding?: number;
|
||||||
|
|
||||||
|
// Income statement (annual USD)
|
||||||
|
revenue?: number;
|
||||||
|
netIncome?: number;
|
||||||
|
operatingIncome?: number;
|
||||||
|
grossProfit?: number;
|
||||||
|
costOfRevenue?: number;
|
||||||
|
|
||||||
|
// Balance sheet (annual USD)
|
||||||
|
assets?: number;
|
||||||
|
liabilities?: number;
|
||||||
|
stockholdersEquity?: number;
|
||||||
|
cash?: number;
|
||||||
|
propertyPlantEquipment?: number;
|
||||||
|
|
||||||
|
// Calculated metrics (requires current price)
|
||||||
|
marketCap?: number; // price × sharesOutstanding
|
||||||
|
priceToEarnings?: number; // price / EPS
|
||||||
|
priceToBook?: number; // marketCap / stockholdersEquity
|
||||||
|
|
||||||
|
// Metadata
|
||||||
|
fiscalYear?: string;
|
||||||
|
fiscalQuarter?: string;
|
||||||
|
filingDate?: Date;
|
||||||
|
form?: '10-K' | '10-Q' | string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Provider interface for fetching fundamental data
|
||||||
|
* Parallel to IStockProvider but for fundamentals instead of prices
|
||||||
|
*/
|
||||||
|
export interface IFundamentalsProvider {
|
||||||
|
name: string;
|
||||||
|
priority: number;
|
||||||
|
fetchData(request: IFundamentalsRequest): Promise<IStockFundamentals | IStockFundamentals[]>;
|
||||||
|
isAvailable(): Promise<boolean>;
|
||||||
|
readonly requiresAuth: boolean;
|
||||||
|
readonly rateLimit?: {
|
||||||
|
requestsPerMinute: number;
|
||||||
|
requestsPerDay?: number;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Configuration for fundamentals providers
|
||||||
|
*/
|
||||||
|
export interface IFundamentalsProviderConfig {
|
||||||
|
enabled: boolean;
|
||||||
|
priority?: number;
|
||||||
|
timeout?: number;
|
||||||
|
retryAttempts?: number;
|
||||||
|
retryDelay?: number;
|
||||||
|
cacheTTL?: number; // Custom cache TTL for this provider
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Registry for managing fundamental data providers
|
||||||
|
*/
|
||||||
|
export interface IFundamentalsProviderRegistry {
|
||||||
|
register(provider: IFundamentalsProvider, config?: IFundamentalsProviderConfig): void;
|
||||||
|
unregister(providerName: string): void;
|
||||||
|
getProvider(name: string): IFundamentalsProvider | undefined;
|
||||||
|
getAllProviders(): IFundamentalsProvider[];
|
||||||
|
getEnabledProviders(): IFundamentalsProvider[];
|
||||||
|
}
|
||||||
@@ -1,16 +1,15 @@
|
|||||||
import type { IStockPrice, IStockQuoteRequest, IStockBatchQuoteRequest } from './stockprice.js';
|
import type { IStockPrice, IStockDataRequest } from './stockprice.js';
|
||||||
|
|
||||||
export interface IStockProvider {
|
export interface IStockProvider {
|
||||||
name: string;
|
name: string;
|
||||||
priority: number;
|
priority: number;
|
||||||
|
|
||||||
fetchPrice(request: IStockQuoteRequest): Promise<IStockPrice>;
|
fetchData(request: IStockDataRequest): Promise<IStockPrice | IStockPrice[]>;
|
||||||
fetchPrices(request: IStockBatchQuoteRequest): Promise<IStockPrice[]>;
|
|
||||||
isAvailable(): Promise<boolean>;
|
isAvailable(): Promise<boolean>;
|
||||||
|
|
||||||
supportsMarket?(market: string): boolean;
|
supportsMarket?(market: string): boolean;
|
||||||
supportsTicker?(ticker: string): boolean;
|
supportsTicker?(ticker: string): boolean;
|
||||||
|
|
||||||
readonly requiresAuth: boolean;
|
readonly requiresAuth: boolean;
|
||||||
readonly rateLimit?: {
|
readonly rateLimit?: {
|
||||||
requestsPerMinute: number;
|
requestsPerMinute: number;
|
||||||
|
|||||||
65
ts/stocks/interfaces/stockdata.ts
Normal file
65
ts/stocks/interfaces/stockdata.ts
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
import type { IStockPrice } from './stockprice.js';
|
||||||
|
import type { IStockFundamentals } from './fundamentals.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Combined stock data with price and fundamentals
|
||||||
|
* All calculated metrics (market cap, P/E, P/B) are automatically included
|
||||||
|
*/
|
||||||
|
export interface IStockData {
|
||||||
|
/** Stock ticker symbol */
|
||||||
|
ticker: string;
|
||||||
|
|
||||||
|
/** Price information */
|
||||||
|
price: IStockPrice;
|
||||||
|
|
||||||
|
/** Fundamental data (optional - may not be available for all stocks) */
|
||||||
|
fundamentals?: IStockFundamentals;
|
||||||
|
|
||||||
|
/** When this combined data was fetched */
|
||||||
|
fetchedAt: Date;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Configuration for StockDataService
|
||||||
|
*/
|
||||||
|
export interface IStockDataServiceConfig {
|
||||||
|
/** Cache configuration */
|
||||||
|
cache?: {
|
||||||
|
/** TTL for price data (default: 24 hours) */
|
||||||
|
priceTTL?: number;
|
||||||
|
/** TTL for fundamentals data (default: 90 days) */
|
||||||
|
fundamentalsTTL?: number;
|
||||||
|
/** Max cache entries (default: 10000) */
|
||||||
|
maxEntries?: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Provider timeouts */
|
||||||
|
timeout?: {
|
||||||
|
/** Timeout for price providers (default: 10000ms) */
|
||||||
|
price?: number;
|
||||||
|
/** Timeout for fundamentals providers (default: 30000ms) */
|
||||||
|
fundamentals?: number;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Request type for getting complete stock data
|
||||||
|
*/
|
||||||
|
export interface ICompleteStockDataRequest {
|
||||||
|
ticker: string;
|
||||||
|
/** Whether to include fundamentals (default: true) */
|
||||||
|
includeFundamentals?: boolean;
|
||||||
|
/** Whether to enrich fundamentals with price calculations (default: true) */
|
||||||
|
enrichFundamentals?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Batch request for multiple stocks
|
||||||
|
*/
|
||||||
|
export interface ICompleteStockDataBatchRequest {
|
||||||
|
tickers: string[];
|
||||||
|
/** Whether to include fundamentals (default: true) */
|
||||||
|
includeFundamentals?: boolean;
|
||||||
|
/** Whether to enrich fundamentals with price calculations (default: true) */
|
||||||
|
enrichFundamentals?: boolean;
|
||||||
|
}
|
||||||
@@ -1,5 +1,4 @@
|
|||||||
import * as plugins from '../../plugins.js';
|
// Enhanced stock price interface with additional OHLCV data
|
||||||
|
|
||||||
export interface IStockPrice {
|
export interface IStockPrice {
|
||||||
ticker: string;
|
ticker: string;
|
||||||
price: number;
|
price: number;
|
||||||
@@ -12,11 +11,20 @@ export interface IStockPrice {
|
|||||||
marketState: 'PRE' | 'REGULAR' | 'POST' | 'CLOSED';
|
marketState: 'PRE' | 'REGULAR' | 'POST' | 'CLOSED';
|
||||||
exchange?: string;
|
exchange?: string;
|
||||||
exchangeName?: string;
|
exchangeName?: string;
|
||||||
|
|
||||||
|
// Phase 1 enhancements
|
||||||
|
volume?: number; // Trading volume
|
||||||
|
open?: number; // Opening price
|
||||||
|
high?: number; // Day high
|
||||||
|
low?: number; // Day low
|
||||||
|
adjusted?: boolean; // If price is split/dividend adjusted
|
||||||
|
dataType: 'eod' | 'intraday' | 'live'; // What kind of data this is
|
||||||
|
fetchedAt: Date; // When we fetched (vs data timestamp)
|
||||||
|
|
||||||
|
// Company identification
|
||||||
|
companyName?: string; // Company name (e.g., "Apple Inc.")
|
||||||
|
companyFullName?: string; // Full company name with exchange (e.g., "Apple Inc. (NASDAQ:AAPL)")
|
||||||
}
|
}
|
||||||
type CheckStockPrice = plugins.tsclass.typeFest.IsEqual<
|
|
||||||
IStockPrice,
|
|
||||||
plugins.tsclass.finance.IStockPrice
|
|
||||||
>;
|
|
||||||
|
|
||||||
export interface IStockPriceError {
|
export interface IStockPriceError {
|
||||||
ticker: string;
|
ticker: string;
|
||||||
@@ -25,12 +33,62 @@ export interface IStockPriceError {
|
|||||||
timestamp: Date;
|
timestamp: Date;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface IStockQuoteRequest {
|
// Pagination support for large datasets
|
||||||
ticker: string;
|
export interface IPaginatedResponse<T> {
|
||||||
includeExtendedHours?: boolean;
|
data: T[];
|
||||||
|
pagination: {
|
||||||
|
currentPage: number;
|
||||||
|
totalPages: number;
|
||||||
|
totalRecords: number;
|
||||||
|
hasMore: boolean;
|
||||||
|
limit: number;
|
||||||
|
offset: number;
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface IStockBatchQuoteRequest {
|
// Phase 1: Discriminated union types for different request types
|
||||||
tickers: string[];
|
export type TIntervalType = '1min' | '5min' | '10min' | '15min' | '30min' | '1hour';
|
||||||
includeExtendedHours?: boolean;
|
export type TSortOrder = 'ASC' | 'DESC';
|
||||||
|
|
||||||
|
// Current price request (latest EOD or live)
|
||||||
|
export interface IStockCurrentRequest {
|
||||||
|
type: 'current';
|
||||||
|
ticker: string;
|
||||||
|
exchange?: string; // MIC code like 'XNAS', 'XNYS', 'XLON'
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Historical price request (date range)
|
||||||
|
export interface IStockHistoricalRequest {
|
||||||
|
type: 'historical';
|
||||||
|
ticker: string;
|
||||||
|
from: Date;
|
||||||
|
to: Date;
|
||||||
|
exchange?: string;
|
||||||
|
sort?: TSortOrder;
|
||||||
|
limit?: number; // Max results per page (default 1000)
|
||||||
|
offset?: number; // For pagination
|
||||||
|
}
|
||||||
|
|
||||||
|
// Intraday price request (real-time intervals)
|
||||||
|
export interface IStockIntradayRequest {
|
||||||
|
type: 'intraday';
|
||||||
|
ticker: string;
|
||||||
|
interval: TIntervalType;
|
||||||
|
exchange?: string;
|
||||||
|
limit?: number; // Number of bars to return
|
||||||
|
date?: Date; // Specific date for historical intraday
|
||||||
|
}
|
||||||
|
|
||||||
|
// Batch current prices request
|
||||||
|
export interface IStockBatchCurrentRequest {
|
||||||
|
type: 'batch';
|
||||||
|
tickers: string[];
|
||||||
|
exchange?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Union type for all stock data requests
|
||||||
|
export type IStockDataRequest =
|
||||||
|
| IStockCurrentRequest
|
||||||
|
| IStockHistoricalRequest
|
||||||
|
| IStockIntradayRequest
|
||||||
|
| IStockBatchCurrentRequest;
|
||||||
|
|||||||
@@ -1,22 +1,37 @@
|
|||||||
import * as plugins from '../../plugins.js';
|
import * as plugins from '../../plugins.js';
|
||||||
import type { IStockProvider, IProviderConfig } from '../interfaces/provider.js';
|
import type { IStockProvider, IProviderConfig } from '../interfaces/provider.js';
|
||||||
import type { IStockPrice, IStockQuoteRequest, IStockBatchQuoteRequest } from '../interfaces/stockprice.js';
|
import type {
|
||||||
|
IStockPrice,
|
||||||
|
IStockDataRequest,
|
||||||
|
IStockCurrentRequest,
|
||||||
|
IStockHistoricalRequest,
|
||||||
|
IStockIntradayRequest,
|
||||||
|
IStockBatchCurrentRequest
|
||||||
|
} from '../interfaces/stockprice.js';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Marketstack API v2 Provider
|
* Marketstack API v2 Provider - Enhanced
|
||||||
* Documentation: https://marketstack.com/documentation_v2
|
* Documentation: https://docs.apilayer.com/marketstack/docs/marketstack-api-v2-v-2-0-0
|
||||||
*
|
*
|
||||||
* Features:
|
* Features:
|
||||||
* - End-of-Day (EOD) stock prices
|
* - End-of-Day (EOD) stock prices with historical data
|
||||||
* - Supports 125,000+ tickers across 72+ exchanges worldwide
|
* - Intraday pricing with multiple intervals (1min, 5min, 15min, 30min, 1hour)
|
||||||
|
* - Exchange filtering via MIC codes (XNAS, XNYS, XLON, etc.)
|
||||||
|
* - Supports 500,000+ tickers across 72+ exchanges worldwide
|
||||||
|
* - OHLCV data (Open, High, Low, Close, Volume)
|
||||||
|
* - Pagination for large datasets
|
||||||
* - Requires API key authentication
|
* - Requires API key authentication
|
||||||
*
|
*
|
||||||
* Rate Limits:
|
* Rate Limits:
|
||||||
* - Free Plan: 100 requests/month (EOD only)
|
* - Free Plan: 100 requests/month (EOD only)
|
||||||
* - Basic Plan: 10,000 requests/month
|
* - Basic Plan: 10,000 requests/month
|
||||||
* - Professional Plan: 100,000 requests/month
|
* - Professional Plan: 100,000 requests/month (intraday access)
|
||||||
*
|
*
|
||||||
* Note: This provider returns EOD data, not real-time prices
|
* Phase 1 Enhancements:
|
||||||
|
* - Historical data retrieval with date ranges
|
||||||
|
* - Exchange filtering
|
||||||
|
* - OHLCV data support
|
||||||
|
* - Pagination handling
|
||||||
*/
|
*/
|
||||||
export class MarketstackProvider implements IStockProvider {
|
export class MarketstackProvider implements IStockProvider {
|
||||||
public name = 'Marketstack';
|
public name = 'Marketstack';
|
||||||
@@ -40,11 +55,34 @@ export class MarketstackProvider implements IStockProvider {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Fetch latest EOD price for a single ticker
|
* Unified data fetching method supporting all request types
|
||||||
*/
|
*/
|
||||||
public async fetchPrice(request: IStockQuoteRequest): Promise<IStockPrice> {
|
public async fetchData(request: IStockDataRequest): Promise<IStockPrice[] | IStockPrice> {
|
||||||
|
switch (request.type) {
|
||||||
|
case 'current':
|
||||||
|
return this.fetchCurrentPrice(request);
|
||||||
|
case 'historical':
|
||||||
|
return this.fetchHistoricalPrices(request);
|
||||||
|
case 'intraday':
|
||||||
|
return this.fetchIntradayPrices(request);
|
||||||
|
case 'batch':
|
||||||
|
return this.fetchBatchCurrentPrices(request);
|
||||||
|
default:
|
||||||
|
throw new Error(`Unsupported request type: ${(request as any).type}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch current/latest EOD price for a single ticker (new API)
|
||||||
|
*/
|
||||||
|
private async fetchCurrentPrice(request: IStockCurrentRequest): Promise<IStockPrice> {
|
||||||
try {
|
try {
|
||||||
const url = `${this.baseUrl}/tickers/${request.ticker}/eod/latest?access_key=${this.apiKey}`;
|
let url = `${this.baseUrl}/tickers/${request.ticker}/eod/latest?access_key=${this.apiKey}`;
|
||||||
|
|
||||||
|
// Add exchange filter if specified
|
||||||
|
if (request.exchange) {
|
||||||
|
url += `&exchange=${request.exchange}`;
|
||||||
|
}
|
||||||
|
|
||||||
const response = await plugins.smartrequest.SmartRequest.create()
|
const response = await plugins.smartrequest.SmartRequest.create()
|
||||||
.url(url)
|
.url(url)
|
||||||
@@ -63,20 +101,101 @@ export class MarketstackProvider implements IStockProvider {
|
|||||||
throw new Error(`No data found for ticker ${request.ticker}`);
|
throw new Error(`No data found for ticker ${request.ticker}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
return this.mapToStockPrice(responseData);
|
return this.mapToStockPrice(responseData, 'eod');
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.logger.error(`Failed to fetch price for ${request.ticker}:`, error);
|
this.logger.error(`Failed to fetch current price for ${request.ticker}:`, error);
|
||||||
throw new Error(`Marketstack: Failed to fetch price for ${request.ticker}: ${error.message}`);
|
throw new Error(`Marketstack: Failed to fetch current price for ${request.ticker}: ${error.message}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Fetch latest EOD prices for multiple tickers
|
* Fetch historical EOD prices for a ticker with date range
|
||||||
*/
|
*/
|
||||||
public async fetchPrices(request: IStockBatchQuoteRequest): Promise<IStockPrice[]> {
|
private async fetchHistoricalPrices(request: IStockHistoricalRequest): Promise<IStockPrice[]> {
|
||||||
|
try {
|
||||||
|
const allPrices: IStockPrice[] = [];
|
||||||
|
let offset = request.offset || 0;
|
||||||
|
const limit = request.limit || 1000; // Max per page
|
||||||
|
const maxRecords = 10000; // Safety limit
|
||||||
|
|
||||||
|
while (true) {
|
||||||
|
let url = `${this.baseUrl}/eod?access_key=${this.apiKey}`;
|
||||||
|
url += `&symbols=${request.ticker}`;
|
||||||
|
url += `&date_from=${this.formatDate(request.from)}`;
|
||||||
|
url += `&date_to=${this.formatDate(request.to)}`;
|
||||||
|
url += `&limit=${limit}`;
|
||||||
|
url += `&offset=${offset}`;
|
||||||
|
|
||||||
|
if (request.exchange) {
|
||||||
|
url += `&exchange=${request.exchange}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (request.sort) {
|
||||||
|
url += `&sort=${request.sort}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const response = await plugins.smartrequest.SmartRequest.create()
|
||||||
|
.url(url)
|
||||||
|
.timeout(this.config?.timeout || 15000)
|
||||||
|
.get();
|
||||||
|
|
||||||
|
const responseData = await response.json() as any;
|
||||||
|
|
||||||
|
// Check for API errors
|
||||||
|
if (responseData.error) {
|
||||||
|
throw new Error(`Marketstack API error: ${responseData.error.message || JSON.stringify(responseData.error)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!responseData?.data || !Array.isArray(responseData.data)) {
|
||||||
|
throw new Error('Invalid response format from Marketstack API');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Map data to stock prices
|
||||||
|
for (const data of responseData.data) {
|
||||||
|
try {
|
||||||
|
allPrices.push(this.mapToStockPrice(data, 'eod'));
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.warn(`Failed to parse historical data for ${data.symbol}:`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check if we have more pages
|
||||||
|
const pagination = responseData.pagination;
|
||||||
|
const hasMore = pagination && offset + limit < pagination.total;
|
||||||
|
|
||||||
|
// Safety check: don't fetch more than maxRecords
|
||||||
|
if (!hasMore || allPrices.length >= maxRecords) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
offset += limit;
|
||||||
|
}
|
||||||
|
|
||||||
|
return allPrices;
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error(`Failed to fetch historical prices for ${request.ticker}:`, error);
|
||||||
|
throw new Error(`Marketstack: Failed to fetch historical prices for ${request.ticker}: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch intraday prices with specified interval (Phase 2 placeholder)
|
||||||
|
*/
|
||||||
|
private async fetchIntradayPrices(request: IStockIntradayRequest): Promise<IStockPrice[]> {
|
||||||
|
throw new Error('Intraday data support coming in Phase 2. For now, use EOD data with type: "current" or "historical"');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch current prices for multiple tickers (new API)
|
||||||
|
*/
|
||||||
|
private async fetchBatchCurrentPrices(request: IStockBatchCurrentRequest): Promise<IStockPrice[]> {
|
||||||
try {
|
try {
|
||||||
const symbols = request.tickers.join(',');
|
const symbols = request.tickers.join(',');
|
||||||
const url = `${this.baseUrl}/eod/latest?access_key=${this.apiKey}&symbols=${symbols}`;
|
let url = `${this.baseUrl}/eod/latest?access_key=${this.apiKey}&symbols=${symbols}`;
|
||||||
|
|
||||||
|
if (request.exchange) {
|
||||||
|
url += `&exchange=${request.exchange}`;
|
||||||
|
}
|
||||||
|
|
||||||
const response = await plugins.smartrequest.SmartRequest.create()
|
const response = await plugins.smartrequest.SmartRequest.create()
|
||||||
.url(url)
|
.url(url)
|
||||||
@@ -98,7 +217,7 @@ export class MarketstackProvider implements IStockProvider {
|
|||||||
|
|
||||||
for (const data of responseData.data) {
|
for (const data of responseData.data) {
|
||||||
try {
|
try {
|
||||||
prices.push(this.mapToStockPrice(data));
|
prices.push(this.mapToStockPrice(data, 'eod'));
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.logger.warn(`Failed to parse data for ${data.symbol}:`, error);
|
this.logger.warn(`Failed to parse data for ${data.symbol}:`, error);
|
||||||
// Continue processing other tickers
|
// Continue processing other tickers
|
||||||
@@ -111,8 +230,8 @@ export class MarketstackProvider implements IStockProvider {
|
|||||||
|
|
||||||
return prices;
|
return prices;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.logger.error(`Failed to fetch batch prices:`, error);
|
this.logger.error(`Failed to fetch batch current prices:`, error);
|
||||||
throw new Error(`Marketstack: Failed to fetch batch prices: ${error.message}`);
|
throw new Error(`Marketstack: Failed to fetch batch current prices: ${error.message}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -165,7 +284,7 @@ export class MarketstackProvider implements IStockProvider {
|
|||||||
/**
|
/**
|
||||||
* Map Marketstack API response to IStockPrice interface
|
* Map Marketstack API response to IStockPrice interface
|
||||||
*/
|
*/
|
||||||
private mapToStockPrice(data: any): IStockPrice {
|
private mapToStockPrice(data: any, dataType: 'eod' | 'intraday' | 'live' = 'eod'): IStockPrice {
|
||||||
if (!data.close) {
|
if (!data.close) {
|
||||||
throw new Error('Missing required price data');
|
throw new Error('Missing required price data');
|
||||||
}
|
}
|
||||||
@@ -174,12 +293,13 @@ export class MarketstackProvider implements IStockProvider {
|
|||||||
// EOD data: previous close is typically open price of the same day
|
// EOD data: previous close is typically open price of the same day
|
||||||
// For better accuracy, we'd need previous day's close, but that requires another API call
|
// For better accuracy, we'd need previous day's close, but that requires another API call
|
||||||
const currentPrice = data.close;
|
const currentPrice = data.close;
|
||||||
const previousClose = data.open;
|
const previousClose = data.open || currentPrice;
|
||||||
const change = currentPrice - previousClose;
|
const change = currentPrice - previousClose;
|
||||||
const changePercent = previousClose !== 0 ? (change / previousClose) * 100 : 0;
|
const changePercent = previousClose !== 0 ? (change / previousClose) * 100 : 0;
|
||||||
|
|
||||||
// Parse timestamp
|
// Parse timestamp
|
||||||
const timestamp = data.date ? new Date(data.date) : new Date();
|
const timestamp = data.date ? new Date(data.date) : new Date();
|
||||||
|
const fetchedAt = new Date();
|
||||||
|
|
||||||
const stockPrice: IStockPrice = {
|
const stockPrice: IStockPrice = {
|
||||||
ticker: data.symbol.toUpperCase(),
|
ticker: data.symbol.toUpperCase(),
|
||||||
@@ -192,9 +312,65 @@ export class MarketstackProvider implements IStockProvider {
|
|||||||
provider: this.name,
|
provider: this.name,
|
||||||
marketState: 'CLOSED', // EOD data is always for closed markets
|
marketState: 'CLOSED', // EOD data is always for closed markets
|
||||||
exchange: data.exchange,
|
exchange: data.exchange,
|
||||||
exchangeName: data.exchange_code || data.name
|
exchangeName: data.exchange_code || data.name,
|
||||||
|
|
||||||
|
// Phase 1 enhancements: OHLCV data
|
||||||
|
volume: data.volume,
|
||||||
|
open: data.open,
|
||||||
|
high: data.high,
|
||||||
|
low: data.low,
|
||||||
|
adjusted: data.adj_close !== undefined, // If adj_close exists, price is adjusted
|
||||||
|
dataType: dataType,
|
||||||
|
fetchedAt: fetchedAt,
|
||||||
|
|
||||||
|
// Company identification
|
||||||
|
companyName: data.company_name || data.name || undefined,
|
||||||
|
companyFullName: this.buildCompanyFullName(data)
|
||||||
};
|
};
|
||||||
|
|
||||||
return stockPrice;
|
return stockPrice;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Build full company name with exchange and ticker information
|
||||||
|
* Example: "Apple Inc (NASDAQ:AAPL)"
|
||||||
|
*/
|
||||||
|
private buildCompanyFullName(data: any): string | undefined {
|
||||||
|
// Check if API already provides full name
|
||||||
|
if (data.full_name || data.long_name) {
|
||||||
|
return data.full_name || data.long_name;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build from available data
|
||||||
|
const companyName = data.company_name || data.name;
|
||||||
|
const exchangeCode = data.exchange_code; // e.g., "NASDAQ"
|
||||||
|
const symbol = data.symbol; // e.g., "AAPL"
|
||||||
|
|
||||||
|
if (!companyName) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
// If we have exchange and symbol, build full name: "Apple Inc (NASDAQ:AAPL)"
|
||||||
|
if (exchangeCode && symbol) {
|
||||||
|
return `${companyName} (${exchangeCode}:${symbol})`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// If we only have symbol: "Apple Inc (AAPL)"
|
||||||
|
if (symbol) {
|
||||||
|
return `${companyName} (${symbol})`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Otherwise just return company name
|
||||||
|
return companyName;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Format date to YYYY-MM-DD for API requests
|
||||||
|
*/
|
||||||
|
private formatDate(date: Date): string {
|
||||||
|
const year = date.getFullYear();
|
||||||
|
const month = String(date.getMonth() + 1).padStart(2, '0');
|
||||||
|
const day = String(date.getDate()).padStart(2, '0');
|
||||||
|
return `${year}-${month}-${day}`;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
429
ts/stocks/providers/provider.secedgar.ts
Normal file
429
ts/stocks/providers/provider.secedgar.ts
Normal file
@@ -0,0 +1,429 @@
|
|||||||
|
import * as plugins from '../../plugins.js';
|
||||||
|
import type {
|
||||||
|
IFundamentalsProvider,
|
||||||
|
IStockFundamentals,
|
||||||
|
IFundamentalsRequest,
|
||||||
|
IFundamentalsCurrentRequest,
|
||||||
|
IFundamentalsBatchRequest
|
||||||
|
} from '../interfaces/fundamentals.js';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Configuration for SEC EDGAR provider
|
||||||
|
*/
|
||||||
|
export interface ISecEdgarConfig {
|
||||||
|
userAgent: string; // Required: Format "Company Name Email" (e.g., "fin.cx info@fin.cx")
|
||||||
|
cikCacheTTL?: number; // Default: 30 days (CIK codes rarely change)
|
||||||
|
fundamentalsCacheTTL?: number; // Default: 90 days (quarterly filings)
|
||||||
|
timeout?: number; // Request timeout in ms
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Rate limiter for SEC EDGAR API
|
||||||
|
* SEC requires: 10 requests per second maximum
|
||||||
|
*/
|
||||||
|
class RateLimiter {
|
||||||
|
private requestTimes: number[] = [];
|
||||||
|
private readonly maxRequestsPerSecond = 10;
|
||||||
|
|
||||||
|
public async waitForSlot(): Promise<void> {
|
||||||
|
const now = Date.now();
|
||||||
|
const oneSecondAgo = now - 1000;
|
||||||
|
|
||||||
|
// Remove requests older than 1 second
|
||||||
|
this.requestTimes = this.requestTimes.filter(time => time > oneSecondAgo);
|
||||||
|
|
||||||
|
// If we've hit the limit, wait
|
||||||
|
if (this.requestTimes.length >= this.maxRequestsPerSecond) {
|
||||||
|
const oldestRequest = this.requestTimes[0];
|
||||||
|
const waitTime = 1000 - (now - oldestRequest) + 10; // +10ms buffer
|
||||||
|
await plugins.smartdelay.delayFor(waitTime);
|
||||||
|
return this.waitForSlot(); // Recursively check again
|
||||||
|
}
|
||||||
|
|
||||||
|
// Record this request
|
||||||
|
this.requestTimes.push(now);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* SEC EDGAR Fundamental Data Provider
|
||||||
|
*
|
||||||
|
* Features:
|
||||||
|
* - Free public access (no API key required)
|
||||||
|
* - All US public companies
|
||||||
|
* - Financial data from 10-K/10-Q filings
|
||||||
|
* - US GAAP standardized metrics
|
||||||
|
* - Historical data back to ~2009
|
||||||
|
* - < 1 minute filing delay
|
||||||
|
*
|
||||||
|
* Documentation: https://www.sec.gov/edgar/sec-api-documentation
|
||||||
|
*
|
||||||
|
* Rate Limits:
|
||||||
|
* - 10 requests per second (enforced by SEC)
|
||||||
|
* - Requires User-Agent header in format: "Company Name Email"
|
||||||
|
*
|
||||||
|
* Data Sources:
|
||||||
|
* - Company Facts API: /api/xbrl/companyfacts/CIK##########.json
|
||||||
|
* - Ticker Lookup: /files/company_tickers.json
|
||||||
|
*/
|
||||||
|
export class SecEdgarProvider implements IFundamentalsProvider {
|
||||||
|
public name = 'SEC EDGAR';
|
||||||
|
public priority = 100; // High priority - free, authoritative, comprehensive
|
||||||
|
public readonly requiresAuth = false; // No API key needed!
|
||||||
|
public readonly rateLimit = {
|
||||||
|
requestsPerMinute: 600, // 10 requests/second = 600/minute
|
||||||
|
requestsPerDay: undefined // No daily limit
|
||||||
|
};
|
||||||
|
|
||||||
|
private logger = console;
|
||||||
|
private baseUrl = 'https://data.sec.gov/api/xbrl';
|
||||||
|
private tickersUrl = 'https://www.sec.gov/files/company_tickers.json';
|
||||||
|
private userAgent: string;
|
||||||
|
private config: Required<ISecEdgarConfig>;
|
||||||
|
|
||||||
|
// Caching
|
||||||
|
private cikCache = new Map<string, { cik: string; timestamp: Date }>();
|
||||||
|
private tickerListCache: { data: any; timestamp: Date } | null = null;
|
||||||
|
|
||||||
|
// Rate limiting
|
||||||
|
private rateLimiter = new RateLimiter();
|
||||||
|
|
||||||
|
constructor(config: ISecEdgarConfig) {
|
||||||
|
// Validate User-Agent
|
||||||
|
if (!config.userAgent) {
|
||||||
|
throw new Error('User-Agent is required for SEC EDGAR provider (format: "Company Name Email")');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate User-Agent format (must contain space and @ symbol)
|
||||||
|
if (!config.userAgent.includes(' ') || !config.userAgent.includes('@')) {
|
||||||
|
throw new Error(
|
||||||
|
'Invalid User-Agent format. Required: "Company Name Email" (e.g., "fin.cx info@fin.cx")'
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
this.userAgent = config.userAgent;
|
||||||
|
this.config = {
|
||||||
|
userAgent: config.userAgent,
|
||||||
|
cikCacheTTL: config.cikCacheTTL || 30 * 24 * 60 * 60 * 1000, // 30 days
|
||||||
|
fundamentalsCacheTTL: config.fundamentalsCacheTTL || 90 * 24 * 60 * 60 * 1000, // 90 days
|
||||||
|
timeout: config.timeout || 30000
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unified data fetching method
|
||||||
|
*/
|
||||||
|
public async fetchData(
|
||||||
|
request: IFundamentalsRequest
|
||||||
|
): Promise<IStockFundamentals | IStockFundamentals[]> {
|
||||||
|
switch (request.type) {
|
||||||
|
case 'fundamentals-current':
|
||||||
|
return this.fetchFundamentals(request);
|
||||||
|
case 'fundamentals-batch':
|
||||||
|
return this.fetchBatchFundamentals(request);
|
||||||
|
default:
|
||||||
|
throw new Error(`Unsupported request type: ${(request as any).type}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch fundamental data for a single ticker
|
||||||
|
*/
|
||||||
|
private async fetchFundamentals(request: IFundamentalsCurrentRequest): Promise<IStockFundamentals> {
|
||||||
|
try {
|
||||||
|
// 1. Get CIK for ticker (with caching)
|
||||||
|
const cik = await this.getCIK(request.ticker);
|
||||||
|
|
||||||
|
// 2. Fetch company facts from SEC (with rate limiting)
|
||||||
|
const companyFacts = await this.fetchCompanyFacts(cik);
|
||||||
|
|
||||||
|
// 3. Parse facts into structured fundamental data
|
||||||
|
return this.parseCompanyFacts(request.ticker, cik, companyFacts);
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error(`Failed to fetch fundamentals for ${request.ticker}:`, error);
|
||||||
|
throw new Error(`SEC EDGAR: Failed to fetch fundamentals for ${request.ticker}: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch fundamental data for multiple tickers
|
||||||
|
*/
|
||||||
|
private async fetchBatchFundamentals(
|
||||||
|
request: IFundamentalsBatchRequest
|
||||||
|
): Promise<IStockFundamentals[]> {
|
||||||
|
const results: IStockFundamentals[] = [];
|
||||||
|
const errors: string[] = [];
|
||||||
|
|
||||||
|
for (const ticker of request.tickers) {
|
||||||
|
try {
|
||||||
|
const fundamentals = await this.fetchFundamentals({
|
||||||
|
type: 'fundamentals-current',
|
||||||
|
ticker
|
||||||
|
});
|
||||||
|
results.push(fundamentals);
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.warn(`Failed to fetch fundamentals for ${ticker}:`, error);
|
||||||
|
errors.push(`${ticker}: ${error.message}`);
|
||||||
|
// Continue with other tickers
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (results.length === 0) {
|
||||||
|
throw new Error(`Failed to fetch fundamentals for all tickers. Errors: ${errors.join(', ')}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get CIK (Central Index Key) for a ticker symbol
|
||||||
|
* Uses SEC's public ticker-to-CIK mapping file
|
||||||
|
*/
|
||||||
|
private async getCIK(ticker: string): Promise<string> {
|
||||||
|
const tickerUpper = ticker.toUpperCase();
|
||||||
|
|
||||||
|
// Check cache first
|
||||||
|
const cached = this.cikCache.get(tickerUpper);
|
||||||
|
if (cached) {
|
||||||
|
const age = Date.now() - cached.timestamp.getTime();
|
||||||
|
if (age < this.config.cikCacheTTL) {
|
||||||
|
return cached.cik;
|
||||||
|
}
|
||||||
|
// Cache expired, remove it
|
||||||
|
this.cikCache.delete(tickerUpper);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fetch ticker list (with caching at list level)
|
||||||
|
const tickers = await this.fetchTickerList();
|
||||||
|
|
||||||
|
// Find ticker in list (case-insensitive)
|
||||||
|
const entry = Object.values(tickers).find((t: any) => t.ticker === tickerUpper);
|
||||||
|
|
||||||
|
if (!entry) {
|
||||||
|
throw new Error(`CIK not found for ticker ${ticker}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const cik = String((entry as any).cik_str);
|
||||||
|
|
||||||
|
// Cache the result
|
||||||
|
this.cikCache.set(tickerUpper, {
|
||||||
|
cik,
|
||||||
|
timestamp: new Date()
|
||||||
|
});
|
||||||
|
|
||||||
|
return cik;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch the SEC ticker-to-CIK mapping list
|
||||||
|
* Cached for 24 hours (list updates daily)
|
||||||
|
*/
|
||||||
|
private async fetchTickerList(): Promise<any> {
|
||||||
|
// Check cache
|
||||||
|
if (this.tickerListCache) {
|
||||||
|
const age = Date.now() - this.tickerListCache.timestamp.getTime();
|
||||||
|
if (age < 24 * 60 * 60 * 1000) { // 24 hours
|
||||||
|
return this.tickerListCache.data;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for rate limit slot
|
||||||
|
await this.rateLimiter.waitForSlot();
|
||||||
|
|
||||||
|
// Fetch from SEC
|
||||||
|
const response = await plugins.smartrequest.SmartRequest.create()
|
||||||
|
.url(this.tickersUrl)
|
||||||
|
.headers({
|
||||||
|
'User-Agent': this.userAgent,
|
||||||
|
'Accept': 'application/json'
|
||||||
|
})
|
||||||
|
.timeout(this.config.timeout)
|
||||||
|
.get();
|
||||||
|
|
||||||
|
const data = await response.json();
|
||||||
|
|
||||||
|
// Cache the list
|
||||||
|
this.tickerListCache = {
|
||||||
|
data,
|
||||||
|
timestamp: new Date()
|
||||||
|
};
|
||||||
|
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch company facts from SEC EDGAR
|
||||||
|
*/
|
||||||
|
private async fetchCompanyFacts(cik: string): Promise<any> {
|
||||||
|
// Pad CIK to 10 digits
|
||||||
|
const paddedCIK = cik.padStart(10, '0');
|
||||||
|
const url = `${this.baseUrl}/companyfacts/CIK${paddedCIK}.json`;
|
||||||
|
|
||||||
|
// Wait for rate limit slot
|
||||||
|
await this.rateLimiter.waitForSlot();
|
||||||
|
|
||||||
|
// Fetch from SEC
|
||||||
|
const response = await plugins.smartrequest.SmartRequest.create()
|
||||||
|
.url(url)
|
||||||
|
.headers({
|
||||||
|
'User-Agent': this.userAgent,
|
||||||
|
'Accept': 'application/json',
|
||||||
|
'Accept-Encoding': 'gzip, deflate',
|
||||||
|
'Host': 'data.sec.gov'
|
||||||
|
})
|
||||||
|
.timeout(this.config.timeout)
|
||||||
|
.get();
|
||||||
|
|
||||||
|
const data = await response.json();
|
||||||
|
|
||||||
|
// Validate response
|
||||||
|
if (!data || !data.facts) {
|
||||||
|
throw new Error('Invalid response from SEC EDGAR API');
|
||||||
|
}
|
||||||
|
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse SEC company facts into structured fundamental data
|
||||||
|
*/
|
||||||
|
private parseCompanyFacts(ticker: string, cik: string, data: any): IStockFundamentals {
|
||||||
|
const usGaap = data.facts?.['us-gaap'];
|
||||||
|
|
||||||
|
if (!usGaap) {
|
||||||
|
throw new Error('No US GAAP data available');
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract latest values for key metrics
|
||||||
|
const fundamentals: IStockFundamentals = {
|
||||||
|
ticker: ticker.toUpperCase(),
|
||||||
|
cik: cik,
|
||||||
|
companyName: data.entityName,
|
||||||
|
provider: this.name,
|
||||||
|
timestamp: new Date(),
|
||||||
|
fetchedAt: new Date(),
|
||||||
|
|
||||||
|
// Per-share metrics
|
||||||
|
earningsPerShareBasic: this.getLatestValue(usGaap, 'EarningsPerShareBasic'),
|
||||||
|
earningsPerShareDiluted: this.getLatestValue(usGaap, 'EarningsPerShareDiluted'),
|
||||||
|
sharesOutstanding: this.getLatestValue(usGaap, 'CommonStockSharesOutstanding'),
|
||||||
|
weightedAverageSharesOutstanding: this.getLatestValue(
|
||||||
|
usGaap,
|
||||||
|
'WeightedAverageNumberOfSharesOutstandingBasic'
|
||||||
|
),
|
||||||
|
|
||||||
|
// Income statement
|
||||||
|
revenue: this.getLatestValue(usGaap, 'Revenues') ||
|
||||||
|
this.getLatestValue(usGaap, 'RevenueFromContractWithCustomerExcludingAssessedTax'),
|
||||||
|
netIncome: this.getLatestValue(usGaap, 'NetIncomeLoss'),
|
||||||
|
operatingIncome: this.getLatestValue(usGaap, 'OperatingIncomeLoss'),
|
||||||
|
grossProfit: this.getLatestValue(usGaap, 'GrossProfit'),
|
||||||
|
costOfRevenue: this.getLatestValue(usGaap, 'CostOfRevenue'),
|
||||||
|
|
||||||
|
// Balance sheet
|
||||||
|
assets: this.getLatestValue(usGaap, 'Assets'),
|
||||||
|
liabilities: this.getLatestValue(usGaap, 'Liabilities'),
|
||||||
|
stockholdersEquity: this.getLatestValue(usGaap, 'StockholdersEquity'),
|
||||||
|
cash: this.getLatestValue(usGaap, 'CashAndCashEquivalentsAtCarryingValue'),
|
||||||
|
propertyPlantEquipment: this.getLatestValue(usGaap, 'PropertyPlantAndEquipmentNet'),
|
||||||
|
|
||||||
|
// Metadata (from latest available data point)
|
||||||
|
fiscalYear: this.getLatestMetadata(usGaap, 'EarningsPerShareDiluted')?.fy,
|
||||||
|
fiscalQuarter: this.getLatestMetadata(usGaap, 'EarningsPerShareDiluted')?.fp,
|
||||||
|
filingDate: this.getLatestMetadata(usGaap, 'EarningsPerShareDiluted')?.filed
|
||||||
|
? new Date(this.getLatestMetadata(usGaap, 'EarningsPerShareDiluted')!.filed)
|
||||||
|
: undefined,
|
||||||
|
form: this.getLatestMetadata(usGaap, 'EarningsPerShareDiluted')?.form
|
||||||
|
};
|
||||||
|
|
||||||
|
return fundamentals;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get the latest value for a US GAAP metric
|
||||||
|
*/
|
||||||
|
private getLatestValue(usGaap: any, metricName: string): number | undefined {
|
||||||
|
const metric = usGaap[metricName];
|
||||||
|
if (!metric?.units) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get the first unit type (USD, shares, etc.)
|
||||||
|
const unitType = Object.keys(metric.units)[0];
|
||||||
|
const values = metric.units[unitType];
|
||||||
|
|
||||||
|
if (!values || !Array.isArray(values) || values.length === 0) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get the latest value (last in array)
|
||||||
|
const latest = values[values.length - 1];
|
||||||
|
return latest?.val;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get metadata from the latest data point
|
||||||
|
*/
|
||||||
|
private getLatestMetadata(usGaap: any, metricName: string): any | undefined {
|
||||||
|
const metric = usGaap[metricName];
|
||||||
|
if (!metric?.units) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
const unitType = Object.keys(metric.units)[0];
|
||||||
|
const values = metric.units[unitType];
|
||||||
|
|
||||||
|
if (!values || !Array.isArray(values) || values.length === 0) {
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
return values[values.length - 1];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if SEC EDGAR API is available
|
||||||
|
*/
|
||||||
|
public async isAvailable(): Promise<boolean> {
|
||||||
|
try {
|
||||||
|
// Test with Apple's well-known CIK
|
||||||
|
const url = `${this.baseUrl}/companyfacts/CIK0000320193.json`;
|
||||||
|
|
||||||
|
const response = await plugins.smartrequest.SmartRequest.create()
|
||||||
|
.url(url)
|
||||||
|
.headers({
|
||||||
|
'User-Agent': this.userAgent,
|
||||||
|
'Accept': 'application/json'
|
||||||
|
})
|
||||||
|
.timeout(5000)
|
||||||
|
.get();
|
||||||
|
|
||||||
|
const data = await response.json();
|
||||||
|
return data && data.facts !== undefined;
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.warn('SEC EDGAR provider is not available:', error);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get cache statistics
|
||||||
|
*/
|
||||||
|
public getCacheStats(): {
|
||||||
|
cikCacheSize: number;
|
||||||
|
hasTickerList: boolean;
|
||||||
|
} {
|
||||||
|
return {
|
||||||
|
cikCacheSize: this.cikCache.size,
|
||||||
|
hasTickerList: this.tickerListCache !== null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clear all caches
|
||||||
|
*/
|
||||||
|
public clearCache(): void {
|
||||||
|
this.cikCache.clear();
|
||||||
|
this.tickerListCache = null;
|
||||||
|
this.logger.log('SEC EDGAR cache cleared');
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,11 @@
|
|||||||
import * as plugins from '../../plugins.js';
|
import * as plugins from '../../plugins.js';
|
||||||
import type { IStockProvider, IProviderConfig } from '../interfaces/provider.js';
|
import type { IStockProvider, IProviderConfig } from '../interfaces/provider.js';
|
||||||
import type { IStockPrice, IStockQuoteRequest, IStockBatchQuoteRequest } from '../interfaces/stockprice.js';
|
import type {
|
||||||
|
IStockPrice,
|
||||||
|
IStockDataRequest,
|
||||||
|
IStockCurrentRequest,
|
||||||
|
IStockBatchCurrentRequest
|
||||||
|
} from '../interfaces/stockprice.js';
|
||||||
|
|
||||||
export class YahooFinanceProvider implements IStockProvider {
|
export class YahooFinanceProvider implements IStockProvider {
|
||||||
public name = 'Yahoo Finance';
|
public name = 'Yahoo Finance';
|
||||||
@@ -17,7 +22,28 @@ export class YahooFinanceProvider implements IStockProvider {
|
|||||||
|
|
||||||
constructor(private config?: IProviderConfig) {}
|
constructor(private config?: IProviderConfig) {}
|
||||||
|
|
||||||
public async fetchPrice(request: IStockQuoteRequest): Promise<IStockPrice> {
|
/**
|
||||||
|
* Unified data fetching method
|
||||||
|
*/
|
||||||
|
public async fetchData(request: IStockDataRequest): Promise<IStockPrice | IStockPrice[]> {
|
||||||
|
switch (request.type) {
|
||||||
|
case 'current':
|
||||||
|
return this.fetchCurrentPrice(request);
|
||||||
|
case 'batch':
|
||||||
|
return this.fetchBatchCurrentPrices(request);
|
||||||
|
case 'historical':
|
||||||
|
throw new Error('Yahoo Finance provider does not support historical data. Use Marketstack provider instead.');
|
||||||
|
case 'intraday':
|
||||||
|
throw new Error('Yahoo Finance provider does not support intraday data yet. Use Marketstack provider instead.');
|
||||||
|
default:
|
||||||
|
throw new Error(`Unsupported request type: ${(request as any).type}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Fetch current price for a single ticker
|
||||||
|
*/
|
||||||
|
private async fetchCurrentPrice(request: IStockCurrentRequest): Promise<IStockPrice> {
|
||||||
try {
|
try {
|
||||||
const url = `${this.baseUrl}/v8/finance/chart/${request.ticker}`;
|
const url = `${this.baseUrl}/v8/finance/chart/${request.ticker}`;
|
||||||
const response = await plugins.smartrequest.SmartRequest.create()
|
const response = await plugins.smartrequest.SmartRequest.create()
|
||||||
@@ -52,7 +78,9 @@ export class YahooFinanceProvider implements IStockProvider {
|
|||||||
provider: this.name,
|
provider: this.name,
|
||||||
marketState: this.determineMarketState(meta),
|
marketState: this.determineMarketState(meta),
|
||||||
exchange: meta.exchange,
|
exchange: meta.exchange,
|
||||||
exchangeName: meta.exchangeName
|
exchangeName: meta.exchangeName,
|
||||||
|
dataType: 'live', // Yahoo provides real-time/near real-time data
|
||||||
|
fetchedAt: new Date()
|
||||||
};
|
};
|
||||||
|
|
||||||
return stockPrice;
|
return stockPrice;
|
||||||
@@ -62,7 +90,10 @@ export class YahooFinanceProvider implements IStockProvider {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public async fetchPrices(request: IStockBatchQuoteRequest): Promise<IStockPrice[]> {
|
/**
|
||||||
|
* Fetch batch current prices
|
||||||
|
*/
|
||||||
|
private async fetchBatchCurrentPrices(request: IStockBatchCurrentRequest): Promise<IStockPrice[]> {
|
||||||
try {
|
try {
|
||||||
const symbols = request.tickers.join(',');
|
const symbols = request.tickers.join(',');
|
||||||
const url = `${this.baseUrl}/v8/finance/spark?symbols=${symbols}&range=1d&interval=5m`;
|
const url = `${this.baseUrl}/v8/finance/spark?symbols=${symbols}&range=1d&interval=5m`;
|
||||||
@@ -101,7 +132,9 @@ export class YahooFinanceProvider implements IStockProvider {
|
|||||||
provider: this.name,
|
provider: this.name,
|
||||||
marketState: sparkData.marketState || 'REGULAR',
|
marketState: sparkData.marketState || 'REGULAR',
|
||||||
exchange: sparkData.exchange,
|
exchange: sparkData.exchange,
|
||||||
exchangeName: sparkData.exchangeName
|
exchangeName: sparkData.exchangeName,
|
||||||
|
dataType: 'live', // Yahoo provides real-time/near real-time data
|
||||||
|
fetchedAt: new Date()
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -119,7 +152,7 @@ export class YahooFinanceProvider implements IStockProvider {
|
|||||||
public async isAvailable(): Promise<boolean> {
|
public async isAvailable(): Promise<boolean> {
|
||||||
try {
|
try {
|
||||||
// Test with a well-known ticker
|
// Test with a well-known ticker
|
||||||
await this.fetchPrice({ ticker: 'AAPL' });
|
await this.fetchData({ type: 'current', ticker: 'AAPL' });
|
||||||
return true;
|
return true;
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.warn('Yahoo Finance provider is not available:', error);
|
console.warn('Yahoo Finance provider is not available:', error);
|
||||||
|
|||||||
Reference in New Issue
Block a user