opendata/readme.md

# @fin.cx/opendata
🚀 **Real-time financial data and German business intelligence in one powerful TypeScript library**

Access live stock prices, cryptocurrencies, forex, commodities AND comprehensive German company data - all through a single, unified API.

## Installation

```bash
npm install @fin.cx/opendata
# or
yarn add @fin.cx/opendata
```

## Quick Start

### 📈 Real-Time Stock Data

Get live market data in seconds:

```typescript
import { StockPriceService, YahooFinanceProvider } from '@fin.cx/opendata';

// Initialize the service
const stockService = new StockPriceService();
stockService.register(new YahooFinanceProvider());

// Get single stock price
const apple = await stockService.getPrice({ ticker: 'AAPL' });
console.log(`Apple: $${apple.price} (${apple.changePercent.toFixed(2)}%)`);

// Get multiple prices at once
const prices = await stockService.getPrices({ 
  tickers: ['AAPL', 'MSFT', 'GOOGL', 'BTC-USD', 'ETH-USD'] 
});

// Market indices, crypto, forex, commodities - all supported!
const marketData = await stockService.getPrices({
  tickers: ['^GSPC', '^DJI', 'BTC-USD', 'EURUSD=X', 'GC=F']
});
```

### 🏢 German Business Data

Access comprehensive data on German companies:

```typescript
import { OpenData } from '@fin.cx/opendata';

const openData = new OpenData();
await openData.start();

// Create a business record
const company = new openData.CBusinessRecord();
company.data = {
  name: "TechStart GmbH",
  city: "Berlin",
  registrationId: "HRB 123456",
  // ... more fields
};
await company.save();

// Search companies by city
const berlinCompanies = await openData.db
  .collection('businessrecords')
  .find({ city: "Berlin" })
  .toArray();

// Import bulk data from official sources
await openData.buildInitialDb();
```

## Features

### 🎯 Stock Market Module

- **Real-time prices** for stocks, ETFs, indices, crypto, forex, and commodities
- **Batch operations** - fetch 100+ symbols in one request
- **Smart caching** - configurable TTL, automatic invalidation
- **Provider system** - easily extensible for new data sources
- **Automatic retries** and fallback mechanisms
- **Type-safe** - full TypeScript support with detailed interfaces

### 🇩🇪 German Business Intelligence

- **MongoDB integration** for scalable data storage
- **Bulk JSONL import** from official German data sources
- **Handelsregister automation** - automated document retrieval
- **CRUD operations** with validation
- **Streaming processing** for multi-GB datasets

## Advanced Examples

### Market Dashboard

Create a real-time market overview:

```typescript
const indicators = [
  // Indices
  { ticker: '^GSPC', name: 'S&P 500' },
  { ticker: '^IXIC', name: 'NASDAQ' },
  
  // Tech Giants
  { ticker: 'AAPL', name: 'Apple' },
  { ticker: 'MSFT', name: 'Microsoft' },
  
  // Crypto
  { ticker: 'BTC-USD', name: 'Bitcoin' },
  { ticker: 'ETH-USD', name: 'Ethereum' },
  
  // Commodities
  { ticker: 'GC=F', name: 'Gold' },
  { ticker: 'CL=F', name: 'Oil' }
];

const prices = await stockService.getPrices({ 
  tickers: indicators.map(i => i.ticker) 
});

// Display with color-coded changes
prices.forEach(price => {
  const indicator = indicators.find(i => i.ticker === price.ticker);
  const arrow = price.change >= 0 ? '↑' : '↓';
  const color = price.change >= 0 ? '\x1b[32m' : '\x1b[31m';
  
  console.log(
    `${indicator.name.padEnd(15)} ${price.price.toFixed(2).padStart(10)} ` +
    `${color}${arrow} ${price.changePercent.toFixed(2)}%\x1b[0m`
  );
});
```

### Handelsregister Integration

Automate German company data retrieval:

```typescript
// Search for a company
const results = await openData.handelsregister.searchCompany("Siemens AG");

// Get detailed information and documents
const details = await openData.handelsregister.getSpecificCompany({
  court: "Munich",
  type: "HRB",
  number: "6684"
});

// Downloaded files include:
// - XML data (SI files)
// - PDF documents (AD files)
for (const file of details.files) {
  await file.writeToDir('./downloads');
}
```

### Combined Data Analysis

Merge financial and business data:

```typescript
// Find all public German companies (AG)
const publicCompanies = await openData.db
  .collection('businessrecords')
  .find({ legalForm: 'AG' })
  .toArray();

// Enrich with stock data
for (const company of publicCompanies) {
  try {
    // Map company to ticker (custom logic needed)
    const ticker = mapCompanyToTicker(company.data.name);
    
    if (ticker) {
      const stock = await stockService.getPrice({ ticker });
      
      // Add financial metrics
      company.data.stockPrice = stock.price;
      company.data.marketCap = stock.price * getSharesOutstanding(ticker);
      company.data.priceChange = stock.changePercent;
      
      await company.save();
    }
  } catch (error) {
    // Handle missing tickers gracefully
  }
}
```

## Configuration

### Stock Service Options

```typescript
const stockService = new StockPriceService({
  ttl: 60000,         // Cache for 1 minute
  maxEntries: 1000    // Max cached symbols
});

// Provider configuration
stockService.register(new YahooFinanceProvider(), {
  enabled: true,
  priority: 100,
  timeout: 10000,
  retryAttempts: 3,
  retryDelay: 1000
});
```

### MongoDB Setup

Set environment variables:

```env
MONGODB_URL=mongodb://localhost:27017
MONGODB_NAME=opendata
MONGODB_USER=myuser
MONGODB_PASS=mypass
```

## API Reference

### Stock Types

```typescript
interface IStockPrice {
  ticker: string;
  price: number;
  currency: string;
  change: number;
  changePercent: number;
  previousClose: number;
  timestamp: Date;
  provider: string;
  marketState: 'PRE' | 'REGULAR' | 'POST' | 'CLOSED';
  exchange?: string;
  exchangeName?: string;
}
```

### Key Methods

**StockPriceService**
- `getPrice(request)` - Single stock price
- `getPrices(request)` - Batch prices
- `register(provider)` - Add data provider
- `clearCache()` - Clear price cache

**OpenData**
- `start()` - Initialize MongoDB connection
- `buildInitialDb()` - Import bulk data
- `CBusinessRecord` - Business record class
- `handelsregister` - Registry automation

## Performance

- **Batch fetching**: Get 100+ prices in <500ms
- **Caching**: Instant repeated queries
- **Concurrent processing**: Handle 1000+ records/second
- **Streaming**: Process GB-sized datasets without memory issues

## Extensibility

The provider architecture makes it easy to add new data sources:

```typescript
class MyCustomProvider implements IStockProvider {
  name = 'My Provider';
  
  async fetchPrice(request: IStockQuoteRequest): Promise<IStockPrice> {
    // Your implementation
  }
  
  // ... other required methods
}

stockService.register(new MyCustomProvider());
```

## Testing

Run the comprehensive test suite:

```bash
npm test
```

View live market data:

```bash
npm test -- --grep "market indicators"
```

## Contributing

We welcome contributions! Please see our contributing guidelines for details.

## License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the [license](license) file within this repository. 

**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

### Company Information

Task Venture Capital GmbH  
Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.