Files

T

jkunz 73801f785a feat(laws,opendata): add local law storage and migrate OpenData persistence to smartdb-backed local storage

2026-04-17 11:51:02 +00:00

13 KiB

Raw Blame History

@fin.cx/opendata

@fin.cx/opendata is a TypeScript toolbox for serious data plumbing: market data, SEC fundamentals, German company registry workflows, and locally synced law corpora. It is built for programmers who want scriptable APIs, local persistence, and real workflows instead of a thin wrapper around one remote endpoint.

It currently gives you four practical lanes in one package:

StockPriceService for stock and crypto prices with caching, retries, failover, historical ranges, and intraday support
FundamentalsService and StockDataService for SEC fundamentals and combined price + fundamentals payloads
OpenData for German company ingestion plus browser-driven Handelsregister lookup and document download
LawService for syncing and searching German, EU, and US law texts in a local embedded database

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Installation

pnpm add @fin.cx/opendata

What You Get

Market Data

StockPriceService Fetches current, batch, historical, and intraday price data.
FundamentalsService Fetches SEC EDGAR fundamentals for US-listed companies.
StockDataService Combines price + fundamentals and enriches fundamentals with metrics like marketCap, priceToEarnings, and priceToBook.
MarketstackProvider Authenticated stock market provider with current, batch, historical, and intraday support.
CoinGeckoProvider Crypto market provider behind the same stock-price interface.
SecEdgarProvider Fundamentals provider backed by SEC EDGAR company facts.

German Company Data

OpenData Starts a local embedded database, imports the public German company JSONL dump, and exposes a handelsregister helper for company lookup and file download.

Laws

LawService Syncs laws into a local embedded database and lets you search them later.
LawRecord The stored law document model returned by sync and search operations.

Before You Start

No stock or fundamentals providers are pre-registered. You must register them before calling the services.
MarketstackProvider requires an API key.
SecEdgarProvider requires a valid User-Agent string in the format Company Name email@example.com.
CoinGeckoProvider works without an API key, but a key gives you better rate limits.
OpenData requires nogitDir, downloadDir, and germanBusinessDataDir.
OpenData.start() is required before using OpenData.handelsregister or buildInitialDb().
LawService stores data locally. It auto-starts on demand, but explicit start() and stop() keep lifecycle handling cleaner.
EU law syncing and Handelsregister access use a headless browser under the hood.
StockDataService defaults to a 24 hour price cache. If you want fresher quote caching, pass cache.priceTTL explicitly or use StockPriceService directly.

Quick Start

Combined Stock Data

Use StockDataService when you want one call that returns both the latest price and SEC fundamentals.

import {
  MarketstackProvider,
  SecEdgarProvider,
  StockDataService,
} from '@fin.cx/opendata';

const stocks = new StockDataService({
  cache: {
    priceTTL: 60_000,
  },
});

stocks.registerPriceProvider(
  new MarketstackProvider(process.env.MARKETSTACK_API_KEY!)
);

stocks.registerFundamentalsProvider(
  new SecEdgarProvider({
    userAgent: 'Acme Inc. dev@acme.dev',
  })
);

const apple = await stocks.getStockData('AAPL');

console.log({
  ticker: apple.ticker,
  price: apple.price.price,
  provider: apple.price.provider,
  companyName: apple.fundamentals?.companyName,
  marketCap: apple.fundamentals?.marketCap,
  priceToEarnings: apple.fundamentals?.priceToEarnings,
  fetchedAt: apple.fetchedAt,
});

You can also batch this:

const batch = await stocks.getBatchStockData(['AAPL', 'MSFT', 'NVDA']);

Price-Only Stocks And Intraday Data

Use StockPriceService if you want smarter per-request caching and direct access to current, historical, and intraday request types.

import { MarketstackProvider, StockPriceService } from '@fin.cx/opendata';

const prices = new StockPriceService();
prices.register(new MarketstackProvider(process.env.MARKETSTACK_API_KEY!));

const current = await prices.getPrice({ ticker: 'AAPL' });

const historical = await prices.getData({
  type: 'historical',
  ticker: 'AAPL',
  from: new Date('2025-01-01'),
  to: new Date('2025-01-31'),
  sort: 'DESC',
});

const intraday = await prices.getData({
  type: 'intraday',
  ticker: 'AAPL',
  interval: '1hour',
  limit: 24,
});

console.log({
  live: current.price,
  candles: intraday.length,
  historicalRows: historical.length,
});

Supported stock request shapes:

{ type: 'current', ticker, exchange? }
{ type: 'batch', tickers, exchange? }
{ type: 'historical', ticker, from, to, exchange?, sort?, limit?, offset? }
{ type: 'intraday', ticker, interval, exchange?, limit?, date? }

StockPriceService also exposes operational helpers like checkProvidersHealth(), getProviderStats(), clearCache(), and getCacheStats().

Crypto Prices With The Same Interface

CoinGeckoProvider plugs into StockPriceService, so crypto lookups use the same request model as stock lookups.

import { CoinGeckoProvider, StockPriceService } from '@fin.cx/opendata';

const prices = new StockPriceService({ ttl: 30_000 });
prices.register(new CoinGeckoProvider(process.env.COINGECKO_API_KEY));

const btc = await prices.getPrice({ ticker: 'BTC' });
const top = await prices.getPrices({ tickers: ['BTC', 'ETH', 'SOL'] });

console.log(btc.price, top.map((item) => item.ticker));

You can also pass CoinGecko ids like bitcoin or ethereum instead of ticker symbols.

Fundamentals Only

Use FundamentalsService when you only care about SEC filing data.

import { FundamentalsService, SecEdgarProvider } from '@fin.cx/opendata';

const fundamentals = new FundamentalsService();
fundamentals.register(
  new SecEdgarProvider({
    userAgent: 'Acme Inc. dev@acme.dev',
  })
);

const apple = await fundamentals.getFundamentals('AAPL');

console.log({
  companyName: apple.companyName,
  revenue: apple.revenue,
  netIncome: apple.netIncome,
  earningsPerShareDiluted: apple.earningsPerShareDiluted,
  filingDate: apple.filingDate,
});

German Company Data And Handelsregister Automation

OpenData is the local workflow layer for German company data. It creates its directories, boots an embedded SmartDB-backed database, and starts a browser automation session for Handelsregister access.

import { OpenData } from '@fin.cx/opendata';
import * as path from 'node:path';

const nogitDir = path.join(process.cwd(), '.nogit');

const openData = new OpenData({
  nogitDir,
  downloadDir: path.join(nogitDir, 'downloads'),
  germanBusinessDataDir: path.join(nogitDir, 'germanbusinessdata'),
});

await openData.start();

const matches = await openData.handelsregister.searchCompany('Siemens AG', 20);

const company = await openData.handelsregister.getSpecificCompany({
  court: 'Munich',
  type: 'HRB',
  number: '6684',
});

console.log({
  searchHits: matches.length,
  downloadedFiles: company.files.map((file) => file.path),
});

await openData.stop();

Useful OpenData workflows today:

await openData.start()
await openData.buildInitialDb() to ingest the public German company JSONL dataset
await openData.handelsregister.searchCompany(name, limit)
await openData.handelsregister.getSpecificCompany({ court, type, number })
await openData.handelsregister.getSpecificCompanyByName(name)

The download workflow currently fetches the Handelsregister files into a temporary folder and returns the files to you as SmartFile objects.

Law Syncing And Search

LawService stores synced laws in a local embedded database and supports three jurisdictions:

Germany via gesetze-im-internet
EU via EUR-Lex
US via GovInfo and Cornell LII

import { LawService } from '@fin.cx/opendata';

const laws = new LawService({
  dbFolderPath: '.nogit/law-smartdb',
  dbName: 'laws',
  govInfoApiKey: process.env.GOVINFO_API_KEY,
});

await laws.start();

const germanLaw = await laws.syncLaw({
  jurisdiction: 'de',
  identifier: 'aeg',
});

const euLaw = await laws.syncLaw({
  jurisdiction: 'eu',
  identifier: '32024R1689',
  language: 'EN',
});

const usCode = await laws.syncLaw({
  jurisdiction: 'us',
  identifier: '8 U.S.C. § 1226',
});

const results = await laws.searchLaws({
  jurisdiction: 'eu',
  query: 'Artificial Intelligence Act',
  limit: 5,
});

console.log({
  germany: germanLaw.title,
  eu: euLaw.title,
  us: usCode.title,
  hits: results.length,
});

await laws.stop();

LawService also supports bulk sync:

await laws.syncLaws({
  jurisdiction: 'us',
  usCollection: 'PLAW',
  limit: 25,
  since: new Date('2025-01-01'),
});

Useful request patterns:

Germany: identifier: 'aeg'
EU: identifier: '32024R1689', language: 'EN'
US public law package: identifier: 'PLAW-119publ1', usCollection: 'PLAW'
US Code citation: identifier: '8 U.S.C. § 1226'

API At A Glance

`StockPriceService`

register(provider, config?)
unregister(providerName)
getPrice({ ticker })
getPrices({ tickers })
getData(request)
checkProvidersHealth()
getProviderStats()
clearCache()
getCacheStats()

`FundamentalsService`

register(provider, config?)
getFundamentals(ticker)
getBatchFundamentals(tickers)
getData(request)
enrichWithPrice(fundamentals, price)
checkProvidersHealth()
getProviderStats()
clearCache()

`StockDataService`

registerPriceProvider(provider, config?)
registerFundamentalsProvider(provider, config?)
getPrice(ticker)
getPrices(tickers)
getFundamentals(ticker)
getBatchFundamentals(tickers)
getStockData(ticker | request)
getBatchStockData(tickers | request)
checkProvidersHealth()
getProviderStats()
clearCache()

`OpenData`

start()
stop()
buildInitialDb()
handelsregister.searchCompany(name, limit?)
handelsregister.getSpecificCompany({ court, type, number })
handelsregister.getSpecificCompanyByName(name)

`LawService`

start()
stop()
getLaw(request)
syncLaw(request)
syncLaws(request)
searchLaws({ query, jurisdiction?, limit? })

Behavior And Caveats

Historical market data is cached aggressively because it is treated as immutable.
Intraday data uses interval-aware caching and tries incremental refreshes for recent data.
StockDataService is convenience-first. StockPriceService is the better low-level choice if you care about request-shape-specific cache behavior.
OpenData is a real workflow layer, not just types. It writes local data, launches a browser, and manages downloads.
Handelsregister and EUR-Lex integrations depend on current site structure. They are practical automation workflows, not guaranteed stable official APIs.
The most useful public OpenData features today are startup, JSONL import, and Handelsregister operations. Some extra validation/search helper methods exist in source but are still stubs.
LawService persists synced laws locally, so repeated searches can stay offline once the data has been synced.
There is no hosted backend in this package. Everything is meant to run from your own TypeScript/Node process.

Testing

pnpm test

Some tests hit live remote services, so they may depend on network access, optional API credentials, or browser automation support.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the LICENSE file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

13 KiB Raw Blame History