# Fonrex Documentation - Full Content Dump

> Concatenated English documentation for LLMs

---

<!-- FILE: /docs/intro.md -->

---
id: "intro"
title: "Introduction to Fonrex"
sidebar_label: "Introduction"
description: "Overview of Fonrex, the open-source self-hosted financial data API"
---

# Introduction to Fonrex

Fonrex is an open-source, **self-hosted** financial data API. One Docker Compose stack — FastAPI, PostgreSQL/TimescaleDB and Redis — collects and serves end-of-day prices, real-time quotes, fundamentals, technical indicators, news and DCF valuations, and watches the quality of the data it collects.

Fonrex is not a hosted service: there is no Fonrex cloud API. Every client (your scripts, Google Sheets, OpenBB Workspace, Zipline) talks to **your own instance**. The data is scraped or fetched from public sources by your instance, under your responsibility.

Fonrex is distributed under the **AGPL-3.0** license.

## What you get

| Area | What Fonrex provides |
|---|---|
| **Prices** | End-of-day history per listing (Yahoo Finance, TradingView fallback), daily/weekly/monthly bars, real-time quotes over WebSocket |
| **Fundamentals** | One document in the EODHD layout built from Yahoo Finance, the stored deep fundamentals and 13 scraped websites, with the source of every figure |
| **Technical indicators** | 18 indicators computed server-side with pandas-ta, multi-indicator requests and a screener |
| **Valuation** | DCF with three models (FCF, EPS, DDM), dynamic WACC, model comparison and sensitivity matrix |
| **News** | 7 news providers, deduplicated by URL and title similarity |
| **Data quality** | Range and consensus checks on every request, a daily canary run against known assets, alerts |
| **Integrations** | OpenBB Workspace widgets, a Google Sheets template, a Zipline data bundle |

## Fonrex compared with a commercial data API

| | Fonrex | Commercial market data API |
|---|---|---|
| **Hosting** | Your machine (Docker) | Vendor cloud |
| **Price** | Free, open source (AGPL-3.0) | Monthly subscription |
| **Storage** | Your PostgreSQL + TimescaleDB | Vendor managed |
| **Real-time** | WebSocket push + Redis Pub/Sub | Often REST polling or a paid tier |
| **European markets** | Native (Euronext, Xetra…), UCITS ETFs via JustETF | Often a higher tier |
| **Sources** | Several providers per figure, source reported | One vendor |
| **Rate limits** | Those of the public sources you query | Vendor quota |

## Quick start

```bash
git clone https://github.com/fonrex/fonrex.git
cd fonrex
cp .env.example .env
# The API rejects every request until a key is configured:
export FONREX_API_KEY="frx_live_$(openssl rand -hex 24)"
sed -i.bak "s/^FONREX_API_KEY=.*/FONREX_API_KEY=$FONREX_API_KEY/" .env && rm .env.bak
mkdir -p logs
docker compose up -d
```

`/health` answers without a key:

```bash
curl http://localhost:5000/health
```

```json
{
  "status": "healthy",
  "service": "FonRex API",
  "timestamp": "2026-10-08T16:34:42.235390",
  "yfinance_available": true,
  "providers": { "loaded": 14, "unavailable": [] },
  "cache": { "enabled": true, "status": "connected", "ttl_seconds": { "eod": 86400, "...": "..." } }
}
```

Every other route needs the key — see [Installation](getting-started/installation.md) and [First API call](getting-started/first-api-call.md).



<!-- FILE: /docs/pathways/quant-trader.md -->

---
id: "quant-trader"
title: "Quant & Algo-Trader Pathway"
sidebar_label: "Quant & Algo-Trader"
description: "From an empty instance to backtests: ingest prices per listing, compute indicators server-side, feed Zipline"
---

# Quant & Algo-Trader Pathway

This pathway takes you from an empty instance to a backtest: daily prices stored per listing in TimescaleDB, indicators computed by the API, and a Zipline bundle reading your database.

| Step | What you use |
|---|---|
| Prices | `POST /historical/ingest`, `scripts/ingest_all.py`, `GET /eod` |
| Indicators | `GET /technical/{ticker}` and `/multi`, 18 indicators (pandas-ta) |
| Screening | `GET /technical/screen` |
| Backtesting | `zipline_bundle` (zipline-reloaded) or pandas |

## 1. Start an instance

```bash
git clone https://github.com/fonrex/fonrex.git && cd fonrex
cp .env.example .env
export FONREX_API_KEY="frx_live_$(openssl rand -hex 24)"
sed -i.bak "s/^FONREX_API_KEY=.*/FONREX_API_KEY=$FONREX_API_KEY/" .env && rm .env.bak
mkdir -p logs && docker compose up -d
AUTH="X-API-KEY: $FONREX_API_KEY"
```

## 2. Import instruments and ingest prices

```bash
docker compose exec fonrex-api python import_assets.py --file data/stocks.csv
curl -s -X POST -H "$AUTH" "http://localhost:5000/historical/ingest?ticker=AIR.PA"
```

The ingestion fetches ten years of daily bars from Yahoo Finance with the symbol verified for the listing (TradingView as a fallback), dated by trading session: the traded prices adjusted for splits, and an `adj_close` adjusted for dividends too, kept on one adjustment basis when a split or a dividend occurs. For the whole catalogue: `docker compose exec fonrex-api python scripts/ingest_all.py`. Details: [Ingesting historical data](../guides/ingest-historical-data.md).

## 3. Compute indicators

```bash
curl -s -H "$AUTH" "http://localhost:5000/technical/AIR.PA?indicator=rsi&period=14"
curl -s -H "$AUTH" "http://localhost:5000/technical/AIR.PA/multi?indicators=sma_50,sma_200,macd,bbands_20"
curl -s -H "$AUTH" "http://localhost:5000/technical/screen?indicator=rsi&operator=lt&value=30"
```

Indicators are computed on the stored prices with pandas-ta and cached in Redis. See the [Technical indicators reference](../api-reference/technical-indicators.md).

## 4. Backtest

With Zipline, register the bundle and ingest it from your database:

```bash
pip install zipline-reloaded
export DATABASE_URL="postgresql://fonrex:<POSTGRES_PASSWORD>@localhost:5432/fonrex"
python -m zipline_bundle ingest --start 2020-01-01 --end 2025-12-31 --tickers AIR.PA,BNP.PA --calendar XPAR
```

Or load the prices into pandas through `GET /eod/{ticker}` — see [Python & Jupyter](../guides/python-jupyter.md), with an example notebook. See [Backtesting with Zipline](../guides/backtesting-zipline.md).

## Next steps

- [Historical prices API](../api-reference/historical.md)
- [Realtime streaming](../api-reference/realtime.md) for 1-minute ticks
- [Data model](../architecture/data-model.md)



<!-- FILE: /docs/pathways/financial-analyst.md -->

---
id: "financial-analyst"
title: "Financial Analyst Pathway"
sidebar_label: "Financial Analyst"
description: "Fundamentals with their sources, DCF valuations, Google Sheets and OpenBB Workspace on your own instance"
---

# Financial Analyst Pathway

This pathway covers the fundamentals and valuation side of Fonrex, and the two no-code front-ends: Google Sheets and OpenBB Workspace. All of them read **your own instance**.

| Need | Where |
|---|---|
| Ratios with their source | `GET /fundamental` (EODHD layout, `Sources` section) |
| Statements, earnings, ratings | `GET /fundamental/deep` |
| Intrinsic value | `GET /dcf/{ticker}` (FCF), `/compare` (three models), `/sensitivity`; `POST /dcf/{ticker}` with your assumptions |
| Spreadsheet | Google Sheets template |
| Dashboards | OpenBB Workspace widgets |

## 1. An instance and a key

Install the instance ([Installation](../getting-started/installation.md)) and create a **read-only key** for the tools that keep it outside your machine:

```
FONREX_READ_ONLY_API_KEYS=frx_live_<random value>
```

## 2. Fundamentals

```bash
curl -s -H "X-API-KEY: $KEY" "http://localhost:5000/fundamental?ticker=AIR.PA"
curl -s -H "X-API-KEY: $KEY" "http://localhost:5000/fundamental/deep?ticker=AIR.PA"
```

`/fundamental` takes each figure from Yahoo Finance (asked with the symbol verified for the listing), then from the stored figures, then from the scraped websites, and tells you where each one comes from. Ratios are ratios (`0.0125` for 1.25 %). See [Fundamentals](../api-reference/fundamentals.md).

## 3. Valuation

The DCF reads the deep fundamentals stored for the instrument: call `/fundamental/deep` first.

```bash
curl -s -H "X-API-KEY: $KEY" "http://localhost:5000/dcf/AIR.PA"
curl -s -H "X-API-KEY: $KEY" "http://localhost:5000/dcf/AIR.PA/sensitivity?model=fcf"
curl -s -X POST -H "X-API-KEY: $KEY" -H "Content-Type: application/json" \
  -d '{"models": ["fcf", "eps", "ddm"], "projection_years": 10, "terminal_growth_rate": 0.02}' \
  http://localhost:5000/dcf/AIR.PA
```

`GET /dcf` computes the FCF model; `/compare` and a `POST` naming several models weigh FCF, EPS and DDM 50/30/20. WACC from CAPM with the FRED 10-year rate. See [Valuation & DCF](../api-reference/valuation-dcf.md).

## 4. Google Sheets

The template refreshes fundamentals, DCF and indicators for a watchlist, and offers `=FONREX_PE()`, `=FONREX_DIVIDEND_YIELD()`, `=FONREX_INTRINSIC_VALUE()` and `=FONREX_RSI()`. Google's servers reach your instance through a tunnel. See the [Google Sheets guide](../guides/google-sheets-connector.md).

## 5. OpenBB Workspace

Add your instance URL as a data source, with your key in the `X-API-KEY` header: 19 widgets and two dashboards (EU Markets, Screener & Macro). See the [OpenBB guide](../guides/openbb-workspace.md).

:::info
Fonrex displays raw financial data and analytical outputs. It does not constitute investment advice.
:::



<!-- FILE: /docs/pathways/app-developer.md -->

---
id: "app-developer"
title: "App Developer Pathway"
sidebar_label: "App Developer"
description: "Integrate a Fonrex instance in an application: authentication, REST routes, WebSocket stream, errors and caching"
---

# App Developer Pathway

This pathway is for developers calling a Fonrex instance from an application (web, mobile, scripts).

| Protocol | Use |
|---|---|
| REST (JSON, OpenAPI) | Catalogue, prices, fundamentals, indicators, valuation, news |
| WebSocket | Realtime ticks, one connection per ticker |

## 1. OpenAPI

Your instance publishes its own specification:

- Swagger UI: `http://localhost:5000/docs`
- ReDoc: `http://localhost:5000/redoc`
- OpenAPI JSON: `http://localhost:5000/openapi.json`

Generate a client from `openapi.json` if your stack supports it.

## 2. Authentication

Send a key with every request: `X-API-KEY: <key>` or `Authorization: Bearer <key>`. `401` means no key, `403` an unknown key or a read-only key on a route that changes something. A browser or mobile application holds its key on the user's device: give it a **read-only** key.

```typescript
const FONREX = "http://localhost:5000";

async function fonrex<T>(path: string, key: string): Promise<T> {
  const response = await fetch(`${FONREX}${path}`, { headers: { "X-API-KEY": key } });
  if (!response.ok) {
    throw new Error(`${response.status}: ${await response.text()}`);
  }
  return response.json() as Promise<T>;
}

type Listing = { id: number; ticker: string; exchange: string; currency: string; isin: string; name: string; is_primary: boolean };

const { listings } = await fonrex<{ count: number; listings: Listing[] }>(
  "/listings?isin=NL0000235190", key,
);
```

## 3. Identify instruments correctly

A ticker is not a global identifier: the same instrument has several listings (currencies, exchanges), and the same ticker can be another instrument elsewhere. Look instruments up by ISIN (`/assets/by-isin/{isin}`, `/listings?isin=`), and pass `currency` or `exchange` to the price routes when several listings share a ticker.

## 4. Realtime

```javascript
const ws = new WebSocket(`ws://localhost:5000/ws/realtime/AIR.PA?token=${key}`);
ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  switch (message.type) {
    case "snapshot":
    case "tick":
      render(message.data.close);
      break;
    case "not_streaming":
      showDelayed(message.error);
      break;
  }
};
```

One connection per ticker. Prices arrive as decimal strings. A read-only key does not start streams; subscribe tickers server-side with `POST /realtime/subscribe`. See [Realtime](../api-reference/realtime.md).

## 5. Errors and caching

- Error bodies are `{"detail": "..."}`, except `/eod` (`{"error", "message", "reason"}`).
- `503` means a service of the instance is not available (database not migrated, Redis down, worker not started).
- Most answers are cached in Redis (EOD 24 h, fundamentals 1 h, DCF 6 h, news 30 min…); `nocache`, `refresh` or `force_refresh` parameters bypass the cache where they exist.
- Fundamentals answers report their sources (`Sources`); there is no response header naming the provider.

## Next steps

- [Assets & listings](../api-reference/assets.md)
- [Realtime configuration](../guides/configure-realtime.md)
- [Layers & ports](../architecture/hexagonal.md)



<!-- FILE: /docs/pathways/devops-admin.md -->

---
id: "devops-admin"
title: "DevOps & Infra Admin Pathway"
sidebar_label: "DevOps & Infra Admin"
description: "Install, secure, upgrade, back up and monitor a Fonrex instance"
---

# DevOps & Infra Admin Pathway

This pathway is for whoever runs the instance: the stack, its security, its upgrades and the health of its data sources.

| Component | Technology | Notes |
|---|---|---|
| API | FastAPI, Gunicorn + Uvicorn (Python 3.12) | One worker: realtime and canary live in the process |
| Database | PostgreSQL 16 + TimescaleDB (`timescaledb-ha:pg16`) | Volume `timescale_data`, `127.0.0.1:5432` |
| Cache | Redis 7, 256 MB, `allkeys-lru` | `127.0.0.1:6379`, no password |
| Monitoring | Validation layer + daily canary | `/health/*` routes |

## 1. Install and secure

```bash
cp .env.example .env
# FONREX_API_KEY, FONREX_READ_ONLY_API_KEYS, POSTGRES_PASSWORD, SEC_EDGAR_EMAIL
mkdir -p logs
docker compose up -d
docker compose ps
```

Keep the API behind a TLS reverse proxy that forwards the WebSocket upgrade. See [Docker production deployment](../deployment/docker.md) and the [production checklist](../deployment/production-checklist.md).

## 2. Upgrade

```bash
docker compose exec -T db pg_dump -U fonrex -d fonrex -Fc > fonrex-$(date +%F).dump
git pull
docker compose up -d --build        # migrations run at start
```

Roll back by restoring the dump with the previous code. See [Database migrations in production](../deployment/database-migrations.md).

## 3. Watch the instance

```bash
curl -s http://localhost:5000/health                                        # no key
curl -s -H "X-API-KEY: $KEY" http://localhost:5000/health/providers
curl -s -H "X-API-KEY: $KEY" "http://localhost:5000/health/alerts?severity=critical"
curl -s -H "X-API-KEY: $KEY" http://localhost:5000/realtime/status
curl -s -H "X-API-KEY: $KEY" http://localhost:5000/database/stats
```

- `/health`: `providers.unavailable` names a provider that failed to load.
- `/health/providers`: result of the daily canary (06:00 UTC), per provider.
- Logs: `docker compose logs -f fonrex-api`.

## 4. Data sources refusing your IP

Websites protected by anti-bot services may refuse a server IP. Route those providers through a proxy:

```env
FONREX_PROXY_URL=http://user:password@proxy.example:8888
FONREX_PROXY_PROVIDERS=Investing,Gurufocus,wallStreetJournal
FONREX_PROVIDER_MAX_CONCURRENCY=4
```

## 5. Storage

- Prices: about ten years per listing on first ingestion, compressed after 14 days.
- `POST /database/cleanup` deletes prices older than `days_to_keep` (730 by default!) — always run it with `dry_run` first.
- Intraday candles and validation logs expire after 30 days; the usage log after `USAGE_LOG_RETENTION_DAYS`; news articles are kept.

## Next steps

- [Docker Compose topology](../getting-started/docker-compose.md)
- [Canary monitor](../monitoring/canary-monitor.md) and [alerts](../monitoring/alerts.md)
- [Environment variables](../deployment/environment-variables.md)



<!-- FILE: /docs/getting-started/installation.md -->

---
id: "installation"
title: "Installation Guide"
sidebar_label: "Installation"
description: "Install and run a self-hosted Fonrex instance with Docker Compose"
---

# Installation Guide

This guide sets up a self-hosted Fonrex instance with Docker Compose.

## Requirements

- Docker Engine and Docker Compose v2
- Linux, macOS, or Windows with WSL2
- 4 GB of RAM at least, 8 GB recommended for large ingestions
- Disk space for the price history you ingest (a first ingestion fetches about ten years per listing)

## 1. Clone the repository

```bash
git clone https://github.com/fonrex/fonrex.git
cd fonrex
```

## 2. Create `.env`

```bash
cp .env.example .env
```

Docker Compose loads `.env` into the API container. Two settings must be set **before the first start**:

**An API key.** Authentication is on by default: until a key is configured, every route except `/health`, the documentation and the OpenBB discovery files answers `401`.

```bash
export FONREX_API_KEY="frx_live_$(openssl rand -hex 24)"
sed -i.bak "s/^FONREX_API_KEY=.*/FONREX_API_KEY=$FONREX_API_KEY/" .env && rm .env.bak
```

**The database password.** Change `POSTGRES_PASSWORD` (letters, digits, `-` and `_` only — it is embedded in a URL). It is stored in the database volume when the database is created; changing it later requires `ALTER USER fonrex PASSWORD '...'` or a new volume.

Keep one `KEY=value` per line and put comments on their own lines: Docker Compose reads a comment placed after an empty value as the value itself.

See [Configuration](configuration.md) for every setting.

## 3. Start the stack

```bash
mkdir -p logs
docker compose up -d
```

This starts three containers:

| Container | Role | Published on |
|---|---|---|
| `fonrex-api` | FastAPI served by Gunicorn | `0.0.0.0:5000` |
| `fonrex-db` | PostgreSQL 16 + TimescaleDB (`timescale/timescaledb-ha:pg16`) | `127.0.0.1:5432` only |
| `fonrex-redis` | Redis 7 (cache and Pub/Sub) | `127.0.0.1:6379` only |

The API container applies the database migrations (`alembic upgrade head`) before starting. A fourth service, `fonrex-migrate`, runs the migrations alone; it belongs to the `migrate` profile and does not start by default.

## 4. Check the instance

```bash
curl http://localhost:5000/health
curl -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/listings?ticker=AIR.PA"
```

`/health` answers without a key; the second request checks that your key is accepted.

## 5. Import instruments

The image contains two catalogues, `data/etf.csv` and `data/stocks.csv`:

```bash
docker compose exec fonrex-api python import_assets.py --file data/etf.csv
```

Alternatives:

- `make db-seed` imports the default catalogue and enriches it from Yahoo Finance.
- `SEED_ON_FIRST_RUN=true` in `.env` imports `data/etf.csv` at the first start when the database is empty.

Prices are ingested the first time a listing is asked for (`GET /eod/{ticker}`), or for the whole catalogue with `scripts/ingest_all.py` — see [Ingesting historical data](../guides/ingest-historical-data.md).

## Updating

```bash
git pull
docker compose up -d --build
```

The image holds the code, the migrations and the seed files: rebuild it after every update. Migrations run at the next start. Back the database up before an upgrade — see [Database migrations in production](../deployment/database-migrations.md).

For development, `docker compose -f docker-compose.yml -f docker-compose.dev.yml up` runs the code of your folder without rebuilding the image.



<!-- FILE: /docs/getting-started/configuration.md -->

---
id: "configuration"
title: "System Configuration"
sidebar_label: "Configuration"
description: "Reference of the Fonrex settings read from .env"
---

# System Configuration

Fonrex reads its settings from environment variables. Copy `.env.example` to `.env` and edit it: Docker Compose loads `.env` into the API container, and `make run` loads it for a local run.

Keep one `KEY=value` per line and put comments on their own lines — a comment written after an empty value is read as the value.

## Authentication

| Variable | Default | Description |
|---|---|---|
| `FONREX_API_KEY` | *(empty)* | Full-access API key. Generate one with `echo "frx_live_$(openssl rand -hex 24)"` |
| `FONREX_API_KEYS` | *(empty)* | Additional full-access keys, comma-separated |
| `FONREX_READ_ONLY_API_KEYS` | *(empty)* | Read-only keys, comma-separated: they read data but cannot clear the cache, clean the database, trigger ingestion or change subscriptions |
| `FONREX_AUTH_REQUIRED` | `true` | `false` opens every route, **only** when no key is configured. Use it only for an instance reachable from no network |

Clients send a key as `Authorization: Bearer <key>` or `X-API-KEY: <key>`. See [First API call](first-api-call.md).

## Database and cache

| Variable | Default | Description |
|---|---|---|
| `DATABASE_URL` | `postgresql://fonrex:fonrex_password@localhost:5432/fonrex` | Address for a local run. With Docker Compose it is replaced by the `db` service address, built from `POSTGRES_PASSWORD` |
| `ASYNC_DATABASE_URL` | *(empty)* | asyncpg address; derived from `DATABASE_URL` when empty (Docker Compose empties it) |
| `POSTGRES_DB` / `POSTGRES_USER` | `fonrex` | Kept for local tools; `docker-compose.yml` uses fixed values (user `fonrex`, database `fonrex` created by `postgres-init.sh`) |
| `POSTGRES_PASSWORD` | `fonrex_password` | Change it before the first start (letters, digits, `-`, `_`) |
| `REDIS_URL` | `redis://localhost:6379/0` | Replaced by the `redis` service address with Docker Compose |
| `CACHE_TTL` | `300` | Default Redis lifetime in seconds (each cached category has its own lifetime, listed by `GET /cache/stats`) |
| `WEB_CONCURRENCY` | `1` | Gunicorn workers (Docker Compose). Keep `1`: the realtime worker and the daily canary live in the API process |

## Historical ingestion

| Variable | Default | Description |
|---|---|---|
| `INGEST_CONCURRENCY` | `5` | Parallel ingestions of a bulk ingestion whose caller gives no `concurrency` |
| `INGEST_YF_DELAY` | `0.5` | Pause in seconds before falling back to TradingView |
| `INGEST_TV_DELAY` | `2.0` | Read but not used by the current code |
| `INGEST_BATCH_SIZE` | `1000` | Rows per database upsert |

## Real-time streaming

| Variable | Default | Description |
|---|---|---|
| `TV_MAX_CONNECTIONS` | `10` | Simultaneous TradingView WebSocket connections |
| `TV_RECONNECT_DELAY` | `5` | First reconnection delay in seconds, doubled up to 60 |
| `REALTIME_QUOTE_TTL` | `60` | Lifetime of a quote snapshot in Redis, in seconds |

## Technical indicators

| Variable | Default | Description |
|---|---|---|
| `TECHNICAL_CACHE_ENABLED` | `true` | Cache indicator results in Redis |
| `TECHNICAL_DEFAULT_LIMIT` | `500` | Bars loaded when a request gives no limit (10 to 5000) |
| `TECHNICAL_MAX_BATCH_TICKERS` | `20` | Tickers accepted by `POST /technical/batch` |
| `TECHNICAL_MAX_BATCH_INDICATORS` | `10` | Indicators accepted by `POST /technical/batch` |

## News

| Variable | Default | Description |
|---|---|---|
| `NEWS_CACHE_TTL` | `1800` | Lifetime of a news answer in Redis, in seconds |
| `NEWS_DEFAULT_LIMIT` | `20` | Articles returned for a ticker when the request gives no limit |
| `NEWS_MAX_LIMIT` | `100` | Largest limit a request may ask for |
| `NEWS_DEDUP_SIMILARITY` | `0.85` | Title similarity above which two articles are one |

## Valuation (DCF) and macro rates

| Variable | Default | Description |
|---|---|---|
| `DCF_CACHE_TTL` | `21600` | Lifetime of a DCF answer in Redis (6 h) |
| `DCF_DEFAULT_PROJECTION_YEARS` | `5` | Projection years (3 to 10) |
| `DCF_RISK_FREE_RATE` | `0.04` | Risk-free rate used when FRED gives none |
| `DCF_EQUITY_RISK_PREMIUM` | `0.055` | Equity risk premium, as a ratio |
| `DCF_TERMINAL_GROWTH_RATE` | `0.025` | Terminal growth rate, as a ratio |
| `FRED_API_KEY` | *(empty)* | Free key from fred.stlouisfed.org; without it the stored rate or `DCF_RISK_FREE_RATE` is used |
| `MACRO_RATES_CACHE_TTL` | `21600` | Lifetime of the macro rates in Redis (6 h) |

## Provider monitoring

| Variable | Default | Description |
|---|---|---|
| `VALIDATION_OUTLIER_THRESHOLD` | `0.50` | Deviation from the median above which a value is an outlier |
| `VALIDATION_MIN_PROVIDERS` | `2` | Providers needed for a consensus check |
| `CANARY_RUN_HOUR` | `6` | UTC hour of the daily canary run |
| `CANARY_PROVIDER_SEMAPHORE` | `3` | Providers checked in parallel by the canary |
| `CANARY_PRICE_RANGE_TTL_SECONDS` | `21600` | Validity of a dynamic price range (6 h) |
| `CANARY_PRICE_RANGE_NEGATIVE_TTL_SECONDS` | `300` | Retry delay after a range could not be computed |
| `ALERT_CANARY_CRITICAL` | `3` | Canary failures that raise a critical alert |
| `ALERT_SUCCESS_RATE_CRITICAL` | `0.70` | Success rate under which an alert is critical |
| `ALERT_SUCCESS_RATE_WARNING` | `0.85` | Success rate under which an alert is a warning |

## Providers and outbound requests

| Variable | Default | Description |
|---|---|---|
| `SEC_EDGAR_EMAIL` | `contact@fonrex.io` | Contact address sent to SEC EDGAR (required by the SEC policy): put your own |
| `OPENFIGI_API_KEY` | *(empty)* | Optional OpenFIGI key (higher rate limit) |
| `BARRONS_TOKEN`, `MARKETWATCH_TOKEN`, `WSJ_TOKEN` | *(empty)* | Optional tokens of these websites |
| `FONREX_PROVIDER_MAX_CONCURRENCY` | `4` | Requests one provider may run at the same time (1 to 64) |
| `FONREX_PROXY_URL` | *(empty)* | Optional outbound HTTP proxy for the scraped websites (not for yfinance or TradingView) |
| `FONREX_PROXY_PROVIDERS` | *(empty)* | Providers that use the proxy, comma-separated; empty means all |
| `LOGO_TOKEN` | *(empty)* | Token for logo downloads from img.logo.dev |

## Usage log and start-up

| Variable | Default | Description |
|---|---|---|
| `USAGE_LOG_IP` | `none` | Part of the caller's IP kept in `usage_logs`: `none`, `truncated` (network only) or `full` |
| `USAGE_LOG_RETENTION_DAYS` | `90` | Days of usage log kept; `0` keeps everything |
| `SEED_ON_FIRST_RUN` | `false` | Import `data/etf.csv` at the first start when the database is empty |
| `OPENBB_ALLOWED_ORIGIN` | `https://pro.openbb.co` | Origins allowed by CORS, comma-separated |



<!-- FILE: /docs/getting-started/first-api-call.md -->

---
id: "first-api-call"
title: "Making Your First API Call"
sidebar_label: "First API Call"
description: "Authenticate and query prices, indicators and real-time quotes with cURL, Python and WebSocket clients"
---

# Making Your First API Call

Every route except `/health`, `/docs`, `/redoc`, `/openapi.json`, `/widgets.json`, `/apps.json` and `/static` requires an API key, sent in one of two headers:

```
Authorization: Bearer frx_live_...
X-API-KEY: frx_live_...
```

A missing key answers `401`, an unknown key `403`. The examples below use the key you set in `.env` during the [installation](installation.md):

```bash
export FONREX_API_KEY="frx_live_..."   # the value of FONREX_API_KEY in .env
AUTH="X-API-KEY: $FONREX_API_KEY"
```

The interactive documentation of your instance is at `http://localhost:5000/docs`.

## 1. End-of-day prices

```bash
curl -s -H "$AUTH" "http://localhost:5000/eod/AIR.PA?period=5d"
```

When nothing is stored yet for the listing, Fonrex ingests its history first (Yahoo Finance, TradingView as a fallback), so the first call can take a few seconds.

```json
{
  "ticker": "AIR.PA",
  "period": "5d",
  "format": "json",
  "count": 3,
  "retrieved_at": "2026-10-08T16:34:42.404598+00:00",
  "data_source": "database",
  "data": [
    {
      "Date": "2026-10-07",
      "Open": 154.46,
      "High": 156.46,
      "Low": 152.46,
      "Close": 155.46,
      "Adj Close": 155.46,
      "Volume": 1400000
    }
  ]
}
```

Add `&fmt=csv` for CSV. When several listings share a ticker, choose one with `currency` or `exchange`.

### Python

```python
import os
import requests

session = requests.Session()
session.headers["X-API-KEY"] = os.environ["FONREX_API_KEY"]

eod = session.get("http://localhost:5000/eod/AIR.PA", params={"period": "1mo"}).json()
for bar in eod["data"]:
    print(bar["Date"], bar["Close"])
```

## 2. A technical indicator

Indicators are computed on the prices stored in the database: ingest the listing first (step 1, or `POST /historical/ingest?ticker=AIR.PA`). Without prices the answer is `404`.

```bash
curl -s -H "$AUTH" "http://localhost:5000/technical/AIR.PA?indicator=rsi&period=14"
```

```python
rsi = session.get(
    "http://localhost:5000/technical/AIR.PA",
    params={"indicator": "rsi", "period": 14},
).json()
last = rsi["series"][0]["values"][-1]
print(f"RSI(14) on {last['t']}: {last['v']}")
```

Values are returned as strings (decimal numbers) or `null` while the indicator has too few bars.

## 3. Fundamentals

```bash
curl -s -H "$AUTH" "http://localhost:5000/fundamental?ticker=AIR.PA"
```

The answer is one document in the EODHD layout (`General`, `Highlights`, `Valuation`, …) with a `Sources` section naming the source of each figure. See [Fundamentals](../api-reference/fundamentals.md).

## 4. Real-time prices over WebSocket

Browsers cannot set headers on a WebSocket: pass the key in the query string (`token`, `api_key` or `key`).

```javascript
const ws = new WebSocket(`ws://localhost:5000/ws/realtime/AIR.PA?token=${FONREX_API_KEY}`);

ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  if (message.type === "tick") console.log(message.data.close, message.data.timestamp);
};
```

```python
import asyncio
import json
import os

import websockets

async def stream(ticker: str) -> None:
    url = f"ws://localhost:5000/ws/realtime/{ticker}?token={os.environ['FONREX_API_KEY']}"
    async with websockets.connect(url) as ws:
        async for raw in ws:
            message = json.loads(raw)
            if message["type"] in ("snapshot", "tick"):
                print(message["type"], message["data"]["close"])

asyncio.run(stream("AIR.PA"))
```

With a full-access key, connecting starts the TradingView stream of the ticker when it is not streamed yet. A read-only key never starts one: it receives a `not_streaming` message and then the ticks, once a full-access client has subscribed the ticker. See [Realtime](../api-reference/realtime.md).



<!-- FILE: /docs/getting-started/docker-compose.md -->

---
id: "docker-compose"
title: "Docker Compose Topology"
sidebar_label: "Docker Compose"
description: "Containers, ports, volumes, health checks and everyday operations of the Fonrex stack"
---

# Docker Compose Topology

`docker-compose.yml` runs the API (`fonrex-api`), the database (`db`) and the cache (`redis`). A fourth service, `fonrex-migrate`, runs the migrations on demand. Commands such as `docker compose exec` take the service name; `docker exec` takes the container name (`fonrex-db`, `fonrex-redis`).

```
┌───────────────────────────── docker compose ─────────────────────────────┐
│                                                                          │
│   fonrex-api  ──────────►  fonrex-db (TimescaleDB)                       │
│   :5000                    127.0.0.1:5432, volume timescale_data         │
│      │                                                                   │
│      └────────────────►  fonrex-redis (Redis 7)                          │
│                            127.0.0.1:6379, volume redis_data             │
│                                                                          │
│   fonrex-migrate (profile "migrate"): alembic upgrade head               │
└──────────────────────────────────────────────────────────────────────────┘
```

## Services

### `fonrex-api`
- **Image**: built from the `Dockerfile` (Python 3.12, non-root user). It holds the code, the Alembic migrations and the seed files (`data/*.csv`).
- **Port**: `5000` on every interface of the host — the API key protects it.
- **Start-up** (`entrypoint.sh`): waits for PostgreSQL and Redis, applies `alembic upgrade head`, optionally imports `data/etf.csv` (`SEED_ON_FIRST_RUN=true`), then starts Gunicorn with Uvicorn workers (`WEB_CONCURRENCY`, 1 by default).
- **Configuration**: `.env` (`env_file`). `DATABASE_URL`, `REDIS_URL` and `ASYNC_DATABASE_URL` are overridden so that the container reaches the `db` and `redis` services, not `localhost`.
- **Mounts**: only what the application writes — `./logs` and `./static/logos`.
- **Health check**: `curl -f http://localhost:5000/health`.

### `db` (container `fonrex-db`)
- **Image**: `timescale/timescaledb-ha:pg16` (PostgreSQL 16 + TimescaleDB).
- **Port**: `127.0.0.1:5432` — reachable from the host (psql, Zipline bundle), never from the network.
- **Volume**: `timescale_data`, mounted on the data directory of this image, `/home/postgres/pgdata/data` (`PGDATA`). The data survives `docker compose down` and a rebuild.
- **Initialisation**: `postgres-init.sh` creates the `fonrex` database.

### `redis` (container `fonrex-redis`)
- **Image**: `redis:7-alpine`, `--appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru`.
- **Port**: `127.0.0.1:6379` (Redis has no password).
- **Role**: cache of the answers, real-time quotes, Pub/Sub channels of the WebSocket stream.

### `fonrex-migrate`
- **Profile**: `migrate` — it does not start with `docker compose up`.
- **Command**: `alembic upgrade head`. Useful to migrate without starting the API.

## Common operations

```bash
docker compose up -d                       # start
docker compose up -d --build               # rebuild after a code update
docker compose logs -f fonrex-api          # API logs
docker compose --profile migrate run --rm fonrex-migrate   # migrations alone
docker compose down                        # stop, keep the data
docker compose down -v                     # stop and DELETE the database and cache volumes
```

Development: `docker compose -f docker-compose.yml -f docker-compose.dev.yml up` mounts the project folder on `/app`, so an edit only needs a restart.

## Backing up the database

```bash
# Backup to one file on the host
docker compose exec -T db pg_dump -U fonrex -d fonrex -Fc > fonrex.dump

# Restore into an empty database
docker compose up -d db
docker compose exec -T db psql -U fonrex -d fonrex \
  -c "CREATE EXTENSION IF NOT EXISTS timescaledb;" -c "SELECT timescaledb_pre_restore();"
docker compose exec -T db pg_restore -U fonrex -d fonrex -Fc < fonrex.dump
docker compose exec -T db psql -U fonrex -d fonrex -c "SELECT timescaledb_post_restore();"
docker compose up -d
```

Restore with the same TimescaleDB version as the one that made the dump. Never commit a dump to Git: it holds the data of your instance.

## Troubleshooting

### `401 Missing API key` or `403` on every request
Set `FONREX_API_KEY` in `.env`, run `docker compose up -d` and send the key with each request. The start-up logs state the authentication mode: `docker compose logs fonrex-api | grep -i auth`.

### Permission denied on `logs`
```bash
mkdir -p logs
chmod 777 logs
docker compose up -d
```

### Container name already in use
```bash
docker rm -f fonrex-db fonrex-redis fonrex-api
docker compose up -d
```

### An installation whose database was not on a volume
Older versions mounted the volume on `/var/lib/postgresql/data`, a path this image does not use: the database lived inside the container. Check before upgrading:

```bash
docker exec fonrex-db psql -U fonrex -d fonrex -tc "show data_directory"
docker inspect fonrex-db --format '{{range .Mounts}}{{.Destination}} {{end}}'
```

If the data directory is not a mounted destination, back the database up **before** `docker compose up -d` with the new `docker-compose.yml`, then restore it.



<!-- FILE: /docs/api-reference/assets.md -->

---
id: "assets"
title: "Asset & Listing API Reference"
sidebar_label: "Assets & Listings"
description: "Instruments, listings and end-of-day prices of the Fonrex catalogue"
---

# Asset & Listing API Reference

Fonrex separates an **instrument** (one ISIN, a row of `assets`) from its **listings** (one row of `asset_listings` per ticker, exchange and currency). The same ETF quoted in EUR and in USD is one instrument with two listings, and each listing has its own price series.

All routes on this page require an API key (`X-API-KEY` or `Authorization: Bearer`).

---

## <span className="api-method get">GET</span> `/assets/by-isin/{isin}`

The instrument of an ISIN, with its preferred listing and all its listings.

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/assets/by-isin/NL0000235190"
```

```json
{
  "asset_id": 1,
  "name": "Airbus SE",
  "ticker": "AIR.PA",
  "exchange": "XPAR",
  "currency": "EUR",
  "sector": "Industrials",
  "industry": "Aerospace & Defense",
  "quote_type": "EQUITY",
  "isin": "NL0000235190",
  "listing_id": 1,
  "listings": [
    {
      "id": 1, "asset_id": 1, "ticker": "AIR.PA", "exchange": "XPAR", "currency": "EUR",
      "isin": "NL0000235190", "name": "Airbus SE", "source": "csv_import",
      "is_primary": true, "is_active": true
    },
    {
      "id": 2, "asset_id": 1, "ticker": "AIR.DE", "exchange": "XETR", "currency": "EUR",
      "isin": "NL0000235190", "name": "Airbus SE", "source": "csv_import",
      "is_primary": false, "is_active": true
    }
  ]
}
```

The answer also holds `display_name`, `official_symbol`, `logo_path`, `ir_website` and `long_business_summary` when they are known. An unknown ISIN answers `404`.

---

## <span className="api-method get">GET</span> `/listings`

Active listings matching the filters. At least one filter is required (`400` otherwise).

| Parameter | Type | Description |
|---|---|---|
| `ticker` | string | Ticker of the listing (e.g. `AIR.PA`) |
| `isin` | string | ISIN of the instrument |
| `exchange` | string | Exchange code as stored in the catalogue |
| `currency` | string | Currency of the listing (e.g. `EUR`) |

```json
{
  "count": 2,
  "listings": [
    { "id": 1, "asset_id": 1, "ticker": "AIR.PA", "exchange": "XPAR", "currency": "EUR", "isin": "NL0000235190", "name": "Airbus SE", "source": "csv_import", "is_primary": true, "is_active": true }
  ]
}
```

---

## <span className="api-method get">GET</span> `/eod/{ticker}`

End-of-day prices of a listing, in JSON or CSV. When nothing is stored for the request, the listing is ingested first (Yahoo Finance with the symbol verified for the listing, TradingView as a fallback).

| Parameter | Type | Default | Description |
|---|---|---|---|
| `ticker` | string | — | Ticker (at most 10 characters: letters, digits, `.` and `-`) |
| `period` | string | — | `1d`, `5d`, `1mo`, `3mo`, `6mo`, `1y`, `2y`, `5y`, `10y`, `ytd`, `max`, `daily`, `weekly`, `monthly`. Required unless `from` and `to` are given |
| `from`, `to` | date | — | Window `YYYY-MM-DD`, given together |
| `fmt` | string | `json` | `json` or `csv` |
| `order` | string | `a` | `a` (oldest first) or `d` (newest first) |
| `currency` | string | — | Currency of the listing, when several listings share the ticker |
| `exchange` | string | — | Exchange of the listing, when several listings share the ticker |
| `isin` | string | — | ISIN of the instrument, when several instruments share the ticker (12 characters, case-insensitive) |

`weekly` returns weekly bars and `monthly` monthly bars; every other period returns daily bars.

### Choosing the listing

A ticker alone does not always designate one instrument: in the default catalogue, `NEM` is Newmont in USD, its Australian line in AUD and Nemetschek in EUR — three ISINs. Fonrex takes, among the listings bearing the ticker, the primary one first, then by currency and exchange in alphabetical order: for `NEM`, the Australian line in AUD.

- `isin` keeps the listings of one instrument only. A listing of another instrument is never returned, even when the ticker with its suffix is not in the catalogue (`MRK.DE` falls back to `MRK` only within the named instrument).
- `currency` and `exchange` choose among the listings of that instrument.

`isin` with `currency` names a listing without ambiguity: `GET /eod/NEM?period=1y&isin=US6516391066&currency=USD`. An ISIN that does not have 12 characters (two letters, then ten letters or digits) is refused with `400`.

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/eod/AIR.PA?period=5d"
```

```json
{
  "ticker": "AIR.PA",
  "listing": { "ticker": "AIR.PA", "isin": "NL0000235190", "currency": "EUR", "exchange": "XPAR" },
  "period": "5d",
  "format": "json",
  "count": 3,
  "retrieved_at": "2026-10-08T16:34:42.404598+00:00",
  "data_source": "database",
  "data": [
    { "Date": "2026-10-06", "Open": 153.44, "High": 155.44, "Low": 151.44, "Close": 154.44, "Adj Close": 154.44, "Volume": 1399000 },
    { "Date": "2026-10-07", "Open": 154.46, "High": 156.46, "Low": 152.46, "Close": 155.46, "Adj Close": 155.46, "Volume": 1400000 }
  ]
}
```

`Open`, `High`, `Low` and `Close` are the traded prices, adjusted for splits; `Adj Close` is the close adjusted for splits and dividends (the close when the source gives none). `listing` is the listing that was read: compare its `isin` with the instrument you expect. Its `exchange` is `null` when the catalogue does not know it (tickers without suffix, such as US stocks). `Date` is the date of the trading session. `data_source` is `database` when the prices were already stored, otherwise the source of the ingestion (`yfinance` or `tradingview`). Answers are cached 24 hours in Redis.

With `fmt=csv`:

```
Date,Open,High,Low,Close,Adj Close,Volume
2026-10-06,153.44,155.44,151.44,154.44,154.44,1399000
2026-10-07,154.46,156.46,152.46,155.46,155.46,1400000
```

### Errors

| Code | Body | When |
|---|---|---|
| `400` | `{"error": "Invalid request", "message": "..."}` | Invalid ticker, period, format, order, dates or ISIN |
| `404` | `{"error": "No data found", "message": "...", "reason": "..."}` | Nothing stored and nothing could be ingested. `reason` explains why, e.g. no listing of the ticker for that ISIN and currency (`No listing found for ticker NEM (ISIN DE0006452907, currency USD)`), or no Yahoo symbol quoted in the currency of the listing |

See [Ingesting historical data](../guides/ingest-historical-data.md) for the way the source symbol of a listing is chosen.



<!-- FILE: /docs/api-reference/fundamentals.md -->

---
id: "fundamentals"
title: "Fundamental Financials API Reference"
sidebar_label: "Fundamentals"
description: "Multi-provider fundamentals in the EODHD layout and stored deep fundamentals"
---

# Fundamental Financials API Reference

Two routes serve fundamentals:

- `GET /fundamental` builds one document from Yahoo Finance, the figures stored in the database and the scraped providers, with the source of every figure.
- `GET /fundamental/deep` returns what the deep enrichment stored: highlights, financial statements, earnings history and analyst ratings.

Ratios are ratios: a 0.32 % dividend yield is `0.0032`.

---

## <span className="api-method get">GET</span> `/fundamental`

| Parameter | Type | Default | Description |
|---|---|---|---|
| `ticker` | string | — | Ticker (e.g. `AIR.PA`). `ticker` or `isin` is required |
| `isin` | string | — | ISIN of the instrument |
| `exchange` | string | — | Exchange, to choose a listing |
| `currency` | string | — | Currency, to choose a listing |
| `provider` | string | all | One provider name, or several separated by commas, instead of all of them |
| `fmt` | string | `eodhd` | `eodhd` (rendered document) or `raw` (the answer of each provider) |
| `nocache` | boolean | `false` | Ignore the cached answer |

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/fundamental?ticker=AIR.PA"
```

### The rendered document (`fmt=eodhd`)

| Section | Content |
|---|---|
| `General` | Name, ISIN, exchange, currency, country, sector and industry, description, address, website |
| `Highlights` | Market capitalisation, EBITDA, P/E, EPS, dividend yield, margins, returns, revenue… |
| `Valuation` | Trailing and forward P/E, price/sales, price/book, enterprise value ratios |
| `SharesStats` | Shares outstanding and float, insider and institution ownership, short interest |
| `Technicals` | Beta, 52-week high and low, moving averages |
| `SplitsDividends` | Dividend rate and yield, payout ratio, dates, last split |
| `AnalystRatings` | Consensus, target price, number of buy/hold/sell ratings |
| `Holders`, `InsiderTransactions`, `ESGScores` | Holders, SEC Form 4 transactions (US shares), ESG scores |
| `Earnings`, `Financials` | Stored earnings history and financial statements |
| `Providers` | What each provider returned |
| `Sources` | The source of each figure, e.g. `{"Highlights": {"PERatio": "YahooFinance", "PEGRatio": "database (2026-10-01)"}}` |
| `ETF_Data` | Only for an ETF |

Each figure is taken, in this order, from:

1. the Yahoo Finance answer of this request;
2. the figures stored by the deep enrichment, reported as `database (date of the fetch)`;
3. for the trailing P/E, the earnings per share and the dividend yield only, the scraped providers publishing the same quantity (Google Finance, Barron's, MarketWatch, WSJ, Investing.com).

Estimates for the current year (Boursorama, ZoneBourse) and quarterly figures (Google Finance) are other quantities: they are never used as a fallback but remain available with `fmt=raw`.

### Which instrument is asked

For a listing of your catalogue, Yahoo Finance is asked with the **symbol verified for the listing** (found from the ISIN and checked against the currency of the listing), never with the bare ticker, which may be another instrument on Yahoo. Without a verified symbol, Yahoo is not asked and its entry says why. Scraped providers are searched by mapping, ISIN or ticker; a provider that answers about another ISIN is reported as an error.

Every value goes through the [validation layer](../monitoring/validation-layer.md) before it is used.

The complete answer is cached one hour; `nocache=true` bypasses it.

---

## <span className="api-method get">GET</span> `/fundamental/deep`

| Parameter | Type | Default | Description |
|---|---|---|---|
| `ticker` / `isin` | string | — | The instrument (one of the two is required) |
| `refresh` | boolean | `false` | Fetch again from Yahoo Finance instead of using the cached answer |
| `sections` | string | `all` | `all`, or a comma-separated list among `highlights`, `statements`, `earnings`, `ratings` |

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" \
  "http://localhost:5000/fundamental/deep?ticker=AIR.PA&sections=highlights,ratings"
```

Answer layout:

```json
{
  "asset_profile": { "isin": "NL0000235190", "ticker": "AIR.PA", "name": "Airbus SE", "exchange": "XPAR", "currency": "EUR" },
  "highlights": { "pe_ratio": 28.5, "dividend_yield": 0.0125, "roe": 0.162, "...": "..." },
  "statements": {
    "income":   { "annual": [ { "period_end": "2025-12-31", "...": "..." } ], "quarterly": [] },
    "balance":  { "annual": [], "quarterly": [] },
    "cashflow": { "annual": [], "quarterly": [] }
  },
  "earnings_history": [ { "...": "..." } ],
  "analyst_ratings": { "...": "..." },
  "meta": { "fetched_at": "2026-10-08T16:40:00+00:00", "source": "yfinance", "cache_hit": false, "symbol": "AIR.PA" }
}
```

The figures are fetched from Yahoo Finance with the verified symbol (`meta.symbol`) and stored. Without a verified symbol nothing is fetched: the answer is what the database already holds, `meta.source` is `database` and `meta.note` gives the reason. Complete answers are cached 24 hours; a request receives only the sections it asked for.

---

## Legacy routes

`GET /stocks` (market overview) and `GET /stocks/{ticker}/financials` remain from earlier versions. They ask Yahoo Finance with the ticker as typed; prefer `/fundamental`.



<!-- FILE: /docs/api-reference/historical.md -->

---
id: "historical"
title: "Historical Ingestion API Reference"
sidebar_label: "Historical Prices"
description: "Ingest end-of-day prices into TimescaleDB and read them back"
---

# Historical Ingestion API Reference

Prices are stored per **listing**, resolution (`1D`, `1W`, `1M`) and trading session in the `prices_eod` hypertable. The ingestion routes change data: they need a **full-access** key (a read-only key gets `403`).

---

## <span className="api-method post">POST</span> `/historical/ingest`

Ingest the history of one listing. The parameters are **query parameters**.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `ticker` | string | — | Ticker to ingest (required) |
| `resolution` | string | `1D` | `1D`, `1W` or `1M` |
| `source` | string | `auto` | `auto` (Yahoo Finance, then TradingView), `yfinance` or `tradingview` |
| `force_refresh` | boolean | `false` | Fetch the requested range and the stored range again in one piece, replace the stored bars and look the source symbol up again |
| `from_date`, `to_date` | date | — | Window `YYYY-MM-DD`. Without them: ten years on a first ingestion, otherwise from the day after the last stored session |
| `currency`, `exchange` | string | — | Choose the listing when several share the ticker (the primary one otherwise) |
| `isin` | string | — | Keep the listings of this instrument only, when several instruments share the ticker. A malformed ISIN is refused with `422` |

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" \
  "http://localhost:5000/historical/ingest?ticker=AIR.PA"
```

```json
{
  "ticker": "AIR.PA",
  "resolution": "1D",
  "status": "success",
  "source_used": "yfinance",
  "provider_symbol": "AIR.PA",
  "records_added": 2531,
  "from_date": "2016-10-10",
  "to_date": "2026-10-07",
  "duration_ms": 1840,
  "error": null,
  "note": null
}
```

| Field | Meaning |
|---|---|
| `status` | `success`, `up_to_date` (nothing to fetch) or `failed` |
| `source_used` | `yfinance` or `tradingview`; on a failure, the source that was asked (`auto`…) |
| `provider_symbol` | The symbol asked to the source — the Yahoo symbol verified for the listing, or the TradingView symbol |
| `note` | Why Yahoo was not the source, when the prices come from TradingView; `Whole history fetched again: ...` when the stored series was replaced (split or dividend since the last ingestion, forced refresh, or a series stored before migration 016) |
| `error` | Why nothing could be ingested, e.g. no Yahoo symbol quoted in the currency of the listing |

---

## <span className="api-method post">POST</span> `/historical/ingest/bulk`

Ingest several tickers in parallel. JSON body:

```json
{
  "tickers": ["AIR.PA", "BNP.PA", "MC.PA"],
  "resolution": "1D",
  "source": "auto",
  "force_refresh": false,
  "concurrency": 5
}
```

`concurrency` is between 1 and 20. Each ticker designates its primary listing. The answer is `{"status": "completed", "results": [...]}` with one result per ticker, in the format above.

---

## <span className="api-method get">GET</span> `/ticker/{symbol}/history`

OHLCV bars of a listing, read from the database only — this route never ingests. Bars are returned newest first.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `symbol` | string | — | Ticker |
| `start_date`, `end_date` | date | — | Window `YYYY-MM-DD` |
| `interval` | string | `1D` | `1D`, `1W`, `1M` (or `daily`, `weekly`, `monthly`) |
| `currency`, `exchange` | string | — | Choose the listing |
| `isin` | string | — | Keep the listings of this instrument only (`422` when malformed) |

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" \
  "http://localhost:5000/ticker/AIR.PA/history?start_date=2026-09-28&end_date=2026-10-02"
```

```json
{
  "ticker": "AIR.PA",
  "listing": { "ticker": "AIR.PA", "isin": "NL0000235190", "currency": "EUR", "exchange": "XPAR" },
  "interval": "1D",
  "count": 5,
  "data": [
    { "time": "2026-10-02T00:00:00Z", "open": 149.07, "high": 151.07, "low": 147.07, "close": 150.07, "adj_close": 150.07, "volume": 1395000 },
    { "time": "2026-10-01T00:00:00Z", "open": 147.97, "high": 149.97, "low": 145.97, "close": 148.97, "adj_close": 148.97, "volume": 1394000 }
  ]
}
```

`close` is the traded close (adjusted for splits); `adj_close` is also adjusted for dividends, and is empty for TradingView bars. `listing` is the listing that was read (`null` when the ticker designates none). `time` is the date of the trading session, at midnight UTC. Answers are cached 24 hours and dropped when the ticker is ingested again.

---

## Weekly and monthly bars

Besides the `1W` and `1M` bars you can ingest, the database maintains two continuous aggregates computed from the daily bars of each listing, `prices_weekly` and `prices_monthly`. They are refreshed daily and answer from the daily bars for the recent period.

See [Ingesting historical data](../guides/ingest-historical-data.md) for the pipeline and the choice of the source symbol.



<!-- FILE: /docs/api-reference/realtime.md -->

---
id: "realtime"
title: "Realtime & WebSocket API Reference"
sidebar_label: "Realtime Streaming"
description: "WebSocket price stream, quote snapshots and stream subscriptions"
---

# Realtime & WebSocket API Reference

The realtime worker of the API streams 1-minute ticks from TradingView, stores the last tick of each ticker in Redis (`quote:{ticker}`, 60 s), publishes it on the Redis channel `price:{ticker}` and saves it in the `prices_intraday` hypertable when the instrument is in the catalogue.

Streams are started by a full-access key: by `POST /realtime/subscribe`, or by connecting to the WebSocket. A **read-only key never starts a stream**; it is served what is already streamed. Subscriptions of tickers that name an instrument of the catalogue are stored in `realtime_subscriptions` and restored when the API starts; other tickers are streamed but not restored after a restart.

---

## <span className="api-method ws">WS</span> `/ws/realtime/{ticker}`

```javascript
const ws = new WebSocket(`ws://localhost:5000/ws/realtime/AIR.PA?token=${FONREX_API_KEY}`);
```

The key is checked during the handshake. A WebSocket client can send it as a header (`Authorization` or `X-API-KEY`) or in the query string as `token`, `api_key` or `key`. A missing or wrong key closes the connection with code `1008`.

### Messages sent by the server

Every message has the form `{"type", "ticker", "data", "error", "ts"}`, except `pong`, sent as `{"type": "pong"}`.

| `type` | When | `data` |
|---|---|---|
| `not_streaming` | Read-only key on a ticker that is not streamed (sent first; `error` explains it) | — |
| `snapshot` | Right after the connection, when a last tick is cached | The last tick |
| `tick` | Each new tick | The tick |
| `pong` | Answer to a `ping` from the client | — |

```json
{
  "type": "tick",
  "ticker": "AIR.PA",
  "data": {
    "ticker": "AIR.PA",
    "timestamp": "2026-10-08T09:31:00Z",
    "open": "155.20",
    "high": "155.48",
    "low": "155.10",
    "close": "155.42",
    "volume": 18250,
    "source": "tradingview",
    "exchange": null,
    "currency": null
  },
  "error": null,
  "ts": "2026-10-08T09:31:02.114Z"
}
```

Prices are decimal numbers serialised as strings.

### Messages sent by the client

| Text | Effect |
|---|---|
| `ping` | The server answers `{"type": "pong"}` |
| `unsubscribe` | The server closes the connection |

The TradingView symbol is derived from the ticker suffix (`AIR.PA` → `EURONEXT:AIR`, `.DE` → `XETRA`); a ticker without suffix is taken for a NASDAQ line.

---

## <span className="api-method get">GET</span> `/quote/{ticker}`

The last known price of a ticker.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `subscribe_if_missing` | boolean | `false` | Also start the stream of the ticker in the background. Ignored for a read-only key |

When the ticker is streamed, the cached tick is returned (`is_realtime: true`, `source: "tradingview"`). Otherwise Fonrex returns the delayed Yahoo Finance price of the ticker **as typed** (`is_realtime: false`, `source: "yfinance"`, `delay_seconds: 900`). Nothing found answers `404`. A tick carries no previous close: for a streamed ticker `change` and `change_pct` are `0` and `previous_close` is `null`; the delayed Yahoo answer fills them.

```json
{
  "ticker": "AIR.PA",
  "price": "155.42",
  "open": "155.20",
  "high": "155.48",
  "low": "155.10",
  "close": "155.42",
  "volume": 18250,
  "change": "0",
  "change_pct": "0",
  "previous_close": null,
  "timestamp": "2026-10-08T09:31:00Z",
  "is_realtime": true,
  "source": "tradingview",
  "delay_seconds": 0
}
```

---

## <span className="api-method get">GET</span> `/quotes`

Quotes of several tickers: `tickers` is a comma-separated list, cut to the first 20. A ticker without a quote is `null`. This route never starts a stream.

```json
{ "count": 2, "tickers": ["AIR.PA", "BNP.PA"], "quotes": { "AIR.PA": { "...": "..." }, "BNP.PA": null } }
```

---

## <span className="api-method post">POST</span> `/realtime/subscribe`

Start the stream of up to 50 tickers. Full-access key only.

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" -H "Content-Type: application/json" \
  -d '{"tickers": ["AIR.PA", "BNP.PA"]}' http://localhost:5000/realtime/subscribe
```

```json
[
  {
    "ticker": "AIR.PA",
    "tv_exchange": "EURONEXT",
    "tv_symbol": "AIR",
    "is_active": true,
    "subscribed_at": "2026-10-08T09:30:00Z",
    "last_tick_at": null,
    "tick_count": 0,
    "is_streaming": true
  }
]
```

---

## <span className="api-method delete">DELETE</span> `/realtime/subscribe/{ticker}`

Stop the stream of a ticker. Full-access key only. Answers `{"status": "unsubscribed", "ticker": "AIR.PA"}`, or `404` when the ticker is not streamed.

---

## <span className="api-method get">GET</span> `/realtime/status`

```json
{
  "streaming_count": 1,
  "active_tickers": ["AIR.PA"],
  "ws_connections": { "AIR.PA": 2 },
  "total_ws_clients": 2,
  "stale_tickers": [],
  "worker_running": true
}
```

`stale_tickers` lists streamed tickers without a fresh tick in Redis.



<!-- FILE: /docs/api-reference/technical-indicators.md -->

---
id: "technical-indicators"
title: "Technical Indicators API Reference"
sidebar_label: "Technical Indicators"
description: "18 server-side technical indicators, multi-indicator requests, chart data and a screener"
---

# Technical Indicators API Reference

Fonrex computes 18 indicators with pandas-ta on the prices **stored in the database**. Ingest a listing before asking for its indicators (`GET /eod/{ticker}` or `POST /historical/ingest`). Without prices, `GET /technical/{ticker}` answers `404`; `/multi` and `/chart` answer with no bars and the reason in `errors`.

| Category | Indicators (default parameters) |
|---|---|
| Trend | `sma` (20), `ema` (20), `wma` (20), `dema` (20), `tema` (20), `vwap` (intraday only) |
| Momentum | `rsi` (14), `macd` (12, 26, 9), `stoch` (14, 3, 3), `cci` (20), `roc` (10), `mom` (10) |
| Volatility | `bbands` (20, 2.0), `atr` (14), `kc` (20) |
| Volume | `obv`, `ad`, `mfi` (14) |

`GET /technical/list` returns this catalogue with the parameters, the output columns and the minimum number of bars of each indicator.

Resolutions `1D` (default), `1W` and `1M` read the end-of-day prices of the listing; an intraday resolution such as `1min` reads the 1-minute candles saved by the realtime stream (`prices_intraday`). Results are cached in Redis (1 hour for daily bars, 60 seconds for 1-minute bars) when `TECHNICAL_CACHE_ENABLED=true`.

---

## <span className="api-method get">GET</span> `/technical/{ticker}`

One indicator.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `indicator` | string | `rsi` | Indicator name |
| `period` | integer | — | Sets the `length` parameter (not used by `macd` and `stoch`, which take `fast`/`slow`/`signal` or their defaults) |
| `fast`, `slow`, `signal` | integer | — | MACD parameters |
| `std` | number | — | Bollinger standard deviations |
| `resolution` | string | `1D` | `1D`, `1W`, `1M`, or intraday (`1min`) |
| `from_date`, `to_date` | date | — | Window `YYYY-MM-DD` |
| `limit` | integer | `500` | Bars loaded (`TECHNICAL_DEFAULT_LIMIT`) |

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" \
  "http://localhost:5000/technical/AIR.PA?indicator=rsi&period=14&limit=60"
```

```json
{
  "ticker": "AIR.PA",
  "indicator": "rsi",
  "params": { "length": 14 },
  "resolution": "1D",
  "category": "momentum",
  "from_date": "2026-07-16T00:00:00Z",
  "to_date": "2026-10-07T00:00:00Z",
  "count": 60,
  "series": [
    {
      "name": "RSI_14",
      "label": "RSI",
      "values": [
        { "t": "2026-07-16T00:00:00Z", "v": null },
        { "t": "2026-10-07T00:00:00Z", "v": "74.81" }
      ]
    }
  ],
  "cached": false,
  "calculated_at": "2026-10-08T16:34:58Z"
}
```

An indicator with several outputs (MACD, Bollinger Bands, Stochastic…) has one entry per output in `series`. Values are decimal strings, `null` while the indicator has too few bars.

| Code | When |
|---|---|
| `400` | Unknown indicator, or VWAP asked on daily bars |
| `404` | No prices stored for the ticker |
| `422` | Not enough bars for the parameters |

---

## <span className="api-method get">GET</span> `/technical/{ticker}/multi`

Several indicators computed on one read of the prices.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `indicators` | string | `sma_20,ema_50,rsi_14,macd` | Comma-separated names; a suffix sets the first parameter (`sma_50`, `bbands_20`) |
| `resolution`, `from_date`, `to_date`, `limit` | | | As above |
| `include_ohlcv` | boolean | `false` | Also return the bars |

The answer holds one result per indicator in `indicators` (same format as above) and the failures in `errors`.

---

## <span className="api-method get">GET</span> `/technical/{ticker}/chart`

Bars and indicator columns aligned on the same timestamps, ready for a charting library.

| Parameter | Default |
|---|---|
| `indicators` | `sma_20,ema_50,volume` |
| `limit` | `200` |

```json
{
  "ticker": "AIR.PA",
  "resolution": "1D",
  "timestamps": ["2026-08-27", "2026-08-28"],
  "ohlcv": { "open": [158.14, 157.45], "high": [160.14, 159.45], "low": [156.14, 155.45], "close": [159.14, 158.45], "volume": [1359000, 1360000] },
  "indicators": { "SMA_20": [null, null], "RSI_14": [null, 0.0] }
}
```

---

## <span className="api-method post">POST</span> `/technical/batch`

Several tickers at once. JSON body:

```json
{
  "tickers": ["AIR.PA", "BNP.PA", "MC.PA"],
  "indicators": ["rsi_14", "sma_50"],
  "resolution": "1D",
  "from_date": null,
  "to_date": null,
  "limit": 500,
  "include_ohlcv": false
}
```

At most `TECHNICAL_MAX_BATCH_TICKERS` tickers (20) and `TECHNICAL_MAX_BATCH_INDICATORS` indicators (10). The answer maps each ticker to a multi-indicator result. This route only computes: a read-only key may call it.

---

## <span className="api-method get">GET</span> `/technical/screen`

Instruments of the catalogue whose last value of an indicator meets a condition.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `indicator` | string | `rsi` | Indicator |
| `operator` | string | `lt` | `lt`, `gt`, `lte`, `gte` |
| `value` | number | `30` | Threshold |
| `resolution` | string | `1D` | Resolution |
| `period` | integer | `14` | Indicator length |
| `limit` | integer | `50` | Maximum matches |

```json
{
  "indicator": "rsi",
  "params": { "length": 14 },
  "operator": "gt",
  "value": 50.0,
  "resolution": "1D",
  "matches": [ { "ticker": "AIR.PA", "name": "Airbus SE", "isin": "NL0000235190", "value": "74.81" } ],
  "total": 1,
  "calculated_at": "2026-10-08T16:34:59Z"
}
```

Screener results are cached 15 minutes.



<!-- FILE: /docs/api-reference/valuation-dcf.md -->

---
id: "valuation-dcf"
title: "DCF Valuation API Reference"
sidebar_label: "Valuation & DCF"
description: "Intrinsic value with three DCF models, dynamic WACC, model comparison, sensitivity matrix and macro rates"
---

# DCF Valuation API Reference

Three models are available. When a request computes several of them, the consensus is their weighted average:

| Model | Basis | Default weight |
|---|---|---|
| `fcf` | Average free cash flow of the last three fiscal years, projected, plus a Gordon terminal value, minus net debt | 50 % |
| `eps` | Earnings per share growth (history and analyst trend) and a terminal P/E (the company's P/E bounded to 10–30, 15 when unknown) | 30 % |
| `ddm` | Dividend discount (Gordon) — only for a company that pays a dividend | 20 % |

The weights are shared among the models computed by the request: with `fcf` and `eps` only, the consensus weighs them 50/30, normalised. `GET /dcf/{ticker}` computes `fcf` alone, so its consensus is the FCF value.

**Inputs come from the database only.** The service reads the highlights, the annual statements (grouped by fiscal year, five years), the earnings trend and the analyst ratings stored by the deep enrichment, and the last daily close of the main listing. Run `GET /fundamental/deep?ticker=...` first: a ticker that was never enriched answers `404` with the name of the missing data.

**WACC.** Cost of equity by CAPM (risk-free rate + beta × equity risk premium), cost of debt from interest and debt, weights from market capitalisation and debt; the WACC is kept within 5–20 %. The risk-free rate is the US 10-year Treasury yield from FRED (see `GET /macro/rates`), otherwise `DCF_RISK_FREE_RATE`. Missing inputs use documented defaults (beta 1, tax rate 25 %, cost of debt = risk-free rate + 2 points) and add a warning.

---

## <span className="api-method get">GET</span> `/dcf/{ticker}`

The **FCF model alone**, with the default assumptions: `consensus_value` is the FCF value. Cached 6 hours; `force_refresh=true` computes again. Use `/compare` for the three models, or `POST` to choose them.

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/dcf/AIR.PA"
```

Answer layout:

```json
{
  "ticker": "AIR.PA",
  "currency": "EUR",
  "current_price": "155.42",
  "shares_outstanding": "790000000",
  "wacc": {
    "wacc": "0.0865",
    "cost_of_equity": "0.0912",
    "cost_of_debt": "0.032",
    "tax_rate": "0.25",
    "weight_equity": "0.93",
    "weight_debt": "0.07",
    "beta_used": "1.1",
    "cost_of_debt_source": "calculated",
    "risk_free_rate_source": "fred_cached"
  },
  "models": {
    "fcf": {
      "model_name": "...",
      "intrinsic_value_per_share": "168.20",
      "upside_pct": "8.22",
      "projected_values": ["..."],
      "terminal_value": "...",
      "present_values": ["..."],
      "pv_terminal": "...",
      "warnings": []
    }
  },
  "solvency": { "debt_to_equity_ratio": "...", "net_debt_to_ebitda": "...", "interest_coverage_ratio": "...", "...": "..." },
  "consensus_value": "168.20",
  "consensus_upside_pct": "8.22",
  "analyst_target": "175.00",
  "computed_at": "2026-10-08T16:50:00Z"
}
```

The figures above are illustrative. Amounts and rates are decimal numbers serialised as **strings**. `models` is keyed by model (`fcf`, `eps`, `ddm`); `warnings` of each model says when a default or a cap was applied. `risk_free_rate_source` is `fred_cached` (FRED rate), `env_fallback` (`DCF_RISK_FREE_RATE`) or `client_override` (your `POST` assumptions).

---

## <span className="api-method post">POST</span> `/dcf/{ticker}`

Valuation with your own assumptions; never cached. It only computes, so a read-only key may call it.

| Field | Type | Default | Description |
|---|---|---|---|
| `models` | list | `["fcf"]` | Among `fcf`, `eps`, `ddm` |
| `projection_years` | integer | `5` | 3 to 10 |
| `terminal_growth_rate` | number | `0.025` | Ratio (0.025 = 2.5 %) |
| `wacc_params` | object | — | `risk_free_rate`, `equity_risk_premium`, `beta_override`, `cost_of_debt_override`, `tax_rate_override` |
| `fcf_growth_override`, `eps_growth_override`, `dividend_growth_override` | number | — | Force the initial growth of a model |
| `model_weights` | object | — | Consensus weights, e.g. `{"fcf": 0.6, "eps": 0.4, "ddm": 0}` |

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" -H "Content-Type: application/json" \
  -d '{"models": ["fcf", "eps"], "projection_years": 10, "terminal_growth_rate": 0.02,
       "wacc_params": {"risk_free_rate": 0.035}}' \
  http://localhost:5000/dcf/AIR.PA
```

A terminal growth rate within half a point of the discount rate is capped (discount rate − 0.5 %) with a warning. Asking for `ddm` for a company that pays no dividend answers `404`.

---

## <span className="api-method get">GET</span> `/dcf/{ticker}/compare`

The three models side by side, with their weighted consensus. A company without a dividend gets a DDM entry valued at zero with a warning, and the consensus is computed from FCF and EPS.

---

## <span className="api-method get">GET</span> `/dcf/{ticker}/sensitivity`

Intrinsic value for a grid of WACC (rows) × terminal growth (columns).

| Parameter | Default |
|---|---|
| `model` | `fcf` |
| `wacc_min`, `wacc_max`, `wacc_step` | `0.06`, `0.16`, `0.02` |
| `growth_min`, `growth_max`, `growth_step` | `0.01`, `0.05`, `0.01` |
| `force_refresh` | `false` |

The answer holds `ticker`, `model`, `wacc_range`, `growth_range` and `matrix`; each cell gives the intrinsic value and the upside or downside against the current price.

---

## <span className="api-method get">GET</span> `/macro/rates`

The risk-free rate used by the valuation:

```json
{
  "risk_free_rate": {
    "series_id": "DGS10",
    "label": "10-Year Treasury Constant Maturity Rate",
    "value": "0.0412",
    "unit": "percent",
    "observation_date": "2026-10-07"
  }
}
```

`value` is a ratio (0.0412 = 4.12 %), serialised as a string, although `unit` says `percent`. It comes from Redis (6 hours), then from the value stored in `macro_rates_cache` when it was read from FRED less than 6 hours ago, then from the FRED API (`FRED_API_KEY`), and finally from an older stored value. Without any value, `risk_free_rate` is `null` and the valuation uses `DCF_RISK_FREE_RATE`.



<!-- FILE: /docs/api-reference/news.md -->

---
id: "news"
title: "News Aggregator API Reference"
sidebar_label: "News"
description: "News of a ticker from 7 providers, deduplicated, and the global feed of stored articles"
---

# News Aggregator API Reference

`GET /news/{ticker}` asks seven providers in parallel — Yahoo Finance, Google Finance, ZoneBourse, Boursorama, Investing.com, MarketWatch and MSN Finance — removes duplicates and stores the articles of instruments that are in the catalogue in `news_articles`. See [News providers](../providers/news-providers.md).

---

## <span className="api-method get">GET</span> `/news/{ticker}`

| Parameter | Type | Default | Description |
|---|---|---|---|
| `ticker` | string | — | Ticker (e.g. `AIR.PA`) |
| `limit` | integer | `20` | Articles returned (at most `NEWS_MAX_LIMIT`, 100) |
| `language` | string | — | Keep only this language (`en`, `fr`…); articles of unknown language are kept. `all` means no filter |
| `force_refresh` | boolean | `false` | Ignore the cached answer |

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/news/AIR.PA?limit=5&language=fr"
```

```json
{
  "ticker": "AIR.PA",
  "isin": "NL0000235190",
  "count": 1,
  "providers": ["boursorama"],
  "cached": false,
  "articles": [
    {
      "title": "Airbus : livraisons en hausse en septembre",
      "summary": "...",
      "url": "https://www.boursorama.com/bourse/actualites/...",
      "image_url": null,
      "source": "Boursorama",
      "provider": "boursorama",
      "author": null,
      "published_at": "2026-10-08T07:45:00Z",
      "related_tickers": [],
      "language": "fr"
    }
  ]
}
```

Each answer is cached 30 minutes (`NEWS_CACHE_TTL`), separately for each `limit` and language. A provider that fails returns nothing without blocking the others.

### Deduplication

1. **URL**: lower case, `utm_*` parameters, fragment and trailing slash removed. The first article received is kept.
2. **Title similarity**: `difflib.SequenceMatcher` on normalised titles; at or above `NEWS_DEDUP_SIMILARITY` (0.85) the most recent article is kept.

Articles are then sorted newest first and cut to `limit` (each provider is asked for twice that number).

---

## <span className="api-method get">GET</span> `/news/feed`

The latest articles stored in `news_articles`, all instruments together. Nothing is fetched from the providers.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `limit` | integer | `50` | Articles returned |
| `language` | string | — | Language filter (`all` = none) |
| `tickers` | string | — | Comma-separated tickers; keeps the articles related to one of them |

```json
{ "count": 0, "from_date": null, "to_date": null, "articles": [] }
```

---

## <span className="api-method post">POST</span> `/news/{ticker}/refresh`

Fetch the news of a ticker again in the background and answer at once: `{"status": "queued", "ticker": "AIR.PA"}`. Full-access key only.

---

## <span className="api-method get">GET</span> `/news/stats`

```json
{
  "total_articles": 0,
  "by_provider": {},
  "by_language": {},
  "last_fetched_at": null,
  "top_assets": {}
}
```



<!-- FILE: /docs/api-reference/monitoring.md -->

---
id: "monitoring"
title: "Provider Monitoring & Administration API Reference"
sidebar_label: "Monitoring & Admin"
description: "Provider health, canary runs, alerts, validation statistics, cache and database administration"
---

# Provider Monitoring & Administration API Reference

The monitoring routes report what the [validation layer](../monitoring/validation-layer.md) and the [daily canary](../monitoring/canary-monitor.md) observed about each fundamentals provider. The administration routes manage the cache and the database of your instance.

Routes that change something (`POST` here) need a **full-access** key.

---

## <span className="api-method get">GET</span> `/health`

Public (no key). Service status, providers loaded at start-up, cache status and lifetimes.

```json
{
  "status": "healthy",
  "service": "FonRex API",
  "timestamp": "2026-10-08T16:34:42.235390",
  "yfinance_available": true,
  "providers": { "loaded": 14, "unavailable": [] },
  "cache": { "enabled": true, "status": "connected", "ttl_seconds": { "eod": 86400, "...": "..." } }
}
```

`providers.unavailable` names a provider that could not be imported (missing dependency): it is skipped by every request until fixed.

---

## <span className="api-method get">GET</span> `/health/providers`

Health summary of the providers, read from Redis (written by the canary, 1 hour) or from the database.

```json
{
  "checked_at": "2026-10-08T06:02:10Z",
  "total_providers": 14,
  "healthy": 13,
  "degraded": 1,
  "down": 0,
  "providers": [
    {
      "name": "ZoneBourse",
      "is_healthy": true,
      "success_rate_7d": 0.98,
      "avg_latency_ms": null,
      "last_check": null,
      "canary_passed": null,
      "active_alerts": 0,
      "status_label": "OK"
    }
  ]
}
```

`status_label` is `OK` (success rate ≥ 85 %), `DEGRADED` (≥ 70 %) or `DOWN`. When the summary comes from Redis, `avg_latency_ms`, `last_check` and `canary_passed` are `null` and `active_alerts` is 0. On a new instance, before the first canary run, the list is empty.

## <span className="api-method get">GET</span> `/health/providers/{provider_name}`

Details of one provider over `days` days (default 7): `status`, `success_rate_7d`, `success_rate_30d`, `avg_latency_ms`, `daily_stats`, `recent_failures` and `active_alerts`.

---

## <span className="api-method get">GET</span> `/health/alerts`

| Parameter | Type | Default | Description |
|---|---|---|---|
| `severity` | string | — | `warning` or `critical` |
| `provider_name` | string | — | One provider |
| `include_resolved` | boolean | `false` | Include resolved alerts |
| `limit` | integer | `50` | Maximum alerts |

Alert types raised by the canary: `canary_failed` (a value outside its expected range) and `high_outlier_rate` (success rate under `ALERT_SUCCESS_RATE_WARNING` / `ALERT_SUCCESS_RATE_CRITICAL`). See [Alerts](../monitoring/alerts.md).

## <span className="api-method post">POST</span> `/health/alerts/{alert_id}/resolve`

Resolve an alert by hand. The note is a **query parameter**:

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" \
  "http://localhost:5000/health/alerts/42/resolve?resolution_note=Parser%20fixed"
```

```json
{ "status": "resolved", "alert_id": 42, "resolved_at": "2026-10-08T10:12:00+00:00" }
```

An alert already resolved answers `{"status": "already_resolved"}`, an unknown one `404`.

---

## <span className="api-method post">POST</span> `/health/canary/run`

Start a canary run in the background, for every provider or for `provider_name` only. Answers `{"status": "queued", "provider": "all", "message": "..."}`.

## <span className="api-method get">GET</span> `/health/canary/history`

Past canary checks. Parameters: `provider_name`, `ticker`, `days` (7), `limit` (100).

## <span className="api-method get">GET</span> `/health/stats`

Validation statistics of the last 7 days:

```json
{
  "period": "last_7_days",
  "total_values_validated": 0,
  "total_valid": 0,
  "total_outliers": 0,
  "total_out_of_range": 0,
  "total_nulls": 0,
  "overall_quality_score": null,
  "most_reliable_providers": [],
  "least_reliable_providers": [],
  "fields_most_often_invalid": []
}
```

---

## Cache administration

| Method | Route | Description |
|---|---|---|
| `GET` | `/cache/stats` | Redis version and memory, lifetime of each cache category, cached tickers |
| `POST` | `/cache/clear` | Delete the cached end-of-day answers (`eod:*` keys only) |
| `POST` | `/cache/clear/{ticker}` | Delete the cached end-of-day answers of a ticker (`eod:{TICKER}:*`). The `period` parameter currently deletes nothing: the keys hold more segments than the pattern it builds |

The other categories (fundamentals, indicators, news, DCF…) expire on their own; an ingestion drops the price-derived answers of its ticker.

## Database administration

| Method | Route | Description |
|---|---|---|
| `GET` | `/database/stats` | Stored prices and tickers, API requests of the last 24 hours |
| `GET` | `/database/tickers` | Every ticker with prices: first and last date, number of bars |
| `GET` | `/database/ticker/{ticker}` | The same for one ticker, with its number of API requests |
| `POST` | `/database/cleanup` | Delete prices older than `days_to_keep` days and logs older than 30 days |

`POST /database/cleanup` takes a JSON body `{"days_to_keep": 730, "dry_run": false}`. `days_to_keep` must be 30 or more (730 by default). A first ingestion fetches ten years: count before deleting with `dry_run`, which deletes nothing and returns what a real run would delete.

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" -H "Content-Type: application/json" \
  -d '{"days_to_keep": 3650, "dry_run": true}' http://localhost:5000/database/cleanup
```

The usage log (`usage_logs`) has its own retention, `USAGE_LOG_RETENTION_DAYS`.



<!-- FILE: /docs/api-reference/specialized.md -->

---
id: "specialized"
title: "Specialized Providers API Reference"
sidebar_label: "Specialized"
description: "SEC insider transactions, UCITS ETF details from JustETF and index constituents"
---

# Specialized Providers API Reference

Three routes query one specialised source each. Every route accepts `refresh=true` to bypass the cache.

---

## <span className="api-method get">GET</span> `/insider-transactions/{ticker}`

Form 4 insider transactions filed with the US SEC (EDGAR). Cached 12 hours.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `ticker` | string | — | A share that files with the SEC (e.g. `AAPL`) |
| `limit` | integer | `20` | Transactions returned |
| `refresh` | boolean | `false` | Ask EDGAR again |

```json
{
  "ticker": "AAPL",
  "cik": "0000320193",
  "company_name": "Apple Inc.",
  "transactions": [
    {
      "filing_date": "2026-10-02",
      "insider_name": "COOK TIMOTHY D",
      "insider_title": "Chief Executive Officer",
      "transaction_date": "2026-10-01",
      "transaction_type": "Sell",
      "transaction_code": "S",
      "shares": 50000,
      "price_per_share": 226.1,
      "total_value": 11305000.0,
      "shares_owned_after": 3280000,
      "sec_filing_url": "https://www.sec.gov/Archives/edgar/data/..."
    }
  ],
  "total_count": 1,
  "source": "SEC EDGAR"
}
```

`transaction_type` is the reading of the Form 4 code (`P` Buy, `S` Sell, `A` Award, `M` Option Exercise, `G` Gift, `F` Tax Withholding…). Set `SEC_EDGAR_EMAIL` to your own address: the SEC requires a contact in the User-Agent of automated clients. Values are illustrative.

---

## <span className="api-method get">GET</span> `/etf/{isin}/details`

UCITS ETF data from JustETF. Cached 24 hours. An ISIN known as something other than an ETF is refused.

```json
{
  "isin": "IE00B4L5Y983",
  "name": "iShares Core MSCI World UCITS ETF USD (Acc)",
  "ticker": "EUNL",
  "net_expense_ratio": "0.0020",
  "total_net_assets": "...",
  "domicile": "Ireland",
  "replication_method": "...",
  "distribution_policy": "Accumulating",
  "index_tracked": "MSCI World",
  "inception_date": "2009-09-25",
  "nb_holdings": 1400,
  "performance": { "...": "..." },
  "top_holdings": [ { "...": "..." } ],
  "allocation": { "...": "..." },
  "provider_url": "https://www.justetf.com/..."
}
```

The answer is not written to the `etf_details` and `etf_holdings` tables.

---

## <span className="api-method get">GET</span> `/index/{index_name}/constituents`

Members of an index, read from Wikipedia. Cached 7 days.

`index_name` is `SP500`, `CAC40`, `NASDAQ100` or `DAX` (case-insensitive). Another name answers `400`.

```json
{
  "index_name": "CAC40",
  "constituents": [
    { "ticker": "AIR.PA", "isin": "NL0000235190", "name": "Airbus", "sector": "Industrie", "sub_sector": null, "weight": null, "country": null, "cik": null }
  ],
  "total_count": 40,
  "source_url": "https://fr.wikipedia.org/wiki/CAC_40",
  "source": "Wikipedia"
}
```



<!-- FILE: /docs/api-reference/openbb.md -->

---
id: "openbb"
title: "OpenBB Workspace Integration API"
sidebar_label: "OpenBB Integration"
description: "The /openbb routes and discovery files that feed OpenBB Workspace widgets"
---

# OpenBB Workspace Integration API

The `/openbb` routes serve the widgets of [OpenBB Workspace](https://openbb.co) from your own instance. Each route calls the Fonrex route it adapts and reshapes the answer into one of the three formats OpenBB expects:

- **metric**: a list of tiles `[{"label": "...", "value": ..., "delta": ...}]`
- **chart**: a Plotly figure `{"data": [...], "layout": {...}}`
- **table**: a flat list of rows `[{...}, {...}]` for AgGrid

Setup is described in the [OpenBB Workspace guide](../guides/openbb-workspace.md).

## Authentication

| Route | Key |
|---|---|
| `GET /widgets.json`, `GET /apps.json` | None — OpenBB fetches them before a key is configured |
| `GET /openbb/...` | Required, in the `X-API-KEY` header (or `Authorization: Bearer`) |

A read-only key (`FONREX_READ_ONLY_API_KEYS`) is enough for every widget. CORS accepts the origins of `OPENBB_ALLOWED_ORIGIN` (`https://pro.openbb.co` by default).

## Discovery files

- `GET /widgets.json` — the 19 widgets: for each one, its name, category, type, route and parameters (`integrations/openbb/widgets.json`).
- `GET /apps.json` — two pre-assembled dashboards, **Fonrex — EU Markets** and **Fonrex — Screener & Macro** (`integrations/openbb/apps.json`).

## Routes

| Widget | Type | Route | Adapts |
|---|---|---|---|
| `fonrex_quote` | metric | `GET /openbb/quote/{ticker}` | `/quote/{ticker}` |
| `fonrex_macro_rates` | metric | `GET /openbb/macro/rates` | `/macro/rates` |
| `fonrex_eod` | chart | `GET /openbb/eod/{ticker}` (`period` 1y by default) | `/eod/{ticker}` |
| `fonrex_history` | chart | `GET /openbb/ticker/{symbol}/history` | `/ticker/{symbol}/history` |
| `fonrex_technical` | chart | `GET /openbb/technical/{ticker}` | `/technical/{ticker}` |
| `fonrex_technical_multi` | chart | `GET /openbb/technical/{ticker}/multi` | `/technical/{ticker}/multi` |
| `fonrex_technical_chart` | chart | `GET /openbb/technical/{ticker}/chart` | `/technical/{ticker}/chart` |
| `fonrex_fundamentals` | table | `GET /openbb/fundamental` | `/fundamental` |
| `fonrex_fundamentals_deep` | table | `GET /openbb/fundamental/deep` | `/fundamental/deep` |
| `fonrex_quotes_batch` | table | `GET /openbb/quotes` | `/quotes` |
| `fonrex_screener` | table | `GET /openbb/technical/screen` | `/technical/screen` |
| `fonrex_news` | table | `GET /openbb/news/{ticker}` | `/news/{ticker}` |
| `fonrex_news_feed` | table | `GET /openbb/news/feed` | `/news/feed` |
| `fonrex_dcf` | table | `GET /openbb/dcf/{ticker}` | `/dcf/{ticker}` |
| `fonrex_dcf_compare` | table | `GET /openbb/dcf/{ticker}/compare` | `/dcf/{ticker}/compare` |
| `fonrex_dcf_sensitivity` | table | `GET /openbb/dcf/{ticker}/sensitivity` | `/dcf/{ticker}/sensitivity` |
| `fonrex_insider_transactions` | table | `GET /openbb/insider-transactions/{ticker}` | `/insider-transactions/{ticker}` |
| `fonrex_etf_details` | table | `GET /openbb/etf/{isin}/details` | `/etf/{isin}/details` |
| `fonrex_index_constituents` | table | `GET /openbb/index/{index_name}/constituents` | `/index/{index_name}/constituents` |

Each route takes the parameters of the route it adapts (see the corresponding API reference page), with a few differences: `/openbb/fundamental` has no `fmt`; `/openbb/technical/{ticker}/multi` has no `include_ohlcv` and defaults to `sma_20,ema_50,rsi_14`; `/openbb/technical/{ticker}/chart` defaults to `sma_20,rsi_14`; `/openbb/news/feed` returns 20 articles by default.

`GET /openbb/quote/{ticker}` never starts a realtime stream: the quote is real time once the ticker is subscribed with `POST /realtime/subscribe`, and the delayed Yahoo Finance price otherwise.

## Example

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/openbb/quote/AIR.PA"
```

```json
[
  { "label": "AIR.PA Price", "value": 154.6, "delta": 0.13 },
  { "label": "Change", "value": "+0.20", "delta": "+0.13%" },
  { "label": "Previous Close", "value": 154.4, "delta": null },
  { "label": "Day High", "value": 155.48, "delta": null },
  { "label": "Day Low", "value": 155.1, "delta": null },
  { "label": "Volume", "value": 18250, "delta": null }
]
```

This example is a delayed Yahoo Finance quote. Tiles without a value (no previous close, no volume…) are left out.



<!-- FILE: /docs/guides/import-assets.md -->

---
id: "import-assets"
title: "Asset CSV Import Pipeline Guide"
sidebar_label: "Import Assets"
description: "Import instruments and listings from CSV files with import_assets.py"
---

# Asset CSV Import Pipeline Guide

`import_assets.py` loads instruments and their listings from CSV files into the catalogue. It is idempotent: importing a file twice updates the rows instead of duplicating them.

## CSV format

```csv
name,ticker,isin,productType,currency
Airbus SE,AIR.PA,NL0000235190,STOCK,EUR
Apple Inc,AAPL,US0378331005,STOCK,USD
Apple Inc,APC.DE,US0378331005,STOCK,EUR
iShares Core MSCI World UCITS ETF,EUNL.DE,IE00B4L5Y983,ETF,EUR
```

| Column | Rule |
|---|---|
| `name` | Not empty |
| `ticker` | At most 20 characters |
| `isin` | 12 characters, `[A-Z]{2}[A-Z0-9]{10}` |
| `productType` | `STOCK` or `ETF` |
| `currency` | 3 upper-case letters |

Rows are deduplicated on `(isin, ticker, currency)`. The exchange of a listing is deduced from the ticker suffix (`AIR.PA` → `XPAR`, `BMW.DE` → `XETR`); a ticker without suffix gets no exchange.

The image ships two catalogues: `data/etf.csv` and `data/stocks.csv`.

## What the import writes

```
CSV file
  │ parse_csv(): validation, deduplication
  ▼
AssetImporter.run()  — batches of 200 rows
  ├─ one row of assets per ISIN
  ├─ one row of asset_listings per (instrument, ticker, exchange, currency)
  └─ default mappings: YahooFinance and GoogleFinance
```

- The first listing of an instrument is its **primary** listing; when none is primary yet, a listing in USD, GBP, JPY, CHF, CAD or AUD becomes primary.
- The import makes **no network call**. The Yahoo Finance mapping it writes is the ticker of the file, not verified: the first ingestion or fundamentals request of the listing replaces it with the Yahoo symbol verified from the ISIN and the currency.

## Commands

```bash
# One file
docker compose exec fonrex-api python import_assets.py --file data/etf.csv

# Simulation, nothing written
docker compose exec fonrex-api python import_assets.py --file data/etf.csv --dry-run

# Every CSV file of a directory
docker compose exec fonrex-api python import_assets.py --dir data/isin_data

# Without --file or --dir: data/etf.csv and data/stocks.csv
docker compose exec fonrex-api python import_assets.py
```

Options: `--batch-size` (rows per transaction, 200), `--verbose`.

A simple file name is looked up in `data/isin_data/`, then in the application folder; a relative or absolute path is used as it is.

## Enrichment from Yahoo Finance

The deep enrichment (highlights, financial statements, earnings, analyst ratings) is a separate step:

```bash
# One instrument
docker compose exec fonrex-api python import_assets.py --enrich-only --isin US0378331005

# The instruments of a file
docker compose exec fonrex-api python import_assets.py --enrich-only --file data/etf.csv --limit 100
```

`--enrich-only` needs `--isin` or `--file`. Yahoo is asked with the symbol verified for the primary listing; an instrument without a verified symbol is skipped and the log says why. `GET /fundamental/deep?ticker=...&refresh=true` does the same for one instrument through the API.

`make db-seed` imports the default catalogue and enriches it (`scripts/seed_database.py --enrich`).

## Removing ISIN duplicates

Databases created before the ISIN uniqueness rule may hold the same ISIN twice. `scripts/clean_isin_duplicates.py` merges them onto the oldest row:

```bash
docker compose exec fonrex-api python scripts/clean_isin_duplicates.py --diagnose-only   # read only
docker compose exec fonrex-api python scripts/clean_isin_duplicates.py --dry-run         # run, then roll back
docker compose exec fonrex-api python scripts/clean_isin_duplicates.py --create-index    # clean + unique index
```

## Next step

Ingest the prices of the imported listings — see [Ingesting historical data](ingest-historical-data.md).



<!-- FILE: /docs/guides/ingest-historical-data.md -->

---
id: "ingest-historical-data"
title: "Ingesting Historical Market Data"
sidebar_label: "Ingest Historical Data"
description: "How Fonrex fetches, verifies and stores end-of-day prices per listing"
---

# Ingesting Historical Market Data

Prices are stored per **listing** (ticker + exchange + currency), resolution and trading session. A listing is ingested:

- automatically, the first time `GET /eod/{ticker}` finds nothing stored for the request;
- on demand, with `POST /historical/ingest` or `POST /historical/ingest/bulk`;
- for the whole catalogue, with `scripts/ingest_all.py`.

## The pipeline

`HistoricalIngestionService` runs these steps:

1. **Series** — the ticker designates a listing: the listing bearing that ticker (the primary one first; `currency` or `exchange` choose another), otherwise the preferred listing of the instrument. An instrument without a listing cannot be ingested.
2. **Gap detection** — with no stored bar, ten years are fetched; when the last stored session is today or yesterday, the listing is `up_to_date`; otherwise only the missing days are fetched. A `from_date` older than the first stored bar fetches the older part too. With `force_refresh`, the requested range and the stored range are fetched again in one piece.
3. **Source symbol** — the ticker of your catalogue is not always a Yahoo symbol (`EUCO` is `SYBC.DE` on Yahoo, and `SPFF` alone is a US fund). Fonrex searches Yahoo by the **ISIN**, keeps the first line quoted in the **currency of the listing** with a price, and stores it as the listing's verified symbol. A listing for which nothing matches is not fetched from Yahoo, and is not searched again for 24 hours.
4. **Fetch** — Yahoo Finance with the verified symbol; TradingView as a fallback, accepted only when the line is quoted in the currency of the listing. `open`, `high`, `low` and `close` are the traded prices, adjusted for splits; `adj_close` is the close adjusted for splits **and dividends** (empty for TradingView bars). Each bar is dated by its trading session. The last stored bars are fetched again with the new ones, to check the adjustment (next section).
5. **Normalisation** — bars without prices dropped, inverted high/low fixed, negative volume set to zero, duplicate dates dropped.
6. **Upsert** — batches of 1,000 rows, `ON CONFLICT (asset_listing_id, resolution, time) DO UPDATE`. With `force_refresh`, or when the whole series was fetched again, the stored bars of the fetched range are replaced.
7. **Cache** — the cached answers computed from the ticker's prices (`eod`, `history`, `technical`, `dcf`) are dropped.
8. **Log** — one row in `ingest_log`: status, source, rows added, range, duration, error.

## Splits and dividends: one adjustment for the whole series

Yahoo adjusts a whole history again after each split (all prices) and each dividend (`adj_close`). If the new bars were simply added to the stored ones, the two parts would be adjusted differently and a false return would appear where they meet: about minus the dividend yield after a dividend, -75 % after a four-for-one split.

So Fonrex keeps each series (listing and resolution) on one adjustment:

- To complete a series, it fetches the new sessions **and the last five stored bars**. If the source gives the same prices for those bars, only the new sessions are written.
- If the prices differ — a split or a dividend since the last ingestion — the **whole series is fetched again** and replaces the stored one. The result says so in `note`: `Whole history fetched again: the source adjusted the stored bars again (split or dividend)`. If that fetch fails, nothing is written and the ingestion fails with the reason.
- The table `price_series_adjustments` records, per series, how its bars are adjusted and when it was last fetched in one piece.

Use `close` for prices as traded (charts, indicators, valuation), `adj_close` for returns that include dividends (performance, beta, backtests).

:::note After upgrading to migration 016
Series stored before migration 016 hold the dividend-adjusted price in `close`. Each one is fetched again in full at its next ingestion. To do it at once for the whole catalogue:

```bash
docker compose exec fonrex-api python scripts/ingest_all.py --force
```
:::

## When a ticker gets no price, or the wrong one

The ingestion result says why:

```json
{
  "ticker": "GOVY",
  "status": "failed",
  "error": "No Yahoo symbol quoted in CHF for ISIN IE00B3S5XW04; Yahoo offers SYBB.DE (EUR)"
}
```

- **Several listings share the ticker**: name the one you want — `POST /historical/ingest?ticker=GOVY&currency=CHF`.
- **Which symbol was used**: `provider_symbol` in the result; `note` says why TradingView was used instead of Yahoo.
- **Look the symbol up again, or replace an old series**: `POST /historical/ingest?ticker=<ticker>&force_refresh=true`.
- **Set the symbol yourself** when you know the right line (it is then trusted as it is):

```bash
docker compose exec -T db psql -U fonrex -d fonrex -c "
  INSERT INTO asset_mappings (asset_id, asset_listing_id, provider_name, provider_ticker,
                              source, is_active, failure_count, created_at, updated_at)
  SELECT l.asset_id, l.id, 'YahooFinance', 'GOVY.SW', 'manual', true, 0, now(), now()
  FROM asset_listings l WHERE l.ticker = 'GOVY' AND l.currency = 'CHF'
  ON CONFLICT (asset_listing_id, provider_name)
  DO UPDATE SET provider_ticker = EXCLUDED.provider_ticker, source = 'manual', is_active = true"
```

## Ingesting the whole catalogue

```bash
docker compose exec fonrex-api python scripts/ingest_all.py
```

| Option | Default | Description |
|---|---|---|
| `--resolution` | `1D` | `1D`, `1W` or `1M` |
| `--source` | `auto` | `auto`, `yfinance` or `tradingview` |
| `--force` | off | Fetch every history again (no gap detection) |
| `--concurrency` | `5` | Parallel ingestions |

A short random pause precedes each ticker so as not to hammer the sources.

## Keeping or deleting old prices

A first ingestion fetches ten years. `POST /database/cleanup` deletes prices older than `days_to_keep` days — **730 by default**, which would remove eight of those ten years. Count first with `dry_run`:

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" -H "Content-Type: application/json" \
  -d '{"days_to_keep": 3650, "dry_run": true}' http://localhost:5000/database/cleanup
```

## Upgrading from shared series (migration 014)

Before migration 014, the listings of one instrument shared one series and European or Asian sessions were dated the day before. The migration rebuilds `prices_eod` per listing and re-dates the existing rows; nothing has to be downloaded again. If a series looks wrong afterwards, replace it with `force_refresh=true`. Back up the database before upgrading.



<!-- FILE: /docs/guides/configure-realtime.md -->

---
id: "configure-realtime"
title: "Realtime Streaming & WebSocket Setup"
sidebar_label: "Configure Realtime"
description: "Start TradingView streams, receive ticks over WebSocket and tune the realtime worker"
---

# Realtime Streaming & WebSocket Setup

The realtime worker runs inside the API process. It keeps TradingView WebSocket streams open for the subscribed tickers and spreads each 1-minute tick through Redis.

```
TradingView WebSocket
        │
        ▼
RealtimePriceWorker (API process)
        ├─► Redis key quote:{ticker}       last tick, 60 s      ──► GET /quote/{ticker}
        ├─► Redis channel price:{ticker}   each tick            ──► WS /ws/realtime/{ticker}
        └─► prices_intraday (TimescaleDB)  1-minute candles, kept 30 days
```

## 1. Subscribe tickers

Streams are started with a **full-access** key:

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" -H "Content-Type: application/json" \
  -d '{"tickers": ["AIR.PA", "BNP.PA"]}' http://localhost:5000/realtime/subscribe
```

Connecting to `WS /ws/realtime/{ticker}` with a full-access key also starts the stream of a ticker that is not streamed. Subscriptions are stored in `realtime_subscriptions` and restored when the API restarts.

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" http://localhost:5000/realtime/status
curl -s -X DELETE -H "X-API-KEY: $FONREX_API_KEY" http://localhost:5000/realtime/subscribe/BNP.PA
```

## 2. Receive the ticks

```javascript
const ws = new WebSocket(`ws://localhost:5000/ws/realtime/AIR.PA?token=${FONREX_API_KEY}`);
ws.onmessage = (event) => {
  const { type, data } = JSON.parse(event.data);
  if (type === "snapshot" || type === "tick") console.log(type, data.close);
};
```

The repository also has a Python example client: `make example-client` (`scripts/example_realtime_client.py`).

**Read-only keys** (for a dashboard or any client outside your machine) never start a stream. They receive a `not_streaming` message for a ticker that is not streamed, then its ticks as soon as a full-access client subscribes it. `GET /quote/{ticker}` and `GET /openbb/quote/{ticker}` never start a stream either: without one they return the delayed Yahoo Finance price.

## 3. Settings

```env
# Simultaneous TradingView connections
TV_MAX_CONNECTIONS=10
# First reconnection delay in seconds, doubled after each failure up to 60
TV_RECONNECT_DELAY=5
# Lifetime of the last tick in Redis, in seconds
REALTIME_QUOTE_TTL=60
```

## Things to know

- **One process.** Streams, subscriptions and WebSocket clients live in the memory of the API process. Keep `WEB_CONCURRENCY=1`: each additional Gunicorn worker would open its own streams.
- **TradingView symbol.** It is derived from the ticker suffix (`AIR.PA` → `EURONEXT:AIR`, `.DE` → `XETRA`); a ticker without suffix is taken for a NASDAQ line. Unlike the end-of-day ingestion, the realtime path does not check the line against the ISIN and currency of a listing.
- **Intraday storage.** 1-minute candles are saved per instrument (not per listing) when the ticker is in the catalogue, and purged after 30 days by a TimescaleDB retention policy.
- **Reverse proxy.** A proxy in front of the API must forward the WebSocket upgrade — see [Docker production deployment](../deployment/docker.md).



<!-- FILE: /docs/guides/backtesting-zipline.md -->

---
id: "backtesting-zipline"
title: "Backtesting with Zipline"
sidebar_label: "Backtesting (Zipline)"
description: "Run zipline-reloaded backtests on the prices of your Fonrex database, or load them into pandas"
---

# Backtesting with Zipline

Fonrex ships a [`zipline-reloaded`](https://github.com/stefan-jansen/zipline-reloaded) data bundle, `zipline_bundle/`, that reads the daily prices of your database directly — no CSV export, no parallel dataset. The API never imports Zipline: install it only where you run the backtests.

## Prerequisites

- Prices ingested in your instance (`POST /historical/ingest`, `scripts/ingest_all.py`).
- The Fonrex repository and `zipline-reloaded` in the same Python environment:

```bash
pip install zipline-reloaded
```

- Access to the database. With Docker Compose it is published on `127.0.0.1:5432` of the host:

```bash
export DATABASE_URL="postgresql://fonrex:<POSTGRES_PASSWORD>@localhost:5432/fonrex"
```

## Register and ingest the bundle

```bash
mkdir -p ~/.zipline
cp zipline_bundle/extension.py ~/.zipline/extension.py
zipline bundles            # fonrex <no ingestions>
zipline ingest -b fonrex
```

Or without the extension file:

```bash
python -m zipline_bundle ingest --start 2020-01-01 --end 2025-12-31 \
    --tickers AAPL,MSFT --calendar NYSE

# What the bundle would contain (Zipline not needed)
python -m zipline_bundle preview --start 2024-01-01 --end 2024-12-31
```

| Variable | Default | Description |
|---|---|---|
| `DATABASE_URL` | the address of `.env.example` | Database read by the bundle (`postgresql+asyncpg://` is accepted) |
| `FONREX_BUNDLE_NAME` | `fonrex` | Bundle name |
| `FONREX_BUNDLE_TICKERS` | *(empty)* | Comma-separated tickers; empty = every instrument with daily prices in the window |
| `FONREX_BUNDLE_CALENDAR` | `NYSE` | Trading calendar: `XPAR`, `XETR`, `XLON`, `XSWX`… for other markets |

For several markets, register one bundle per calendar:

```python
from zipline_bundle import register_fonrex_bundle

register_fonrex_bundle(bundle_name="fonrex_us", tickers=["AAPL", "MSFT"], calendar_name="NYSE")
register_fonrex_bundle(bundle_name="fonrex_paris", tickers=["AIR.PA", "BNP.PA"], calendar_name="XPAR")
```

## Run a backtest

```python
import pandas as pd
from zipline import run_algorithm
from zipline.api import order_target_percent, symbol

def initialize(context):
    context.asset = symbol("AIR.PA")

def handle_data(context, data):
    order_target_percent(context.asset, 1.0)

result = run_algorithm(
    start=pd.Timestamp("2024-01-02"),
    end=pd.Timestamp("2024-12-31"),
    initialize=initialize,
    handle_data=handle_data,
    capital_base=100_000,
    bundle="fonrex_paris",
)
```

## What the bundle contains

- **Daily bars only**, from `prices_eod`.
- **One listing per instrument**: the primary one, then an active one; the other listings (other currencies) are not exposed.
- **Adjusted prices**: `adj_close` (adjusted for splits and dividends) is used as the Zipline close, and the open, high and low of the bar are scaled by the same factor; a bar without `adj_close` (TradingView) keeps its prices. Split and dividend tables are written empty.
- **Calendar alignment**: bars outside the sessions of the calendar are dropped.
- **Stable `sid`s**: assigned in the alphabetical order of the symbols.

## Without Zipline: pandas

For Backtrader, vectorbt or your own code, read the prices through the API:

```python
import os
import pandas as pd
import requests

def fonrex_ohlcv(ticker: str, period: str = "5y") -> pd.DataFrame:
    response = requests.get(
        f"http://localhost:5000/eod/{ticker}",
        params={"period": period},
        headers={"X-API-KEY": os.environ["FONREX_API_KEY"]},
        timeout=60,
    )
    response.raise_for_status()
    frame = pd.DataFrame(response.json()["data"])
    frame["Date"] = pd.to_datetime(frame["Date"])
    return frame.set_index("Date").rename(columns=str.lower)

df = fonrex_ohlcv("AIR.PA")
print(df.tail())
```

The columns are `open`, `high`, `low`, `close`, `adj close` and `volume`, one row per trading session.



<!-- FILE: /docs/guides/python-jupyter.md -->

---
id: "python-jupyter"
title: "Using Fonrex from Python and Jupyter"
sidebar_label: "Python & Jupyter"
description: "Load prices from your Fonrex instance into pandas, and replace OpenBB price calls in a notebook"
---

# Using Fonrex from Python and Jupyter

Fonrex is an HTTP API: there is no `fonrex` Python package to import, and Fonrex is not an OpenBB Platform provider (`obb`). From Python, you call your instance with `requests` and build pandas objects from the JSON answers. A dozen lines replace a call such as `obb.equity.price.historical(...)`.

## Prerequisites

- A running instance — see [Installation](../getting-started/installation.md).
- An API key in the environment, set before starting Jupyter. A **read-only** key is enough to read prices:

```bash
export FONREX_URL=http://localhost:5000
export FONREX_API_KEY=frx_live_...
jupyter lab
```

- The listings you need in the catalogue (next section).

## 1. Put the listings in the catalogue

Fonrex stores prices per **listing** of an instrument known by its ISIN. A ticker that is not in the catalogue has no prices: `GET /eod/{ticker}` answers `404` with `"reason": "No listing found for ticker ..."`.

The example below uses eight US stocks and the SPY ETF. Write them in a CSV file ([download it](pathname:///notebooks/us-hedging-listings.csv)):

```csv
name,ticker,isin,productType,currency
Newmont Corporation,NEM,US6516391066,STOCK,USD
Royal Gold Inc,RGLD,US7802871084,STOCK,USD
SSR Mining Inc,SSRM,CA7847301032,STOCK,USD
Coeur Mining Inc,CDE,US1921085049,STOCK,USD
Eli Lilly and Co,LLY,US5324571083,STOCK,USD
UnitedHealth Group Inc,UNH,US91324P1021,STOCK,USD
Johnson & Johnson,JNJ,US4781601046,STOCK,USD
Merck & Co Inc,MRK,US58933Y1055,STOCK,USD
SPDR S&P 500 ETF Trust,SPY,US78462F1030,ETF,USD
```

Send the file to the container through standard input, then import it. The file is created by the user of the container, which can read it; with `docker compose cp` it would keep the permissions of your machine and the import could fail with `Permission denied`.

```bash
docker compose exec -T fonrex-api sh -c 'cat > /tmp/us-hedging-listings.csv' < us-hedging-listings.csv
docker compose exec fonrex-api python import_assets.py --file /tmp/us-hedging-listings.csv
```

See [Importing assets](import-assets.md) for the CSV rules.

## 2. Read prices into pandas

`GET /eod/{ticker}` with `from` and `to` returns the daily bars of a window. When the database has no bar in that window, the route first ingests it from Yahoo Finance (with the symbol verified for the listing), so the first call of a ticker takes a few seconds.

A ticker alone does not always designate one instrument. In the default catalogue, `NEM` is Newmont in USD, its Australian line in AUD and Nemetschek in EUR; `MRK` is also Merck KGaA, and `CDE` also City Developments. Name each instrument with its **ISIN** (`isin`) and the listing with its **currency** (`currency`): together they designate one listing.

```python
import os

import pandas as pd
import requests

FONREX_URL = os.environ.get("FONREX_URL", "http://localhost:5000")
FONREX_API_KEY = os.environ["FONREX_API_KEY"]


def fonrex_prices(instruments, start_date, end_date, currency="USD"):
    """Daily closing prices of several listings, one column per ticker.

    ``instruments`` maps each ticker to the ISIN of its instrument.
    """
    session = requests.Session()
    session.headers["X-API-KEY"] = FONREX_API_KEY
    closes = {}
    for ticker, isin in instruments.items():
        response = session.get(
            f"{FONREX_URL}/eod/{ticker}",
            params={"from": start_date, "to": end_date, "isin": isin, "currency": currency},
            timeout=120,  # the first call ingests the prices from Yahoo Finance
        )
        if response.status_code != 200:
            raise RuntimeError(f"{ticker}: {response.status_code} {response.text}")
        bars = pd.DataFrame(response.json()["data"])
        closes[ticker] = bars.set_index(pd.to_datetime(bars["Date"]))["Close"]
    data = pd.DataFrame(closes).sort_index(axis=1)
    data.index.name = "date"
    data.columns.name = "symbol"
    return data


data = fonrex_prices(
    {"NEM": "US6516391066", "LLY": "US5324571083", "SPY": "US78462F1030"},
    "2020-01-01",
    "2022-12-31",
)
```

Each bar has `Date`, `Open`, `High`, `Low`, `Close`, `Adj Close` and `Volume`. The answer also gives the `listing` that was read (`ticker`, `isin`, `currency`, `exchange`) — see [`GET /eod/{ticker}`](../api-reference/assets.md).

:::tip Without the ISIN
`isin` is optional. Without it, Fonrex takes, among the listings bearing the ticker, the primary one first, then by currency in alphabetical order: `/eod/NEM` without `isin` nor `currency` returns the Australian line in AUD. With `currency="USD"` alone, the nine tickers of the example happen to be unique, but 196 ticker and currency pairs of the default catalogue still belong to several instruments. Check `listing.isin` in the answer when you do not pass `isin`.
:::

## Replacing OpenBB in a notebook

| OpenBB | Fonrex |
|---|---|
| `from openbb import obb` | `import requests` and the `fonrex_prices()` function above |
| `obb.equity.price.historical(symbols, start_date=..., end_date=..., provider="yfinance")` | `fonrex_prices(instruments, start_date, end_date)`, with `instruments` mapping each ticker to its ISIN |
| `.pivot(columns="symbol", values="close")` | Already done: one column per symbol, index `date` |
| `obb.user.preferences.output_type = "dataframe"` | Not needed |

The rest of a notebook that works on the DataFrame (`pct_change()`, regressions, plots) does not change.

:::note Close or Adj Close
`Close` is the traded close, adjusted for splits only — the default of OpenBB's `yfinance` provider, so the notebook gives the same numbers as with OpenBB. `Adj Close` is also adjusted for dividends: use it for returns that include dividends, the usual choice to measure alpha and beta.
:::

## Example notebook: beta hedging

[beta-hedging-fonrex.ipynb](pathname:///notebooks/beta-hedging-fonrex.ipynb) builds an equally weighted portfolio of gold stocks (NEM, RGLD, SSRM, CDE) and healthcare stocks (LLY, UNH, JNJ, MRK), estimates its alpha and beta against SPY with an OLS regression (`statsmodels`), then builds a beta-hedged portfolio whose beta is about zero.

Its only Fonrex-specific part is the price loading:

```python
instruments = {
    "NEM": "US6516391066",   # Newmont
    "RGLD": "US7802871084",  # Royal Gold
    "SSRM": "CA7847301032",  # SSR Mining
    "CDE": "US1921085049",   # Coeur Mining
    "LLY": "US5324571083",   # Eli Lilly
    "UNH": "US91324P1021",   # UnitedHealth
    "JNJ": "US4781601046",   # Johnson & Johnson
    "MRK": "US58933Y1055",   # Merck & Co
    "SPY": "US78462F1030",   # SPDR S&P 500 ETF
}
data = fonrex_prices(instruments, start_date="2020-01-01", end_date="2022-12-31")

benchmark_returns = data.pop("SPY").pct_change().dropna()
portfolio_returns = data.pct_change().dropna().sum(axis=1)
```

Import the listings of step 1, set `FONREX_API_KEY`, then open the notebook in Jupyter.

## Troubleshooting

| Answer | Cause |
|---|---|
| `401` | `FONREX_API_KEY` is missing or not a key of the instance |
| `400` with `L'ISIN ... n'est pas valide` | The ISIN does not have 12 characters (two letters, then ten letters or digits) |
| `404` with `No listing found for ticker` | No listing of the catalogue has this ticker with this ISIN and currency: import it, or check the ISIN |
| `404` with another `reason` | The ingestion failed, e.g. no Yahoo symbol quoted in the currency of the listing |
| `404` `No data found` without `reason`, or fewer rows than expected | The database already holds other dates of the listing: `/eod` ingests only an empty window, and the ingestion completes after the last stored date. Fetch the window again with a full-access key: `POST /historical/ingest?ticker=NEM&isin=US6516391066&currency=USD&from_date=2020-01-01&to_date=2022-12-31&force_refresh=true` |

## Related pages

- [Historical prices API](../api-reference/historical.md)
- [Ingesting historical data](ingest-historical-data.md)
- [Quant & Algo-Trader pathway](../pathways/quant-trader.md)



<!-- FILE: /docs/guides/adding-providers.md -->

---
id: "adding-providers"
title: "Guide: Adding a Fundamentals Provider"
sidebar_label: "Adding Providers"
description: "Every step a new fundamentals provider needs: code, registration, units, canary, tests and coverage floor"
---

# Guide: Adding a Fundamentals Provider

A provider of `/fundamental` is a class of `financials/providers/` that turns a website or an API into a `FinancialMetrics` object. The repository enforces each step below with a test: a provider that misses one fails `make ci`. The rules come from `AGENTS.md`.

## 1. Write the provider

Create `financials/providers/MySite_provider.py`, a subclass of `BaseFinancialProvider` implementing `get_financials(ticker)`. The [reference page](../providers/adding-custom-provider.md) has a complete skeleton.

- **One HTTP layer.** Never create an HTTP client: use `self._get()`, `self._get_json()`, `self._post_json()` or `async with self._session()` (cookies shared between a search and a page). Retries, pauses, the per-provider concurrency limit and the proxy live there. *(`tests/test_provider_http_policy.py`)*
- **Numbers are read in one place.** Turn a displayed text into a number with `parse_number` (a cell) or `find_number` (a sentence) of `financials/numbers.py`: signs, thousands separators, scales (`k`, `M`, `Md`, `B`) and currencies are handled there. No `float()` on a scraped text. *(`tests/test_numbers.py`)*
- **Return the ISIN when the page shows it.** The runner rejects an answer about another ISIN — a site searched by ticker may return a homonym.

The runner gives your provider, in this order: its `provider_url` mapping, its `provider_ticker` mapping, the ISIN (for providers searched by ISIN), or the ticker.

## 2. Register it

Add a line to `PROVIDER_SPECS` in `main.py`:

```python
PROVIDER_SPECS = (
    ...
    ("MySite", "financials.providers.MySite_provider", "MySiteProvider"),
)
```

A provider that cannot be imported is listed in `providers.unavailable` of `GET /health` instead of disappearing silently. *(`tests/test_docs_consistency.py`)*

## 3. Declare its units

Monitoring ranges are ratios (a 3.45 % yield is `0.0345`). If your provider returns displayed percentages (`3.45`), declare those fields in `PROVIDER_PERCENT_FIELDS` of `monitoring/units.py`; a provider returning ratios is declared with an empty set. *(`tests/test_provider_units.py`)*

## 4. Add it to the canary

Add the provider to `MONITORED_PROVIDERS` and `_PROVIDER_IMPORTS` in `monitoring/canary_catalog.py`, so that the daily canary checks it against the canary assets. An EU-only provider is tested on EU tickers only.

## 5. Test it without network

Tests never reach a real website. Save a reduced copy of a real page in `tests/fixtures/providers/` and serve it with the `fake_network` fixture of `tests/conftest.py`:

```python
async def test_my_site_reads_the_displayed_figures(fake_network):
    page = (FIXTURES / "mysite_airbus.html").read_text(encoding="utf-8")
    fake_network.get("mysite.example/search", httpx.Response(200, json={"url": "/airbus"}))
    fake_network.get("mysite.example/airbus", httpx.Response(200, text=page))

    metrics = await MySiteProvider().get_financials("AIR.PA")

    assert metrics.pe_ratio == pytest.approx(24.1)
    assert fake_network.calls("mysite.example") == 2
```

A request without a canned response fails the test.

## 6. Give it a coverage floor

Every module of `financials/providers/` has a coverage floor in `scripts/check_coverage_distribution.py`. Add yours; floors only go up. *(`tests/test_coverage_gate.py`)*

## 7. Update the documents

The provider count of `README.md` is checked against the code, and `ARCHITECTURE.md` lists the providers. Then run the whole gate:

```bash
make ci
```



<!-- FILE: /docs/guides/google-sheets-connector.md -->

---
id: google-sheets-connector
title: Google Sheets Connector
sidebar_label: Google Sheets Connector
description: "Fill a Google Sheets template with fundamentals, DCF valuations and indicators from your own Fonrex instance"
---

# Google Sheets Connector

The Fonrex Sheets template fills a spreadsheet with fundamentals, DCF valuations and technical indicators read from **your own Fonrex instance**. No data goes through a Fonrex-operated service, and no paid plan is involved.

![Fonrex Sheets Connector Preview](/img/template-preview.png)

:::info
**This template displays raw financial data for informational purposes only.** It does not constitute investment advice. All displayed values (including DCF valuations) are analytical outputs. Always do your own research before making any investment decisions.
:::

## How it works

Google runs the script of the spreadsheet on its own servers, which cannot reach `localhost`. You expose your instance through a tunnel that gives it a public HTTPS URL, and the spreadsheet calls that URL with a key you give it.

## Prerequisites

- A running Fonrex instance ([Installation](../getting-started/installation.md))
- A tunnel giving it an HTTPS URL (zrok, Cloudflare Tunnel, Tailscale Funnel, ngrok…)
- A Google account

## Step 1 — Create a read-only key for the spreadsheet

The key will be stored in your Google account, outside the machine running Fonrex: give the spreadsheet a key that can read data but cannot clear the cache, clean the database or trigger ingestion.

```bash
echo "frx_live_$(openssl rand -hex 24)"
```

In the `.env` of your instance:

```
FONREX_READ_ONLY_API_KEYS=frx_live_<the generated value>
```

Then restart the API: `docker compose up -d`.

## Step 2 — Expose your instance through a tunnel

Example with [zrok](https://zrok.io), once your environment is enabled (`zrok2 enable <token>`):

```bash
# temporary URL, valid until you stop the command
zrok2 share public localhost:5000

# or a stable URL: reserve a name once, then share with it
zrok2 create name -n public myfonrex
zrok2 share public localhost:5000 -n public:myfonrex
# → https://myfonrex.share.zrok.io
```

With zrok 1.x the command is `zrok` instead of `zrok2`. Prefer a stable URL: with a temporary one, you reconfigure the spreadsheet each time the tunnel restarts. Check the URL from another network:

```bash
curl https://myfonrex.share.zrok.io/health
```

:::warning
While the tunnel runs, your instance is reachable from the Internet. Only `/health`, the API documentation, the OpenBB discovery files and static files answer without a key. Keep `FONREX_AUTH_REQUIRED` enabled, never share a full-access key, and stop the tunnel when you do not need the spreadsheet.
:::

## Step 3 — Copy the template

👉 **[Open the Fonrex Sheets template](https://docs.google.com/spreadsheets/d/1PUBLISHED_TEMPLATE_ID_XYZ_1234567890/copy)**

You get a personal copy in your Google Drive; the original is never modified.

## Step 4 — Connect the spreadsheet

1. In your copy, open the **Fonrex** menu (next to "Help").
2. **Configure Instance URL** → paste the HTTPS URL of your tunnel.
3. **Configure API Key** → paste the read-only key of step 1.

The URL and the key are stored in the Apps Script *user properties* of your Google account — never in a cell, and not shared when you share the document.

Then add tickers in column A of the **Watchlist** sheet (from row 2: `AAPL`, `AIR.PA`, `MC.PA`…), or with **Fonrex > Add Ticker to Watchlist**, and refresh:

- **Fonrex > Refresh Fundamentals** → *Fundamentals* sheet
- **Fonrex > Refresh DCF Valuations** → *DCF* sheet
- **Fonrex > Refresh Technical Indicators** → *Technicals* sheet

## Template sheets

| Sheet | Content |
|---|---|
| **Config** | Connection status, last refresh date, legal warning |
| **Watchlist** | Tracked tickers (column A, from row 2) |
| **Fundamentals** | P/E, ROE, ROA, market cap, dividend yield, beta, 52-week high/low… |
| **DCF** | Consensus value, current price, consensus upside, WACC, FCF value |
| **Technicals** | RSI 14, MACD, SMA 50/200, EMA 20, Bollinger Bands, ATR 14 |
| **Charts** | Native charts based on the imported data |

## Custom formulas

| Formula | Description |
|---|---|
| `=FONREX_PE("AIR.PA")` | P/E ratio |
| `=FONREX_DIVIDEND_YIELD("AIR.PA")` | Dividend yield (ratio) |
| `=FONREX_INTRINSIC_VALUE("AIR.PA")` | DCF consensus value |
| `=FONREX_RSI("AAPL")` | 14-period RSI |

Google caches custom formulas for 30 minutes; use the **Refresh** menu items for fresh data.

## What the script calls

Only `GET` requests, with `Authorization: Bearer <key>`, to your instance: `/fundamental/deep`, `/dcf/{ticker}` and `/technical/{ticker}/multi` for the menu refreshes, and `/fundamental` for two of the formulas. The DCF needs the deep fundamentals of the ticker: refresh the *Fundamentals* sheet first.

## Limitations

| Limitation | Detail |
|---|---|
| **Manual refresh** | No real-time updates — Apps Script has no WebSocket |
| **Formula cache** | 30 minutes, imposed by Google |
| **Instance and tunnel must run** | A refresh fails when either is stopped |
| **One call per ticker and route** | A refresh makes your instance query its providers for each ticker |

## Troubleshooting

| Error | Probable cause | Solution |
|---|---|---|
| `Configure your instance URL first` | URL not configured | Fonrex > Configure Instance URL |
| `Configure API key first` | Key not configured | Fonrex > Configure API Key |
| `API key refused by your instance` | Key not in `.env`, or API not restarted | Check `FONREX_READ_ONLY_API_KEYS`, then `docker compose up -d` |
| `Instance unreachable` | Tunnel stopped or URL changed | Restart the tunnel, update the URL |
| `The tunnel answered instead of Fonrex` | The tunnel runs but the instance does not | `docker compose ps`, then `docker compose up -d` |
| `Instance error or tunnel down (status 5xx)` | Error in the instance, or tunnel without backend | `docker compose logs fonrex-api` |
| `Ticker not found` | Symbol unknown to the API | Check the format (`AIR.PA`, not `AIR`) |
| "Fonrex" menu missing | Script not authorised | Reload the page, accept the permissions |

## Security

- The script only calls the URL **you** configure, over HTTPS, with `GET` requests.
- The manifest (`appsscript.json`) requests access to the current spreadsheet and to external requests; it has no fixed list of domains, because the URL of your tunnel is yours.
- With a read-only key, a leaked key cannot clear the cache, clean the database, trigger ingestion or change subscriptions.



<!-- FILE: /docs/guides/openbb-workspace.md -->

---
id: "openbb-workspace"
title: "Connecting Fonrex to OpenBB Workspace"
sidebar_label: "OpenBB Workspace Guide"
description: "Connect your self-hosted Fonrex instance to OpenBB Workspace widgets and dashboards"
---

# Connecting Fonrex to OpenBB Workspace

[OpenBB Workspace](https://openbb.co) can use your Fonrex instance as a custom backend: European fundamentals, DCF valuations, technical indicators and news appear as OpenBB widgets.

## Prerequisites

1. A running Fonrex instance that OpenBB can reach. OpenBB Workspace in the browser (`pro.openbb.co`) calls your instance from your browser: `http://localhost:5000` works when the browser runs on the same machine; otherwise expose the instance through a tunnel or your network.
2. An API key of the instance. A **read-only** key (`FONREX_READ_ONLY_API_KEYS`) is enough for every widget and is the one to use.

## Step 1 — Add Fonrex as a data source

1. In OpenBB Workspace, right-click on the dashboard and select **Add data** (or open the backend connections).
2. Enter the URL of your instance, e.g. `http://localhost:5000` or `https://myfonrex.share.zrok.io`.
3. OpenBB reads `/widgets.json` and lists the 19 widgets. This file and `/apps.json` answer without a key.

## Step 2 — Add the key

Add a custom header to the connection:

- **Name**: `X-API-KEY`
- **Value**: `frx_live_...`

Every `/openbb/...` route requires it.

## Step 3 — Import the dashboards

`/apps.json` holds two dashboards:

**Fonrex — EU Markets** — one ticker:
- *Overview*: quote, macro rates, EOD chart, deep fundamentals, news
- *Valuation*: DCF valuation, model comparison and sensitivity matrix
- *Technical*: technical chart and multi-indicator chart
- *News*: news of the ticker and global feed
- *Watchlist*: batch quotes

**Fonrex — Screener & Macro** — discovery:
- *Screener*: technical screener (e.g. RSI < 30)
- *Macro Context*: FRED rates and index constituents

Import them from the Apps menu of OpenBB, or add widgets one by one to your own dashboard.

## Widgets

| Widget | Name | Category | Type |
|---|---|---|---|
| `fonrex_fundamentals` | Fonrex Fundamentals | Fundamentals | table |
| `fonrex_fundamentals_deep` | Fonrex Deep Fundamentals | Fundamentals | table |
| `fonrex_insider_transactions` | Fonrex Insider Transactions | Fundamentals | table |
| `fonrex_etf_details` | Fonrex ETF Details | Fundamentals | table |
| `fonrex_eod` | Fonrex EOD History | Historical | chart |
| `fonrex_history` | Fonrex OHLCV History | Historical | chart |
| `fonrex_quote` | Fonrex Quote | Market Data | metric |
| `fonrex_quotes_batch` | Fonrex Batch Quotes | Market Data | table |
| `fonrex_index_constituents` | Fonrex Index Constituents | Market Data | table |
| `fonrex_technical` | Fonrex Technical Indicator | Technical | chart |
| `fonrex_technical_multi` | Fonrex Multi-Indicator | Technical | chart |
| `fonrex_technical_chart` | Fonrex Technical Chart | Technical | chart |
| `fonrex_screener` | Fonrex Technical Screener | Technical | table |
| `fonrex_news` | Fonrex News | News | table |
| `fonrex_news_feed` | Fonrex News Feed | News | table |
| `fonrex_dcf` | Fonrex DCF Valuation | Valuation | table |
| `fonrex_dcf_compare` | Fonrex DCF Models Comparison | Valuation | table |
| `fonrex_dcf_sensitivity` | Fonrex DCF Sensitivity Matrix | Valuation | table |
| `fonrex_macro_rates` | Fonrex Macro Rates | Macro | metric |

The routes behind them are listed in the [OpenBB API reference](../api-reference/openbb.md).

## Good to know

- **Quotes**: the quote widget never starts a realtime stream. It shows the real-time price once the ticker is subscribed (`POST /realtime/subscribe` with a full-access key), the delayed Yahoo Finance price otherwise.
- **Prices and indicators** need the ticker's prices in the database; the EOD widget ingests them on first use.
- **DCF** needs the deep fundamentals of the ticker: open the deep fundamentals widget first.

## Troubleshooting

- **Connection refused**: check `docker compose ps` and that OpenBB can reach the URL.
- **401 / 403**: the `X-API-KEY` header is missing or does not match a key of your `.env` (restart the API after changing it).
- **CORS error**: the browser calls your instance from the OpenBB origin. `OPENBB_ALLOWED_ORIGIN` (default `https://pro.openbb.co`) lists the allowed origins, comma-separated.



<!-- FILE: /docs/architecture/overview.md -->

---
id: "overview"
title: "Architecture Overview"
sidebar_label: "System Overview"
description: "Components, data flows and start-up of a Fonrex instance"
---

# Architecture Overview

A Fonrex instance is one FastAPI process, a PostgreSQL/TimescaleDB database and a Redis server. Everything that collects data — the fundamentals providers, the news providers, the realtime worker, the daily canary — runs inside the API process.

```mermaid
flowchart TD
    subgraph Clients
        Client[HTTP / WebSocket clients]
        OpenBB[OpenBB Workspace]
        Sheets[Google Sheets, through a tunnel]
    end

    subgraph API [FastAPI process]
        Routers[routers/]
        Worker[RealtimePriceWorker]
        VL[ValidationLayer]
        Canary[CanaryMonitor - 06:00 UTC]
        News[NewsService]
        DCF[DCFService]
    end

    subgraph Sources [Public sources]
        YF[Yahoo Finance]
        TV[TradingView]
        Scraped[13 scraped websites]
        NewsSites[7 news sources]
        Specialised[SEC EDGAR, JustETF, Wikipedia, FRED]
    end

    Redis[(Redis: cache + Pub/Sub)]
    DB[(PostgreSQL + TimescaleDB)]

    Client --> Routers
    OpenBB --> Routers
    Sheets --> Routers
    Routers --> Redis
    Routers --> DB
    Routers --> YF
    Routers --> Scraped
    Routers --> Specialised
    Scraped --> VL
    VL --> DB
    Worker --> TV
    Worker --> Redis
    Worker --> DB
    News --> NewsSites
    News --> DB
    DCF --> DB
    Canary --> Scraped
    Canary --> DB
```

## Main flows

- **End-of-day prices**: `GET /eod` reads `prices_eod`; when nothing is stored, the listing is ingested from Yahoo Finance (with the symbol verified for the listing), or TradingView as a fallback.
- **Fundamentals**: `GET /fundamental` calls the providers in parallel, validates their values (range and consensus checks), and renders one document choosing each figure from Yahoo, the stored figures, then the scraped providers.
- **Realtime**: the worker streams TradingView ticks into Redis (`quote:{ticker}`, `price:{ticker}`) and `prices_intraday`; each WebSocket client listens to the Redis channel of its ticker.
- **Indicators and valuation** are computed from what the database holds.

## Start-up

`entrypoint.sh` waits for PostgreSQL and Redis, applies `alembic upgrade head`, optionally imports `data/etf.csv` (`SEED_ON_FIRST_RUN`), then starts Gunicorn with `WEB_CONCURRENCY` workers (1 by default).

`main.py` then creates the services and publishes them in `app.state`: database and Redis clients, ingestion, indicators, realtime worker (restores the stored subscriptions), news, FRED, DCF, validation layer, canary monitor and its daily scheduler, usage recorder. The start is tolerant: a service that fails to start leaves its routes answering `503` while the rest of the API runs. A provider that cannot be imported is listed by `GET /health`.

`main.py` never changes the schema: it compares the database revision with the Alembic head and marks the database unavailable when they differ.

**Keep one worker.** Realtime streams, WebSocket clients and the daily canary live in the memory of the process: each extra Gunicorn worker would open its own streams and run its own canary.

## Security

Every route except `/health`, `/docs`, `/redoc`, `/openapi.json`, `/widgets.json`, `/apps.json`, `/favicon.ico` and `/static/*` requires an API key. Read-only keys can call the `GET` routes and the two computation routes `POST /technical/batch` and `POST /dcf/{ticker}`; they cannot clear the cache, clean the database, ingest or start streams. With no key configured, every protected request is refused.

## Code layout

| Package | Role |
|---|---|
| `routers/` | HTTP adapters, one module per feature |
| `use_cases/` | Application logic of fundamentals, specialised providers and realtime, behind ports |
| `historical/`, `technical/`, `news/`, `valuation/`, `monitoring/`, `macro/` | Feature services |
| `database/`, `cache/` | SQLAlchemy repositories, Redis |
| `financials/providers/`, `news/providers/` | Providers, all built on `BaseFinancialProvider` |
| `realtime/` | Realtime worker and WebSocket connection manager |
| `integrations/openbb/` | OpenBB widgets, dashboards and adapters |
| `zipline_bundle/` | Zipline data bundle (not imported by the API) |

The repository's `ARCHITECTURE.md` is the detailed reference: module map, every route, every migration, known limits.



<!-- FILE: /docs/architecture/hexagonal.md -->

---
id: "hexagonal"
title: "Layers & Ports"
sidebar_label: "Layers & Ports"
description: "How Fonrex separates HTTP adapters, application logic and adapters to the outside, and how far each feature follows it"
---

# Layers & Ports

The code is organised by feature, and inside a feature in three levels:

1. **HTTP adapters** (`routers/`) — parse the request, call the level below, translate errors into HTTP statuses.
2. **Application logic** — use cases (`use_cases/`) or feature services (`historical/`, `technical/`, `news/`, `valuation/`, `monitoring/`, `macro/`).
3. **Adapters to the outside** — SQLAlchemy repositories (`database/`), Redis (`cache/`), providers (`financials/providers/`, `news/providers/`, `historical/providers.py`).

```
┌──────────────── routers/ (FastAPI) ────────────────┐
│  parse request → call use case → map errors        │
└──────────────────────────┬─────────────────────────┘
                           ▼
┌──────── use_cases/ — depends on ports only ────────┐
│  GetFundamentals, GetDeepFundamentals, GetQuote…   │
│  use_cases/ports.py: repository, cache, providers  │
└──────────────────────────┬─────────────────────────┘
                           ▼ implemented by
┌──── database/, cache/, financials/providers/ ──────┐
│  SQLAlchemy, Redis, HTTP                           │
└────────────────────────────────────────────────────┘
```

## How far each feature follows it

| Feature | Router | Application logic | Behind ports? |
|---|---|---|---|
| Fundamentals | `routers/fundamentals.py` | `use_cases/fundamentals.py` | Yes (`use_cases/ports.py`) |
| Specialised providers | `routers/specialized.py` | `use_cases/specialized.py` | Yes |
| Realtime | `routers/realtime.py` | `use_cases/realtime.py` | Partly — the WebSocket protocol is in the router |
| Technical indicators | `routers/technical.py` | `technical/indicator_service.py` | Yes (`technical/contracts.py`) |
| Monitoring | `routers/monitoring.py` | `monitoring/` | Partly — the canary and the validation layer use `monitoring/ports.py`; the read queries of the routes are written in the router |
| History and EOD | `routers/historical.py`, `routers/assets.py` | `historical/ingestion_service.py`, `database/query.py` | No |
| Valuation | `routers/valuation.py` | `valuation/dcf_service.py` | No |
| News | `routers/news.py` | `news/news_service.py` | No |
| Macro, operations | `routers/macro.py`, `routers/admin.py` | `macro/`, `database/maintenance.py`, `cache/` | No |

The use-case layer is the target model; the other features call their services directly.

## Rules held by tests

- `technical/` imports neither FastAPI, SQLAlchemy, Redis nor the ORM models; `monitoring/` neither SQLAlchemy nor the ORM models (`tests/test_exception_boundaries.py`).
- Blocking code (SQLAlchemy sessions, pandas, yfinance) is reached through `concurrency.run_sync()` from the asynchronous code (`tests/test_async_boundary.py`) — see [Concurrency](concurrency.md).
- Routers translate application errors with `routers/errors.py`.

## Error mapping

| Application error (`use_cases/errors.py`) | HTTP status |
|---|---|
| `InvalidInput` | `400 Bad Request` |
| `ResourceNotFound` | `404 Not Found` |
| `DependencyUnavailable` | `503 Service Unavailable` |
| `UpstreamFailure` | `500 Internal Server Error` |

The technical indicators have their own errors: unknown indicator `400`, no prices `404`, too few bars `422`.



<!-- FILE: /docs/architecture/data-model.md -->

---
id: "data-model"
title: "Data Model & Schema Reference"
sidebar_label: "Data Model"
description: "Instruments, listings and provider mappings, per-listing prices on TimescaleDB, fundamentals and monitoring tables"
---

# Data Model & Schema Reference

The identity of an instrument has three levels:

- `assets` — the instrument, one per ISIN;
- `asset_listings` — where it is quoted: ticker, exchange, currency;
- `asset_mappings` — the identifier of the instrument or of a listing at a given provider (Yahoo symbol, page URL…).

The same ISIN is listed under several tickers and currencies; the same ticker can name different instruments on different markets; and providers do not accept the same identifiers. `models.py` is the reference for every column.

```mermaid
erDiagram
    ASSETS ||--o{ ASSET_LISTINGS : "is quoted as"
    ASSETS ||--o{ ASSET_MAPPINGS : "global mappings"
    ASSET_LISTINGS |o--o{ ASSET_MAPPINGS : "listing mappings"
    ASSET_LISTINGS ||--o{ PRICES_EOD : "price series"
    ASSETS ||--o{ PRICES_INTRADAY : "1-minute candles"
    ASSETS ||--o| FUNDAMENTALS_HIGHLIGHTS : "highlights"
    ASSETS ||--o{ FINANCIAL_STATEMENTS : "statements"
    ASSETS ||--o{ EARNINGS_HISTORY : "EPS history"
    ASSETS ||--o| ANALYST_RATINGS : "ratings"
    ASSETS |o--o{ NEWS_ARTICLES : "news"

    ASSETS {
        int id PK
        string isin "unique when not null"
        string name
        string sector
        string industry
        string quote_type
    }
    ASSET_LISTINGS {
        int id PK
        int asset_id FK
        string ticker
        string exchange
        string currency
        bool is_primary
        bool is_active
    }
    ASSET_MAPPINGS {
        int id PK
        int asset_id FK
        int asset_listing_id FK "nullable"
        string provider_name
        string provider_ticker
        string provider_url
        string source
    }
    PRICES_EOD {
        int asset_listing_id PK
        string resolution PK "1D 1W 1M"
        timestamptz time PK "session date, midnight UTC"
        int asset_id
        float open
        float high
        float low
        float close
        float adj_close
        bigint volume
    }
```

## Identity

| Table | Rule |
|---|---|
| `assets` | One row per ISIN: partial unique index `uq_assets_isin_not_null` (`WHERE isin IS NOT NULL`) |
| `asset_listings` | Unique on `(asset_id, ticker, exchange, currency)` (`uq_asset_listing_identity`); `is_primary` marks the default listing |
| `asset_mappings` | Unique on `(asset_listing_id, provider_name)`. A mapping without listing applies to every listing of the instrument. `source` says where the identifier comes from: `csv_import`, `manual`, `isin_search`, `ticker_check`, `symbol_not_found` |

The Yahoo Finance mapping of a listing holds its **verified symbol** — found from the ISIN and checked against the listing's currency — or, with `source = 'manual'`, a symbol you set by hand.

## Prices

| Table | Description |
|---|---|
| `prices_eod` | TimescaleDB hypertable. Key `(asset_listing_id, resolution, time)`: one series per listing and resolution. `time` is the trading session date at midnight UTC. `open`, `high`, `low`, `close` are traded prices adjusted for splits; `adj_close` is also adjusted for dividends. Chunks older than 14 days are compressed (segmented by listing and resolution) |
| `price_series_adjustments` | One row per series (listing and resolution): how its bars are adjusted (`scheme`) and when it was last fetched in one piece (`fetched_at`). A series without a row is fetched again in full at its next ingestion |
| `prices_weekly`, `prices_monthly` | Continuous aggregates of the daily bars, per listing, refreshed daily; used when no `1W`/`1M` row is stored for the listing |
| `prices_intraday` | Hypertable of 1-minute candles from the realtime stream, per instrument, one-day chunks, purged after 30 days |
| `realtime_subscriptions` | Streamed tickers, restored at start-up |
| `ingest_log` | One row per ingestion: status, source, rows, range, duration, error |

## Fundamentals

| Table | Description |
|---|---|
| `fundamentals_highlights` | Last snapshot of an instrument (valuation, profitability, dividend, short interest, solvency). `dividend_yield` is a ratio |
| `financial_statements` | One row per statement type (income, balance, cash flow), fiscal period and frequency. A fiscal year is three rows; calculations put them together with `financials/fiscal_years.py` |
| `earnings_history`, `earnings_trend` | Actual vs estimated EPS; analyst estimates for `0q`, `+1q`, `0y`, `+1y` |
| `analyst_ratings` | Consensus, target price, rating counts |
| `esg_scores` | E/S/G scores and 15 controversy flags |
| `outstanding_shares_history` | Share count history |
| `etf_details`, `etf_holdings` | Read for ETFs but not written by the application today |
| `fundamentals` | Legacy table, no longer written |

These tables are written by the deep enrichment from Yahoo Finance (`/fundamental/deep`, `import_assets.py --enrich-only`).

## News, macro and usage

| Table | Description |
|---|---|
| `news_articles` | Unique on `url`; indexes for the feed and the statistics. Old articles are not purged automatically |
| `macro_rates_cache` | Series read from FRED, unique on `(series_id, observation_date)` |
| `usage_logs` | One row per API request, written in background batches; IP not kept unless `USAGE_LOG_IP` asks for it; purged after `USAGE_LOG_RETENTION_DAYS` |

## Monitoring

| Table | Description |
|---|---|
| `provider_health_log` | Hypertable, one row per checked value (`check_type` `canary`, `realtime` or `consensus`), 30-day retention |
| `provider_health_daily` | Daily aggregate per provider, unique on `(provider_name, date)` |
| `provider_alerts` | Alerts `canary_failed` and `high_outlier_rate`, active or resolved |



<!-- FILE: /docs/architecture/migrations.md -->

---
id: "migrations"
title: "Schema Migrations (Alembic)"
sidebar_label: "Schema Migrations"
description: "The Alembic migration chain, how it runs and how to add a migration"
---

# Schema Migrations (Alembic)

Alembic owns the schema, including the TimescaleDB hypertables, compression and continuous aggregates. The chain is linear, with a single head.

## Migration history

| Revision | File | Changes |
|---|---|---|
| 001 | `001_initial_schema.py` | Initial schema: `assets` (unique ISIN index), `asset_listings`, `asset_mappings`, `prices_eod`, `fundamentals`, `usage_logs`, and the legacy tables `stock_data`, `data_requests`, `cache_status` |
| 002 | `002_refonte_fundamentals.py` | `fundamentals_highlights`, `financial_statements`, `earnings_history`, `analyst_ratings`, `etf_details`, `etf_holdings` |
| 003 | `003_index_constituents.py` | `index_constituents` table (not used by the code) |
| 004 | `004_fix_assets_columns.py` | Profile columns of `assets` |
| 005 | `005_premium_fields.py` | Short interest, TTM and growth columns; GICS columns; `earnings_trend`, `esg_scores`, `outstanding_shares_history` |
| 006 | `006_prices_eod_resolution.py` | `resolution`, `adjusted`, `source` on `prices_eod`; `ingest_log` |
| 007 | `007_realtime_tables.py` | `prices_intraday` hypertable (30-day retention), `realtime_subscriptions` |
| 008 | `008_drop_legacy_tables.py` | **Destructive**: drops the legacy price tables |
| 009 | `009_fix_assets_isin_unique.py` | Merges ISIN duplicates, unique ISIN index and listing identity constraint |
| 010 | `010_news_articles.py` | `news_articles` (unique `url`) |
| 011 | `011_provider_health.py` | `provider_health_log` hypertable, `provider_health_daily`, `provider_alerts` |
| 012 | `012_alembic_schema_authority.py` | Alembic takes over hypertables, compression and weekly/monthly aggregates |
| 013 | `013_solvency_ratios.py` | Solvency ratios and cost of debt; `macro_rates_cache` |
| 014 | `014_prices_per_listing.py` | `prices_eod` rebuilt per listing: key `(asset_listing_id, resolution, time)`, rows re-dated to their session; compression and aggregates per listing |
| 015 | `015_dividend_yield_as_ratio.py` | Stored dividend yields converted from percentages to ratios |
| 016 | `016_price_series_adjustments.py` | `price_series_adjustments`: how each stored price series is adjusted and when it was last fetched in one piece. Series stored before are fetched again in full at their next ingestion |

## How migrations run

1. The API container runs `alembic upgrade head` in `entrypoint.sh` before starting the application. The `fonrex-migrate` service (profile `migrate`) does the same alone.
2. `main.py` compares the revision stored in `alembic_version` with the head. A database behind the code is marked unavailable and the routes that need it answer `503` — the application never changes the schema itself.

Migration 014 first deletes the TimescaleDB jobs of the price tables (waiting for one that is running) and locks `prices_eod`: a compression or refresh job running at the same time would otherwise deadlock with it. The jobs are created again by the migration.

## Adding a migration

```bash
alembic revision -m "describe_the_change"
```

Rename the new file of `alembic/versions/` and set its identifiers after the last migration (`revision = "017"`, `down_revision = "016"`, file `017_describe_the_change.py`), then:

```bash
alembic upgrade head
make migration-check     # one head only
```

- Add the migration to the migrations table of `ARCHITECTURE.md` (`tests/test_docs_consistency.py`).
- A migration that moves or rewrites data comes with a test in `tests/test_timescale_integration.py`, run on a real TimescaleDB (`make test-db`).
- Write `downgrade()` too: the integration tests go down and up again.



<!-- FILE: /docs/architecture/concurrency.md -->

---
id: "concurrency"
title: "Concurrency & Async Execution"
sidebar_label: "Concurrency & Async"
description: "How Fonrex runs blocking calls without blocking the FastAPI event loop"
---

# Concurrency & Async Execution

FastAPI serves every request on one `asyncio` event loop. A blocking call made on that loop — a synchronous SQLAlchemy query, a pandas calculation, a `yfinance` download — stops every other request and WebSocket of the process until it returns.

## `run_sync()`

`concurrency.py` provides one way to run blocking code from asynchronous code:

```python
from concurrency import run_sync

result = await run_sync(database.get_asset_context, ticker=ticker)
```

`run_sync` runs the function in a worker thread (`asyncio.to_thread`), propagates the context variables and accepts keyword arguments. `tests/test_async_boundary.py` checks that `main.py`, the routers, the use cases and the feature packages reach blocking code only through it.

```
event loop ──► async route ──► await run_sync(blocking_call) ──► worker thread
     │                                                               │
     └── keeps serving other requests and WebSockets ◄───────────────┘
```

## What is asynchronous already

- History queries, news, monitoring and the realtime worker use the asynchronous SQLAlchemy engine (asyncpg), derived from `DATABASE_URL`.
- Providers use `httpx.AsyncClient` through `BaseFinancialProvider`; the providers of one request run in parallel (`asyncio.gather`), each with its own limit of simultaneous requests.
- Caches use the asynchronous Redis client, except `CacheService` (synchronous, called through `run_sync`).

## Background work

| Work | How it runs |
|---|---|
| Realtime streams | TradingView clients in a thread pool, at most `TV_MAX_CONNECTIONS` at once; ticks handed back to the event loop |
| Daily canary | APScheduler `AsyncIOScheduler`, `CANARY_RUN_HOUR` UTC |
| Usage log | Queued by the middleware, written in batches by a background task; a response never waits for it |
| News refresh, canary run on demand | FastAPI background tasks |

## Guidelines for contributors

1. Write routes with `async def`.
2. Call a synchronous service with `await run_sync(service.method, ...)`; never call it directly from a coroutine.
3. Await asynchronous services (Redis asyncio, `httpx`, asyncpg) directly.
4. Keep one Gunicorn worker: the streams, the WebSocket clients and the canary live in the process memory.



<!-- FILE: /docs/providers/overview.md -->

---
id: "overview"
title: "Providers Architecture Overview"
sidebar_label: "Overview"
description: "How Fonrex queries its providers in parallel, chooses the search term, validates the values and shares one HTTP layer"
---

# Providers Architecture Overview

Fonrex collects data from public sources through **providers**: 14 for fundamentals, 4 specialised ones, 7 for news, plus Yahoo Finance and TradingView for prices. All of them run inside your instance, from your IP address (or your proxy).

## A `/fundamental` request

```
GET /fundamental?ticker=AIR.PA
      │
      ▼
GetFundamentals  ── listing, ISIN, mappings, verified Yahoo symbol
      │
      ▼
FinancialProviderRunner.run()  ── all providers in parallel (12 s each)
      │      ZoneBourse, GoogleFinance, Boursorama, Barrons, WSJ, MarketWatch,
      │      MorningStar, Investing, Gurufocus, Fortuneo, BourseDirect, MSN,
      │      InvestirLesEchos, YahooFinance  (+ SEC EDGAR insider data, 15 s max)
      ▼
ValidationLayer.validate_results()  ── range and consensus checks, outliers → None
      ▼
FinancialsFormatter.to_eodhd()  ── each figure: Yahoo → stored figures → scraped providers
      ▼
JSON document with a Sources section (cached 1 hour)
```

## The search term of each provider

The runner gives each provider the most reliable identifier it has, in this order:

1. For Yahoo Finance, the **symbol verified for the listing** (from the ISIN, quoted in the listing's currency). A listing without one is not sent to Yahoo — its bare ticker may be another instrument.
2. For Google Finance, the ticker built from the exchange of the listing (`EPA:AIR`); for Gurufocus, the ticker with its Yahoo suffix (`AIR.PA`).
3. An active `provider_url` mapping, then an active `provider_ticker` mapping.
4. The ISIN, for the providers that search by ISIN (ZoneBourse, Investing, WSJ, MarketWatch, Fortuneo, BourseDirect, Boursorama, Gurufocus, InvestirLesEchos).
5. The requested ticker.

The term used is reported per provider in `raw_providers` (`fmt=raw`).

## Resilience

- **Independent providers.** A provider that fails or times out returns an error entry; the others answer.
- **Homonyms rejected.** A scraped provider answering with another ISIN than the instrument's is reported as an error, not merged.
- **Validated values.** The [validation layer](../monitoring/validation-layer.md) discards out-of-range values and consensus outliers before the document is built.
- **One HTTP layer.** Every provider goes through `BaseFinancialProvider`: three attempts with growing pauses on network errors and 429/5xx, final failure on 401/403/404, at most `FONREX_PROVIDER_MAX_CONCURRENCY` simultaneous requests per provider, optional proxy (`FONREX_PROXY_URL`, limited to some providers with `FONREX_PROXY_PROVIDERS`).

## When a website refuses your requests

Websites protected by an anti-bot service increasingly refuse requests from a personal connection (typically `403`). The provider then reports an error and the others answer. Route those providers through an HTTP proxy of your choice:

```env
FONREX_PROXY_URL=http://user:password@proxy.example:8888
FONREX_PROXY_PROVIDERS=Investing,Gurufocus,wallStreetJournal
```

The proxy applies to the scraped websites, not to the `yfinance` and TradingView libraries.

## Related pages

- [Fundamentals providers](fundamentals-providers.md)
- [News providers](news-providers.md)
- [Adding a provider](../guides/adding-providers.md)



<!-- FILE: /docs/providers/fundamentals-providers.md -->

---
id: "fundamentals-providers"
title: "Fundamental Providers List"
sidebar_label: "Fundamentals Providers"
description: "The providers queried by /fundamental and the specialised providers"
---

# Fundamental Providers List

## Providers of `/fundamental`

These names are the keys of the answer and the values accepted by the `provider` parameter (any case).

| Provider | Source | Searched by | Region |
|---|---|---|---|
| `YahooFinance` | Yahoo Finance (`yfinance` library) | Verified symbol of the listing | Global |
| `ZoneBourse` | zonebourse.com | ISIN | Europe |
| `Boursorama` | boursorama.com | ISIN | France / Europe |
| `Fortuneo` | fortuneo.fr | ISIN | France |
| `BourseDirect` | boursedirect.fr | ISIN | France |
| `InvestirLesEchos` | investir.lesechos.fr | ISIN | France |
| `GoogleFinance` | google.com/finance | Ticker with the exchange of the listing (`EPA:AIR`) | Global |
| `Msn` | msn.com/money | Ticker | Global |
| `MorningStar` | morningstar | Ticker | Global |
| `Investing` | investing.com | ISIN | Global |
| `Barrons` | barrons.com | Ticker | US |
| `wallStreetJournal` | wsj.com | ISIN | US |
| `Marketwatch` | marketwatch.com | ISIN | US |
| `Gurufocus` | gurufocus.com | Ticker with the exchange suffix (`AIR.PA`), otherwise ISIN | Global |

An active mapping of the listing (`provider_url`, then `provider_ticker`) takes precedence over the ISIN or the ticker; MSN receives the ticker resolved from the ISIN when the request names an ISIN. Each provider returns a `FinancialMetrics` object (P/E, EPS, dividend yield, margins, revenue, net income, ESG score, Gurufocus scores…). Some publish displayed percentages: they are declared in `monitoring/units.py` and converted before validation.

In the rendered document, the Yahoo answer and the stored deep fundamentals come first; the scraped providers fill only the trailing P/E, the EPS and the dividend yield when those are missing (Google Finance, Barron's, MarketWatch, WSJ, Investing.com). Every provider answer remains available with `fmt=raw`.

Optional tokens: `BARRONS_TOKEN`, `MARKETWATCH_TOKEN`, `WSJ_TOKEN`.

## Specialised providers

| Provider | Source | Route |
|---|---|---|
| `SECEdgar` | SEC EDGAR, Form 4 | `/insider-transactions/{ticker}`, and the `InsiderTransactions` section of `/fundamental` for US shares |
| `JustETF` | justetf.com | `/etf/{isin}/details` |
| `IndexConstituents` | Wikipedia | `/index/{index_name}/constituents` |
| `OpenFIGI` | openfigi.com | Loaded, not used by any route today |

## Health

Each fundamentals provider is checked every day by the [canary monitor](../monitoring/canary-monitor.md); `GET /health/providers` shows the result. A provider that cannot be imported at start-up is listed in `providers.unavailable` of `GET /health`.



<!-- FILE: /docs/providers/news-providers.md -->

---
id: "news-providers"
title: "News Aggregator Providers"
sidebar_label: "News Providers"
description: "The 7 news providers, their mappings and the deduplication of articles"
---

# News Aggregator Providers

`NewsService` (`news/news_service.py`) asks the seven providers of `news/providers/` in parallel for `GET /news/{ticker}`.

| Module | Source | Method | Language |
|---|---|---|---|
| `yfinance_news.py` | Yahoo Finance | `yfinance` (`ticker.news`), ticker as typed | `en` |
| `google_finance_news.py` | Google Finance | Embedded JSON, HTML fallback | Deduced from the ticker suffix |
| `zonebourse_news.py` | ZoneBourse | HTML | `fr` |
| `boursorama_news.py` | Boursorama | HTML | `fr` |
| `investing_news.py` | Investing.com | HTML, needs a mapping | `en` |
| `marketwatch_news.py` | MarketWatch | HTML | `en` |
| `msn_finance_news.py` | MSN Finance | JSON endpoint, HTML fallback | `en` |

## Mappings

Three providers read a mapping of the instrument, by provider name in lower case: ZoneBourse (`zonebourse`) and Boursorama (`boursorama`) share the mapping of the fundamentals provider of the same name; Investing.com needs a mapping named `investing_com` and returns nothing without it. The four others build their request from the ticker.

## Deduplication

1. **URL** — lower case, `utm_*` parameters, fragment and trailing slash removed; the first article received is kept.
2. **Title** — `difflib.SequenceMatcher` on normalised titles (lower case, no punctuation); at or above `NEWS_DEDUP_SIMILARITY` (0.85) the most recent article is kept.

With `language`, articles whose language is known and different are removed before deduplication. Articles are sorted newest first and cut to `limit`; each provider is asked for twice that number.

## Storage

Articles of instruments that are in the catalogue are upserted into `news_articles` (`ON CONFLICT (url) DO UPDATE`); `GET /news/feed` and `GET /news/stats` read that table. Old articles are not purged automatically.

A failing provider returns an empty list without blocking the others.



<!-- FILE: /docs/providers/adding-custom-provider.md -->

---
id: "adding-custom-provider"
title: "Custom Provider Implementation Reference"
sidebar_label: "Adding Custom Provider"
description: "Skeleton of a fundamentals provider and the helpers of BaseFinancialProvider"
---

# Custom Provider Implementation Reference

The steps around a new provider (registration, units, canary, tests, coverage) are in the [guide](../guides/adding-providers.md). This page is the code.

## Skeleton

```python
"""MySite: fundamentals read from https://mysite.example."""

import logging
from typing import Optional

from selectolax.parser import HTMLParser

from financials.models import FinancialMetrics
from financials.numbers import parse_number
from financials.providers.base import BaseFinancialProvider

logger = logging.getLogger(__name__)


class MySiteProvider(BaseFinancialProvider):
    name = "MySite"
    timeout = 10.0

    SEARCH_URL = "https://mysite.example/api/search"
    BASE_URL = "https://mysite.example"

    async def get_financials(self, ticker: str) -> Optional[FinancialMetrics]:
        """``ticker`` is the search term chosen by the runner (mapping, ISIN or ticker)."""
        async with self._session() as client:  # cookies kept between the two requests
            found = await client.get(self.SEARCH_URL, params={"q": ticker})
            if found.status_code != 200 or not found.json().get("results"):
                return None
            path = found.json()["results"][0]["url"]
            page = await client.get(self.BASE_URL + path)
            if page.status_code != 200:
                return None
        metrics = self._parse_page(HTMLParser(page.text), ticker)
        metrics.provider_url = self.BASE_URL + path
        return metrics

    def _parse_page(self, parser: HTMLParser, ticker: str) -> FinancialMetrics:
        metrics = FinancialMetrics(ticker=ticker)
        for row in parser.css("table.key-figures tr"):
            label = row.css_first("th")
            value = row.css_first("td")
            if not label or not value:
                continue
            text = value.text(strip=True)
            match label.text(strip=True):
                case "P/E":
                    metrics.pe_ratio = parse_number(text, decimal=".")
                case "Dividend yield":
                    metrics.dividend_yield = parse_number(text, decimal=".")  # "3.45 %" -> 3.45
                case "ISIN":
                    metrics.isin = text
        return metrics
```

`dividend_yield` here is a displayed percentage (`3.45`): declare it for `MySite` in `monitoring/units.py`.

## `FinancialMetrics`

Fields of `financials/models.py`: `revenue`, `ebitda`, `net_income`, `eps`, `payout_ratio`, `dividend_yield`, `debt_to_equity`, `isin`, `pe_ratio`, `profit_margin`, `operating_margin`, `esg_score`, `risk_level`, `morningstar_rating`, `eligibility`, `piotroski_score`, `beneish_m_score`, `roic`, `gf_score`, `provider_url`, `ticker`.

## Helpers of `BaseFinancialProvider`

| Helper | Use |
|---|---|
| `await self._get(url, headers=, params=)` | Text of a page, or `None` |
| `await self._get_json(url, ...)` / `await self._post_json(url, ...)` | Decoded JSON, or `None` |
| `async with self._session() as client` | Several requests sharing cookies (`client.get`, `client.post`) |
| `self._get_headers(extra)` | Browser-like headers with a rotating User-Agent |
| `self._safe_float(value)`, `self._safe_int(value)` | Tolerant conversions |

Class attributes: `name`, `timeout` (seconds), `max_retries` (3), `retry_delay` (1 s, doubled at each attempt), `_semaphore` (a provider-specific concurrency limit).

Every request goes through the same policy: network errors and the statuses 429, 500, 502, 503, 504 are retried with a growing pause; 400, 401, 403, 404 and 410 are final; a `Retry-After` longer than 10 seconds is capped. At most `FONREX_PROVIDER_MAX_CONCURRENCY` requests (4) run at the same time per provider, and `FONREX_PROXY_URL` routes them through a proxy when set.

## Numbers

```python
from financials.numbers import find_number, parse_number

parse_number("1 234,5 M€", decimal=",")   # 1234500000.0
parse_number("(12.3)", decimal=".")      # -12.3  (accounting parentheses)
parse_number("80,95 Md", decimal=",")     # 80950000000.0
find_number("Dividend yield: 3.45 % (2025)")   # 3.45
```

`decimal` is the decimal separator of the page: `,` (the default of `parse_number`) for French pages, `.` for English ones. A text that is more than a number is rejected rather than half read.



<!-- FILE: /docs/monitoring/validation-layer.md -->

---
id: "validation-layer"
title: "Validation Layer Architecture"
sidebar_label: "Validation Layer"
description: "Range and consensus checks applied to every provider value before it reaches an answer"
---

# Validation Layer Architecture

Thirteen of the fundamentals providers read web pages. When a page changes, a provider may keep answering — with a wrong number (`0.8` instead of `24.0` for a P/E). The `ValidationLayer` (`monitoring/validation_layer.py`) catches such values on every `/fundamental` request, after the providers answered and before the document is built. A rejected value becomes `None`, and the document takes the figure from another source.

## 1. Unit normalisation

Ranges are ratios: a 3.45 % dividend yield is `0.0345`. Scraped providers often return displayed percentages (`3.45`). `monitoring/units.py` declares, per provider, the fields returned as percentages (`PROVIDER_PERCENT_FIELDS`); they are converted before any check. The answer of the provider keeps its own unit; the logs hold the converted ratio.

## 2. Range checks

| Field | Min | Max |
|---|---|---|
| `pe_ratio` | 0.5 | 1000 |
| `pe_forward` | 0.5 | 500 |
| `pb_ratio` | 0 | 100 |
| `ps_ratio` | 0 | 200 |
| `peg_ratio` | −10 | 50 |
| `ev_ebitda` | 0 | 500 |
| `price`, `target_price`, `week_52_high`, `week_52_low` | 0.001 | 1,000,000 |
| `dividend_yield` | 0 | 0.50 |
| `dividend_rate` | 0 | 1000 |
| `payout_ratio` | 0 | 10 |
| `roe` | −5 | 10 |
| `roa` | −2 | 2 |
| `net_margin`, `operating_margin` | −5 | 1 |
| `gross_margin` | −1 | 1 |
| `quarterly_revenue_growth_yoy` | −0.99 | 10 |
| `quarterly_earnings_growth_yoy` | −0.99 | 20 |
| `eps`, `eps_trailing`, `eps_forward` | −1000 | 10,000 |
| `beta` | −3 | 5 |
| `short_percent_float` | 0 | 1 |

A value outside its range is `out_of_range` and set to `None`.

## 3. Consensus check

When at least `VALIDATION_MIN_PROVIDERS` (2) providers give a value in range for the same field:

1. the median `M` of those values is computed;
2. each value `V` deviates by `|V − M| / M`;
3. above `VALIDATION_OUTLIER_THRESHOLD` (0.50), the value is an `outlier` and set to `None`.

## 4. Logging

Every checked value is written to the `provider_health_log` hypertable (30-day retention) with its status: `ok`, `out_of_range`, `outlier` or null. `GET /health/stats` summarises the last 7 days.

The validation layer never raises: an internal error is logged and the answer goes on unvalidated rather than failing.

## Settings

```env
VALIDATION_OUTLIER_THRESHOLD=0.50
VALIDATION_MIN_PROVIDERS=2
```



<!-- FILE: /docs/monitoring/canary-monitor.md -->

---
id: "canary-monitor"
title: "Canary Monitor Suite"
sidebar_label: "Canary Monitor"
description: "Daily check of every fundamentals provider against known assets and expected ranges"
---

# Canary Monitor Suite

The `CanaryMonitor` (`monitoring/canary_monitor.py`) asks every fundamentals provider, once a day, for a few well-known assets whose values are expected in known ranges. A provider whose page changed is detected even when no user asked for it.

## Schedule

- APScheduler (`AsyncIOScheduler`) inside the API process, every day at `CANARY_RUN_HOUR` UTC (6 by default).
- `POST /health/canary/run` starts a run at once (all providers, or `provider_name`).
- Providers are checked `CANARY_PROVIDER_SEMAPHORE` at a time (3); each call is limited to 15 seconds and the whole run to 120 seconds.

## Canary assets

| Ticker | Fields | Expected ranges |
|---|---|---|
| `AAPL` | P/E, dividend yield, beta, price | P/E 20–45, yield 0.003–0.01, beta 0.8–1.5, price 100–500 |
| `AIR.PA` | P/E, dividend yield, beta, price | P/E 15–60, yield 0.005–0.04, beta 0.8–1.8, price 80–300 |
| `BNP.PA` | P/E, dividend yield, P/B, price | P/E 4–15, yield 0.04–0.12, P/B 0.3–1.5, price 30–100 |
| `MSFT` | P/E, beta, price | P/E 25–50, beta 0.7–1.3, price 200–600 |
| `TSLA` | P/E, beta, price | P/E 30–300, beta 1.5–3.5, price 100–600 |

The price ranges above are a fallback. When the database holds at least ten daily closes of the last 90 days for the asset, the expected range is their mean ± 3 standard deviations, kept for `CANARY_PRICE_RANGE_TTL_SECONDS` (6 hours).

Providers that only cover Europe (Boursorama, Fortuneo, BourseDirect, InvestirLesEchos) are only checked on `AIR.PA` and `BNP.PA`. Percentages are converted to ratios as in the [validation layer](validation-layer.md).

## A run

```
06:00 UTC  ──►  CanaryMonitor.run_all()
                  ├─ for each provider (3 in parallel):
                  │     canary assets × expected fields → ok / out_of_range / null / timeout
                  │     results → provider_health_log
                  ├─ daily aggregate → provider_health_daily (counters, success rate, is_healthy)
                  ├─ alerts → provider_alerts (created, or canary_failed auto-resolved)
                  └─ summary → Redis provider:health:summary (1 hour) → GET /health/providers
```

## Reading the results

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" http://localhost:5000/health/providers
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/health/providers/ZoneBourse?days=30"
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/health/canary/history?provider_name=ZoneBourse"
```

## Settings

```env
CANARY_RUN_HOUR=6
CANARY_PROVIDER_SEMAPHORE=3
CANARY_PRICE_RANGE_TTL_SECONDS=21600
CANARY_PRICE_RANGE_NEGATIVE_TTL_SECONDS=300
```

The canary runs in the API process: with several Gunicorn workers, each one would run it.



<!-- FILE: /docs/monitoring/alerts.md -->

---
id: "alerts"
title: "Provider Alerts & Resolution"
sidebar_label: "Alerts & Resolution"
description: "The alerts raised by the canary, their severity, automatic and manual resolution"
---

# Provider Alerts & Resolution

The canary run creates alerts in `provider_alerts`. An alert of a given type is not duplicated while one is active for the same provider.

## Alert types

| Type | Raised when | Severity | Resolution |
|---|---|---|---|
| `canary_failed` | A canary value is out of its expected range (or an outlier) | `critical` from `ALERT_CANARY_CRITICAL` failures in the run (3), `warning` below | Automatic when every check of a later run passes |
| `high_outlier_rate` | The success rate of the provider in the run is below `ALERT_SUCCESS_RATE_WARNING` (0.85) | `critical` below `ALERT_SUCCESS_RATE_CRITICAL` (0.70), `warning` otherwise | Manual |

`consecutive_nulls` and `latency_spike` exist in the schema, but no code raises them today.

An alert records the provider, the ticker and field of the first failure, the value received and the expected range.

## Listing alerts

```bash
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/health/alerts?severity=critical"
curl -s -H "X-API-KEY: $FONREX_API_KEY" "http://localhost:5000/health/alerts?provider_name=Investing&include_resolved=true"
```

## Resolving an alert by hand

The note is a query parameter; a full-access key is required.

```bash
curl -s -X POST -H "X-API-KEY: $FONREX_API_KEY" \
  "http://localhost:5000/health/alerts/42/resolve?resolution_note=Parser%20updated"
```

## What to do with an alert

1. Look at the failing values: `GET /health/canary/history?provider_name=<name>`.
2. Ask the provider directly: `GET /fundamental?ticker=AIR.PA&provider=<name>&fmt=raw&nocache=true`.
3. A `403` or an empty answer usually means the website refuses your connection: route the provider through a proxy (`FONREX_PROXY_URL`, `FONREX_PROXY_PROVIDERS`).
4. A wrong number usually means the page changed: the parser of the provider needs an update ([adding providers](../guides/adding-providers.md) describes the tests with saved pages).
5. Meanwhile, the validation layer keeps the provider's suspicious values out of the answers.

## Settings

```env
ALERT_CANARY_CRITICAL=3
ALERT_SUCCESS_RATE_WARNING=0.85
ALERT_SUCCESS_RATE_CRITICAL=0.70
```



<!-- FILE: /docs/deployment/docker.md -->

---
id: "docker"
title: "Docker Production Deployment"
sidebar_label: "Docker Deployment"
description: "Run Fonrex on a server: Compose overrides, reverse proxy with TLS and WebSocket, what to expose"
---

# Docker Production Deployment

Fonrex is a self-hosted application for your own use. On a server, run the same `docker-compose.yml` as locally and put a reverse proxy with TLS in front of the API.

## What is exposed

| Service | Published on | Reach it from |
|---|---|---|
| `fonrex-api` | `0.0.0.0:5000` | The reverse proxy only — bind it to `127.0.0.1` on a server (override below) |
| `db` | `127.0.0.1:5432` | The host only |
| `redis` | `127.0.0.1:6379` | The host only (Redis has no password) |

Every API route except `/health`, the documentation, the OpenBB discovery files and `/static` requires a key. Give clients outside the machine (dashboards, Google Sheets) a **read-only** key.

## Compose override

Create `docker-compose.prod.yml`:

```yaml
services:
  fonrex-api:
    ports: !override
      - "127.0.0.1:5000:5000"
    deploy:
      resources:
        limits:
          memory: 4g

  db:
    deploy:
      resources:
        limits:
          memory: 4g
```

```bash
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
```

`!override` replaces the port list instead of adding to it; it needs Docker Compose 2.24 or later.

Keep `WEB_CONCURRENCY=1`: the realtime streams, the WebSocket clients and the daily canary live in the API process, and each extra worker would duplicate them.

## Reverse proxy (NGINX)

```nginx
server {
    listen 443 ssl http2;
    server_name fonrex.example.com;

    ssl_certificate     /etc/letsencrypt/live/fonrex.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/fonrex.example.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:5000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 120s;
    }

    location /ws/ {
        proxy_pass http://127.0.0.1:5000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;
        proxy_read_timeout 3600s;
    }
}
```

Gunicorn runs with a 120-second timeout: a first ingestion or a `/fundamental` request querying every provider can take several seconds.

Without a server, a tunnel (zrok, Cloudflare Tunnel, Tailscale Funnel) gives a local instance an HTTPS URL — see the [Google Sheets guide](../guides/google-sheets-connector.md).

## Outbound requests

Your server's IP address makes the requests to the data sources. Websites protected by an anti-bot service may refuse a datacenter IP; route those providers through a proxy:

```env
FONREX_PROXY_URL=http://user:password@proxy.example:8888
FONREX_PROXY_PROVIDERS=Investing,Gurufocus,wallStreetJournal
```

## Data and backups

The database is in the `timescale_data` volume. Back it up with `pg_dump` (see [Docker Compose](../getting-started/docker-compose.md#backing-up-the-database)) before every upgrade, and keep dumps out of Git.



<!-- FILE: /docs/deployment/environment-variables.md -->

---
id: "environment-variables"
title: "Environment Variables Deployment Reference"
sidebar_label: "Environment Variables"
description: "The settings that matter when deploying Fonrex, and how .env reaches the containers"
---

# Environment Variables Deployment Reference

Every setting is described in [Configuration](../getting-started/configuration.md). This page covers what matters for a deployment.

## How `.env` reaches the containers

- `docker-compose.yml` loads `.env` into the API container (`env_file`).
- It then **overrides** the service addresses written for a local run: `DATABASE_URL` (built from `POSTGRES_PASSWORD`, host `db`), `REDIS_URL` (host `redis`) and `ASYNC_DATABASE_URL` (emptied, so it is derived from `DATABASE_URL`). It also sets `WEB_CONCURRENCY` and clears `HTTP_PROXY` / `HTTPS_PROXY`.
- `.env` is never copied into the image.
- One `KEY=value` per line; a comment after an empty value is read as the value.

Every variable of `.env.example` is read by the code (`tests/test_env_settings.py`); an invalid value falls back to its default with a warning instead of stopping the API.

## Must be set

| Variable | Why |
|---|---|
| `FONREX_API_KEY` | Without a key, every protected route answers `401` |
| `FONREX_READ_ONLY_API_KEYS` | Keys for clients outside the machine (Sheets, dashboards) |
| `POSTGRES_PASSWORD` | Before the first start; stored in the volume at initialisation |
| `SEC_EDGAR_EMAIL` | Your contact address: the SEC refuses anonymous automated clients |

## Never in production

| Setting | Why |
|---|---|
| `FONREX_AUTH_REQUIRED=false` | Opens every route, including cache and database administration (only effective when no key is configured) |
| `WEB_CONCURRENCY` > 1 | Duplicates realtime streams and the daily canary |
| `USAGE_LOG_IP=full` | Keeps full caller IP addresses in `usage_logs`; prefer `none` or `truncated` |

## Often adjusted

| Variable | Default | When |
|---|---|---|
| `FRED_API_KEY` | *(empty)* | Live risk-free rate for the DCF |
| `FONREX_PROXY_URL`, `FONREX_PROXY_PROVIDERS` | *(empty)* | Websites refusing your server's IP |
| `FONREX_PROVIDER_MAX_CONCURRENCY` | `4` | Fewer simultaneous requests per website |
| `OPENBB_ALLOWED_ORIGIN` | `https://pro.openbb.co` | Another OpenBB origin (CORS) |
| `USAGE_LOG_RETENTION_DAYS` | `90` | Usage log retention |
| `CANARY_RUN_HOUR` | `6` | Hour (UTC) of the daily provider check |



<!-- FILE: /docs/deployment/database-migrations.md -->

---
id: "database-migrations"
title: "Database Migrations in Production"
sidebar_label: "Database Migrations"
description: "Upgrade an instance safely: back up, migrate, check, and restore if needed"
---

# Database Migrations in Production

The API container applies `alembic upgrade head` at every start. An upgrade of Fonrex is therefore an upgrade of the schema: prepare it.

## Upgrade procedure

1. **Back up the database**:

   ```bash
   docker compose exec -T db pg_dump -U fonrex -d fonrex -Fc > fonrex-$(date +%F).dump
   ```

2. **Update the code**: `git pull`.
3. **Migrate alone** (optional, to see the migrations run before the API starts):

   ```bash
   docker compose --profile migrate build fonrex-migrate
   docker compose --profile migrate run --rm fonrex-migrate
   ```

4. **Start the new version**: `docker compose up -d --build`.
5. **Check**: `docker compose logs fonrex-api` shows the migrations applied, then `curl http://localhost:5000/health` and a few requests with your key. A database left behind the code makes its routes answer `503`.

## Rolling back

Restore the backup taken in step 1 with the previous version of the code (see [Docker Compose](../getting-started/docker-compose.md#backing-up-the-database)). Prefer it to `alembic downgrade`: some downgrades cannot give back what the upgrade removed (migration 008 drops tables; migration 014 keeps one bar per instrument and date when going back).

## Notable migrations

| Migration | What to know |
|---|---|
| 014 — prices per listing | Rebuilds `prices_eod` with one series per listing and re-dates the existing bars to their trading session. Runs by itself; nothing is downloaded again. If a series looks wrong afterwards: `POST /historical/ingest?ticker=<ticker>&force_refresh=true` |
| 015 — dividend yields as ratios | Converts stored dividend yields from percentages to ratios |
| 016 — adjustment of price series | Creates `price_series_adjustments`. Prices are not touched: each series stored before is fetched again in full at its next ingestion, with the traded close in `close` and the dividend-adjusted close in `adj_close`. To do it at once: `docker compose exec fonrex-api python scripts/ingest_all.py --force` |

The full list is in [Schema migrations](../architecture/migrations.md).

## Testing migrations

The database tests apply the migrations to a real TimescaleDB, on existing data, down and up again:

```bash
make test-db     # throwaway container of the image of docker-compose.yml, port 54329
```

To run them against your own server (a temporary database is created and dropped):

```bash
FONREX_TEST_DATABASE_URL=postgresql://fonrex:<password>@localhost:5432/fonrex \
    pytest tests/test_timescale_integration.py
```



<!-- FILE: /docs/deployment/production-checklist.md -->

---
id: "production-checklist"
title: "Production Deployment Checklist"
sidebar_label: "Production Checklist"
description: "Security, configuration, monitoring and backup checks before exposing an instance"
---

# Production Deployment Checklist

## Security
- [ ] `FONREX_API_KEY` set to a random key (`echo "frx_live_$(openssl rand -hex 24)"`); `FONREX_AUTH_REQUIRED` left at `true`.
- [ ] Read-only keys (`FONREX_READ_ONLY_API_KEYS`) for every client outside the machine: Google Sheets, dashboards, OpenBB.
- [ ] `POSTGRES_PASSWORD` changed before the first start.
- [ ] PostgreSQL and Redis published on `127.0.0.1` only (the default); the API reachable only through the reverse proxy.
- [ ] TLS on the reverse proxy, WebSocket upgrade forwarded for `/ws/`.
- [ ] `USAGE_LOG_IP` at `none` or `truncated`.
- [ ] `.env` and database dumps never committed.

## Configuration
- [ ] `SEC_EDGAR_EMAIL` set to your own contact address.
- [ ] `WEB_CONCURRENCY=1`.
- [ ] `FRED_API_KEY` set if you use the DCF.
- [ ] A proxy (`FONREX_PROXY_URL`) for the websites that refuse your server's IP, if needed.

## Data
- [ ] Instruments imported (`import_assets.py`) and prices ingested (`scripts/ingest_all.py`).
- [ ] `POST /database/cleanup` never run with the default `days_to_keep` (730) unless you mean to delete eight of the ten ingested years — count first with `dry_run`.
- [ ] Daily `pg_dump` backup of the whole database, restore tested once.

## Monitoring
- [ ] `/health` answers, `providers.unavailable` is empty.
- [ ] `/health/providers` filled after the first canary run (06:00 UTC by default).
- [ ] Critical alerts checked regularly: `GET /health/alerts?severity=critical`.
- [ ] Container health check green: `docker compose ps`.



<!-- FILE: /docs/contributing/setup.md -->

---
id: "setup"
title: "Development Environment Setup"
sidebar_label: "Dev Setup"
description: "Run Fonrex from source with locked dependencies, a local database and the quality gate"
---

# Development Environment Setup

## Prerequisites

- Python 3.12
- Docker and Docker Compose (database and Redis)
- Git, and `make`

## 1. Clone and create a virtual environment

```bash
git clone https://github.com/fonrex/fonrex.git
cd fonrex
python3.12 -m venv venv
source venv/bin/activate          # Windows: venv\Scripts\activate
```

## 2. Install the locked dependencies

```bash
make install-dev
```

This installs `requirements-dev.lock`: exact versions, each checked against its hash — the same packages as the Docker image and the CI. After editing `requirements*.txt`, refresh the locks with `make lock` (needs `uv`); never edit the `.lock` files by hand.

## 3. Start the database and Redis

```bash
cp .env.example .env              # set FONREX_API_KEY
docker compose up -d db redis
```

Both are published on `127.0.0.1`, so the `localhost` addresses of `.env.example` reach them.

## 4. Migrate and run the API

```bash
alembic upgrade head
make run                          # uvicorn --reload on port 5000, loads .env
```

The interactive documentation is at `http://localhost:5000/docs`.

To run your working copy inside Docker instead, without rebuilding at each change:

```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up
```

## 5. Run the quality gate

```bash
make ci
```

See [Testing](testing.md). Before opening a pull request, read [Architecture rules](architecture-rules.md): each rule is held by a test. Pull request titles follow Conventional Commits (`feat`, `fix`, `docs`, `chore`, `refactor`, `perf`, `test`), subject in lower case.



<!-- FILE: /docs/contributing/testing.md -->

---
id: "testing"
title: "Testing Guidelines & Execution"
sidebar_label: "Testing Guidelines"
description: "The quality gate, the database tests and how to test providers without network"
---

# Testing Guidelines & Execution

## The quality gate

```bash
make ci
```

`make ci` is what GitHub Actions runs on every pull request and push to `main`:

| Step | Command | Blocks |
|---|---|---|
| Lint | `make lint` (Ruff) | Strict Ruff violations |
| Annotations | `make typecheck` | Untyped boundaries of the listed modules |
| Syntax | `make syntax` | Python files that do not compile |
| Migrations | `make migration-check` | More than one Alembic head |
| Tests and coverage | `make test-cov` | Failing tests, warnings (`PYTHONWARNINGS=error`), global coverage under 70 %, a module under its own floor |

Coverage floors are listed per module in `scripts/check_coverage_distribution.py`, including every data provider; a provider without a floor fails the gate. Floors only go up.

## Running tests

```bash
PYTHONPATH=. pytest                                   # everything
PYTHONPATH=. pytest tests/test_technical_indicators.py -v
PYTHONPATH=. pytest -k "cache_key"
```

The suite never talks to a real service: `tests/conftest.py` clears the credentials of your shell, points Redis and the database to unreachable addresses and makes any real Yahoo lookup fail.

## Database tests

`tests/test_timescale_integration.py` runs the real migrations on a real TimescaleDB: migration of existing prices, compressed hypertable, upserts, cleanup, downgrade and upgrade, and a migration that waits for a TimescaleDB job. Without a server they are skipped.

```bash
make test-db     # throwaway container of the image of docker-compose.yml, port 54329
```

Or against any TimescaleDB server — each run creates and drops its own database:

```bash
FONREX_TEST_DATABASE_URL=postgresql://user:password@127.0.0.1:5432/postgres make ci
```

The CI runs them on a `timescaledb` service of the same image as `docker-compose.yml`.

## Writing tests

- **No real network.** Provider tests use the `fake_network` fixture (`tests/conftest.py`): every `httpx` request is answered by a canned response, and a request without one fails the test.
- **Real pages.** Parsers are tested on reduced copies of real pages in `tests/fixtures/providers/`.
- **Cache keys.** A route that gains a parameter gets a test with two requests differing by that parameter only (`tests/test_cache_keys.py`).
- **Statements by fiscal year.** Test a calculation on rows stored the way the enrichment stores them: three rows per fiscal year.
- **A test must fail without the fix.** Check it by reverting the change once.

Some guards read the documents: `tests/test_docs_consistency.py` compares the tables of `ARCHITECTURE.md` (routes, migrations, modules, cache lifetimes, canary assets) and the figures of `README.md` with the code.



<!-- FILE: /docs/contributing/architecture-rules.md -->

---
id: "architecture-rules"
title: "Architecture Rules & Guidelines"
sidebar_label: "Architecture Rules"
description: "The contribution rules of AGENTS.md and the test that holds each one"
---

# Architecture Rules & Guidelines

`AGENTS.md`, at the root of the repository, is the contract for every contribution, human or automated. Each rule is held by a test: breaking one fails `make ci`.

| # | Rule | Held by |
|---|---|---|
| 1 | **One HTTP layer.** A provider never creates an HTTP client; it uses the helpers of `financials/providers/base.py`, where retries, pauses, concurrency limit and proxy live | `tests/test_provider_http_policy.py` |
| 2 | **Settings are real.** Read settings with the helpers of `settings.py`; every variable of `.env.example` is read by the code; an invalid value falls back with a warning | `tests/test_env_settings.py` |
| 3 | **Documents tell the truth.** Figures of `README.md` match the code; the tables of `ARCHITECTURE.md` list every route, migration and module | `tests/test_docs_consistency.py` |
| 4 | **Providers load, or the failure is visible** (`PROVIDER_SPECS`, `/health`) | `tests/test_docs_consistency.py` |
| 5 | **Units are declared** in `monitoring/units.py` | `tests/test_provider_units.py` |
| 6 | **Coverage floors only go up**, one per provider | `tests/test_coverage_gate.py` |
| 7 | **Secure by default.** Every route needs a key unless declared public; a route that changes something is never a `GET` | `tests/test_auth_defaults.py` |
| 8 | **The Docker image is self-contained**; `.env` never goes into it | `tests/test_docker_image.py` |
| 9 | **The usage log never delays a response**; no IP stored unless asked | `tests/test_usage_recorder.py` |
| 10 | **No real network in tests** (`fake_network`, saved pages) | `tests/conftest.py` |
| 11 | **Versions are locked** (`requirements*.lock`, hash-checked) | `tests/test_dependency_lock.py` |
| 12 | **Prices belong to a listing**: key `(asset_listing_id, resolution, time)`, session date at midnight UTC, tickers resolved by `database/price_series.py` | `tests/test_price_series.py` |
| 13 | **Source symbols are verified, never guessed** (`historical/yahoo_symbols.py`) | `tests/test_yahoo_symbols.py` |
| 14 | **Displayed numbers are read in one place** (`financials/numbers.py`) | `tests/test_numbers.py` |
| 15 | **A rendered figure names its source** (`Sources` of `/fundamental`) | `tests/test_financials_formatter.py` |
| 16 | **Nothing read from Redis is executed, nothing is deleted without bounds** (JSON cache, bounded cleanup with `dry_run`) | `tests/test_cache_service.py`, `tests/test_database_cleanup.py` |
| 17 | **The CI tests on the database of an installation** (same TimescaleDB image) | `tests/test_ci_workflow.py` |
| 18 | **Statements are read by fiscal year** (`financials/fiscal_years.py`) | `tests/test_fiscal_years.py` |
| 19 | **A cache key holds every parameter that changes the answer** | `tests/test_cache_keys.py` |

## Layering

- `routers/` parse the request and translate application errors into HTTP statuses.
- `use_cases/` depend on the ports of `use_cases/ports.py`, never on FastAPI, SQLAlchemy or a provider.
- Blocking code is called through `concurrency.run_sync()`.

See [Layers & ports](../architecture/hexagonal.md).

## Identity

- Never take a ticker for a global identifier: `SPFF` is a bond ETF in EUR in a catalogue and a US fund on Yahoo. Resolve a listing (ticker, exchange, currency) and use the symbol verified for it.
- Never mask a failure with a fallback value or a silent `except`: report why something is missing (`reason`, `note`, `warnings`).



<!-- FILE: /docs/contributing/changelog.md -->

---
id: "changelog"
title: "Fonrex Version Changelog"
sidebar_label: "Changelog"
description: "Project history, feature additions, schema migrations, and version updates"
---

# Fonrex Version Changelog

## Next release

### Prices
- **`close` is the traded close again**, adjusted for splits only; **`adj_close`** is adjusted for splits and dividends. Before, both held the dividend-adjusted price.
- **One adjustment per series.** A split or a dividend after the last ingestion used to leave a false return where the stored and the new bars met (about minus the dividend yield, -75 % after a 4-for-1 split). The ingestion now compares the last stored bars with the source and fetches the whole series again when they differ. Migration 016 adds `price_series_adjustments`; series stored before are fetched again at their next ingestion (`scripts/ingest_all.py --force` for all at once).
- **`isin` parameter** on `GET /eod/{ticker}`, `GET /ticker/{symbol}/history` and `POST /historical/ingest`: names the instrument when several share a ticker. The answers give the `listing` they read.

## October 2026 — Secure defaults, per-listing prices, verified provider data

Merged on `main` on 8 October 2026 (pull request #15).

### ⚠️ Breaking changes
- **An API key is required by default.** Every route except `/health`, the documentation, `/widgets.json`, `/apps.json` and `/static` answers `401` until `FONREX_API_KEY` is set. `FONREX_AUTH_REQUIRED=false` only opens an instance with no key configured. New **read-only keys** (`FONREX_READ_ONLY_API_KEYS`) for clients outside the machine.
- **`GET /quote` and `GET /openbb/quote` no longer start a realtime stream.** Use `POST /realtime/subscribe`; `subscribe_if_missing=true` remains on `/quote` for full-access keys.
- **Prices are stored per listing** (migration 014): `prices_eod` is keyed by `(asset_listing_id, resolution, time)` and dated by trading session. The migration converts existing rows; back up before upgrading.
- **Dividend yields are ratios** everywhere, including stored values (migration 015).

### Security and operations
- Docker Compose loads `.env` and overrides the service addresses; PostgreSQL and Redis published on `127.0.0.1` only; database volume mounted on the right data directory.
- Usage log written in background batches, without the caller's IP by default (`USAGE_LOG_IP`), purged after `USAGE_LOG_RETENTION_DAYS`.
- `POST /database/cleanup` bounded (`days_to_keep` ≥ 30) with a `dry_run`.
- Redis cache entries are JSON only.

### Data quality
- Prices and fundamentals of a listing are fetched with a **Yahoo symbol verified** from the ISIN and the currency of the listing; a listing without one is not ingested, and the answer says why.
- Providers read the figures their pages really display (`financials/numbers.py`); percentages are normalised before validation; an answer about another ISIN is rejected.
- `/fundamental` builds each figure from Yahoo, then the stored figures, then the scraped providers, and names the source in a `Sources` section.
- Financial statements are read by fiscal year (DCF, solvency ratios).
- Every request parameter is part of its cache key (fundamentals, news, insider transactions, technical indicators per listing).
- The stored risk-free rate is refreshed from FRED; each realtime tick reaches each WebSocket client once.
- Migration 014 waits for running TimescaleDB jobs instead of deadlocking with them.

### Quality
- Locked, hash-checked dependencies; a coverage floor per module (global 70 %); database tests on TimescaleDB in the CI; guards keeping `ARCHITECTURE.md` and `AGENTS.md` in step with the code.


## v1.6.0 (2026-09)

### Major Features
- **OpenBB Workspace Integration**: Native backend adapter router (`/openbb`) supporting 19 interactive widgets and 2 pre-assembled application dashboards (`Fonrex — EU Markets` & `Fonrex — Screener & Macro`).
- **Dual Header Authentication**: Added support for OpenBB's native `X-API-KEY` custom header alongside standard `Authorization: Bearer` token validation (`auth/dependencies.py`).
- **Plotly & AgGrid Adapters**: Standardized data transformations for Plotly figures (candlestick charts, technical indicator overlays) and AgGrid tables (deep fundamentals, DCF sensitivity matrix, news, index constituents).

### 🐛 Bug Fixes & Improvements
- **Docker Volume Logging**: Solved volume permission issues on host environments by ensuring `mkdir -p logs` setup step and added permission troubleshooting guides.
- **DCF & Provider Updates**: Improved caching and data normalization for DCF valuation models and index constituent providers.

## v2.0.0 (2026-08)


### Major Features
- **Docusaurus v3 Documentation Suite**: Complete technical documentation structure generated under `documentation/`.
- **Provider Health Monitoring (Phase 12)**: Implemented `ValidationLayer`, `CanaryMonitor`, `provider_health_log` TimescaleDB hypertable, daily consensus aggregation, and 7 REST health endpoints (`/health/*`).
- **Valuation & DCF Engine (Phase 11)**: Integrated FCF, EPS, and DDM intrinsic value models with dynamic WACC calculation and sensitivity matrices (`/dcf/*`).
- **News Aggregator (Phase 10)**: Multi-provider scraping engine (`NewsService`) across 7 sources with `ON CONFLICT (url)` deduplication and Redis caching (`/news/*`).
- **ISIN Asset Architecture (Phase 9)**: Refactored asset database schema into `assets`, `asset_listings`, and `asset_mappings` with partial unique ISIN index.

### 🐛 Bug Fixes & Refactoring
- Legacy codebase cleanup (`eod/`, `record/`, `seed_assets.py` purged).
- Standalone ISIN deduplication tool `scripts/clean_isin_duplicates.py`.
- Thread pool executor concurrency boundaries via `concurrency.py`.