Agrégateur d'Actualités Providers
The NewsService module aggregates news articles from 7 scraping connectors:
Matrice des Scrapers d'Actualités
| Scraper Module | Provider Name | Method | Language | Deduplication Key |
|---|---|---|---|---|
yfinance_news.py | Yahoo Finance | JSON API (ticker.news) | en | URL |
google_finance_news.py | Google Finance | HTML / Embedded JSON | en | URL |
zonebourse_news.py | ZoneBourse | HTML BeautifulSoup | fr | URL |
boursorama_news.py | Boursorama | HTML BeautifulSoup | fr | URL |
investing_news.py | Investing.com | HTML Scraper | en | URL |
marketwatch_news.py | MarketWatch | HTML Scraper | en | URL |
msn_finance_news.py | MSN Finance | JSON API Endpoint | en / fr | URL |
Moteur de Déduplication d'Actualités
To eliminate cross-posted articles across multiple news outlets, Fonrex enforces a two-tier deduplication algorithm:
- Exact URL Deduplication: Database unique constraint
ON CONFLICT (url) DO UPDATEin tablenews_articles. - Title Similarity Matching: Before inserting new articles into the database feed, Fonrex compares article titles using Python's
difflib.SequenceMatcher. Titles with a similarity ratio exceedingNEWS_DEDUP_SIMILARITY(default0.85) are merged under the earliest publication timestamp.