Metadata-Version: 2.5
Name: bearishpy
Version: 0.56.0
Summary: Query financial and price data from various sources and persist it into a well-structured SQLite database for further analysis.
Author-email: aan <andoludovic.andriamamonjy@gmail.com>
License: MIT
License-File: LICENSE
Requires-Python: <3.13,>=3.11
Requires-Dist: alembic<2,>=1.14.0
Requires-Dist: html5lib<2,>=1.1
Requires-Dist: numpy<3,>=2.2
Requires-Dist: openpyxl>=3.1.5
Requires-Dist: pandas<4,>=2.1.4
Requires-Dist: pydantic<3,>=2.5.3
Requires-Dist: pypdf>=6.21.2
Requires-Dist: python-dotenv<2,>=1.0.1
Requires-Dist: requests>=2.31
Requires-Dist: rich>=13
Requires-Dist: sec-cik-mapper<3,>=2.1.0
Requires-Dist: sec-edgar-downloader<6,>=5.0.3
Requires-Dist: setuptools>=78.1.0
Requires-Dist: sqlmodel<0.1,>=0.0.22
Requires-Dist: tenacity<10,>=9.0.0
Requires-Dist: typer<1,>=0.13.1
Requires-Dist: xlrd>=2.0.2
Requires-Dist: yahooquery<3,>=2.4.1
Requires-Dist: yfinance<2,>=0.2.66
Description-Content-Type: text/markdown

# 🐻 Bearish

**Bearish** is a package for **querying financial and price data** from various equities across different countries, and **persisting it into a well-structured SQLite database** for further data analysis and stock screening.

---

## 🎯 Use Case

This package is intended for **active retail investors** with a strong passion for **data analysis**, who require well-structured, clean, and large datasets locally to:

- Run **custom technical and fundamental analysis**
- Perform in-depth **data analysis**
- Identify patterns and investment **insights**
- Build **custom screeners**

Bearish is **not designed for real-time trading**, but for extensive analysis of historical and current financial data.

---

## 🌍 Global Scope

Bearish is designed to collect large-scale data from exchanges in different countries. It **fetches data politely** — meaning it does not make large-scale concurrent API calls and instead respects the rate limits and policies of each data provider. So, **patience is required** during large data retrieval.

The data fetched includes:

- Historical **price data**
- Company **fundamentals**
- **Balance sheets**, **income statements**, and more

All data is stored in a **local SQLite database** from which you can query, analyze, and build insights.

---

## 📊 Data Sources

Bearish pulls data from multiple sources:

- [📦 FinanceDatabase](https://github.com/JerBouma/FinanceDatabase) for basic ticker information
- [🌐 InvestPy](https://github.com/alvarobartt/investpy) for country-level equity listings
- [📉 yFinance](https://pypi.org/project/yfinance/) as the primary source of prices and fundamentals
- [🔎 YahooQuery](https://pypi.org/project/yahooquery/) as a fallback for prices and fundamentals
- [📈 Tiingo](https://www.tiingo.com/) for prices (API key required)
- [🏛️ SEC EDGAR](https://www.sec.gov/) for 13F institutional holdings

> Ticker information is **enriched** using additional sources like yFinance, YahooQuery, etc.  
> Bearish is also **extensible** — you can add support for any data source or API.
>
> ℹ️ Support for **Financial Modeling Prep (FMP)** was removed: FMP discontinued
> its legacy API endpoints for subscriptions created after August 31, 2025.

By default, Bearish relies primarily on **yFinance** due to limitations in free-tier APIs of other providers. However, if you have a **premium subscription**, you can use other sources more fully.

---

## 📥 Installation

Install Bearish with pip:

```bash
pip install bearishpy
```

---

## 🚀 Fetch & Store Data

### 🏛️ Country-Level Data

Fetch and store stock data for selected countries:

```bash
bearish run /path/to/sqlite/db Belgium France --api-keys=config.json
```

✅ This command:

- Loads tickers from FinanceDatabase and InvestPy
- Filters relevant equities from the selected countries
- Enriches the data with fundamentals and prices
- Stores everything in your local SQLite database

> ⏱️ This operation can take some time depending on the size of the country’s exchange — data is fetched "politely", not in bulk.

Once your database is populated, future updates are quicker.

![img.png](docs/img/img.png)

---


The `config.json` contains the API keys of the different providers (if needed):

```json
{
  "Tiingo": "your Tiingo API key"
}
```

---

## 🗂️ Coverage census

Before judging a stock it helps to know whether the database can support
the judgement. `bearish coverage` (also run at the end of `bearish run` and
`bearish update`) writes one row per `(symbol, table_name, source)` into
**`coverage`** with the first date, last date and row count of every dated
table (prices, statements, earnings, analyst rows, filings, insiders, 13F
summaries, 13D/G, snapshots, index membership). Grading it ("enough years
of statements?", "prices fresh?") is downstream work.

```bash
bearish coverage /path/to/sqlite/db              # full census
bearish coverage /path/to/sqlite/db NVDA AAPL    # refresh two symbols
```

---

## 🧩 Index constituents

`bearish members` stores who is in the major indices (S&P 500, Nasdaq-100,
Dow Jones, DAX, CAC 40, BEL 20, AEX, FTSE 100, Euro Stoxx 50, SMI, IBEX 35,
FTSE MIB, Nikkei 225) from the Wikipedia constituent tables into
**`indexmember`** as a dated snapshot `(index_symbol, symbol, as_of)`, with
symbols in Yahoo Finance notation (`BRK-B`, `ADS.DE`, `III.L`). Breadth
measures and ETF tracking need this; `bearish run` and `bearish update`
refresh it with the index prices.

```bash
bearish members /path/to/sqlite/db                 # every configured index
bearish members /path/to/sqlite/db ^GSPC ^GDAXI    # a subset
```

---

## 💱 FX rates

Assets are priced in their listing currency. `bearish fx` stores the daily
Yahoo Finance quotes of every `USD<CCY>=X` pair for the currencies found in
the `equity`, `etf` and `index` tables (plus a core set: EUR, GBP, JPY, CHF,
CAD, AUD, CNY, ...) in **`pricecurrency`**; the **`fxrate`** view reads them
as `(date, base, quote, rate)` with `rate` = units of `quote` per one `base`.
Any cross rate is the ratio of two USD pairs. `bearish run` and
`bearish update` include this step (`--no-fx` to skip).

```bash
bearish fx /path/to/sqlite/db            # every currency in use
bearish fx /path/to/sqlite/db EUR JPY    # a subset
```

---

## 📸 Daily snapshots of asset fields

The `equity` and `etf` tables are overwritten on every detailed fetch, so
Bearish also appends a dated copy of their fast-moving fields on each run:

- **`equitysnapshot`** — per `(symbol, source, as_of)`: market capitalisation,
  shares outstanding, float, **short interest** (`short_shares`,
  `short_ratio`), valuation ratios (P/E, P/B, P/S, PEG, dividend yield),
  margins and growth as the provider reports them that day.
- **`etfsnapshot`** — per `(symbol, source, as_of)`: **assets under
  management** (`total_net_assets`), `nav_price`, expense ratio, turnover,
  valuation ratios, top holdings and sector weights.

Nothing is derived; fund flows, premium / discount to NAV, short-interest and
valuation trends are downstream calculations on these histories.

---

## 🧑‍💼 Analyst data & filings

Alongside statements and prices, Bearish records raw analyst data from
yFinance on every financials run:

- **`pricetarget`** — dated snapshots of analyst price targets
  (mean/high/low/median, number of analysts, recommendation mean/key).
  One row per fetch date, so a history accumulates across runs.
- **`analystrecommendation`** — the provider's recommendation trend
  (strong buy / buy / hold / sell / strong sell counts) per relative period
  bucket (`0m`, `-1m`, …), snapshotted per fetch date.

For US tickers, SEC EDGAR filing metadata can be fetched into the
**`filing`** table (form type, period of report, filed date, accession
number) and the 10-K / 10-Q filing date is copied onto financial statement
rows whose `(symbol, period_end)` matches exactly (`filed_date` column).
8-K current reports are stored too, with their EDGAR **item codes** in
`items` (`2.02` results, `4.01` auditor change, `4.02` restatement, `5.02`
officer departure, `1.03` bankruptcy, ...) and the event date in
`period_end`, so downstream analysis can tell a routine results filing from
a structural red flag without parsing the documents:

```bash
bearish filings /path/to/sqlite/db US --filters NVDA,AAPL
```

Earnings rows (`earningsdate`) also store `is_forward`, a fetch-time flag
marking rows whose earnings date was still in the future when fetched, and
price rows include dividends and stock splits.

---

## 🌐 Macro, commodity & regime data

Beyond tickers, Bearish stores **open macro-economic data keyed by concept
and entity** so that countries and commodities can be compared on the same
footing and later turned into KPIs:

- a **concept** is a normalised meaning (`CPI_YOY`, `POLICY_RATE`, `CLI`,
  `HY_SPREAD`, `COT_NET_SPEC`, …; see `bearish/macro/catalog/concepts.py`);
- an **entity** is an ISO-3 country (`FRA`), a region (`EA20`, `WLD`) or a
  commodity code (`BRENT`, `COPPER`; see `bearish/macro/commodities.py`);
- the **catalog** (`bearish/macro/catalog/`) maps each `concept.entity` to
  exactly one provider series. Providers are keyless open APIs: OECD, FRED,
  IMF, World Bank, BIS, ECB, Eurostat, policy-uncertainty and geopolitical
  risk indices, CFTC positioning, the World Bank Pink Sheet, Ken French
  factors (FRED also accepts an optional `FRED_API_KEY`).

Observations are stored **point-in-time**: every value carries the `as_of`
date on which it was known, and a provider revision is written as a new
vintage instead of overwriting the old one. `macrolatest` is a view of the
newest vintage per `(series_id, date)`; `read_observations(as_of=...)` answers
"what was known on that day", which is what keeps a backtest of
LLM-generated recommendations honest.

```bash
# every priority country (G7 + China + India + the countries supported by the
# equity pipeline), every category
bearish macro /path/to/sqlite/db

# a subset: two countries, only rates and credit, from 2015
bearish macro /path/to/sqlite/db FRA DEU --categories rates --categories credit --start 2015-01-01
```

Three more commands fill the tables around the macro store:

```bash
# commodity reference + futures prices (pricecommodity), World Bank Pink Sheet
# monthly history and CFTC positioning (macroobservation), and each country's
# export/import share per commodity from UN Comtrade (countrycommodityexposure)
bearish commodities /path/to/sqlite/db --year 2024

# SEC Form 4 insider transactions for US filers (insidertransaction)
bearish insiders /path/to/sqlite/db US --filters NVDA,AAPL --since 2025-01-01

# linking tables: country references/aliases (equitycountry view maps
# equity.country to ISO-3), curated sector <-> commodity sensitivities, and ETF
# exposure to countries, regions, commodities and sectors
bearish link /path/to/sqlite/db
```

Institutional ownership comes from two more SEC sources:

```bash
# Form 13F: every institutional manager's quarter-end positions, from the SEC
# bulk data sets (one ~100 MB zip per filing window, all filers at once).
# Per-manager rows go to holding13f for --managers plus the --top-managers
# largest filers; holding13fsummary holds, for EVERY security and quarter,
# the number of holders, total shares/value and the flows against the prior
# quarter (new holders, closed holders, share and value change).
bearish holdings /path/to/sqlite/db --windows 2 --managers 0001067983

# Schedule 13D/13G: beneficial owners crossing 5 % (13D = activists, filed
# within days; 13G = passive), parsed from EDGAR's structured XML (filings
# since December 2024) into ownership13dg.
bearish ownership /path/to/sqlite/db US --filters AAPL,ASPN
```

13F positions are published up to 45 days after quarter end, so treat them as
a lagging, quarterly signal; `holding13f.filed_date` and
`holding13fsummary.latest_filed_date` record when each snapshot became public.

US financial statements are now taken from **SEC XBRL companyfacts** first
(as reported, with the real `filed_date`), falling back to Yahoo for tickers
the SEC does not cover.

Tables: `macroseries` (catalog mirror), `macroobservation`
(`series_id, date, value, as_of`), `macrotracker` (last fetched date per
series, used by `bearish update --macro` to refresh only what is due and to
re-request a revision window). From Python:

```python
from bearish.main import Bearish
from bearish.macro.models import MacroQuery

bearish = Bearish(path="/path/to/sqlite/db")
frame = bearish.read_macro(MacroQuery(entities=["FRA", "USA"], categories=["inflation", "rates"]))
known_then = bearish.read_observations(MacroQuery(concepts=["GDP_YOY"], as_of="2025-06-30"))
```

---

## 🗓️ Events, conflicts & early-winner signals

Three groups of commands feed event studies and per-stock signals. Everything
is stored point-in-time (`as_of`), so a screen or study can be re-run "as
known on" any past date.

**Elections and policy**

```bash
bearish events /path/to/sqlite/db          # US elections 1946-2026, 28 curated conflict onsets, FOMC meetings
bearish midterms /path/to/sqlite/db        # returns around every midterm: ^GSPC + sector SPDRs, with bootstrap CIs
bearish predictions /path/to/sqlite/db     # Polymarket / Kalshi odds (elections, geopolitics)
bearish congress /path/to/sqlite/db        # House members' trades (STOCK Act periodic transaction reports)
bearish lobbying /path/to/sqlite/db US --filters NVDA,LMT
```

Tables: `marketevent`, `eventreturn`, `predictionquote`, `congresstrade`,
lobbying spend in `tickersignal` (`LOBBYING_USD`). Senate trade reports are
not fetched (the Senate site requires interactive terms acceptance).

**Conflict and supply-chain shocks**

```bash
bearish conflicts /path/to/sqlite/db
```

Daily vessel transits through 28 chokepoints (IMF PortWatch), GDELT daily
conflict counts / Goldstein score / tone for 15 country pairs, monthly new
OFAC sanctions designations per country, SIPRI military spending, plus
sanctions-wave events. GDELT (~8 MB per day) and OFAC (~110 MB) only run here;
PortWatch and SIPRI are also part of `bearish macro`.

**Early-winner signals ("the next NVIDIA")**

```bash
bearish themes /path/to/sqlite/db US --theme ai_accelerator --theme ai_datacenter
bearish shorts /path/to/sqlite/db US --filters NVDA,SMCI
bearish awards /path/to/sqlite/db US --filters PLTR,LMT
bearish screen /path/to/sqlite/db --as-of 2025-06-30 --output screen.csv
```

`tickersignal` holds `THEME_MENTIONS:<theme>` (SEC filings mentioning a
theme, per quarter), `CAPEX_USD` / `CAPEX_YOY` (with a `HYPERSCALERS`
aggregate), `SHORT_INTEREST`, `SHORT_VOLUME_RATIO`, `FED_AWARDS_USD`. The
screen ranks stocks on revenue acceleration, gross-margin change, 13F holder
growth, insider buying, theme-mention growth, short-interest change, relative
strength and AI-capex tailwind, using only data public on `--as-of`.

---

## 🔄 Updating Data

### 💵 Update Prices

To update only the price data:

```bash
bearish prices /path/to/sqlite/db Belgium France --api-keys=config.json
```
![img_2.png](docs/img/img_2.png)
---

### 🧾 Update Financials

To update financial and fundamental data:

```bash
bearish financials /path/to/sqlite/db Belgium France --api-keys=config.json
```
![img_1.png](docs/img/img_1.png)
---

## 🎯 Fetch Specific Tickers

To fetch and store data for specific tickers:

```bash
bearish run /path/to/sqlite/db US --filters NVDA,TSLA,RHM.DE --api-keys=config.json
```

> You must always provide the country where each ticker is traded along with the desired tickers as filters.

---

## 📚 Summary

| Feature | Description |
|--------|-------------|
| 🌍 Country-level support | Fetch equities from any exchange |
| 🧠 Fundamental data | Balance sheets, cash flow, income statements |
| 📉 Price history | Up-to-date historical prices |
| 🗄️ Local database | Data saved in SQLite for offline analysis |
| 🔌 Extensible | Plug in your own APIs or providers |
| 🧪 Designed for analysis | Ideal for custom screeners and research |

---

## 🛠️ Contributing

Contributions are welcome! Feel free to open issues or submit pull requests to improve Bearish.

### Development

The project is managed with [uv](https://github.com/astral-sh/uv):

```bash
uv sync                                      # install dependencies
uv run pytest -m "not network and not lfs"   # offline, deterministic tests
uv run pytest                                # full suite (live APIs + git-lfs test db)
uv run ruff check                            # lint
uv run mypy                                  # type check
```

Database schema changes are managed with **Alembic** (migrations live in
`bearish/database/alembic/versions`):

```bash
uv run python -m bearish.database.scripts.create_revision   # autogenerate a revision
uv run python -m bearish.database.scripts.upgrade           # upgrade a database to head
```

---

## 📄 License

Bearish is released under the **MIT License**.