Metadata-Version: 2.4
Name: jing
Version: 0.3.11
Summary: Download stock data and perform data analisys
Author-email: sai <dawangfei@gmail.com>
License: MIT
Keywords: jing,sai,stock
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
Requires-Dist: requests
Requires-Dist: pandas
Requires-Dist: baostock
Requires-Dist: yfinance
Requires-Dist: akshare
Requires-Dist: pyarrow
Requires-Dist: importlib-metadata; python_version < "3.10"

# jing

jing is a Python library for downloading stock price data to local CSV files and organizing market data.

## Installation

```bash
pip install jing
```

All required dependencies (`requests`, `pandas`, `baostock`, `yfinance`, `akshare`, `pyarrow`) will be installed automatically.

## Quickstart

```python
import jing

# Download AAPL data to local CSV
jing.D('us').download('AAPL')
```

## Main APIs

The core workflow uses the `D` class for downloading market data:

| Class | Purpose | Network? | Typical Use |
|-------|---------|----------|-------------|
| `D` | **Download** market data | Yes (API calls) | Fetch one stock or a batch list to local CSV |

### `D` — Downloader

Download stock data from Yahoo Finance, BaoStock, or AkShare into local CSV files.

```python
import jing

# US stock via Yahoo Finance
d = jing.D('us')
d.download('AAPL')

# CN stock via BaoStock (default source for CN)
d = jing.D('cn')
d.download('sz.000807')

# HK stock via AkShare
d = jing.D('hk')
d.download('00700')

# Batch download all stocks in the default CN BaoStock list
d = jing.D('cn')
d.download()  # downloads all stocks in ~/data/jing/list/cn_baostock.txt

# Batch download CN via AkShare instead
d = jing.D('cn', _source='akshare')
d.download()  # downloads all stocks in ~/data/jing/list/cn_ak.txt
```

## Data Storage

The project uses a source-first layout under `~/data/jing/` (override with `JING_DATA` env var):

```text
~/data/jing/
  raw/
    yahoo/us/
    baostock/cn/
    akshare/cn/
    akshare/hk/
  list/
    us.txt
    cn_ak.txt
    cn_baostock.txt
    hk.txt
```

Raw CSV examples:

- US Yahoo CSV: `~/data/jing/raw/yahoo/us/AAPL.csv`
- CN BaoStock CSV: `~/data/jing/raw/baostock/cn/sh.601088.csv`
- HK AkShare CSV: `~/data/jing/raw/akshare/hk/00700.csv`

### List files

List files are used only for batch downloads, for example `jing.D('us').download()` or `jing.D('cn').download()`. Single-stock downloads use the code you pass directly.

On first use, jing creates `~/data/jing/list/` and copies the bundled default lists there. Edit the files under `~/data/jing/list/`; do not edit the package files under `jing/lists/` unless you are changing the project defaults.

| Market/source | Downloader call | List file | Code format |
|---------------|-----------------|-----------|-------------|
| US Yahoo | `jing.D('us')` | `us.txt` | Yahoo ticker, e.g. `AAPL` |
| CN BaoStock (default CN) | `jing.D('cn')` | `cn_baostock.txt` | BaoStock code with exchange prefix, e.g. `sh.601088` or `sz.000807` |
| CN AkShare | `jing.D('cn', _source='akshare')` | `cn_ak.txt` | 6-digit A-share code, e.g. `601088` |
| HK AkShare | `jing.D('hk')` | `hk.txt` | 5-digit HK code, e.g. `00700` |

For CN, use `cn_baostock.txt` unless you explicitly pass `_source='akshare'`. The default `jing.D('cn')` downloader is BaoStock, and BaoStock requires exchange-prefixed codes.

Minimal examples:

`~/data/jing/list/cn_baostock.txt`

```text
sh.601088
sz.000807
sh.600519
```

`~/data/jing/list/cn_ak.txt`

```text
601088
000807
600519
```

`~/data/jing/list/us.txt`

```text
AAPL
MSFT
NVDA
```

`~/data/jing/list/hk.txt`

```text
00700
09988
03690
```

Batch download calls:

```python
import jing

jing.D('cn').download()                       # reads cn_baostock.txt
jing.D('cn', _source='akshare').download()    # reads cn_ak.txt
jing.D('us').download()                       # reads us.txt
jing.D('hk').download()                       # reads hk.txt
```

### Download cache

Downloaded data is cached as Parquet files to avoid repeated API calls on the same day:

```text
~/.cache/jing/data/
  cn_baostock_sz_000807_2025-05-25.parquet
  us_yahoo_AAPL_2025-05-25.parquet
```

Cache pattern: `<market>_<source>_<code>_<date>.parquet`

- If cache exists for today → read from Parquet (fast, ~0.01s)
- If no cache → download from API, save CSV, and write Parquet cache
- New day → automatic fresh download (old cache ignored)
- Override the cache root with `JING_CACHE`

### Data flow

#### Single stock

```mermaid
flowchart LR
    A["d.download('AAPL')"] --> B{Cache exists?}
    B -->|Yes| C[Read Parquet]
    B -->|No| D[Download from API]
    D --> E[Save CSV]
    E --> F[Write Parquet cache]
    C --> G[Return DataFrame]
    F --> G
```

#### Batch download

```mermaid
flowchart LR
    A["d.download()"] --> B[Read stock list]
    B --> C[Queue stocks]
    C --> D[Download each]
    D --> E[Print summary]
```

### Incremental Download

jing supports smart incremental downloads to save time and reduce API calls. When you already have historical data locally, jing will:

1. Fetch only new data plus an overlap period (default 5 days)
2. Check if the overlapping data matches your local CSV
3. If consistent → merge the new data
4. If inconsistent (e.g., due to dividend/split adjustments) → automatically perform a full refresh

```python
import jing

d = jing.D('us')

# Default: smart incremental download
d.download('AAPL')

# Force full refresh
d.download('AAPL', incremental=False)

# Custom overlap days and tolerance
d.download('AAPL', overlap_days=10, tolerance=0.002)
```

**Parameters:**

| Parameter | Default | Description |
|-----------|---------|-------------|
| `incremental` | `True` | Enable incremental download |
| `overlap_days` | `5` | Days to overlap for consistency check |
| `tolerance` | `0.001` | Price difference tolerance (0.1%) |

**Why overlap days?**

Stock prices are adjusted for dividends and splits. When these events occur, historical prices change. By overlapping a few days and comparing, jing can detect if adjustments have occurred and automatically refresh the entire history to maintain data consistency.

#### Incremental download flow

```mermaid
flowchart TD
    A[download\('AAPL'\)] --> B{Local CSV exists?}
    B -->|No| C[Full download]
    B -->|Yes| D[Fetch from last date - 5 days]
    D --> E{Overlapping data consistent?}
    E -->|No| C
    E -->|Yes| F[Merge new data]
    C --> G[Save CSV]
    F --> G
```

## Configuration

### Data directory

Priority (highest first):

1. **Function call** — programmatic override
   ```python
   from jing.data_paths import set_data_root
   set_data_root('/custom/data/path')
   ```

2. **Environment variable**
   ```bash
   export JING_DATA=/custom/data/path
   ```

3. **Default** — `~/data/jing`

## Release to PyPI

1. Bump the version in `pyproject.toml`:
   ```toml
   version = "0.3.9"
   ```

2. Build the distribution packages:
   ```bash
   python -m build
   ```

3. Upload to PyPI:
   ```bash
   python -m twine upload dist/*
   ```

   > You will need a PyPI account and [API token](https://pypi.org/manage/account/token/). Configure twine once with `python -m twine upload --repository testpypi dist/*` to test first if desired.
