Metadata-Version: 2.4
Name: gcn-circular-llm
Version: 0.4.0
Summary: Extract structured astrophysical information from NASA GCN circulars using an LLM via DeepSeek.
Author: LAujust
License: MIT
Project-URL: Homepage, https://github.com/LAujust/gcn_parser
Project-URL: Issues, https://github.com/LAujust/gcn_parser/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests
Requires-Dist: beautifulsoup4
Requires-Dist: pydantic
Requires-Dist: python-dotenv
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# GCN Circular LLM Parser (gcn-circular-llm)

[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A lightweight Python package for extracting structured astrophysical information from NASA GCN (Gamma-ray Coordinates Network) circulars using an LLM via any OpenAI-compatible API (DeepSeek, OpenAI, Minimax, etc.).

## Features

- **Fetch circulars** for a given transient event from the NASA GCN archive
- **Extract structured fields** with an LLM:
  - Event coordinates (RA, Dec, system, error)
  - Redshift and host galaxy
  - Classification (e.g. Long GRB, SGRB, Supernova)
  - Flux / magnitude measurements with UTC timestamps and optional MJD
  - Observations and instrument references
  - A one-sentence summary per circular
- **Post-extraction validation** automatically checks completeness and fills gaps:
  - Cross-fills missing coordinates, redshift, event type from sibling circulars
  - Propagates flux measurement times across matching bands/instruments
  - Regex fallback extracts RA/Dec, redshift, and UTC timestamps from raw text
  - Flags circulars needing manual review in `needs_review.json`
- **Organized output** — all results land in `output/<event>/` with JSON, CSV, full extraction log, and review file
- **Export light-curve CSV** that aggregates all measurements across circulars with automatic UTC/MJD cross-fill and standardized photometric band names

## Installation

### From PyPI (recommended)

```bash
pip install gcn-circular-llm
```

### From source

```bash
git clone git@github.com:LAujust/gcn_parser.git
cd gcn_parser
pip install .
```

### Requirements

- Python >= 3.10
- An API key for any OpenAI-compatible LLM provider

## Configuration

Set up your LLM provider via environment variables or a `.env` file.

### Supported env vars

| Env Var | Default | Description |
|---------|---------|-------------|
| `LLM_API_KEY` | *(falls back to `DEEPSEEK_API_KEY`)* | API key for your LLM provider |
| `LLM_BASE_URL` | `https://api.deepseek.com` | API base URL |
| `LLM_MODEL` | `deepseek-v4-flash` | Default model name |

### Provider examples

Create a `.env` file in your working directory (see `.env.example` for the template):

**DeepSeek (default):**

```bash
LLM_API_KEY=sk-...
# Uses default base URL and model — no additional config needed
```

**OpenAI:**

```bash
LLM_BASE_URL=https://api.openai.com/v1
LLM_API_KEY=sk-...
LLM_MODEL=gpt-4o
```

**Minimax:**

```bash
LLM_BASE_URL=https://api.minimax.chat/v1
LLM_API_KEY=your-key-here
LLM_MODEL=minimax-m2.5
```

For backward compatibility, `DEEPSEEK_API_KEY` is still supported as a fallback for `LLM_API_KEY`.

## Usage

### CLI

After installation, the `gcn-parser` command is available globally:

```bash
# Basic extraction — writes to output/GRB250101A/
gcn-parser GRB250101A

# Specify output directory
gcn-parser GRB250101A -d my_results/GRB250101A

# Use a different model
gcn-parser GRB250101A -m deepseek-v4-pro

# Use OpenAI directly (requires LLM_BASE_URL + LLM_API_KEY in .env)
gcn-parser GRB250101A -m gpt-4o

# Enable debug logging
gcn-parser GRB250101A -v

# List all options
gcn-parser --help
```

**CLI options**

| Option | Description |
|--------|-------------|
| `event` | Event identifier, e.g. `GRB250101A` or `EP260321a` |
| `-d, --outdir` | Output directory (default: `output/<event>`) |
| `-m, --model` | Override the default model (`LLM_MODEL` env var or `deepseek-v4-flash`) |
| `-v, --verbose` | Enable debug-level logging |

**Output directory structure** (after a run):

```
output/GRB250101A/
  ├── GRB250101A.json      # Full extraction results with _filled_from annotations
  ├── GRB250101A.csv       # Light-curve CSV
  ├── extraction.log       # Timestamped log of the full run
  └── needs_review.json    # Circulars requiring manual inspection
```

You can also run it as a Python module:

```bash
python -m gcn_parser GRB250101A -d output/GRB250101A
```

### Python API

Use the package programmatically in your own scripts:

```python
from gcn_parser import (
    fetch_event_circulars,
    extract_circular,
    cross_fill,
    cross_fill_flux_time,
    check_completeness,
    regex_fallback,
    OutputManager,
)

# Fetch all circulars for an event
circulars = fetch_event_circulars("EP260321a")

# Extract each circular
results = []
for c in circulars:
    extraction = extract_circular(c["text"], model="deepseek-v4-flash")
    results.append({
        "gcn": c["gcn"],
        "extraction": extraction.model_dump(by_alias=True),
        "raw_text": c["text"],
    })

# Post-extraction validation
results = cross_fill(results)            # fill top-level fields from siblings
results = cross_fill_flux_time(results)  # propagate flux times across bands

for r in results:
    missing = check_completeness(r["extraction"])
    if missing:
        fb = regex_fallback(r["raw_text"])
        # apply fb["coordinates"], fb["redshift"], fb["utc_times"] ...

# Write organized output
out = OutputManager("EP260321a")
out.attach_log()
out.write_json(results)
out.write_csv(results)
out.detach_log()
```

## Output

### JSON

Each circular produces an object with the following schema. Fields auto-filled from sibling circulars or regex carry a `_filled_from` annotation tracing their provenance:

```json
[
  {
    "gcn": "GCN 12345",
    "extraction": {
      "event_type": "GRB",
      "coordinates": {
        "ra": "12:34:56.78",
        "dec": "+12:34:56.7",
        "system": "J2000",
        "error_arcsec": 1.2
      },
      "redshift": 0.5,
      "host_galaxy": "Host Galaxy Name",
      "classification": "Long GRB",
      "flux_magnitude": [
        {
          "value": 18.5,
          "err": null,
          "band": "r",
          "instrument": "ZTF",
          "utc": "2025-01-01T12:00:00Z",
          "mjd": null,
          "unit": "AB mag",
          "upper_limit": false
        }
      ],
      "observations": ["Swift/XRT follow-up"],
      "references": ["GCN 12344"],
      "confidence": "high",
      "summary": "...",
      "_filled_from": {
        "coordinates": "GCN 12346",
        "redshift": "GCN 12344"
      }
    }
  }
]
```

### needs_review.json

Circulars that still have missing critical fields after cross-fill and regex fallback are listed with the reason and a snippet of the raw text:

```json
[
  {
    "gcn": "GCN Circular 12347",
    "reason": "missing: flux_magnitude (empty)",
    "raw_text_snippet": "We report on the detection of..."
  }
]
```

### CSV Light-Curve

The `--csv` option flattens all `flux_magnitude` entries across circulars into a single table. Before writing, the exporter performs two clean-up steps automatically:

1. **UTC / MJD cross-fill** — if one timestamp field is missing, it is computed from the other (UTC ↔ MJD conversion using the MJD epoch).
2. **Band standardization** — common aliases are normalized (e.g. `r-band`, `R band`, `VT_R` → `r` or `R`).

| Column | Description |
|--------|-------------|
| `gcn_number` | Source circular number |
| `utc` | Observation time in ISO-8601 UTC |
| `mjd` | Modified Julian Date (optional) |
| `value` | Numeric measurement |
| `err` | Measurement error; for upper limits, `value` is set to `0` and `err` stores the limit |
| `unit` | e.g. `mag`, `AB mag`, `uJy`, `mJy/beam` |
| `band` | Standardized photometric band or frequency |
| `instrument` | Telescope or instrument name |

Rows are sorted chronologically (by UTC, with MJD as fallback).

## Models & Rate Limiting

The default model is `deepseek-v4-flash` (via DeepSeek) with a default delay of `2s` between batches. If you hit rate limits (`429 Too Many Requests`), the package automatically retries with exponential backoff.

For faster or higher-accuracy analysis, switch models:

```bash
# Use a different DeepSeek model
gcn-parser EP260321a -m deepseek-v4-pro

# Via OpenAI (set LLM_BASE_URL and LLM_API_KEY in .env first)
gcn-parser EP260321a -m gpt-4o
```

**Suggested models**

| Model | Speed | Cost | Notes |
|-------|-------|------|-------|
| `deepseek-v4-flash` | Fast | Low | Default; good balance of speed and accuracy |
| `deepseek-v4-pro` | Medium | Medium | Higher accuracy; supports thinking mode |
| `gpt-4o` | Fast | Low | Higher accuracy for complex circulars |

## Troubleshooting

| Issue | Cause | Solution |
|-------|-------|----------|
| `429 Too Many Requests` | Rate limit hit | Add delay between requests or switch to a different model |
| `LLM returned an empty response` | Model did not produce valid output | Retry with a different model (e.g. `deepseek-v4-pro`) |
| `LLM_API_KEY is not set` | Missing API key | Set `LLM_API_KEY` (or `DEEPSEEK_API_KEY` as fallback) in `.env` or export it. Get your key at https://platform.deepseek.com/api_keys |
| `No circulars found` | Event name typo or GCN has no circulars | Check the exact event name on [gcn.nasa.gov](https://gcn.nasa.gov/) |

## License

This project is licensed under the MIT License. See [LICENSE](LICENSE) for details.
