Metadata-Version: 2.4
Name: astro-colibri-circular-parser
Version: 1.0.0
Summary: AI-assisted parsing of GCN circulars into structured multi-wavelength follow-up observations (Astro-COLIBRI).
Author: Astro-COLIBRI team
License: BSD 3-Clause License
        
        Copyright (c) 2026, the Astro-COLIBRI team
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived from
           this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
        
Project-URL: Homepage, https://astro-colibri.com
Project-URL: Documentation, https://astro-colibri.science/followupdoc
Project-URL: Repository, https://github.com/astro-transients/astro_colibri_circular_parser
Keywords: GCN,GRB,gamma-ray bursts,afterglow,multi-messenger,NLP,LLM
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Astronomy
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai<3,>=2.41
Requires-Dist: requests<3,>=2.31
Requires-Dist: python-dateutil<3,>=2.8
Requires-Dist: python-dotenv<2,>=1.0
Provides-Extra: examples
Requires-Dist: matplotlib; extra == "examples"
Requires-Dist: jupyter; extra == "examples"
Provides-Extra: tests
Requires-Dist: pytest; extra == "tests"
Dynamic: license-file

# astro-colibri-circular-parser

AI-assisted parsing of [GCN circulars](https://gcn.nasa.gov/circulars) into
structured, machine-readable multi-wavelength follow-up observations.

This is the extraction pipeline behind the GRB optical-afterglow feature of the
[Astro-COLIBRI](https://astro-colibri.com) multi-messenger astronomy platform,
published as a standalone library. Given a circular (by number, URL, or raw
text) it extracts reported observations — detections, upper limits,
non-detections, X-ray/radio fluxes, redshifts — into a JSON record, with times
normalised to seconds since trigger and optical photometry converted to a
common observed-frame R<sub>c</sub>-equivalent AB magnitude. As with any
automated extraction, completeness and scientific values should be checked
against the original circular before publication.

> **Associated manuscript:** F. Schüssler et al., “AI-Assisted Extraction of
> Follow-up Observations from GCN Circulars in Astro-COLIBRI” (prepared for
> PASP; publication link to be added when available).
> **Live service:** the parsed results power the afterglow light curves at
> https://astro-colibri.com (documentation: https://astro-colibri.science/followupdoc).

## How it works

```
circular number / URL ──► fetch (gcn.nasa.gov JSON archive)
                              │
                              ▼
              deterministic regex pre-analysis
        (source names, photometry hints, contact emails)
                              │
                              ▼
          event resolution (optional, pluggable)
     Astro-COLIBRI public API: trigger time, position, E(B-V)
                              │
                              ▼
         LLM structured extraction (OpenAI Responses API)
   strict JSON schema; the regex hints are advisory input only
                              │
                              ▼
                 photometric enrichment
   filter normalisation → common Rc band (AB), Galactic-extinction
   handling, absolute/relative time reconciliation vs. trigger
                              │
                              ▼
        follow-up payload + consistency checks (JSON)
```

Key design points:

- **Structured outputs, not free text.** The LLM must return JSON conforming to
  a strict schema (`PHOTOMETRY_RESPONSE_SCHEMA`); every object is closed
  (`additionalProperties: false`) and fully required, so missing values are
  explicit `null`s.
- **Deterministic scaffolding around the LLM.** Callers can use the exported
  regex helpers as cheap prefilters; `parse_circular(...)` itself always runs
  the configured extraction provider. Regex hints point the model at
  magnitudes, limits and redshifts, but the prompt requires the model to verify
  them against the circular. Consistency checks flag extractions that need
  human review (magnitudes without filters, unparseable times, negative
  times-since-trigger, ...).
- **Physics in code, not in the LLM.** All photometric conversions (filter →
  common R<sub>c</sub> band, Vega↔AB, extinction re-reddening, time
  arithmetic) are classical Python (`circular_parser/photometry.py`),
  unit-tested and independent of the model.
- **No hidden state.** The pipeline returns a plain dict; it performs no
  database writes and sends no notifications.

## Install

```bash
git clone https://github.com/astro-transients/astro_colibri_circular_parser
cd astro_colibri_circular_parser
pip install .                # library + CLI
pip install ".[examples]"    # + notebook/plotting extras
```

Python ≥ 3.9. Dependencies: `openai`, `requests`, `python-dateutil`, and
`python-dotenv` (for CLI `.env` loading).

## Configure

Copy `.env.example` to `.env` and set your OpenAI API key
(`openAI_key` or `OPENAI_API_KEY`). **Never commit the key.** All other
settings are optional; see the table in `.env.example` and
`circular_parser/settings.py`. A successful parse normally uses one LLM
request. Transient failures can trigger the configured retries, and semantic
time validation can invoke an optional fallback model. Provider billing
therefore depends on the selected model and number of attempts.

## Quickstart

```python
from circular_parser import parse_circular

result = parse_circular(45049)   # fetches https://gcn.nasa.gov/circulars/45049

print(result["source_name"])                 # EP260626a
print(result["consistency_issues"])          # []
for obs in result["payload"]["observations"]:
    print(obs["observation_type"], obs["filter_raw"], obs["mag_raw"],
          obs["time_since_trigger_s"], obs["corrected"]["mag_rc_ab_gal"])
```

Command line:

```bash
python -m circular_parser 45049 --pretty                 # full result to stdout
python -m circular_parser 45049 --output result.json     # write to a file
python -m circular_parser 45049 --no-event-lookup        # skip event linking
python -m circular_parser 45049 --offline examples/cached/extraction_45049.json  # replays the LLM result
```

## Cached extraction replay

The example notebook and the `--offline` CLI flag replay a cached extraction
(`examples/cached/extraction_45049.json`) without an OpenAI API key. Enrichment,
payload construction and consistency checks are identical to a live run.

The CLI flag replaces only the LLM request: when given a circular number or URL,
the CLI still retrieves the circular from GCN and performs event lookup unless
`--no-event-lookup` is also supplied. For a completely network-free run, use
the Python API with cached circular content, a replay provider, and either a
cached `event=` dictionary or `resolve_events=False`. The example notebook and
the test suite demonstrate this pattern; `pytest tests/` needs no network or
credentials.

## Human validation

Human review of stored reports belongs to the surrounding Astro-COLIBRI
platform rather than this standalone parser. The parser returns
`consistency_issues` and preserves the original circular text and provenance so
callers can implement their own review workflow. The live service is described
in the [Astro-COLIBRI follow-up documentation](https://astro-colibri.science/followupdoc).

## Output data model

`parse_circular(...)` returns:

| key | content |
|---|---|
| `source_name` | resolved event name (e.g. `GRB 260101A`, `EP260626a`) |
| `event_resolved` / `event` | whether/which known event the circular was linked to |
| `circular` | number, subject, archive URL |
| `regexp_hints` | deterministic pre-analysis (advisory input to the LLM) |
| `extraction` | raw structured LLM output (`PHOTOMETRY_RESPONSE_SCHEMA`) |
| `payload` | enriched follow-up record: report metadata (observatory, instrument, authors, contacts, GCN link) + observations |
| `consistency_issues` | human-review flags |

Each observation in `payload["observations"]` carries the reported values
(`filter_raw`, `mag_raw`, `mag_err`, `flux`/`flux_unit`, `time_raw`, ...) plus
derived quantities: `time_since_trigger_s`, `is_upper_limit`, and `corrected`
(`mag_ab`, `mag_rc_ab_gal` — the observed-frame R<sub>c</sub>-equivalent AB
magnitude used for light curves — `beta`, `correction_status`).

## Event linking (Astro-COLIBRI)

To convert absolute observation times to times since trigger — and to attach
positions/extinction — the pipeline can resolve the circular against the
public, unauthenticated Astro-COLIBRI API (`/event`, `/source_details`). This
is optional: pass `resolve_events=False` to disable event lookup (relative
times like "26.5 hours after the trigger" still work), supply your own
`event=` dict, or plug in any other backend via `EventLookup` (three
callables). Avoiding all network access also requires passing circular content
directly instead of a circular number or URL. See `circular_parser/events.py`.

## Examples

`examples/parse_circular_demo.ipynb` walks through: fetching a real circular,
the regex pre-analysis, an offline replay of a cached extraction (renders
fully without an API key), an optional live LLM extraction, and a light-curve
plot of the enriched photometry.

## Tests

```bash
pip install ".[tests]"
pytest tests/
```

The test suite uses cached data and test doubles; no network access or API keys
are required.

## License and citation

BSD 3-Clause (see `LICENSE`). If you use this code in a publication, please use
the metadata in `CITATION.cff` to cite the associated manuscript (the final
publication reference will be added when available) and cite the Astro-COLIBRI
platform ([Reichherzer et al. 2021, ApJS 256, 5](https://doi.org/10.3847/1538-4365/ac1517)).
