Metadata-Version: 2.4
Name: pe-automator
Version: 0.22.0
Summary: PE Automator is a Python package for automating the setup and execution of parameter estimation (PE) runs using Bilby Pipe.
Home-page: https://git.ligo.org/yumeng.xu/pe-automator
Author: Yumeng Xu
Author-email: Yumeng Xu <yumeng.xu@ligo.org>
Keywords: ligo,gravitational waves,parameter estimation
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: bilby-pipe
Requires-Dist: python-gitlab
Requires-Dist: jinja2
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: pandas
Requires-Dist: matplotlib
Requires-Dist: gwpy
Requires-Dist: h5py
Requires-Dist: streamlit
Requires-Dist: streamlit-aggrid
Requires-Dist: plotly
Requires-Dist: GitPython
Requires-Dist: click
Requires-Dist: paramiko
Requires-Dist: pymannkendall
Dynamic: author
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python

# PE Automator

A plugin-based workflow tool for gravitational-wave parameter estimation using
[bilby-pipe](https://lscsoft.docs.ligo.org/bilby_pipe/).  Three run modes are
supported: real event data, Gaussian-noise injections, and real-noise
injections from GWOSC.

---

## Installation

```bash
conda create -n pe_automator python=3.10 setuptools_scm
conda activate pe_automator
make install
```

---

## Project directory layout

### Real-data runs

```
data_path/
├── configs/
│   └── {eventname}_config.ini      # bilby-pipe config with event metadata
├── framefiles/
│   ├── {eventname}_H1.gwf          # pre-downloaded strain frames (git-lfs)
│   ├── {eventname}_L1.gwf
│   └── {eventname}_V1.gwf
├── psds/
│   └── {eventname}_{det}_psd.txt
├── spline_cal_envs/
├── templates/
│   └── bilby_config.tpl.ini        # Jinja2 template shared by all modes
└── project/
    ├── project.json                 # gitlab_url, gitlab_project
    └── allocations.json             # cluster allocation profiles
```

The framefiles are stored as `git-lfs` objects.  Install and initialise
git-lfs before cloning:

```bash
git lfs install
```

### Injection runs (`gaussian_noise` / `real_noise`)

```
data_path/
├── inj_config/
│   └── {inj_set}/
│       ├── injection_params.json   # list of N injection parameter dicts
│       ├── injection_points.json   # (real_noise only) list of M GPS points
│       └── sample_config.ini       # bilby-pipe base config for this inj set
├── psds/
│   └── {inj_set}_{det}_psd.txt    # per-detector PSD (2-column: freq, PSD)
├── templates/
│   └── bilby_config.tpl.ini
└── project/
    ├── project.json
    └── allocations.json
```

GWF frame files for injections are generated at run time and written directly
into `runs/{label}/framefiles/`; `data_path` is never modified.

---

## CLI reference

### `setup` — create and submit PE runs

```
pe_automator setup <name> --mode <mode> [options]
```

`<name>` is the event name (real_data) or injection set name
(gaussian_noise / real_noise).

**Common options**

| Flag | Default | Description |
|------|---------|-------------|
| `--mode` | `real_data` | `real_data`, `gaussian_noise`, or `real_noise` |
| `--data_path` | `./data` | Project data directory |
| `--run_label` | *(required)* | Label appended to the run directory name |
| `--approximant` | *(required)* | LALSim waveform approximant |
| `--allocation` | *(required)* | Allocation key from `allocations.json` |
| `--user` | *(required)* | SSH username on the cluster |
| `--conda_env` | *(required)* | Conda environment on the cluster |
| `--private_token` | *(required)* | GitLab personal access token (`api` scope) |
| `--account` | | SLURM account |
| `--partition` | | SLURM partition |
| `--qos` | | SLURM QoS |
| `--memory` | `300` | Memory per node (GB) |
| `--walltime` | `71:40:00` | SLURM wall-clock limit |
| `--cpu` | allocation default | CPUs per node |
| `--npoint` | `1000` | Dynesty `npoints` |
| `--nact` | `50` | Dynesty `nact` |
| `--naccept` | `60` | Dynesty `naccept` |
| `--maxmcmc` | `20000` | Dynesty `maxmcmc` |
| `--dry_run` | `False` | Generate files locally; skip upload and submission |
| `--distance_marginalization` / `--no-distance_marginalization` | | Enable/disable distance marginalisation |
| `--priors` | | Path to custom priors file |
| `--mode_array` | | Waveform mode array override |
| `--wf_min_f` | | Waveform minimum frequency (Hz) |
| `--wf_ref_f` | | Waveform reference frequency (Hz) |
| `--min_f` | | Analysis minimum frequency (Hz) |
| `--waveform_arguments_dict` | | Waveform-argument dictionary override, e.g. `{'N_harmonics': 12}` |
| `--comment` | | Free-text comment added to the GitLab issue |

**Injection-only options** (`gaussian_noise` and `real_noise`)

| Flag | Default | Description |
|------|---------|-------------|
| `--flow` | `20.0` | Low-frequency cutoff for noise/waveform generation (Hz) |
| `--f_ref` | `20.0` | Reference frequency for waveform generation (Hz) |
| `--force_regenerate` | `False` | Regenerate GWF files even if they already exist |

**real_noise only**

| Flag | Default | Description |
|------|---------|-------------|
| `--fetch_buffer` | `16` | Extra seconds fetched around the injection window |

> **GitLab token**: select the `api` scope.
> See [GitLab docs](https://docs.gitlab.com/user/profile/personal_access_tokens/#personal-access-token-scopes).

---

## Quick start examples

### 1. Real-data run

```bash
pe_automator setup GW150914 \
    --mode real_data \
    --data_path ./data \
    --run_label run1 \
    --approximant IMRPhenomXPNR \
    --account uib107 --partition gpp --qos gp_resa \
    --user resh000428 --conda_env pe_env \
    --private_token <token> \
    --allocation AECT-2025-2-0029
```

### 2. Gaussian-noise injection runs

Generates **N × M** jobs where N = entries in `injection_params.json` and
M = `noise_seeds` listed in each entry.

```bash
pe_automator setup inj_set1 \
    --mode gaussian_noise \
    --data_path ./data \
    --run_label run1 \
    --approximant IMRPhenomXPNR \
    --flow 20.0 --f_ref 20.0 \
    --account uib107 --partition gpp --qos gp_resa \
    --user resh000428 --conda_env pe_env \
    --private_token <token> \
    --allocation AECT-2025-2-0029
```

Add `--dry_run` to generate configs and GWF files locally without uploading.

Required files under `data_path`:

| Path | Description |
|------|-------------|
| `inj_config/{inj_set}/injection_params.json` | N injection parameter dicts (each with a `noise_seeds` list) |
| `inj_config/{inj_set}/sample_config.ini` | Base bilby-pipe config (detectors, priors, …) |
| `psds/{inj_set}_{det}_psd.txt` | PSD for each detector |

### 3. Real-noise injection runs (GWOSC)

Generates **N × M** jobs where N = entries in `injection_params.json` and
M = entries in `injection_points.json`.

```bash
pe_automator setup inj_set1 \
    --mode real_noise \
    --data_path ./data \
    --run_label run1 \
    --approximant IMRPhenomXPNR \
    --flow 20.0 --f_ref 20.0 --fetch_buffer 16 \
    --account uib107 --partition gpp --qos gp_resa \
    --user resh000428 --conda_env pe_env \
    --private_token <token> \
    --allocation AECT-2025-2-0029
```

Additional required file:

| Path | Description |
|------|-------------|
| `inj_config/{inj_set}/injection_points.json` | M GPS injection points with per-detector time shifts |

---

## Injection parameter format (`injection_params.json`)

```json
[
  {
    "mass_1": 35.6,
    "mass_2": 30.4,
    "luminosity_distance": 450.0,
    "geocent_time": 1187008882.43,
    "ra": 1.375,
    "dec": -1.211,
    "psi": 0.0,
    "theta_jn": 0.4,
    "chi_1": 0.0,
    "chi_2": 0.0,
    "noise_seeds": [1000, 2000, 3000]
  }
]
```

`noise_seeds` (Gaussian-noise mode) controls how many noise realisations are
generated per injection.  Spin can be given as aligned (`chi_1`/`chi_2`),
bilby spherical (`a_1`, `tilt_1`, …), or Cartesian (`spin_1x/y/z`).

## Injection point format (`injection_points.json`) — real_noise only

```json
[
  {
    "gps_time": 1187008882.43,
    "shifts": {"H1": 0.0, "L1": 3.14, "V1": 7.0},
    "channel": {
      "H1": "H1:GWOSC-4KHZ_R1_STRAIN",
      "L1": "L1:GWOSC-4KHZ_R1_STRAIN",
      "V1": "V1:GWOSC-4KHZ_R1_STRAIN"
    }
  }
]
```

Non-zero `shifts` fetch each detector's background from a different GPS time,
breaking coherence for background estimation while the injected signal stays
coherent at `gps_time`.

---

## Bundled PSDs

```python
from pe_automator.injection_generator.psds import get_psd_path
psd = get_psd_path("AplusDesign_O5")
```

| Name | Detector |
|------|----------|
| `AplusDesign_O5` | A+ (LIGO O5 design) |
| `AdV_DESIGN` | Advanced Virgo design |
| `aLIGO_ZERO_DET_high_P` | aLIGO zero-det high power |
| `ET_D` | Einstein Telescope D |

---

## Other CLI commands

### `monitor` — track job status

```bash
pe_automator monitor \
    --private_token <token> \
    --ssh_key ~/.ssh/id_rsa \
    --data_path ./data
```

### `rescue` — resubmit a failed job

```bash
pe_automator rescue <issue_number> \
    --data_path ./data \
    --private_token <token> \
    --walltime 47:00:00
```

### `setup_env` — deploy a conda environment on the cluster

```bash
pe_automator setup_env pe-0.0.1beta1 \
    --source_env my_env.tar.gz \
    --source_remote resh000428@picasso.scbi.uma.es \
    --data_dir ./data
```

### `post_process` — post-process PE results

```bash
pe_automator post_process \
    --results_dir ./results \
    --data_dir ./data \
    --output_dir .
```

### `dlogz` — check sampler convergence

```bash
pe_automator dlogz \
    --private_token <token> \
    --ssh_key ~/.ssh/id_rsa \
    --data_path ./data
```
