Metadata-Version: 2.4
Name: dqrxfer
Version: 0.3.0
Summary: Utility to transfer dqr results to centralized cluster
Author-email: Joseph Areeda <joseph.areeda@ligo.org>
Project-URL: Bug Tracker, https://git.ligo.org/o4-dqr/dqrxfer/-/issues
Project-URL: Source Code, https://git.ligo.org/o4-dqr/dqrxfer
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ciecplib
Requires-Dist: lxml
Dynamic: license-file

# dqrxfer

[![PyPI version](https://img.shields.io/pypi/v/dqrxfer.svg)](https://pypi.org/project/dqrxfer/)
[![Python versions](https://img.shields.io/badge/python-3.11%20%7C%203.12-blue)](https://pypi.org/project/dqrxfer/)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Docs](https://img.shields.io/badge/docs-dqr--builder-blue)](https://dqr-builder.docs.ligo.org/dqrxfer/)

Utility to transfer Data Quality Report (DQR) results from remote sites to the central site.

## Overview

`dqrxfer` is a client-side package used by DQRbuild nodes (e.g., LLO, LHO, Virgo, KAGRA, etc.) to push Data Quality Report files to a central DQRbuild node. It supports both HTTPS (with Shibboleth/ECP authentication) and SCP/SSH transfer methods.

## Requirements

- A valid Kerberos ticket (TGT) or keytab for authentication
- Network access to the DQRbuild node

## Installation

```bash
pip install dqrxfer
```

## Commands

The package provides three command-line tools:

### `dqr-ping`

Tests connectivity and authentication with the DQR server. Useful for verifying your Kerberos credentials and network access before attempting an upload.

```
usage: dqr-ping [-h] [-v] [-V] [-q] [--server SERVER] [--user USER]
```

Example:

```
$ dqr-ping
OK
INFO:dqr-xfer:Login: 1.33, Ping: 0.30, status:200
OK
INFO:dqr-xfer:Login: 1.33, Ping: 0.27, status:200
OK
INFO:dqr-xfer:Login: 1.33, Ping: 0.27, status:200
INFO:dqr-xfer:Success = 100.0%
INFO:dqr-xfer:Elapsed time: 2.7s
```

### `dqr-upload`

Uploads files or directories for a single DQR event to the central site. Supports both SCP (default) and HTTPS transfer.

```
usage: dqr-upload [-h] [-v] [-V] [-q] [-s SERVER] [--port PORT]
                  [--retries RETRIES] [--retry-wait RETRY_WAIT]
                  [-g GRACEID] [-u UPLOADER] [-p PROJECT] [--user USER]
                  [--revision REVISION] [--subdir SUBDIR] [--config CONFIG]
                  [--set-dir SET_DIR] [--use-https] [--upload-path UPLOAD_PATH]
                  [--remote-command REMOTE_COMMAND] [--temp-dir TEMP_DIR]
                  [files ...]
```

Key arguments:

| Argument | Description |
|---|---|
| `-g`/`--graceid` | Event ID (e.g. `S230601a`) or GPS time |
| `-u`/`--uploader` | Site ID (e.g. `LLO`, `LHO`, `Virgo`, `Kagra`) |
| `-p`/`--project` | Project name (e.g. `o4`, `replay`, `mdc`, `personal`) |
| `--revision` | Integer run/revision number (default: `1`) |
| `--use-https` | Use HTTPS upload instead of SCP |

### `dqr-batch-upload`

Scans a DQR result directory and calls `dqr-upload` for each event found within it. Useful for bulk transfers.

```
usage: dqr-batch-upload [-h] [-v] [-V] [-q] [-d INDIR] [-p PROJECT]
                        [-u UPLOADER] [--set-dir] [-s SERVER] [--test]
```

## Authentication

`dqrxfer` uses [ciecplib](https://github.com/duncanmmacleod/ciecplib) for Shibboleth/ECP authentication over HTTPS. For SCP transfers, standard Kerberos credentials are used.

Obtain a Kerberos ticket before running any command:

```bash
kinit albert.einstein@LIGO.ORG
```

If no Kerberos ticket is available, you will see:

```
klist: No credentials cache found
```

## License

This project is licensed under the [GNU General Public License v3.0](LICENSE).
