Metadata-Version: 2.4
Name: ovro_lwa_portal
Version: 0.1.0
Summary: Python library for radio astronomy data processing and visualization for the Owens Valley Radio Observatory - Long Wavelength Array (OVRO-LWA)
Project-URL: Homepage, https://github.com/uw-ssec/ovro-lwa-portal
Project-URL: Repository, https://github.com/uw-ssec/ovro-lwa-portal.git
Project-URL: Issues, https://github.com/uw-ssec/ovro-lwa-portal/issues
License: BSD 3-Clause License
        
        Copyright (c) 2024 University of Washington, eScience Institute, Scientific Software Engineering Center
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
        2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
        3. Neither the name of the University of Washington nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE UNIVERSITY OF WASHINGTON AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE UNIVERSITY OF WASHINGTON OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: <3.14,>=3.12
Requires-Dist: astropy<8,>=7.1.0
Requires-Dist: dask<2026,>=2025.9.1
Requires-Dist: matplotlib<4,>=3.8
Requires-Dist: numcodecs<0.16,>=0.15
Requires-Dist: portalocker>=2.8.0
Requires-Dist: rich>=13.7.0
Requires-Dist: typer>=0.9.0
Requires-Dist: universal-pathlib>=0.2
Requires-Dist: xarray<2026,>=2025.9.1
Requires-Dist: xradio[all]<0.1,>=0.0.59
Requires-Dist: zarr<3,>=2.16
Provides-Extra: all
Requires-Dist: bokeh<4,>=3.3; extra == 'all'
Requires-Dist: holoviews<2,>=1.18; extra == 'all'
Requires-Dist: ipyaladin>=0.5.0; extra == 'all'
Requires-Dist: ipywidgets-bokeh>=1.5; extra == 'all'
Requires-Dist: ipywidgets<9,>=8.0; extra == 'all'
Requires-Dist: jupyter-bokeh>=4.0; extra == 'all'
Requires-Dist: panel<2,>=1.3; extra == 'all'
Requires-Dist: param<3,>=2.0; extra == 'all'
Requires-Dist: pre-commit; extra == 'all'
Requires-Dist: pytest-cov; extra == 'all'
Requires-Dist: pytest-mock; extra == 'all'
Requires-Dist: pytest-xdist; extra == 'all'
Requires-Dist: pytest>=6; extra == 'all'
Provides-Extra: ci
Requires-Dist: python-dotenv<2,>=1.2.1; extra == 'ci'
Requires-Dist: s3fs>=2024.6.0; extra == 'ci'
Requires-Dist: tqdm<5,>=4.67.1; extra == 'ci'
Provides-Extra: dev
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: pytest-mock; extra == 'dev'
Requires-Dist: pytest-xdist; extra == 'dev'
Requires-Dist: pytest>=6; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
Requires-Dist: mkdocs>=1.6.0; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25.0; extra == 'docs'
Provides-Extra: prefect
Requires-Dist: prefect>=3.0.0; extra == 'prefect'
Provides-Extra: remote
Requires-Dist: caltechdata-api>=1.0.0; extra == 'remote'
Requires-Dist: fsspec>=2024.6.0; extra == 'remote'
Requires-Dist: gcsfs>=2024.6.0; extra == 'remote'
Requires-Dist: requests>=2.31.0; extra == 'remote'
Requires-Dist: s3fs>=2024.6.0; extra == 'remote'
Provides-Extra: visualization
Requires-Dist: bokeh<4,>=3.3; extra == 'visualization'
Requires-Dist: holoviews<2,>=1.18; extra == 'visualization'
Requires-Dist: ipyaladin>=0.5.0; extra == 'visualization'
Requires-Dist: ipywidgets-bokeh>=1.5; extra == 'visualization'
Requires-Dist: ipywidgets<9,>=8.0; extra == 'visualization'
Requires-Dist: jupyter-bokeh>=4.0; extra == 'visualization'
Requires-Dist: panel<2,>=1.3; extra == 'visualization'
Requires-Dist: param<3,>=2.0; extra == 'visualization'
Description-Content-Type: text/markdown

# OVRO-LWA Portal

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.XXXXXXX.svg)](https://doi.org/10.5281/zenodo.XXXXXXX)
[![PyPI](https://img.shields.io/pypi/v/ovro-lwa-portal)](https://pypi.org/project/ovro-lwa-portal/)

A Python library for radio astronomy data processing and visualization for the
Owens Valley Radio Observatory - Long Wavelength Array (OVRO-LWA).

## Features

- **Unified Data Loading**: Load OVRO-LWA data from local paths, remote URLs
  (S3, HTTPS), or DOI identifiers with a single `open_dataset()` function
- **FITS to Zarr Conversion**: Convert OVRO-LWA FITS image files to
  cloud-optimized Zarr format
- **Command-Line Interface**: User-friendly `ovro-ingest` CLI with progress
  tracking
- **WCS Coordinate Preservation**: Maintain celestial coordinates (RA/Dec) for
  FITS-free analysis
- **Incremental Processing**: Append new observations to existing Zarr stores
- **Concurrent Write Protection**: File locking prevents data corruption from
  simultaneous processes
- **Optional Workflow Orchestration**: Prefect integration for production
  deployments

## Prerequisites

This project uses [Pixi](https://pixi.sh) for dependency management and task
execution. Install Pixi by following the
[installation instructions](https://pixi.sh/latest/#installation).

## Getting Started

### Installation

#### For Users

Install the package using pip:

```bash
pip install git+https://github.com/uw-ssec/ovro-lwa-portal.git
```

Or install from a local clone:

```bash
pip install .
```

#### For Developers

Install dependencies using Pixi:

```bash
# Install dependencies (Pixi will automatically create the environment)
pixi install
```

For detailed installation and development instructions, see
[CONTRIBUTING.md](CONTRIBUTING.md).

### Onboarding

For first-time setup, use the onboarding environment to configure your
development environment:

```bash
pixi run -e onboard onboard
```

This will:

- Install pre-commit hooks in your git repository
- Set up shell completion for ssec-cli
- Run the SSEC onboarding process

## Project Structure

This project is organized using Pixi features for modular dependency management:

- **`pre-commit`**: Code quality and consistency checks
- **`gh-cli`**: GitHub CLI for repository interactions
- **`onboard`**: Tools for project onboarding and setup

## Available Environments

- **`default`**: Standard development environment with pre-commit hooks and
  GitHub CLI
- **`onboard`**: Extended environment including onboarding tools

## Development

### Using Different Environments

Switch between environments as needed:

```bash
# Use default environment
pixi shell

# Use onboard environment
pixi shell -e onboard
```

### Adding Dependencies

Edit `pyproject.toml` to add new dependencies in the `[tool.pixi.dependencies]`
section:

```toml
[tool.pixi.dependencies]
your-package = ">=1.0.0"
```

Then run:

```bash
pixi install
```

or

Directly add packages (this will edit the pyproject.toml and install):

```bash
pixi add your-package
```

## Quick Start

### Loading OVRO-LWA Data

Load data from various sources with a unified interface:

```python
import ovro_lwa_portal

# Load from local zarr store
ds = ovro_lwa_portal.open_dataset("/path/to/observation.zarr")

# Load from remote URL
ds = ovro_lwa_portal.open_dataset("s3://ovro-lwa-data/obs_12345.zarr")

# Load via DOI
ds = ovro_lwa_portal.open_dataset("doi:10.5281/zenodo.1234567")

# Customize chunking for large datasets
ds = ovro_lwa_portal.open_dataset(
    "path/to/data.zarr",
    chunks={"time": 100, "frequency": 50}  # or chunks="auto" (default), chunks=None
)
```

For remote data access, install with remote extras:

```bash
pip install 'ovro_lwa_portal[remote]'
```

See the [open_dataset documentation](docs/open_dataset.md) for more details.

### Using the FITS to Zarr Ingest CLI

After installation, convert OVRO-LWA FITS files to Zarr format:

```bash
# Basic conversion
ovro-ingest convert /path/to/fits /path/to/output

# With custom options
ovro-ingest convert /path/to/fits /path/to/output \
    --zarr-name my_data.zarr \
    --chunk-lm 2048 \
    --rebuild

# Show help
ovro-ingest convert --help
```

For detailed documentation on the ingest module, see the
[Ingest Module README](src/ovro_lwa_portal/ingest/README.md).

### Using the Python API

```python
from pathlib import Path
from ovro_lwa_portal.ingest import FITSToZarrConverter
from ovro_lwa_portal.ingest.core import ConversionConfig

# Configure conversion
config = ConversionConfig(
    input_dir=Path("/path/to/fits"),
    output_dir=Path("/path/to/output"),
    zarr_name="ovro_lwa_data.zarr",
    chunk_lm=1024,
)

# Execute conversion
converter = FITSToZarrConverter(config)
result = converter.convert()
print(f"Created: {result}")
```

## Technology Stack

- **Core**: Python 3.12, xarray, dask, zarr
- **Astronomy**: astropy, xradio, python-casacore
- **CLI**: typer, rich (progress bars and formatted output)
- **Workflow**: prefect (optional orchestration)
- **Storage**: Zarr format optimized for cloud access
- **Environment Management**: pixi

## Contributing

Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of
conduct and the process for submitting pull requests.

## License

This project is licensed under the terms specified in the [LICENSE](LICENSE)
file.

## Project Resources

- eScience Slack channel: 🔒
  [#ssec-ovro-lwa-portal](https://escience-institute.slack.com/archives/C098GJYLNBW)
- SSEC Sharepoint (**INTERNAL SSEC ONLY**): 🔒
  [Projects/OVROXarrraySciPlt](https://uwnetid.sharepoint.com/:f:/r/sites/og_ssec_escience/Shared%20Documents/Projects/OVROXarrraySciPlt?csf=1&web=1&e=P5QKAc)
- Shared Sharepoint Directory: 🔒
  [UW SSEC Caltech OVRO-LWA Portal Shared Folder](https://uwnetid.sharepoint.com/:f:/r/sites/og_ssec_escience/Shared%20Documents/Projects/OVROXarrraySciPlt/UW%20SSEC%20Caltech%20OVRO-LWA%20Portal%20Shared%20Folder?csf=1&web=1&e=siXUk2)
- [User Stories Document 🔒](https://uwnetid.sharepoint.com/:w:/r/sites/og_ssec_escience/Shared%20Documents/Projects/OVROXarrraySciPlt/UW%20SSEC%20Caltech%20OVRO-LWA%20Portal%20Shared%20Folder/SSEC%20OVRO-LWA%20Portal%20User%20Stories.docx?d=w15624ab2d3c0475e95a2865a346e359b&csf=1&web=1&e=ImDH96)

## General Discussions

For general discussion, ideas, and resources please use the
[GitHub Discussions](https://github.com/uw-ssec/ovro-lwa-portal/discussions).
However, if there's an internal discussion that need to happen, please use the
slack channel provided.

- Meeting Notes in GitHub:
  [discussions/meetings](https://github.com/uw-ssec/ovro-lwa-portal/discussions/categories/meetings)

## Citation

If you use this software in your research, please cite it:

```bibtex
@software{ovro_lwa_portal,
  title = {OVRO-LWA Portal},
  author = {Core, Cordero and Setiawan, Don and Tambay, Anshul T. and Kosogorov, Nikita and Johari, Ishika},
  url = {https://github.com/uw-ssec/ovro-lwa-portal},
  license = {BSD-3-Clause}
}
```

## Questions

If you have any questions about our process, or locations of SSEC resources,
please ask [Anshul Tambay](https://github.com/atambay37).
