Metadata-Version: 2.4
Name: roman-wfi-reference-file-pipeline
Version: 0.0.3
Summary: Reference file pipeline for the Roman WFI instrument.
Author: STScI
Project-URL: Source Code, https://grit.stsci.edu/roman/wfi_reference_pipeline
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: astropy<7.2.0,>=7.0.0
Requires-Dist: crds>=12.0.0
Requires-Dist: h5py
Requires-Dist: matplotlib<3.11.0,>=3.10.0
Requires-Dist: numpy<2.4.0,>=2.1.0
Requires-Dist: pandas<2.4.0,>=2.2.1
Requires-Dist: photutils>=2.3.0
Requires-Dist: psutil
Requires-Dist: pysiaf<0.28.0,>=0.27.0
Requires-Dist: PyYAML<6.1.0,>=6.0.1
Requires-Dist: requests<2.33.0,>=2.32.0
Requires-Dist: romancal>=1.1.0
Requires-Dist: scipy<1.17.0,>=1.15.0
Requires-Dist: soc_roman_tools<0.2.0,>=0.1.0
Requires-Dist: stpsf>=2.2.0
Provides-Extra: docs
Requires-Dist: sphinx; extra == "docs"
Requires-Dist: stsci_rtd_theme; extra == "docs"
Requires-Dist: sphinx_automodapi; extra == "docs"
Provides-Extra: test
Requires-Dist: pytest-cov; extra == "test"
Requires-Dist: pytest; extra == "test"
Requires-Dist: ruff; extra == "test"
Requires-Dist: pytest-mock; extra == "test"
Requires-Dist: genbadge[coverage]; extra == "test"
Dynamic: license-file

# Roman WFI Reference File Pipeline

> [!CAUTION]
> The code in this repository is under heavy development and not currently intended for widespread use.
>
> If you have questions, see the CONTRIBUTING.md.

![Coverage](https://raw.githubusercontent.com/spacetelescope/roman-wfi-reference-pipeline/coverage-info/coverage-badge.svg)

## Installation

It is recommended to begin with a clean environment, such as:
```buildoutcfg
conda create -n wfirefpipe python=3.12
```
To install the package for general usage:
```buildoutcfg
pip install .
```
OR
To install the package for development and testing:
```buildoutcfg
pip install -e .[docs,test]
```
OR
To install the package utilizing any database functionality (needed for automation or QC checks) please follow the [Additional instructions below](#installation-with-rtb-database-access)

This will also install all of the dependencies.

> NOTE: Installing this way uses the dependencies outlined in `pyproject.toml`.
> These dependencies mirror `requirements.txt`.
> If you would like to install with custom dependencies,
> just update them in requirements.txt and `pip install -r requrements.txt`

Users must create a config.yml living at:
```/wfi_reference_pipeline/src/wfi_reference_pipeline/config/config.yml```
Use example_config.yml in the same directory as a template.


## Installation with RTB database access

RTB Database is utilized for automated processing of the reference file pipeline at STScI.  If you are not setting up automated instances at STScI then you will not need access to the RTB Database and should skip this section.

To install with the RTB database on most platforms, the main additional step is to install the SQL driver packages with anaconda. This is the easiest way to ensure the correct SQL drivers are installed regardless of operating system. The one exception are Macs with M1 processors. For these installations, please see [macOS M1 installation](#notes-on-macos-machine-with-an-m1-chip-to-be-tested).


Notes:
    1) You will need access to the Roman Grit group and might need to configure git on your machine to use a username and password that are registered with the Roman Grit group.
    2) If you have 2FA enabled for your Grit account, you will need to use an access token to install rtbdb.  Follow the instructions to [create an access token](https://grit.stsci.edu/help/user/profile/personal_access_tokens.md#create-a-personal-access-token).  This access token can now be used as your user password when prompted during the install.

Using conda, please run the following in the specified order:
```
conda activate wfirefpipe
"pip install git+https://grit.stsci.edu/roman/rtb-database"
pip install .
conda install freetds
```

The last step is creating some SQL configuration files.
You will need to set some environmental variables and create some driver configuration files. Please follow the instructions in the "Configuring SQL drivers" section of the README.md file from the [RTB Database repository](https://grit.stsci.edu/roman/rtb-database#configuring-sql-drivers).

If experiencing any issues connecting to the server, please check out the [rtbdb troubleshooting](https://grit.stsci.edu/roman/rtb-database#connecting-to-the-server-and-troubleshooting).


## Contributing
> [!WARNING]
> We are not currently accepting external Pull Requests. However, we plan to
> use the guidelines below in the near future.

To contribute to this project, please become familiar with our [Contributing Guide](https://github.com/spacetelescope/roman-wfi-reference-pipeline/blob/main/CONTRIBUTING.md)

## Documentation

To get the full documentation, install the package as described above, install sphinx,
then go to the /src/docs/ directory and run:

```buildoutcfg
make html
```

The documentation can then be found in doc/build/html/index.html.

## Slack Integration

To use the Slack notifications, you must set up a Slack token.
When you have a token, you can point to it with the environment variable WFI_SLACK_TOKEN.

## Updating reference files on CRDS

Go to your .bash_profile and set environment variables to determine the crds state.

This is for the Roman test instance of CRDS.
```buildoutcfg
export CRDS_SERVER_URL="https://roman-crds.stsci.edu"
export CRDS_PATH="/PATH/TO/ops_crds/"
```

Start with a clean environment where you will get the latest versions of romancal, roman attribute
dictionary, and roman data models.
```buildoutcfg
conda create -yn VMdevRFP_update_refs ipython
conda activate VMdevRFP_update_refs
pip install romancal crds
```
By doing this pip install you will get the latest released versions since this workflow is setup
to replace old reference files with the most current.

Now sync crds to get all of the mappings updated.
```buildoutcfg
crds sync --all
```

See the update_reference_files.py script in examples that was done for Build 17 in April 2025. Also see
```buildoutcfg
navigate to central store's roman directory, then:
./RFP/DEV/py_scripts_notebooks/build_pyscripts/update_all_TVAC_CRDS_ref_files.py
```

## Creating a New (SOC) Calibration Reference File Type in this Repository

These instructions are currently for adding new calibration reference file types are that Science Operations Center (SOC) will utilize in development with the Roman Calibration Pipeline (romancal). Instructions for external community derived products will be provided at a later date.
<!-- TODO: replace with updated information -->


To create a new reference file type, please see [New Reference Type Meta Data Workflow Example](src/docs/new_reftype_meta_example_workflow.md) This allows a new
effect or calibration reference file type to be defined within the structure of the repository.

For a detailed walkthrough on how to create a new reference type class, see the
[New Reference Type Class Example](src/docs/new_reftype_class_example.md).

Once your module is implemented, you can follow the testing guidelines and examples here:
[Basic Testing for New Reference Types](src/docs/new_reftype_class_basic_tests_example.md).

These resources provide step-by-step examples to help ensure both your implementation and test coverage follow the expected structure and conventions used throughout the pipeline.

