Metadata-Version: 2.4
Name: xpcs-correlator
Version: 0.3.0
Summary: XPCS correlation calculations for synchrotron experiments
Author-email: Maciej Jankowski <maciej.jankowski@esrf.fr>
License-Expression: MIT
Project-URL: Repository, https://gitlab.esrf.fr/mj/xpcs_developments/xpcscorr
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: numba
Requires-Dist: dask
Requires-Dist: dask_jobqueue
Requires-Dist: h5py
Requires-Dist: hdf5plugin
Requires-Dist: threadpoolctl
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: memory_profiler; extra == "dev"
Requires-Dist: line_profiler; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: scipy; extra == "dev"
Provides-Extra: doc
Requires-Dist: sphinx; extra == "doc"
Requires-Dist: sphinx-rtd-theme; extra == "doc"
Requires-Dist: sphinxcontrib-napoleon; extra == "doc"
Requires-Dist: nbsphinx; extra == "doc"
Requires-Dist: nbclient; extra == "doc"
Requires-Dist: nbformat; extra == "doc"
Requires-Dist: nbconvert; extra == "doc"
Requires-Dist: jupyter; extra == "doc"
Requires-Dist: pandoc; extra == "doc"
Requires-Dist: numpy; extra == "doc"
Dynamic: license-file

# xpcs-correlator

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) [![Repository](https://img.shields.io/badge/Repo-GitLab-ff69b4.svg)](https://gitlab.esrf.fr/mj/xpcs_developments/xpcscorr)

## Table of contents
- About
- Documentation
- Features
- Requirements
- Installation
- Quickstart
- Configuration / Logging
- Tests
- Contributing
- License
- Contact

## About
This package consolidates ongoing development of correlators for XPCS data analysis at ESRF, with a focus on the ID02 and ID10-coh beamlines.

## Documentation
The documentation is hosted online: [https://mj.gitlab-pages.esrf.fr/xpcs_developments/xpcscorr/](https://mj.gitlab-pages.esrf.fr/xpcs_developments/xpcscorr/)

## Features
- Dense frames data reference and chunked correlator implementations.
- Calculates g2, g2 errors, and ttcf (2-time correlation function).
- The ttcf calculations support linear binning for t1,t2 format and hybrid linear log binning for age,lag format.
- Designed to handle large frame stacks via chunked (partitioned) processing.
- Supports Dask for both cluster (SLURM) and local parallel execution

## Requirements
- Python 3.10+ (recommended: 3.10, 3.11, 3.12)
- numpy
- numba
- dask
- dask_jobqueue
- h5py
- hdf5plugin
- threadpoolctl

## Installation
Install in editable/develop mode (recommended during development):

```bash
pip install -e .
```

Install with development extras (for running tests and linters):

```bash
pip install -e .[dev]
```

Install from PyPI the package for regular use:

```bash
pip install xpcs-correlator
```

## Quickstart
For a step-by-step walkthrough with examples and runnable code, see the Tutorial in the online documentation: [Tutorial](https://mj.gitlab-pages.esrf.fr/xpcs_developments/xpcscorr/notebooks/basics.html).

Basic usage example — adapt to your data shape and correlator options:

```python
import numpy as np
from xpcscorr import correlator_dense_reference, correlator_dense_chunked

# Replace with your frames array; shape here is (n_frames, nx_pixels, ny_pixels)
frames = np.random.random((100,512, 512))
roimask= np.ones((512,512), dtype=bool)

# Run reference  correlator
result_ref = correlator_dense_reference(frames, roimask)

# Run chunked correlator (handles large data in chunks)
extra_options = {'chunks_N': 3}
result_chunked = correlator_dense_chunked(frames, roimask, extra_options=extra_options)

print(type(result_ref), type(result_chunked))
```

Notes:
- Replace the synthetic `frames` with your real dataset (HDF5 dataset or numpy array).
- Check correlator function docstrings for exact argument names and options.

## Configuration / Logging

### Standard Approach (Recommended)

`xpcscorr` follows Python library best practices and does not configure logging handlers by default. This means your application controls all logging configuration:

```python
import logging
import xpcscorr

# Configure logging in your application
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s %(name)s %(levelname)s: %(message)s'
)

# Now xpcscorr logs will appear according to your configuration
result = xpcscorr.correlator_dense_reference(frames, roimask)
```

### Convenience Function with Dask Worker Support

For quick setup that also configures **Dask worker logging**, use the `setup_logging()` convenience function:

```python
import xpcscorr

# Enable both file and console logging (including Dask workers)
xpcscorr.setup_logging(log_to_file=True, log_to_cli=True)

# Or let it read from environment variables (if not set, uses defaults below)
xpcscorr.setup_logging()
```

**Important:** `setup_logging()` sets environment variables that Dask workers inherit, enabling logging in worker processes. Without calling this function (or manually setting environment variables), worker processes will not log.

### Environment Variables

The following environment variables control logging behavior:

- **`XPCSCORR_LOG_TO_FILE`**: `{'0', '1', 'true', 'false', 'yes', 'no'}`  
  Enable file logging. Default when calling `setup_logging()` without arguments: `'1'` (enabled)

- **`XPCSCORR_LOG_TO_CLI`**: `{'0', '1', 'true', 'false', 'yes', 'no'}`  
  Enable console logging to stdout. Default when calling `setup_logging()` without arguments: `'0'` (disabled)

- **`XPCSCORR_DEBUG_MEMORY`**: `{'0', '1', 'true', 'false', 'yes', 'no'}`  
  Enable memory profiling with `tracemalloc`. Default: `'0'` (disabled)

You can set these before running your script:

```bash
export XPCSCORR_LOG_TO_CLI=1
export XPCSCORR_LOG_TO_FILE=0
export XPCSCORR_DEBUG_MEMORY=1
python my_script.py
```

Or in your Python code before importing:

```python
import os
os.environ['XPCSCORR_LOG_TO_CLI'] = '1'
import xpcscorr
```

### Log File Locations

When file logging is enabled, log files are created in platform-specific locations:
- **Linux:** `~/.cache/xpcscorr/xpcscorr.log`
- **macOS:** `~/Library/Logs/xpcscorr/xpcscorr.log`
- **Windows:** `%LOCALAPPDATA%\xpcscorr\xpcscorr.log`

**Dask workers** on LocalCluster create additional log files in `./dask-worker-local-logs/worker_{pid}.log` when `XPCSCORR_LOG_TO_FILE=1`. SLURM workers log to stdout (captured in SLURM `.out` files).

## Tests
Run tests with pytest:

```bash
pip install -e .[dev]
pytest -q
```

There are unit tests under `tests/` that exercise correlator behavior and core utilities.

## Contributing
- Open issues for bugs or feature requests.
- Fork the repo, create a feature branch, add tests, and submit a pull request.
- Keep changes small, document API changes, and add tests for new behavior.

## License
This project is licensed under the MIT License — see the `LICENSE` file for details.

## Contact
Maintainer: Maciej Jankowski — maciej.jankowski@esrf.fr


<!-- EOF -->

