Metadata-Version: 2.4
Name: illustris
Version: 0.1.0
Summary: Python tools for accessing and analyzing Illustris simulation data
Project-URL: Homepage, https://www.illustris-project.org/
Project-URL: Documentation, https://your-docs-url.com
Project-URL: Repository, https://github.com/bjoernbethge/illustris
Project-URL: Issues, https://github.com/bjoernbethge/illustris/issues
Project-URL: Changelog, https://github.com/bjoernbethge/illustris/releases
Author-email: Björn Bethge <bjoern.bethge@gmail.com>
Maintainer-email: Björn Bethge <bjoern.bethge@gmail.com>
License: Copyright (c) 2017, illustris & illustris_python developers
        Copyright (c) 2024, illustris modernized fork contributors
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.
        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
        ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
        WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
        ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
        (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
        LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
        ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
        (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
        SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
        
        The views and conclusions contained in the software and documentation are those
        of the authors and should not be interpreted as representing official policies,
        either expressed or implied, of the FreeBSD Project.
License-File: LICENSE.md
Keywords: astronomy,astrophysics,cosmology,illustris,simulation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Astronomy
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: >=3.11
Requires-Dist: h5py>=3.14.0
Requires-Dist: httpx>=0.28.1
Requires-Dist: numpy>=2.2.6
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: six>=1.17.0
Description-Content-Type: text/markdown

# Illustris (Modernized Fork)

**Modern Python package for reading and analyzing Illustris simulation data.**

This is a modernized and extended fork of the original [illustris_python](https://github.com/illustristng/illustris_python) package, featuring:

- 🚀 **Modern CLI tools** with async data downloading
- 🧪 **Comprehensive pytest test suite** with fixtures
- 📚 **Complete documentation** with examples and guides  
- ⚡ **Performance optimizations** and better error handling
- 🔧 **Developer-friendly** setup with uv and modern Python practices

> **Note**: This fork maintains full API compatibility with the original package while adding significant new functionality.

## Features

- **Data Loading**: Load snapshots, group catalogs, and merger trees
- **CLI Tools**: Command-line interface for data management and documentation
- **Modern Testing**: Comprehensive test suite with pytest fixtures
- **Documentation**: Auto-generated Sphinx documentation

## Installation

```bash
# Install with uv (recommended)
uv add illustris
```
## Development

```bash
# Clone the repository
git clone https://github.com/bjoernbethge/illustris.git
cd illustris

# Install with uv (recommended)
uv sync --dev
```

## Quick Start

### CLI Usage

The `illustris` command provides tools for data management and documentation:

```bash
# Documentation
illustris -docs -generate          # Build documentation
illustris -docs -serve             # Serve documentation (port 8000)
illustris -docs -serve -p 8080     # Serve on custom port

# Data Management
illustris -data -test              # Download complete test data (TNG50-4, all files)
illustris -data -load TNG50-4      # Download latest snapshot
illustris -data -load TNG50-4 -snap 99  # Download specific snapshot
illustris -data -list-sims         # List available simulations
illustris -data -list-snaps TNG50-4 # List snapshots for simulation
```

### Python API

```python
import illustris

# Load snapshot data
gas_data = illustris.snapshot.loadSubset(
    basePath="data/TNG50-4/output", 
    snapNum=99, 
    partType='gas',
    fields=['Coordinates', 'Masses']
)

# Load group catalog
halos = illustris.groupcat.loadHalos(
    basePath="data/TNG50-4/output",
    snapNum=99,
    fields=['GroupMass', 'GroupPos']
)

# Load subhalos
subhalos = illustris.groupcat.loadSubhalos(
    basePath="data/TNG50-4/output", 
    snapNum=99,
    fields=['SubhaloMass', 'SubhaloPos']
)
```

## Configuration

Create a `.env` file for TNG API access:

```bash
# Copy example configuration
cp env.example .env

# Edit with your API key
ILLUSTRIS_API_KEY=your_api_key_here
ILLUSTRIS_DATA_DIR=./data  # Optional: custom data directory
```

Get your API key from: https://www.tng-project.org/users/register/

## Testing

The project includes comprehensive tests using pytest:

```bash
# Run all tests
uv run pytest

# Run specific test modules
uv run pytest tests/test_snapshot.py
uv run pytest tests/test_groupcat.py

# Run with coverage (automatically included)
uv run pytest

# Run with detailed coverage report
uv run pytest --cov-report=term-missing

# Run only tests that don't require data
uv run pytest -m "not requires_data"
```

### Test Data

**Complete Test Data**:
```bash
illustris -data -test
```
- Downloads ~5.3 GB of complete TNG50-4 dataset
- Enables all functionality tests including:
  - All 11 snapshot chunks for `loadSubset` testing
  - Group catalog files for halo/subhalo analysis
  - Offsets files for `loadHalo` testing  
  - SubLink trees for merger analysis
- Recommended for development and full validation

### Test Results

With complete test data:
- ✅ **26 tests pass** - Full functionality tested
- ⏭️ **0 tests skipped** - All features available

Tests automatically detect available data and skip appropriately if files are missing.

## Development

### Project Structure

```
illustris/
├── src/illustris/          # Main package
│   ├── cli.py             # Command-line interface
│   ├── snapshot.py        # Snapshot data loading
│   ├── groupcat.py        # Group catalog loading
│   ├── sublink.py         # Merger tree loading
│   └── util.py            # Utility functions
├── tests/                 # Test suite
│   ├── conftest.py        # Pytest fixtures
│   ├── test_basic.py      # Basic functionality tests
│   ├── test_snapshot.py   # Snapshot loading tests
│   ├── test_groupcat.py   # Group catalog tests
│   └── test_sublink.py    # Merger tree tests
├── docs/                  # Documentation
└── data/                  # Downloaded simulation data
```

### Adding Tests

Tests use pytest fixtures for consistent data access:

```python
import pytest
import illustris

@pytest.mark.requires_data
def test_my_function(test_data_path, test_snapshot):
    """Test with real simulation data."""
    data = illustris.snapshot.loadSubset(
        test_data_path, test_snapshot, 'gas'
    )
    assert data is not None

def test_basic_function():
    """Test without data dependencies."""
    result = illustris.snapshot.partTypeNum('gas')
    assert result == 0
```

### Building Documentation

```bash
# Generate API documentation
illustris -docs -generate

# Serve locally with auto-reload
illustris -docs -serve

# Documentation will be available at http://localhost:8000
```

## API Reference

### Snapshot Module

- `loadSubset(basePath, snapNum, partType, fields=None)` - Load particle data
- `loadHalo(basePath, snapNum, haloID, partType, fields=None)` - Load halo particles
- `partTypeNum(partType)` - Get particle type number
- `snapPath(basePath, snapNum, chunkNum=0)` - Get snapshot file path

### GroupCat Module

- `loadHalos(basePath, snapNum, fields=None)` - Load halo catalog
- `loadSubhalos(basePath, snapNum, fields=None)` - Load subhalo catalog
- `loadSingle(basePath, snapNum, haloID=None, subhaloID=None)` - Load single object
- `gcPath(basePath, snapNum, chunkNum=0)` - Get group catalog path

### SubLink Module

- `loadTree(basePath, snapNum, id, fields=None, onlyMPB=False)` - Load merger tree
- `numMergers(tree, minMassRatio=1e-10, massPartType='stars')` - Count mergers
- `treePath(basePath, chunkNum=0)` - Get tree file path

## License

This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details.

## Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Add tests for new functionality
5. Run the test suite (`uv run pytest`)
6. Commit your changes (`git commit -m 'Add amazing feature'`)
7. Push to the branch (`git push origin feature/amazing-feature`)
8. Open a Pull Request

## Changelog

### Latest Changes

- **Simplified CLI**: Streamlined data management commands
- **Complete Test Data**: Single `-test` option for full dataset downloads
- **Modern Testing**: Migrated from nose to pytest with fixtures
- **API Integration**: Direct TNG API access for data downloads
- **Improved Documentation**: Auto-generated Sphinx docs with CLI integration
- **Better Error Handling**: Robust error handling for missing data
- **Smart Test Skipping**: Tests automatically adapt to available data

### Data Download Features

- **Complete Downloads**: Full dataset for comprehensive testing (~5.3 GB)
- **Intelligent Chunk Detection**: Automatically detects all available file chunks
- **Automatic TNG API authentication**
- **Progress indicators for large downloads**
- **Smart file organization** (snapshots, group catalogs, offsets, trees)
- **Support for all TNG simulations** (TNG50, TNG100, TNG300, Illustris)
- **Robust timeout handling** for large files

### Testing Improvements

- **Session-scoped fixtures** for efficient testing
- **Automatic data directory detection**
- **Separate tests for data-dependent and basic functionality**
- **Comprehensive validation** of file structures and data integrity
- **Smart skip mechanisms** for unavailable data
- **Detailed test output** with progress indicators

## Support

- **Documentation**: https://bjoernbethge.github.io/illustris
- **TNG Project**: https://www.tng-project.org/
- **API Documentation**: https://www.tng-project.org/data/docs/api/

## Troubleshooting

### Common Issues

**Permission Denied Errors**:
```bash
# Make sure you have write permissions to the data directory
chmod 755 data/
```

**Incomplete Downloads**:
```bash
# Remove corrupted files and re-download
rm -rf data/TNG50-4/output/snapdir_099/
illustris -data -test
```

**Missing API Key**:
```bash
# Set up your environment file
cp env.example .env
# Edit .env and add your TNG API key
```

**Test Failures**:
```bash
# Download complete test data for full functionality
illustris -data -test
uv run pytest tests/ -v
```
