Metadata-Version: 2.4
Name: skordinal
Version: 0.0.1
Summary: Ordinal classification for Python, scikit-learn compatible
Author-email: Francisco Bérchez-Moreno <fberchez@uco.es>, Víctor Manuel Vargas <vvargas@uco.es>, Javier Barbero-Gómez <jbarbero@uco.es>, Rafael Ayllón-Gavilán <rayllong@uco.es>, David Guijo-Rubio <dguijo@uco.es>, Ángel Sevilla Molina <asevilla@uco.es>
License: BSD-3-Clause
Project-URL: Homepage, https://github.com/ayrna/skordinal
Project-URL: Repository, https://github.com/ayrna/skordinal
Project-URL: Bug Tracker, https://github.com/ayrna/skordinal/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: POSIX
Classifier: Operating System :: Unix
Classifier: Programming Language :: C
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: numpy>=1.21
Requires-Dist: pandas>=1.0.1
Requires-Dist: scipy>=1.7
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-xdist[psutil]; extra == "dev"
Requires-Dist: pytest-benchmark; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.9; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Dynamic: license-file

# skordinal

| Overview  |                                                                                                                                          |
|-----------|------------------------------------------------------------------------------------------------------------------------------------------|
| **CI/CD** | [![Unit tests](https://github.com/ayrna/skordinal/actions/workflows/unit-tests.yml/badge.svg?branch=main)](https://github.com/ayrna/skordinal/actions/workflows/unit-tests.yml) [![!python](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/) |
| **Code**  | [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![License - BSD 3-Clause](https://img.shields.io/pypi/l/pandas.svg)](https://github.com/ayrna/skordinal/blob/main/LICENSE) |


## What is skordinal?

**skordinal** is an experimental framework built on Python that integrates with scikit-learn to automate machine learning experiments through simple JSON configuration files. Initially designed for ordinal classification, it supports regular classification algorithms as long as they are compatible with scikit-learn, making it easy to run reproducible experiments across multiple datasets and classification methods.

## Table of Contents

- [Installation](#installation)
    - [Requirements](#requirements)
    - [Setup](#setup)
    - [Testing Installation](#testing-installation)
- [Quick Start](#quick-start)
- [Configuration Files](#configuration-files)
    - [general-conf](#general-conf)
    - [configurations](#configurations)
- [Running Experiments](#running-experiments)
    - [Basic Usage](#basic-usage)
    - [Example Output](#example-output)
- [License](#license)

## Installation

### Requirements

skordinal requires Python 3.9 or higher and is tested on Python 3.9, 3.10, 3.11, and 3.12.

All dependencies are managed through `pyproject.toml` and include:
- numpy (>=1.21)
- pandas (>=1.0.1)
- scikit-learn (>=1.3.0)
- scipy (>=1.7)

### Setup

1. **Clone the repository**:
   ```bash
   git clone https://github.com/ayrna/skordinal
   cd skordinal
   ```

2. **Install the framework**:
   ```bash
   pip install .
   ```

   For development purposes, use editable installation:
   ```bash
   pip install -e .
   ```

   Optional dependencies for development:
   ```bash
   pip install -e .[dev]
   ```

> **Note:** The editable mode is required for running tests due to automatic dependency resolution.

### Testing Installation

Test your installation with the provided example:

```bash
python config.py with skordinal/configurations/full_functionality_test.json
```

## Quick Start

skordinal includes sample datasets with pre-partitioned train/test splits using a 30-holdout experimental design.

**Basic experiment configuration:**

```json
{
    "general_conf": {
        "basedir": "skordinal/datasets/data",
        "datasets": ["balance-scale", "contact-lenses", "tae"],
        "hyperparam_cv_nfolds": 3,
        "output_folder": "results/",
        "metrics": ["ccr", "mae", "amae"],
        "cv_metric": "mae"
    },
    "configurations": {
        "SVM": {
            "classifier": "SVC",
            "parameters": {
                "C": [0.001, 0.1, 1, 10, 100],
                "gamma": [0.1, 1, 10]
            }
        },
        "SVMOP": {
            "classifier": "OrdinalDecomposition",
            "parameters": {
                "dtype": "ordered_partitions",
                "decision_method": "frank_hall",
                "base_classifier": "SVC",
                "parameters": {
                    "C": [0.01, 0.1, 1, 10],
                    "gamma": [0.01, 0.1, 1, 10],
                    "probability": ["True"]
                }
            }
        }
    }
}
```

**Run the experiment:**
```bash
python config.py with my_experiment.json
```

Results are saved in `results/` folder with performance metrics for each dataset-classifier combination. The framework automatically performs cross-validation, hyperparameter tuning, and evaluation on test sets.

## Configuration Files

Experiments are defined using JSON configuration files with two main sections: general_conf for experiment settings and configurations for classifier definitions.

### general-conf

Controls global experiment parameters.

**Required parameters:**
- **`basedir`**: folder containing all dataset subfolders, it doesn't allow more than one folder at a time. It can be indicated using a full path, or a relative one to the framework folder.
- **`datasets`**: name of datasets that will be experimented with. A subfolder with the same name must exist inside `basedir`.

**Optional parameters:**
- **`hyperparam_cv_folds`**: number of folds used while cross-validating.
- **`jobs`**: number of jobs used for GridSearchCV during cross-validation.
- **`input_preprocessing`**: data preprocessing (`"std"` for standardization, `"norm"` for normalization, `""` for none)
- **`output_folder`**: name of the folder where all experiment results will be stored.
- **`metrics`**: name of the accuracy metrics to measure the train and test performance of the classifier.
- **`cv_metric`**: error measure used for GridSearchCV to find the best set of hyper-parameters.

### configurations

Defines classifiers and their hyperparameters for GridSearchCV. Each configuration has a name and consists of:

- **`classifier`**: scikit-learn or built-in skordinal classifier
- **`parameters`**: hyperparameters for grid search (nested for ensemble methods)

## Running Experiments

### Basic Usage

```bash
python config.py with experiment_file.json
```

### Example Output

Results are stored in the specified output folder with detailed performance metrics and hyperparameter information for each dataset and configuration combination.

## License
[BSD 3](LICENSE)

<hr>

[Go to Top](#table-of-contents)
