Metadata-Version: 2.4
Name: FranKGraphBench
Version: 0.2.0
Summary: pip install package for frankgraphbench.
Home-page: https://github.com/AKSW/frankgraphbench/tree/main/
Author: Paulo do Carmo and Alvaro Lopes
Author-email: paulo.carmo@htwk-leipzig.de
License: Apache-2.0
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: gensim
Requires-Dist: joblib
Requires-Dist: networkx
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: python-dateutil
Requires-Dist: pytz
Requires-Dist: PyYAML
Requires-Dist: scikit-learn
Requires-Dist: scipy
Requires-Dist: six
Requires-Dist: smart-open
Requires-Dist: threadpoolctl
Requires-Dist: tqdm
Requires-Dist: tzdata
Requires-Dist: pykeen
Requires-Dist: torch
Requires-Dist: py-cpuinfo
Requires-Dist: gputil
Requires-Dist: psutil
Requires-Dist: sentence-transformers
Requires-Dist: isodate
Requires-Dist: Levenshtein
Requires-Dist: pyparsing
Requires-Dist: python-Levenshtein
Requires-Dist: rapidfuzz
Requires-Dist: rdflib
Requires-Dist: SPARQLWrapper
Requires-Dist: thefuzz
Requires-Dist: beautifulsoup4
Requires-Dist: matplotlib
Requires-Dist: contourpy
Requires-Dist: multiprocess
Requires-Dist: ollama
Requires-Dist: polars
Requires-Dist: pyarrow
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: requires-dist
Dynamic: summary

# FranKGraphBench: Reproducible Benchmarking for Knowledge Graph-based Recommendation
[![DOI](https://zenodo.org/badge/646290327.svg)](https://doi.org/10.5281/zenodo.14513776)

FranKGraphBench is a framework for reproducible benchmarking of Knowledge Graph-based recommender systems.
It was first created on Google Summer of Code 2023 for Data Integration between DBpedia and some standard RS datasets in a reproducible framework and was accepted at the Resource Track from the 35th ACM International Conference on Information and Knowledge Management in Rome, Italy from the 7th to 11th November 2026.

Check the [docs](https://frankgraphbench.readthedocs.io/en/latest/index.html) for more information.

* For more info, check the [project proposal](https://github.com/AlvaroJoseLopes/GSoC-2023) and the project progress with weekly (as possible) [updates](https://github.com/AKSW/frankgraphbench/wiki).

# Data Integration

## pip

We recommend using a python 3.11 virtual environment:

```shell
pip install frankgraphbench
```

Install the full dataset using bash scripts located at `datasets/`:

```shell
cd datasets
bash ml-100k.sh # Downloaded at `datasets/ml-100k` folder
bash ml-1m.sh   # Downloaded at `datasets/ml-1m` folder
```

## Source

Install from the repository root using `pip install .`. We recommend a python 3.11 virtual environment:

```shell
python3 -m venv venv_data_integration/
source venv_data_integration/bin/activate
pip install .
```

To install the exact dependency set manually, use `pip install -r requirements.txt`.

Install the full dataset using bash scripts located at `datasets/`:

```shell
cd datasets
bash ml-100k.sh # Downloaded at `datasets/ml-100k` folder
bash ml-1m.sh   # Downloaded at `datasets/ml-1m` folder
```

## Usage
```shell
data_integration [-h] -d DATASET -i INPUT_PATH -o OUTPUT_PATH [-ci] [-cu] [-cr] [-cs] [-map] [-enrich] [-w]
```
Arguments:
- **-h:** Shows the help message.
- **-d:** Name of a supported dataset. It will be the same name of the folder created by the bash script provided for the dataset. For now, check `data_integration/dataset2class.py` to see the supported ones.
- **-i:** Input path where the full dataset is placed.
- **-o:** Output path where the integrated dataset will be placed.
- **-ci:** Use this flag if you want to convert item data.
- **-cu:** Use this flag if you want to convert user data.
- **-cr:** Use this flag if you want to convert rating data.
- **-cs:** Use this flag if you want to convert social link data.
- **-map:** Use this flag if you want to map dataset items with DBpedia. At least the item data should be already converted.
- **-enrich:** Use this flag if you want to enrich dataset with DBpedia.
- **-w:** Choose the number of workers(threads) to be used for parallel queries.

pip example:

```shell
data_integration -d 'ml-100k' -i 'datasets/ml-100k' -o 'datasets/ml-100k/processed' \
    -ci -cu -cr -map -enrich -w 8
```

source example:
```shell
python3 src/data_integration.py -d 'ml-100k' -i 'datasets/ml-100k' -o 'datasets/ml-100k/processed' \
    -ci -cu -cr -map -enrich -w 8
```

Check [Makefile](Makefile) for more examples.

## Supported datasets

| Dataset | #items matched | #items |
|---------|---------------|---|
|[MovieLens-100k](https://grouplens.org/datasets/movielens/100k/)|1411|1681|
|[MovieLens-1M](https://grouplens.org/datasets/movielens/1m/)|3253|3883|
|[LastFM-hetrec-2011](https://grouplens.org/datasets/hetrec-2011/)|8628|17632|
|[Douban-Movie-Short-Comments-Dataset](https://www.kaggle.com/datasets/utmhikari/doubanmovieshortcomments/data)|24|28|douban-movie|
|[MIND-small](https://msnews.github.io/)|30409|51282|mind-small|
|[Yelp-Dataset](https://www.yelp.com/dataset/download)|---|150348|yelp|
|[Amazon-Video-Games-5](https://nijianmo.github.io/amazon/index.html)|---|21106|amazon-video_games-5|

Dataset enrichment, except for the MIND-small dataset, is done through a fixed DBpedia endpoint available at https://dbfk25.aksw.org/sparql, with raw files download available at https://dbfk25.aksw.org/.

# Framework for reproducible experiments

## pip

We recommend using a python 3.11 virtual environment for all platforms.

`pip install frankgraphbench` installs the regular `tensorflow` package on every platform. No manual install is therefore required:

```shell
pip install frankgraphbench
```

If you run into trouble with the automatic install, you can still install manually with the matching requirements file:

```shell
pip install -r requirements.txt
```

## source

Install from the repository root using `pip install .`. We recommend a python 3.11 virtual environment:

```shell
python3 -m venv venv_framework/
source venv_framework/bin/activate
pip install .
```

## Usage

pip:
```shell
framework -c 'config_files/test.yml'
```

source:
```shell
python3 src/framework.py -c 'config_files/test.yml'
```

Arguments:
- **-c:** Experiment configuration file path.

The experiment config file should be a .yaml file like this:

```yaml
experiment:
  dataset:
    name: ml-100k
    item:
      path: datasets/ml-100k/processed/item.csv
      extra_features: [movie_year, movie_title]
    user:
      path: datasets/ml-100k/processed/user.csv
      extra_features: [gender, occupation]
    ratings:
      path: datasets/ml-100k/processed/rating.csv
      timestamp: True
    enrich:
      map_path: datasets/ml-100k/processed/map.csv
      enrich_path: datasets/ml-100k/processed/enriched.csv
      remove_unmatched: False
      properties:
        - type: subject
          grouped: True
          sep: "::"
        - type: director
          grouped: True
          sep: "::"

  preprocess:
    - method: filter_kcore
      parameters:
        k: 20
        iterations: 1
        target: user

  split:
    seed: 42
    test:
      method: k_fold
      k: 2
      level: 'user'


  models:
    - name: deepwalk_based
      config:
        save_weights: True
      parameters:
        walk_len: 10
        p: 1.0
        q: 1.0
        n_walks: 50
        embedding_size: 64
        epochs: 1

  evaluation:
    k: 5
    relevance_threshold: 3
    metrics: [MAP, nDCG]

  report:
    file: 'experiment_results/ml100k_enriched/run1.csv'
```

See the [config_files/](/config_files/) directory for more examples.

## Supported Pre-processing Methods

Those are the currently supported pre-processing methods:
* Binarize ratings.
* Filtering by k-core

## Supported Splitting Methods

Currently the supported Splitting methods are:
* Random by Ratio
* Timestamp by Ratio
* Fixed Timestamp
* K-Fold

## Supported Evaluation Metrics

Those are the already implemented metrics:
* MAP@k
* nDCG@k
* Precision@k
* Recall@k
* F-score@k

# Chart generation

Chart generation module based on: https://github.com/hfawaz/cd-diagram

## pip

We recommend using a python 3.11 virtual environment

```shell
pip install frankgraphbench
```

## source

Install from the repository root using `pip install .`. We recommend a python 3.11 virtual environment:

```shell
python3 -m venv venv_chart_generation/
source venv_chart_generation/bin/activate
pip install .
```

After obtaining results from some experiments, you can use this module to generate charts for them. The module is based on the CD-Diagram, which is a chart to compare multiple methods across multiple datasets and metrics. It is based on the Critical Difference diagram, but with some modifications to allow it to be more informative and easier to read.


## Usage
pip:
```shell
chart_generation [-h] -c CHART -p PERFORMANCE_METRIC -f INPUT_FILES -i INPUT_PATH -o OUTPUT_PATH -n FILE_NAME
```

source:
```shell
python3 src/chart_generation.py [-h] -c CHART -p PERFORMANCE_METRIC -f INPUT_FILES -i INPUT_PATH -o OUTPUT_PATH -n FILE_NAME
```

Arguments:
- **-h:** Shows the help message.
- **-p:** Name of the performance metric within the file to use for chart generation.
- **-f:** List of .csv files to use for generating the chart.
- **-i:** Path where results data to generate chart is located in .csv files.
- **-o:** Path where generated charts will be placed.
- **-n:** Add a name (and file extension) to the chart that will be generated.

Examples:

pip:
```shell
chart_generation -c 'cd-diagram' -p 'MAP@5' -f "['ml-100k.csv', 'ml-1m.csv', 'lastfm.csv', 'ml-100k_enriched.csv', 'ml-1m_enriched.csv', 'lastfm_enriched.csv']" -i 'experiment_results' -o 'charts' -n 'MAP@5.pdf'
```

source:
```shell
python3 src/chart_generation.py -c 'cd-diagram' -p 'MAP@5' -f "['ml-100k.csv', 'ml-1m.csv', 'lastfm.csv', 'ml-100k_enriched.csv', 'ml-1m_enriched.csv', 'lastfm_enriched.csv']" -i 'experiment_results' -o 'charts' -n 'MAP@5.pdf'
```

## Supported charts

| Chart |
|-------|
|[CD-Diagram](https://github.com/hfawaz/cd-diagram)|
