Metadata-Version: 2.4
Name: django-ixp-tracker
Version: 2rc6
Summary: Library to retrieve and manipulate data about IXPs
Author: Internet Society
License: Apache-2.0
Project-URL: Homepage, https://github.com/InternetSociety/django-ixp-tracker
Project-URL: Changelog, https://github.com/InternetSociety/django-ixp-tracker/releases
Project-URL: Issues, https://github.com/InternetSociety/django-ixp-tracker/issues
Project-URL: CI, https://github.com/InternetSociety/django-ixp-tracker/actions
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Framework :: Django
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: django>=5.2
Requires-Dist: django-countries>7.6
Requires-Dist: python-dateutil
Requires-Dist: requests
Requires-Dist: typing-extensions
Provides-Extra: test
Requires-Dist: django-stubs; extra == "test"
Requires-Dist: factory-boy; extra == "test"
Requires-Dist: mypy; extra == "test"
Requires-Dist: pre-commit; extra == "test"
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-django; extra == "test"
Requires-Dist: responses; extra == "test"
Requires-Dist: types-python-dateutil; extra == "test"
Requires-Dist: types-requests; extra == "test"
Dynamic: license-file

# django-ixp-tracker

[![PyPI](https://img.shields.io/pypi/v/django-ixp-tracker.svg)](https://pypi.org/project/django-ixp-tracker/)
[![Tests](https://github.com/InternetSociety/django-ixp-tracker/actions/workflows/test.yml/badge.svg)](https://github.com/InternetSociety/django-ixp-tracker/actions/workflows/test.yml)
[![Changelog](https://img.shields.io/github/v/release/InternetSociety/django-ixp-tracker?include_prereleases&label=changelog)](https://github.com/InternetSociety/django-ixp-tracker/releases)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/InternetSociety/django-ixp-tracker/blob/main/LICENSE)

Library to retrieve and manipulate data about IXPs

## v2 warning

Note the v2 is a complete rewrite to use [Event Sourcing](docs/event-sourcing-rewrite.md). The first time you upgrade you will need to re-run any historical data you have as the history is not migrated between major versions.

v2 does not by default preserve any internal ids for IXPs. If you need to preserve them, you will need to preload the `IXPIdMap` model table with the mapping to Peering DB ids (use a dummy `aggregate_id`), e.g. if you use Postgres, by running these queries
```sql
INSERT INTO ixp_tracker_ixpidmap (id, peeringdb_id, aggregate_id)
SELECT id, peeringdb_id, concat('aaaabbbb-cccc-dddd-eeee-', lpad(id::text, 12, '0'))::uuid FROM ixp_tracker_ixp;

SELECT setval(pg_get_serial_sequence('ixp_tracker_ixpidmap', 'id')
           , COALESCE(max(id) + 1, 1)
           , false)
FROM   ixp_tracker_ixpidmap;
```

The legacy implementation is preserved in v2 for now but will be removed in a future minor version. If you would like to upgrade to v2 but continue with the legacy implementation be sure to set `IXP_TRACKER_ENABLE_EVENT_SOURCING` to `False`. Note that there are no changes to the legacy implementation in v2 though.

## Installation

Install this library using `pip`:
```bash
pip install django-ixp-tracker
```
## Usage

1. Add to your INSTALLED_APPS setting like this:
```
   INSTALLED_APPS = [
   ...,
   "ixp_tracker",
   ]
```

 Note: this app has no web-facing components so you don't need to add anything to `urls.py` etc

2. Run `python manage.py migrate` to create the models.
3. Add the relevant settings to your config.
4. Add `IXP_TRACKER_DATA_LOOKUP_FACTORY` to config with the path to your factory (see below).
5. Run the management command to import the data: `python manage.py ixp_tracker_import` (This will sync the current data, if you want historical data you need to backfill first)

## ASN country and status data

The lib uses an external component to look up the country of registration (why?) and the status of an ASN. This status is used for the logic to identify when members have left an IXP.

If you don't provide this service yourself, it will default to a noop version. This will mean you will get no country of registration data and the marking of members having left an IXP will not be as efficient.

In order to implement such a component yourself, you should implement the Protocol `ixp_tracker.data_lookup.AdditionalDataSources` and provide a factory function for your class.

## Backfilling data

You have the option of backfilling data from archived PeeringDb data. This can be done by running the import command with the `--backfill` option for each month you want to backfill:
```shell
python manage.py ixp_tracker_import --backfill <YYYMM>
```
The backfill currently process a single month at a time and will look for the earliest file for the relevant month at https://publicdata.caida.org/datasets/peeringdb/

IMPORTANT NOTE: due to the way the code tries to figure out when a member left an IXP, you should run the backfill strictly in date order and *before* syncing the current data.

## IXP stats

The import process also generates monthly stats per IXP and per country. These are generated as of the 1st of the month used to import the data.

## Running programmatically

If you'd like to run the import from code, rather than from the management command, you can call `importers.import_data()` and `stats.generate_stats()` directly.

It's not recommended to call any other functions yourself.

## Development

To contribute to this library, first checkout the code. Then create a new virtual environment:
```bash
cd django-ixp-tracker
python -m venv .venv
source .venv/bin/activate
```
Now install the dependencies and test dependencies:
```bash
pip install -e '.[test]'
```
To run the tests:
```bash
pytest
```
We use [pre-commit](https://pre-commit.com/) for linting etc on push. Run:
```bash
pre-commit install
```
from the repo top-level dir to set it up.

## Releases

For now, releasing a new version is manual and can be done by running the following commands from the repo:
```bash
python -m build
python -m twine upload --repository pypi dist/*
```

## Peering Db libraries

PeeringDb provide their own [vanilla Python](https://github.com/peeringdb/peeringdb-py) and [Django](https://github.com/peeringdb/django-peeringdb) libs, but we have decided not to use these.

Both libs are designed to keep a local copy of the current data and to keep that copy in sync with the central copy via the API.

As we need to keep a historical record (e.g. for IXP growth stats over time), we would have to provide some sort of wrapper over those libs anyway.

In addition to that, the [historical archives of PeeringDb data](https://publicdata.caida.org/datasets/peeringdb/) use flat lists of the different object types in json. We can retrieve the data from the API directly in the same way, so it makes it simpler to implement.
