Metadata-Version: 2.4
Name: cikit
Version: 0.3.0
Summary: CI helper toolkit
Author-email: Sven Schlender <svesch@gmx.de>
License-Expression: GPL-3.0-or-later
Project-URL: Documentation, https://cikit-a4de14.gitlab.io
Project-URL: Repository, https://gitlab.com/svesch/cikit
Keywords: ci,gitlab,devops,pipeline,artifacts,sphinx
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.34
Requires-Dist: anybadge>=1.16
Requires-Dist: pygal>=3.1.0
Dynamic: license-file

# cikit

![python](https://cikit-a4de14.gitlab.io/_images/python.svg)
![ai assisted](https://cikit-a4de14.gitlab.io/_images/ai_assisted.svg)
![ruff](https://cikit-a4de14.gitlab.io/_images/ruff.svg)
![pylint](https://cikit-a4de14.gitlab.io/_images/pylint.svg)
![unittests](https://cikit-a4de14.gitlab.io/_images/unittests.svg)
![unittests](https://cikit-a4de14.gitlab.io/_images/systemtests.svg)
![coverage](https://cikit-a4de14.gitlab.io/_images/coverage.svg)

**cikit** is a lightweight Python toolkit for implementing local development and CI workflows
with reusable Python building blocks instead of large shell scripts or provider-specific pipeline
logic.

It combines a project-specific command framework with CI environment abstraction, reusable macros,
run data, artifact management, reports, timeline charts, and Sphinx integration.

[Documentation](https://cikit-a4de14.gitlab.io/) ·
[Repository](https://gitlab.com/svesch/cikit)

## Highlights

* CI commands from installed provider packages with optional project-local overrides
* Unified execution locally and in CI
* Provider support for local Git, GitLab, and Azure DevOps
* Structured environment handling with variable expansion and secret masking
* Ruff, Pylint, pytest, coverage, CMake, CTest, PlatformIO, and Sphinx helpers
* Run-bound metrics and persistent artifact metadata
* Build reports with timings, warnings/errors, memory metrics, artifacts, bundles, and logs
* Local and optional remote cikit data storage for historical metrics
* Configurable SVG timeline charts with line, filled-line, bar, and stacked-bar presets
* Sphinx extensions for inline metrics, interactive charts, collapsible logs, and compact SHA-256 hashes

## Installation

cikit requires Python 3.12 or newer.

```bash
pip install cikit
```

For a project using uv, cikit is typically added as a development dependency:

```bash
uv add --dev cikit
```

## CI Command Framework

The `cikit` console command builds its effective command set from installed
command providers and optional project-local commands.

Reusable command packages register themselves through the Python
`cikit.commands` entry-point group. A project can then install such a package as
a normal development dependency and immediately use its commands.

A project may additionally provide a root-level `ci_commands` package:

```text
project/
├── ci_commands/
│   ├── __init__.py
│   └── all_.py
├── docs/
├── src/
├── tests/
└── pyproject.toml
```

The local package is optional. It is intended for project-specific commands,
orchestration, and deliberate overrides of shared provider commands.

Every discovered command module declares a logical command name:

```python
import argparse

from cikit import CiSection
from cikit.ci import run_ci_script
from cikit.macros import ruff


COMMAND_NAME = "ruff"


def run(env, status, args):
    with CiSection("Running Ruff", status=status):
        ruff.ruff(env, args.paths)

    with CiSection("Generating Ruff report", status=status):
        ruff.write_rst(env)


def command(args):
    return run_ci_script(
        COMMAND_NAME,
        lambda env, status: run(env, status, args),
        args=args,
    )


def register(subparsers):
    parser = subparsers.add_parser(COMMAND_NAME, help="Run Ruff checks.")
    parser.add_argument("paths", nargs="+")
    parser.set_defaults(func=command)
```

`COMMAND_NAME` is independent of the Python filename and is used by the loader
for conflict and override resolution.

Installed providers must not expose duplicate command names. A project-local
command may intentionally use the same name as one provider command; the local
implementation then wins.

The resulting command is executed through the shared runner:

```bash
cikit ruff src tests
```

The logical run name can be overridden globally. It is used for cikit data and
artifact metadata:

```bash
cikit --name quality ruff src tests
```

The command set is not built into cikit itself. It is supplied by installed
command providers and optional project-local modules. The `ci_commands` package
in this repository is the project-specific workflow used to build and test
cikit itself.

For complete provider-package examples and discovery rules, see the CI Commands
guide in the documentation.

## Environment and Providers

`cikit.env` resolves configuration once and exposes it consistently to commands and macros.
Environment variables may reference previously resolved values using `${VAR}` syntax and can be
converted to types such as `Path`.

```python
from cikit.env import CiEnvironment, get_env_vars

env = CiEnvironment(get_env_vars())

print(env["PROJECT_ROOT"])
print(env.pipeline_id)
print(env.branch_name)
print(env.commit_sha)
```

Supported providers are:

* `LOCALGIT`
* `GITLAB`
* `AZURE_DEVOPS`

Outside a supported CI system, local Git metadata is used as the fallback. Provider metadata includes
pipeline, branch, commit, and provider project information without requiring provider-specific logic
inside reusable macros.

For cikit data, the remote project key uses the logical `PROJECT_NAME` with an optional
`tool.cikit.data_scope` prefix from `pyproject.toml`.

## Reusable Macros

The `cikit.macros` package contains the reusable workflow implementation. Command modules should
mainly parse arguments, open CI sections, and compose these macros.

| Module | Purpose |
| --- | --- |
| `ruff` | Ruff execution, reports, badges, timelines, and GitLab Code Quality output |
| `pylint` | Pylint execution, reports, badges, timelines, and rule-specific reports |
| `pytest` | pytest execution, JUnit XML, and optional parallel coverage collection |
| `coverage` | Python/gcovr coverage, Cobertura XML, HTML reports, badges, and timelines |
| `cmake` | CMake configure/build, CTest, CMake/compiler metadata, compilation databases, build metrics, logs, and reports |
| `platformio` | PlatformIO builds, compiler metadata, build metrics, memory parsing, logs, and reports |
| `reports` | Shared test and build report generation |
| `sphinx` | Documentation preparation, build, and artifact integration |
| `bundle` | ZIP, TAR, and TAR.GZ bundle creation and registration |

All run-bound macros operate on a `CiRunEnvironment`, so generated metrics and artifacts belong to a
well-defined logical run.

## Build Metrics and Reports

CMake and PlatformIO builds can collect data such as:

* configure, build, and total build time
* CMake version
* compiler name, version, and type
* compiler optimization and LTO settings when available
* compiled file counts
* compiler warning and error counts
* memory usage where the toolchain exposes it
* command logs

The generic build report adds the information available for the active run, including timeline
charts, memory tables, downloadable build artifacts and bundles with size and SHA-256 metadata,
and collapsible logs.

Typical usage:

```python
from pathlib import Path

from cikit.macros import cmake

build_dir = cmake.get_build_dir(env)

cmake.gather_cmake_meta(env)

cmake.cmake_configure(
    env,
    source_dir=Path("."),
    build_dir=build_dir,
    generator="Ninja",
    clean=True,
    compile_commands=True,
)
cmake.gather_compiler_meta(env, build_dir, "CXX")
cmake.cmake_build(env, build_dir)
cmake.write_rst(env)
```

PlatformIO follows the same run-data and report model:

```python
from cikit.macros import platformio

platformio.build(env, clean=True)
platformio.write_rst(env)
```

## Artifacts and Bundles

Artifacts are owned by `CiRunEnvironment`. Generated files are copied into cikit-managed storage and
registered together with their logical run, artifact type, file size, and hash metadata.

```python
from pathlib import Path

from cikit.artifacts import ArtifactType

artifact = env.copy_artifact(Path("build/firmware.bin"), ArtifactType.BUILD)

print(artifact.path)
print(artifact.size)
print(artifact.hash)
```

For file artifacts, cikit records the size and calculates a SHA-256 hash automatically unless a hash
is supplied explicitly to `copy_artifact()`. Folder artifacts use `None` for size and hash by
default.

Runtime artifact paths are absolute, while the persistent registry stores portable paths relative to
the configured artifact root. Artifacts can later be queried by run and type, which allows separate
pipeline steps to exchange generated reports and build outputs without depending on physical output
paths.

Bundles can package one file or a complete directory:

```python
from pathlib import Path

from cikit.macros import bundle

bundle.create(env, "firmware-package.zip", Path("build/release"))
```

Supported bundle formats are `.zip`, `.tar`, and `.tar.gz`. Bundle artifacts can be included directly
in generated build reports.

## cikit Data

`cikit.data` stores structured data per logical run and provides historical queries for reports,
badges, and timeline charts.

It supports three operating modes:

* local JSON cache only
* remote cikit Data API only
* local cache with remote synchronization

Run-bound access is normally provided by `CiRunEnvironment`:

```python
env.data_set("build_time", 4.2)
env.data_update({"warnings": 0, "errors": 0})

data = env.data
```

Timeline data can be queried from the remote backend using reference keys such as timestamps, pipeline
IDs, commit hashes, or relative indices.

The repository also contains the small PHP/MySQL reference backend used by the cikit Data API,
including schema initialization scripts.

## Timeline Charts

`cikit.timeline` renders cikit data as SVG charts using Pygal. `TimelineChart` validates timeline data
and can render directly or register the result as an artifact.

```python
from cikit.timeline import TimelineChart, TimelinePreset

chart = TimelineChart(data, "Build duration", TimelinePreset.LINE_REPORT)
chart.create(env, "build-duration")
```

Available presets cover general, compact, and report-oriented line and bar charts, a filled report line
chart, and stacked bars. Chart options, style, and individual series options can be overridden after
selecting a preset.

For development and documentation, all presets can be rendered with deterministic preview data:

```bash
python -m cikit.timeline
```

## Sphinx Integration

The Sphinx macro consumes registered artifacts instead of relying on fixed public paths. During
preparation it places generated content into the documentation source tree:

* RST artifacts → `docs/_generated`
* badge artifacts → `docs/_static/badges`
* other documentation artifacts → `docs/_static/artifacts`

The completed HTML documentation is registered again as a folder artifact.

cikit provides a bundled Sphinx extension with reusable roles and directives:

| Extension | RST interface | Purpose |
| --- | --- | --- |
| `cikit.sphinx_ext.metrics` | `:cikit-metric:` | Render a value from the local cikit data cache |
| `cikit.sphinx_ext.chart` | `.. cikit-chart::` | Embed interactive SVG charts in HTML with a non-HTML fallback |
| `cikit.sphinx_ext.log` | `.. cikit-log::` | Embed collapsible command logs with a non-HTML fallback |
| `cikit.sphinx_ext.hash` | `:cikit-hash:` | Render compact SHA-256 hashes with tooltip and clipboard support |


The recommended configuration enables all cikit extensions through the package extension:

```python
extensions = [
    "cikit.sphinx_ext",
]
```

Individual extensions can still be enabled separately if required.

Example RST:

```rst
Build time: :cikit-metric:`build.build_time` seconds.

.. cikit-chart:: ../_static/artifacts/chart_build.svg
   :alt: Build timeline

.. cikit-log:: ../_static/artifacts/build.log
   :title: Build log
   
Package hash: :cikit-hash:`0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef`
```

## Console Entry Points

The package currently installs three console entry points:

| Command | Purpose |
| --- | --- |
| `cikit` | Discover and run project-specific commands from `ci_commands` |
| `cikit-data` | Access local/remote cikit data operations directly |
| `cikit-env` | Inspect, export, document, and prepare the resolved cikit environment |

Use `--help` on each command for the available options. Concrete `cikit` subcommands depend on the
current project's `ci_commands` package.

## Design Goals

* Keep project workflow logic in readable Python
* Share the same commands between local development and CI
* Separate provider metadata from reusable workflow implementation
* Keep generated data and artifacts associated with explicit logical runs
* Make pipeline steps composable through persistent artifact metadata
* Keep reports and documentation generated from the same underlying data
* Remain lightweight and testable with a small runtime dependency set

## Documentation

The full documentation covers environment configuration, providers, project metadata, command
architecture, local workflows, GitLab integration, cikit data, artifacts, build reports, and Sphinx
integration:

https://cikit-a4de14.gitlab.io/

Repository:

https://gitlab.com/svesch/cikit

## License

GPL-3.0-or-later
