API Reference

Projects

A “project” is a directory of stuff, with associated metadata to tell you what that stuff is and how to use it.

Base Classes

`proj.base.Project`(path[, storage_options, ...])	Top level representation of a project directory
`proj.base.ProjectSpec`(proj)	A project specification
`proj.base.ProjectExtra`(proj)	A special subcategory of project types with content but no structure.

Top level representation of a project directory

This holds any parsed project metadata specs, top level contents and artifacts and any project details from nested child directories.

add_to_library(path=None)[source]: Add this project to the current session library

all_artifacts(names: str | None = None) → list[source]: A flat list of all the artifact objects nested in this project.

all_contents(names=None) → list[source]: A flat list of all the content objects nested in this project.

create(name: str) → list[str][source]

Make this project conform to the given project spec type.

Returns a list of files that were created.

has_artifact_type(types: Iterable[type]) → bool[source]

Answers ‘does this project support outputting the given artifact type’

This is an experimental example of filtering through projects

has_content_type(types: Iterable[type]) → bool[source]

Answers ‘does this project support outputting the given content type’

This is an experimental example of filtering through projects

is_local() → bool[source]: Did we read this from the local filesystem?

make(qname: str, **kwargs)[source]

Make an artifact of the given type

qname: str: Format is [<spec>.]<artifact-type>[.<name>]. If <spec> or <name> are not given will use the first artifact found of the given type - so include them to be more explicit.

property pyproject: Contents of top-level pyproject.toml, if found

resolve(walk: bool | None = None, types: set[str] | None = None, xtypes: set[str] | None = None) → None[source]

Fill out project specs in this directory

Parameters:

walk – if None (default) only try subdirectories if root has no specs, and don’t descend further. If True, recurse all directories; if False, don’t descend at all.
types – names of types to allow while parsing. If empty or None, allow all
xtypes – names of types to disallow while parsing.

text_summary(bare=False) → str[source]: Only shows project types, not what they contain

class projspec.proj.base.ProjectSpec(proj: Project)[source]

A project specification

Subclasses of this define particular project types, and if a project conforms to the given type, then parsing it with the class will succeed and the contents and artifacts will be populated.

Checking if a path _might_ meet a spec (with .match()) should be cheap, and parsing (with .parse()) should normally only require reading a few text files of metadata.

Subclasses are automatically added to the registry on import, and any Project will attempt to parse its given path with every class (which is why making .match() fast is important).

property artifacts: AttrDict

A mapping of types and in each a list of objects from this project

Artifacts are things a project can make/do. See projspec.artifact.

clean() → None[source]: Remove any artifacts and runtimes produced by this project

property contents: AttrDict

A mapping of types and in each a list of objects from this project

Contents means the things that are within a project as part of its description, see projspec.content.

classmethod create(path: str)[source]: Make the target directory compliant with this project type, if not already

match() → bool[source]: Whether the given path might be interpreted as this type of project

classmethod snake_name() → str[source]: Convert a project name to snake-case

class projspec.proj.base.ProjectExtra(proj: Project)[source]

A special subcategory of project types with content but no structure.

Subclasses of this are special: they are not free standing projects, but add contents onto the root project. Examples include data catalog specification, linters and CI/CD, that may be run against the root project without using a project-oriented tool.

These classes do not appear in a Project’s .specs, but do contribute .contents or .artifacts. They are still referenced when filtering spec types by name.

Commonly, subclasses are tied 1-1 to a particular content/artifact class.

User Classes

`artifact.container.Docker`(proj)	A Dockerfile in a project directory, which defines how to build an image.
`artifact.linter.PreCommitted`(proj)	A project with pre-commit conf.
`content.environment.CondaEnv`(proj)
`content.environment.PythonRequirements`(proj)
`content.metadata.Licensed`(proj)	A license for the repo, as an isolated top-level text file
`proj.ai.AIEnabled`(proj)	This project has text files intended for LLM/AI to read.
`proj.briefcase.Briefcase`(proj)
`proj.cicd.GitHubActions`(proj)	GitHub Actions CI/CD workflows
`proj.cicd.GitLabCI`(proj)	GitLab CI/CD pipeline defined in .gitlab-ci.yml.
`proj.cicd.CircleCI`(proj)	CircleCI pipeline defined in .circleci/config.yml.
`proj.cicd.Taskfile`(proj)	Task runner using Taskfile (go-task).
`proj.cicd.JustFile`(proj)	Task runner using Just (justfile / Justfile).
`proj.cicd.Tox`(proj)	Python test automation using tox.
`proj.conda_package.CondaRecipe`(proj)	Recipe package for conda-build
`proj.conda_package.RattlerRecipe`(proj)	Recipe project for rattler-build
`proj.conda_project.CondaProject`(proj)	Tool for encapsulating, running, and reproducing projects with conda environments.
`proj.datapackage.DVCRepo`(proj)	Git management of data assets within a repo
`proj.datapackage.DataPackage`(proj)	A FrictionlessData datapackage spec
`proj.dataworkflows.Dbt`(proj)	dbt (data build tool) project.
`proj.dataworkflows.Quarto`(proj)	Quarto publishing system project.
`proj.dataworkflows.Prefect`(proj)	Prefect workflow orchestration project.
`proj.dataworkflows.Dagster`(proj)	Dagster data orchestration project.
`proj.dataworkflows.Kedro`(proj)	Kedro data science pipeline project.
`proj.dataworkflows.Metaflow`(proj)	Metaflow ML/data science workflow project.
`proj.dataworkflows.MLFlow`(proj)	MLflow project, defined by an `MLproject` (or `MLFlow`) file.
`proj.dataworkflows.Airflow`(proj)	Apache Airflow workflow orchestration project/DAG spec.
`proj.dataworkflows.Snakemake`(proj)	Snakemake workflow management system project.
`proj.dataworkflows.Nox`(proj)	Nox Python automation project.
`proj.documentation.Docusaurus`(proj)	Docusaurus documentation/website project.
`proj.documentation.MDBook`(proj)	mdBook is a command line tool to create books with Markdown.
`proj.documentation.MkDocs`(proj)	MkDocs documentation project.
`proj.documentation.RTD`(proj)	Documentation to be processes by ReadTheDocs
`proj.documentation.Sphinx`(proj)	Sphinx documentation project (standalone, without ReadTheDocs config).
`proj.git.GitRepo`(proj)	A version controlled repository utilising git
`proj.golang.Golang`(proj)	A Go module project, identified by the presence of a go.mod file.
`proj.helm.HelmChart`(proj)	A Kubernetes application packaged as a Helm chart.
`proj.hf.HuggingFaceDataset`(proj)	A dataset repository hosted on the Hugging Face Hub.
`proj.hf.HuggingFaceRepo`(proj)
`proj.ide.JetbrainsIDE`(proj)
`proj.ide.NvidiaAIWorkbench`(proj)
`proj.ide.VSCode`(proj)
`proj.ide.Zed`(proj)
`proj.infra.DockerCompose`(proj)	Docker Compose multi-service project.
`proj.infra.Terraform`(proj)	Terraform infrastructure-as-code project.
`proj.infra.Ansible`(proj)	Ansible automation project.
`proj.infra.Pulumi`(proj)	Pulumi infrastructure-as-code project.
`proj.infra.CDK`(proj)	AWS Cloud Development Kit (CDK) project.
`proj.infra.Earthfile`(proj)	Earthly build project.
`proj.infra.Nixpacks`(proj)	Nixpacks build configuration project.
`proj.infra.Vagrant`(proj)	Vagrant virtual machine project.
`proj.jsframeworks.NextJS`(proj)	Next.js React framework project.
`proj.jsframeworks.NuxtJS`(proj)	Nuxt.js Vue framework project.
`proj.jsframeworks.SvelteKit`(proj)	SvelteKit project.
`proj.jsframeworks.Vite`(proj)	Vite-based project (any frontend framework using Vite as the build tool).
`proj.jsframeworks.Pnpm`(proj)	Node project managed with pnpm.
`proj.jsframeworks.Bun`(proj)	Node project managed with Bun.
`proj.jsframeworks.Deno`(proj)	Deno project.
`proj.node.JLabExtension`(proj)	A node variant specific to Jupyter-Lab
`proj.node.Node`(proj)	Node.js project, managed by NPM
`proj.node.Yarn`(proj)	A node project that uses `yarn` for building
`proj.pixi.Pixi`(proj)	A project using https://pixi.sh/
`proj.poetry.Poetry`(proj)	Python packaging and dependency management
`proj.published.Cited`(proj)	A github-specified format to say how this project should be cited.
`proj.published.Zenodo`(proj)	This project has been published on Zenodo
`proj.pyscript.PyScript`(proj)	PyScript is an open source platform for Python in the browser.
`proj.python_code.PythonCode`(proj)	Code directly importable by python
`proj.python_code.PythonLibrary`(proj)	Complete buildable python project
`proj.rust.Rust`(proj)	A directory, which can build a binary executable or library with Cargo.
`proj.rust.RustPython`(proj)	A rust project designed for importing with python, perhaps with mixed rust/python code trees.
`proj.uv.Uv`(proj)	UV-runnable project
`proj.uv.UvScript`(proj)	Single-file project runnable by UV as a script
`proj.webapp.Django`(proj)	A python web app using the django framework
`proj.webapp.Gradio`(proj)	Gradio machine learning demo and web app.
`proj.webapp.Marimo`(proj)	Reactive Python notebook and webapp served in the browser
`proj.webapp.Panel`(proj)	Interactive data dashboarding using panel, with holoviz/bokeh components.
`proj.webapp.Shiny`(proj)	Shiny for Python web application.
`proj.webapp.Streamlit`(proj)	Interactive graphical app served in the browser, with streamlit components
`proj.dataworkflows.MLFlow`(proj)	MLflow project, defined by an `MLproject` (or `MLFlow`) file.

class projspec.artifact.container.Docker(proj: Project)[source]: A Dockerfile in a project directory, which defines how to build an image.

class projspec.artifact.linter.PreCommitted(proj: Project)[source]: A project with pre-commit conf.

class projspec.content.environment.CondaEnv(proj: Project)[source]

class projspec.content.environment.PythonRequirements(proj: Project)[source]

class projspec.content.metadata.Licensed(proj: Project)[source]: A license for the repo, as an isolated top-level text file

class projspec.proj.ai.AIEnabled(proj: Project)[source]: This project has text files intended for LLM/AI to read.

class projspec.proj.briefcase.Briefcase(proj: Project)[source]

class projspec.proj.cicd.GitHubActions(proj: Project)[source]

GitHub Actions CI/CD workflows

Each YAML file under .github/workflows/ defines one workflow.

class projspec.proj.cicd.GitLabCI(proj: Project)[source]: GitLab CI/CD pipeline defined in .gitlab-ci.yml.

class projspec.proj.cicd.CircleCI(proj: Project)[source]: CircleCI pipeline defined in .circleci/config.yml.

class projspec.proj.cicd.Taskfile(proj: Project)[source]: Task runner using Taskfile (go-task).

class projspec.proj.cicd.JustFile(proj: Project)[source]

Task runner using Just (justfile / Justfile).

A justfile defines named recipes that can be run with just.

class projspec.proj.cicd.Tox(proj: Project)[source]

Python test automation using tox.

A set of environments and run commands to be run as a workflow.

class projspec.proj.conda_package.CondaRecipe(proj: Project)[source]: Recipe package for conda-build

class projspec.proj.conda_package.RattlerRecipe(proj: Project)[source]: Recipe project for rattler-build

class projspec.proj.conda_project.CondaProject(proj: Project)[source]: Tool for encapsulating, running, and reproducing projects with conda environments.

class projspec.proj.datapackage.DVCRepo(proj: Project)[source]: Git management of data assets within a repo

class projspec.proj.datapackage.DataPackage(proj: Project)[source]: A FrictionlessData datapackage spec

class projspec.proj.dataworkflows.Dbt(proj: Project)[source]

dbt (data build tool) project.

dbt is used for data ingestion, validation, and transform.

The spec dbt about the context of your project and how to transform your data (build your data sets).

class projspec.proj.dataworkflows.Quarto(proj: Project)[source]: Quarto publishing system project.

class projspec.proj.dataworkflows.Prefect(proj: Project)[source]: Prefect workflow orchestration project.

class projspec.proj.dataworkflows.Dagster(proj: Project)[source]: Dagster data orchestration project.

class projspec.proj.dataworkflows.Kedro(proj: Project)[source]: Kedro data science pipeline project.

class projspec.proj.dataworkflows.Metaflow(proj: Project)[source]

Metaflow ML/data science workflow project.

Metaflow has no project-level config file; detection is done by scanning Python files for from metaflow import (or import metaflow) combined with a FlowSpec subclass definition.

Each py file containing a flow becomes a separate Command / Process pair keyed by the file stem. If a @project(name=...) decorator is found, the project name is captured in metadata. If @schedule or @trigger decorators are present, deployment commands for Argo Workflows and AWS Step Functions are added alongside the local run command.

No explicit parsing of Config files, since they are designed as defaults, and often overridden (and hard to detect).

class projspec.proj.dataworkflows.Airflow(proj: Project)[source]: Apache Airflow workflow orchestration project/DAG spec.

class projspec.proj.dataworkflows.Snakemake(proj: Project)[source]: Snakemake workflow management system project.

class projspec.proj.dataworkflows.Nox(proj: Project)[source]

Nox Python automation project.

Often used for testing, linting, and packaging. Nox is a Python environment management tool that allows you to define multiple CI runs in one execution.

class projspec.proj.documentation.Docusaurus(proj: Project)[source]: Docusaurus documentation/website project.

class projspec.proj.documentation.MDBook(proj: Project)[source]

mdBook is a command line tool to create books with Markdown.

mdBook is used by the Rust programming language project, and The Rust Programming Language book is an example.

class projspec.proj.documentation.MkDocs(proj: Project)[source]: MkDocs documentation project.

class projspec.proj.documentation.RTD(proj: Project)[source]

Documentation to be processes by ReadTheDocs

RTD is commonly used by open-source python projects and others. Documentation is typically built automatically from github repos using sphinx or mkdocs.

General description of the platform: https://docs.readthedocs.com/platform/stable/

class projspec.proj.documentation.Sphinx(proj: Project)[source]: Sphinx documentation project (standalone, without ReadTheDocs config).

class projspec.proj.git.GitRepo(proj: Project)[source]

A version controlled repository utilising git

git is a very common version control system for code projects.

class projspec.proj.golang.Golang(proj: Project)[source]: A Go module project, identified by the presence of a go.mod file.

class projspec.proj.helm.HelmChart(proj: Project)[source]

A Kubernetes application packaged as a Helm chart.

A Helm chart is a directory tree containing a Chart.yaml manifest, a templates/ directory of Kubernetes resource manifests, and an optional values.yaml file with default configuration values. Dependency charts may be declared in Chart.yaml under the dependencies key; pinned versions are recorded in Chart.lock.

class projspec.proj.hf.HuggingFaceDataset(proj: Project)[source]

A dataset repository hosted on the Hugging Face Hub.

A HuggingFace dataset repo is identified by a README.md whose YAML front-matter contains at least one dataset-specific key (e.g. task_categories, dataset_info, size_categories).

class projspec.proj.hf.HuggingFaceRepo(proj: Project)[source]

class projspec.proj.ide.JetbrainsIDE(proj: Project)[source]

class projspec.proj.ide.NvidiaAIWorkbench(proj: Project)[source]

class projspec.proj.ide.VSCode(proj: Project)[source]

class projspec.proj.ide.Zed(proj: Project)[source]

class projspec.proj.infra.DockerCompose(proj: Project)[source]

Docker Compose multi-service project.

Designed to launch a set of runtimes (specific images with config), volumes and networks, and expose ports.

class projspec.proj.infra.Terraform(proj: Project)[source]: Terraform infrastructure-as-code project.

class projspec.proj.infra.Ansible(proj: Project)[source]: Ansible automation project.

class projspec.proj.infra.Pulumi(proj: Project)[source]: Pulumi infrastructure-as-code project.

class projspec.proj.infra.CDK(proj: Project)[source]: AWS Cloud Development Kit (CDK) project.

class projspec.proj.infra.Earthfile(proj: Project)[source]: Earthly build project.

class projspec.proj.infra.Nixpacks(proj: Project)[source]: Nixpacks build configuration project.

class projspec.proj.infra.Vagrant(proj: Project)[source]: Vagrant virtual machine project.

class projspec.proj.jsframeworks.NextJS(proj: Project)[source]: Next.js React framework project.

class projspec.proj.jsframeworks.NuxtJS(proj: Project)[source]: Nuxt.js Vue framework project.

class projspec.proj.jsframeworks.SvelteKit(proj: Project)[source]: SvelteKit project.

class projspec.proj.jsframeworks.Vite(proj: Project)[source]

Vite-based project (any frontend framework using Vite as the build tool).

Note: SvelteKit also has a svelte.config, so SvelteKit takes priority via its more-specific match.

class projspec.proj.jsframeworks.Pnpm(proj: Project)[source]: Node project managed with pnpm.

class projspec.proj.jsframeworks.Bun(proj: Project)[source]: Node project managed with Bun.

class projspec.proj.jsframeworks.Deno(proj: Project)[source]

Deno project.

Note: this is a separate runtime, not a Node project.

class projspec.proj.node.JLabExtension(proj: Project)[source]

A node variant specific to Jupyter-Lab

https://jupyterlab.readthedocs.io/en/latest/developer/contributing.html: #installing-node-js-and-jlpm

These projects have at least a front-end component, but build to python wheels for distribution. A running jupyter-lab process might be considered an output.

Some jlab projects will also have python components (i.e., server extensions https://jupyter-server.readthedocs.io/en/latest/developers/extensions.html).

class projspec.proj.node.Node(proj: Project)[source]

Node.js project, managed by NPM

This is a project that contains a package.json file.

class projspec.proj.node.Yarn(proj: Project)[source]: A node project that uses yarn for building

class projspec.proj.pixi.Pixi(proj: Project)[source]

A project using https://pixi.sh/

pixi is a conda-stack, project-oriented (aka “workspace”) env and execution manager.

class projspec.proj.poetry.Poetry(proj: Project)[source]

Python packaging and dependency management

Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on, and it will manage (install/update) them for you. Poetry offers a lockfile to ensure repeatable installs, and can build your project for distribution.

class projspec.proj.published.Cited(proj: Project)[source]: A github-specified format to say how this project should be cited.

class projspec.proj.published.Zenodo(proj: Project)[source]: This project has been published on Zenodo

class projspec.proj.pyscript.PyScript(proj: Project)[source]

PyScript is an open source platform for Python in the browser.

This spec is the canonical way to provide configuration, and included in new template apps on pyscript.com.

class projspec.proj.python_code.PythonCode(proj: Project)[source]

Code directly importable by python

This applies to directories with __init__.py (i.e., not isolated .py files, or eggs). Could include .zip in theory.

Such a structure does not declare any envs, deps, etc. It contains nothing interesting _except_ code.

A package is executable if it contains a __main__.py file.

class projspec.proj.python_code.PythonLibrary(proj: Project)[source]

Complete buildable python project

Defined by the existence of pyproject.toml or setup.py.

class projspec.proj.rust.Rust(proj: Project)[source]: A directory, which can build a binary executable or library with Cargo.

class projspec.proj.rust.RustPython(proj: Project)[source]

A rust project designed for importing with python, perhaps with mixed rust/python code trees.

This version assumes the build tool is maturin, which may not be the only possibility.

class projspec.proj.uv.Uv(proj: Project)[source]

UV-runnable project

Note: uv can run any python project, but this tests for uv-specific config.

class projspec.proj.uv.UvScript(proj: Project)[source]

Single-file project runnable by UV as a script

Metadata are declared inline in the script header

class projspec.proj.webapp.Django(proj: Project)[source]: A python web app using the django framework

class projspec.proj.webapp.Gradio(proj: Project)[source]

Gradio machine learning demo and web app.

Detected by scanning Python files for import gradio or gr.Interface / gr.Blocks.

class projspec.proj.webapp.Marimo(proj: Project)[source]: Reactive Python notebook and webapp served in the browser

class projspec.proj.webapp.Panel(proj: Project)[source]: Interactive data dashboarding using panel, with holoviz/bokeh components.

class projspec.proj.webapp.Shiny(proj: Project)[source]

Shiny for Python web application.

Detected by scanning Python files for from shiny import combined with app = App( or @app. decorator usage. Also detects app.py at root.

class projspec.proj.webapp.Streamlit(proj: Project)[source]: Interactive graphical app served in the browser, with streamlit components

class projspec.proj.dataworkflows.MLFlow(proj: Project)[source]

MLflow project, defined by an MLproject (or MLFlow) file.

An MLproject file is a YAML document that declares the project name, the environment (conda or pip), and one or more named entry points.

Contents

A contents item is something defined by a project spec, a core component of what that project is.

Base Classes

content.base.BaseContent(proj)

A descriptive piece of information declared in a project

class projspec.content.base.BaseContent(proj: Project)[source]

A descriptive piece of information declared in a project

Content classes tell you something fundamental about a project, but do not have any other functionality than to allow introspection. We use dataclasses to define what information a given Content subclass should provide.

User Classes

`content.cicd.CIWorkflow`(proj, name, ...)	A CI/CD workflow or pipeline definition.
`content.cicd.PipelineStage`(proj, name, cmd, ...)	A named stage or step in a data/ML/workflow pipeline.
`content.cicd.ServiceDependency`(proj[, name, ...])	An external service that a project depends on at runtime.
`content.data.IntakeSource`(proj, name)	A catalog of data assets, including basic properties (location) and how to load/process them.
`content.data.TabularData`(proj, name, ...)	A tabular dataset, columns and rows
`content.data.DataResource`(proj, path, ...)	A data resource found inside a data-only directory.
`content.env_var.EnvironmentVariables`(proj, ...)	A set of environment variable key/value pairs, typically used with new processes.
`content.environment.Environment`(proj, stack, ...)	Definition of a python runtime environment
`content.executable.Command`(proj, cmd)	The simplest runnable thing; we don't know what it does/outputs.
`content.metadata.DescriptiveMetadata`(proj, ...)	Miscellaneous descriptive information
`content.metadata.License`(proj[, shortname, ...])	A legal description of what the given project (code and other assets) can be used for.
`content.package.NodePackage`(proj, name)	Buildable nodeJS source
`content.package.PythonPackage`(proj, package_name)	Importable python directory, i.e., containing an __init__.py file.
`content.package.RustModule`(proj, name)	Usually a directory with a Cargo.toml file

class projspec.content.cicd.CIWorkflow(proj: ~projspec.proj.base.Project, name: str = '', triggers: list = <factory>, jobs: list = <factory>, provider: str = '')[source]

A CI/CD workflow or pipeline definition.

Captures the name, triggering events, and high-level job/stage names from CI configuration files (GitHub Actions, GitLab CI, CircleCI, etc.).

class projspec.content.cicd.PipelineStage(proj: ~projspec.proj.base.Project, name: str = '', cmd: list = <factory>, depends_on: list = <factory>)[source]: A named stage or step in a data/ML/workflow pipeline.

class projspec.content.cicd.ServiceDependency(proj: Project, name: str = '', service_type: str = '', version: str = '', image: str = '')[source]

An external service that a project depends on at runtime.

Typically exposed via an open TCP port, e.g., as used in container orchestration.

class projspec.content.data.IntakeSource(proj: Project, name: str)[source]

A catalog of data assets, including basic properties (location) and how to load/process them.

See https://intake.readthedocs.io/en/latest/

class projspec.content.data.DataResource(proj: ~projspec.proj.base.Project, path: str, format: str, modality: str = '', layout: str = '', file_count: int = 0, total_size: int = 0, schema: dict | list = <factory>, sample_path: str = '', metadata: dict = <factory>)[source]

A data resource found inside a data-only directory.

Describes one logical dataset — which may be a flat collection of files, a Hive-partitioned tree, an Iceberg/Delta table, a Zarr store, or any other recognised on-disk layout.

The path field is a human-readable basename that identifies the resource:

Single file: "data.csv"
Multi-file series: "part*.parquet" (glob-style, common prefix + * + ext)
Directory-as-dataset (Hive partition, Zarr store, …): "year=2024/"

The modality field classifies the broad nature of the data using the vocabulary established by intake’s structure tags and napari’s layer type system:

"tabular" — row/column data (CSV, Parquet, ORC, Excel, …)
"array" — N-dimensional arrays (NumPy, HDF5, NetCDF, Zarr, …)
"image" — 2-D/3-D images (PNG, JPEG, TIFF, DICOM, NIfTI, …)
"timeseries" — time-indexed signals (WAV, GRIB, …)
"geospatial" — vector/raster geodata (Shapefile, GeoJSON, GeoTIFF, …)
"model" — ML model weights (GGUF, SafeTensors, PyTorch, …)
"nested" — hierarchical / JSON-like (Avro, YAML, XML, …)
"document" — human-readable documents (PDF, DOCX, …)
"video" — video streams (MP4, AVI, …)
"archive" — compressed bundles (ZIP, tar.gz, …)
"" — unknown / mixed

The schema field is format-specific:

Tabular (Parquet, Arrow, CSV, …): {column_name: dtype_str, …}
Image / array: {"width": int, "height": int, "channels": int, "mode": str}
Audio: {"sample_rate": int, "channels": int, "frames": int}
HDF5 / Zarr / NetCDF: {"variables": [...], "dims": {...}, "attrs": {...}}
Unknown / library not available: {}

class projspec.content.data.TabularData(proj: ~projspec.proj.base.Project, name: str, metadata: dict = <factory>, schema: dict | list = <factory>)[source]

A tabular dataset, columns and rows

This lists loadable tabular files with defined schema, typically from formats such as JSON, CSV, and parquet.

class projspec.content.env_var.EnvironmentVariables(proj: ~projspec.proj.base.Project, variables: dict[str, str | None] = <factory>)[source]: A set of environment variable key/value pairs, typically used with new processes.

class projspec.content.environment.Environment(proj: ~projspec.proj.base.Project, stack: ~projspec.content.environment.Stack, precision: ~projspec.content.environment.Precision, packages: list[str], channels: list[str] = <factory>)[source]: Definition of a python runtime environment

class projspec.content.executable.Command(proj: Project, cmd: list[str] | str)[source]: The simplest runnable thing; we don’t know what it does/outputs.

class projspec.content.metadata.DescriptiveMetadata(proj: ~projspec.proj.base.Project, meta: dict[str, str] = <factory>)[source]

Miscellaneous descriptive information

Typically includes authors, tags, and text.

class projspec.content.metadata.License(proj: Project, shortname: str = 'unknown', fullname: str = 'unknown', url: str = '')[source]

A legal description of what the given project (code and other assets) can be used for.

This could be one of the typical open-source permissive licenses (see https://spdx.org/licenses/), specified either just by its name or by a link. Some projects will have custom or restrictive conditions on their replication and use.

class projspec.content.package.NodePackage(proj: Project, name: str)[source]: Buildable nodeJS source

class projspec.content.package.PythonPackage(proj: Project, package_name: str)[source]: Importable python directory, i.e., containing an __init__.py file.

class projspec.content.package.RustModule(proj: Project, name: str)[source]: Usually a directory with a Cargo.toml file

Artifacts

An artifact item is a thing that a project can do or make.

Base Classes

`artifact.base.BaseArtifact`(proj[, cmd])	A thing that a project can o or make
`artifact.base.FileArtifact`(proj, fn, **kw)	Specialised artifacts, where the output is one or more files

class projspec.artifact.base.BaseArtifact(proj: Project, cmd: list[str] | None = None, **kwargs)[source]

A thing that a project can o or make

Artifacts are the “actions” of a project spec. Most typically, they involve calling the external tool associated with the project type in a subprocess.

clean()[source]: Remove artifact

make(*args, **kwargs)[source]: Create the artifact and any runtime it depends on

remake()[source]: Recreate the artifact and any runtime it depends on

to_dict(compact=True)[source]

Distil the instance to JSON compatible dict

compact: if True, will produce condensed output, perhaps justa string.

class projspec.artifact.base.FileArtifact(proj: Project, fn: str, **kw)[source]

Specialised artifacts, where the output is one or more files

Ideally, we can know beforehand the path expected for the output.

User Classes

`artifact.container.DockerImage`(proj[, cmd, tag])	Filesystem snapshot as created by bocker-build, which can be instantiated into a container.
`artifact.container.DockerRuntime`(proj[, ...])	Running container in Docker, tied to a certain image and command.
`artifact.deployment.Deployment`(proj[, cmd, ...])	A named release deployed to an external orchestrator (e.g. a Kubernetes cluster).
`artifact.deployment.HelmDeployment`(proj, ...)	A Helm release deployed to the active Kubernetes cluster.
`artifact.infra.ComposeStack`(proj[, file])	A multi-service stack managed by Docker Compose.
`artifact.infra.StaticSite`(proj, fn, **kw)	A static website produced by a build tool (MkDocs, Sphinx, Docusaurus, Quarto, etc.).
`artifact.infra.TerraformPlan`(proj[, plan_file])	A saved Terraform execution plan file (`terraform plan -out plan.tfplan`).
`artifact.installable.CondaPackage`([fn, name])	An installable python wheel file
`artifact.installable.SystemInstallablePackage`(...)	An Installable system package
`artifact.installable.Wheel`(proj[, fn])	An installable python wheel file
`artifact.linter.PreCommit`(proj[, cmd])	Typically used as a git hook, this lists a set of linters that a project uses.
`artifact.process.Process`(proj[, cmd])	A simple process where we know nothing about what it does, only if it's running.
`artifact.process.Server`(proj[, cmd])	A process that is designed to stay running and serve requests, usually over HTTP
`artifact.python_env.CondaEnv`(proj, fn, **kw)	Path to a project conda-built env
`artifact.python_env.EnvPack`(proj, fn, **kw)	Archival form of a python environment
`artifact.python_env.LockFile`(proj, fn, **kw)	File containing exact environment specification
`artifact.python_env.VirtualEnv`(proj, fn, **kw)	Path to a project virtual environment

class projspec.artifact.container.DockerImage(proj: Project, cmd=None, tag=None)[source]: Filesystem snapshot as created by bocker-build, which can be instantiated into a container.

class projspec.artifact.container.DockerRuntime(proj: Project, cmd=None, tag=None)[source]: Running container in Docker, tied to a certain image and command.

class projspec.artifact.deployment.Deployment(proj: Project, cmd: list[str] | None = None, release: str = '', clean_cmd: list[str] | None = None, **kwargs)[source]

A named release deployed to an external orchestrator (e.g. a Kubernetes cluster).

Unlike a local Process, a Deployment has no local subprocess handle. “Done” is inferred by querying the orchestrator; “clean” means the release has been uninstalled.

Subclasses should override _is_done(), _is_clean(), and clean() for their specific orchestrator. The default implementations here are suitable for a Helm release:

make() runs helm upgrade --install <release> .
clean() runs helm uninstall <release>
_is_done() / _is_clean() query helm status <release>

clean()[source]: Tear down the deployment (e.g. helm uninstall <release>).

class projspec.artifact.deployment.HelmDeployment(proj: Project, release: str, **kwargs)[source]

A Helm release deployed to the active Kubernetes cluster.

Parameters:: release – the Helm release name passed to helm upgrade --install.

make() runs:

helm upgrade --install <release> .

clean() runs:

helm uninstall <release>

state is resolved by running helm status <release>:

"done" — release exists and is deployed (exit code 0)
"clean" — release does not exist (exit code non-zero / not found)

class projspec.artifact.infra.ComposeStack(proj: Project, file: str = 'docker-compose.yml', **kwargs)[source]

A multi-service stack managed by Docker Compose.

make() runs docker compose up -d clean() runs docker compose down state is inferred by docker compose ps (checks for running services).

class projspec.artifact.infra.StaticSite(proj: Project, fn: str, **kw)[source]

A static website produced by a build tool (MkDocs, Sphinx, Docusaurus, Quarto, etc.).

fn should be the glob pattern for the output index file, e.g. <proj>/site/index.html.

class projspec.artifact.infra.TerraformPlan(proj: Project, plan_file: str = 'plan.tfplan', **kwargs)[source]

A saved Terraform execution plan file (terraform plan -out plan.tfplan).

make() runs terraform plan -out plan.tfplan clean() deletes the plan file

class projspec.artifact.installable.CondaPackage(fn=None, name=None, **kwargs)[source]

An installable python wheel file

Note that in general, there may be a set of wheels for different platforms. The actual name of the wheel file depends on the platform, vcs config and maybe other factors. We just check if the dist/ directory is populated.

This output is intended to be _local_ - pushing to a remote location (e.g., pypi) is call publishing.

class projspec.artifact.installable.SystemInstallablePackage(proj, ext: str, fn=None, arch=None, **kw)[source]: An Installable system package

class projspec.artifact.installable.Wheel(proj, fn=None, **kw)[source]

An installable python wheel file

Note that in general there may be a set of wheels for different platforms. The actual name of the wheel file depends on platform, vcs config and maybe other factors. We just check if the dist/ directory is populated.

This output is intended to be _local_ - pushing to a remote location (e.g., pypi) is call publishing.

class projspec.artifact.linter.PreCommit(proj: Project, cmd=None)[source]: Typically used as a git hook, this lists a set of linters that a project uses.

class projspec.artifact.process.Process(proj: Project, cmd: list[str] | None = None, **kwargs)[source]

A simple process where we know nothing about what it does, only if it’s running.

Can include batch jobs and long-running services.

While running the process, the output can be “enqueued”, meaning that the .queue attribute will be populated with lines of output (from stdout and stderr, by default). This can be controlled by passing enqueue= to the .make() method; or disabled by setting the config value capture_artifact_output to False.

class projspec.artifact.process.Server(proj: Project, cmd: list[str] | None = None, **kwargs)[source]

A process that is designed to stay running and serve requests, usually over HTTP

When calling make(), some instances will accept port= and address= arguments to specify listening, but only if the instance was initially configured with port_arg= and address_arg=.

After creating the process, is scan is True, the actual listening address and port will attempt to be inferred.

class projspec.artifact.python_env.CondaEnv(proj: Project, fn: str, **kw)[source]

Path to a project conda-built env

Contains both python itself and any other binaries, as well as linked libraries.

In the case of a project having an environment.yaml with a named output, the path may be outside the project tree.

class projspec.artifact.python_env.EnvPack(proj: Project, fn: str, **kw)[source]

Archival form of a python environment

conda-pack: https://conda.github.io/conda-pack/
pixi-pack: https://pixi.sh/latest/deployment/pixi_pack/

class projspec.artifact.python_env.LockFile(proj: Project, fn: str, **kw)[source]: File containing exact environment specification

class projspec.artifact.python_env.VirtualEnv(proj: Project, fn: str, **kw)[source]

Path to a project virtual environment

Some tools like pipenv put these environments in a global location.

Utilities

`library.ProjectLibrary`([library_path, auto_save])	Stores scanned project objects at a given path in JSON format
`utils.AttrDict`(data, *kw)	Contains a dict but allows attribute read access for compliant keys.
`utils.Enum`(new_class_name, /, names, *[, ...])	Named enum values, so that str(x) looks like the label.
`utils.IsInstalled`()	Checks if we can call commands, as a function of the current environment.
`utils.class_infos`()	Gather all the class info for documentation
`utils.get_cls`(name[, registry])	Find class by name and type
`utils.make_and_copy`(path[, sub, mkdir])	Provide a temporary directory to create and write into, and then copy to destination
`proj.base.ParseFailed`	Exception raised when parsing fails: a directory does not meet the given spec.
`config.get_conf`(name)	Fetch the value of the given conf parameter from the current config or defaults

class projspec.library.ProjectLibrary(library_path: str | None = None, auto_save: bool = True)[source]

Stores scanned project objects at a given path in JSON format

An instance of this library library is created on import.

In the future, alternative serialisations will be implemented.

class projspec.utils.AttrDict(*data, **kw)[source]: Contains a dict but allows attribute read access for compliant keys.

class projspec.utils.Enum(new_class_name, /, names, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]: Named enum values, so that str(x) looks like the label.

projspec.utils.class_infos()[source]: Gather all the class info for documentation

projspec.utils.get_cls(name: str, registry: str = 'proj') → type[source]

Find class by name and type

name: str: Class name in camel case (the typical real name) or snake equivalent
registry: projspec|content|artifact|enum: Category of class to find

projspec.utils.make_and_copy(path, sub=None, mkdir=False)[source]: Provide a temporary directory to create and write into, and then copy to destination

class projspec.utils.IsInstalled[source]

Checks if we can call commands, as a function of the current environment.

Typical usage:

>>> "python" in IsInstalled()
True

Results are cached by command and python executable, so that in the future we may be able to persist these for future sessions.

An instance of this class is created at import: projspec.utils.is_installed.

exists(cmd: str, refresh=False)[source]

Test if command can be called by starting a subprocess

This is more costly what some PATH lookup (i.e., what which() does), but also more rigorous. We cache the result - currently for the session, and eventually persistently.

class projspec.proj.base.ParseFailed[source]: Exception raised when parsing fails: a directory does not meet the given spec.

projspec.config.get_conf(name: str)[source]: Fetch the value of the given conf parameter from the current config or defaults