GITHUB CONTRIBUTIONS

DATADOC is an open-source project and we welcome contributions! Before creating a Pull Request, please ensure you follow our core tenets.

Core Tenets

  • No Black Boxes: Data cleaning must be deterministic. Do not propose plugins that use generative AI to alter dataset rows blindly. Mathematical standardizations only.
  • Polars First: The engine runs entirely on polars. Do not import pandas. Do not convert to pandas and back. All transformations must use Polars Expressions for zero-copy efficiency.
  • Zero Leakage: If building scaling or encoding plugins, ensure logic is strictly applied per-column without row-wise leakage.

Building Custom Plugins

To add a new feature to the DATADOC pipeline, you must create a class that inherits from BasePlugin.

from datadoc.plugins.base import BasePlugin
import polars as pl

class MyCustomPlugin(BasePlugin):
    @property
    def name(self) -> str:
        return "MyCustomPlugin"

    @property
    def priority(self) -> int:
        return 50 # Execution order

    def analyze(self, df: pl.DataFrame) -> dict:
        # Diagnostic logic
        return {"has_work": True}

    def apply(self, df: pl.DataFrame) -> pl.DataFrame:
        # Transformation logic
        return df

GitHub Workflow

  1. Fork the repository.
  2. Create a new branch (git checkout -b feat/my-plugin).
  3. Write tests in the tests/ directory.
  4. Run pytest tests/ and ensure 100% pass rate.
  5. Submit a Pull Request.