OMR

Omni Data Refinement — Professional API Reference

Overview: Welcome to the definitive API reference for OMR. This documentation covers every class, method, argument, and schema available in the library to help you build robust data intelligence pipelines.

1. Core API: `Dataset`

The Dataset class is the main entry point to OMR. It wraps a Pandas DataFrame and attaches intelligent engines to it.

from omr import Dataset
import pandas as pd

df = pd.read_csv("data.csv")
dataset = Dataset(df)

Core & Quality Assessment

`Dataset.summary()`

Prints a highly optimized, single-line summary of your dataset. Ideal for quick pulse checks.

Returns: Dataset (returns self for chaining)
Under the hood: Computes essential metadata (shape, total missing cells, duplicate rows) and runs a fast heuristic health score.

dataset.summary()
# > [Dataset] 1000 rows × 5 cols | Missing: 12 (1.2%) | Duplicates: 0 | Health: 98.5/100

`Dataset.health()`

Executes the comprehensive 5-Pillar Data Quality assessment (Completeness, Uniqueness, Consistency, Validity, Conformity).

Returns: HealthReport object containing the .score and .issues list.
Under the hood: Spawns a HealthEngine that analyzes null density, duplication rates, and type uniformity.

report = dataset.health()
print(f"Dataset Health: {report.score}/100")

`Dataset.profile()`

Generates a full statistical profile for every column in the dataset.

Returns: Dataset (returns self for chaining)
Under the hood: Extracts data types, missing counts, unique cardinalities, means, modes, minimums, and maximums. Replaces Pandas .info().

dataset.profile()

Cleaning & Transformation

Transformation

`Dataset.clean()`

Automatically resolves all data quality issues detected during the health() check.

Returns: Dataset (returns self for chaining)
Under the hood: Intelligently imputes missing values (using mode for categorical, median/mean for continuous), drops pure duplicate rows, and standardizes formats.

dataset.health()  # Identifies the issues
dataset.clean()   # Fixes them automatically

`Dataset.explain_changes()`

Prints a detailed "Transformation Log" showing exactly what clean() (or any other mutation) changed.

Returns: Dataset (returns self for chaining)
Under the hood: Reads from DatasetMetadata.transformations to provide an auditable log of operations.

dataset.clean()
dataset.explain_changes()

Advanced Analytics

Analytics

`Dataset.analyze()`

Runs deep statistical analysis to detect complex machine learning hazards.

Returns: Dataset (returns self for chaining)
Under the hood: The StatisticsEngine detects extreme outliers (via IQR/Z-score), highly skewed distributions, multicollinearity between features, and class imbalances.

dataset.analyze()

Analytics

`Dataset.compare(other: Dataset)`

Detects data drift by comparing the statistical distributions of the current dataset against a reference dataset.

Arguments: other (The reference dataset, e.g., historical data).
Returns: Dataset (returns self for chaining)
Under the hood: Utilizes Population Stability Index (PSI), Kolmogorov-Smirnov (KS) Tests, and Jensen-Shannon Divergence.

prod_data = Dataset(pd.read_csv("prod.csv"))
dataset.compare(prod_data)

`Dataset.explain(issue: str)`

Acts as a built-in AI Data Science tutor.

Arguments: issue (The name of the statistical issue, e.g., "data_leakage", "class_imbalance").
Returns: dict containing explanation keys.

dataset.explain("class_imbalance")

Schema Validation

`Dataset.validate(schema: Dict)`

Enforces strict business rules against the dataset.

Arguments: schema (A dictionary mapping column names to OMR schema constraints).
Returns: Dataset (returns self for chaining)

from omr import schemas
rules = {
    "age": schemas.PositiveInteger(max=120),
    "email": schemas.Email()
}
dataset.validate(rules)

Ops, Versioning & Export

Ops

`Dataset.snapshot(name: str = "", description: str = "")`

Saves a checkpoint of your dataset's current state in memory.

Returns: int (The unique Version ID).

v_id = dataset.snapshot(name="v1", description="Pre-cleaning state")

Ops

`Dataset.rollback(version_id: int)`

Reverts the dataset back to a previous snapshot in memory.

Returns: Dataset (returns self for chaining)

dataset.rollback(1)

Ops

`Dataset.export()`

Returns the cleaned, refined underlying Pandas DataFrame.

Returns: pd.DataFrame

clean_df = dataset.export()
model.fit(clean_df)

2. Validation Schemas: `omr.schemas`

OMR provides built-in constraint types for strict schema validation. Every schema accepts a not_null boolean argument (default: True), which dictates whether missing values are permitted.

PositiveInteger(min=1, max=None): Validates that values are integers ≥ min and ≤ max.
PositiveFloat(min=0.0, max=None): Validates that values are floats > min and ≤ max.
NonNegative(): Validates that all numerical values are ≥ 0.
OneOf(*values): Validates that values belong to an explicit allowed set (e.g. OneOf("active", "inactive")).
Email(): Validates standard email address formats via Regex.
Regex(pattern: str): Validates strings against a custom Regular Expression pattern.
NotNull(): Ensures the column contains absolutely no missing or null values.
MinLength(length: int): Ensures string lengths are at least length characters long.
MaxLength(length: int): Ensures string lengths do not exceed length characters.

3. Continuous Monitoring: `Monitor`

The Monitor class is used in production pipelines to track dataset health over time and issue alerts when anomalies occur.

`Monitor.watch(dataset: Dataset)`

Registers a baseline dataset to establish the mathematical "norm" (means, volumes, null frequencies).

from omr import Monitor
monitor = Monitor()
monitor.watch(historical_dataset)

`Monitor.check(new_data: pd.DataFrame)`

Evaluates incoming data against the registered baseline to detect sudden drops in volume, massive shifts in column means, or spikes in missing values.

Returns: list[Alert]

alerts = monitor.check(new_daily_batch)
for alert in alerts:
    print(f"[{alert.severity}] {alert.check}: {alert.message}")

1. Core API: Dataset

Core & Quality Assessment

Dataset.summary()

Dataset.health()

Dataset.profile()

Cleaning & Transformation

Dataset.clean()

Dataset.explain_changes()

Advanced Analytics

Dataset.analyze()

Dataset.compare(other: Dataset)

Dataset.explain(issue: str)