⚙millwrightdocs
01 · data & EDA

The boundary type,
and the typed layer in front of it.

Frame is the numeric boundary the whole API speaks. Table (feature eda) is the polars-backed, dtype-aware world that ingests real CSV/Parquet and lowers into it.

Frame & Dataset

One contiguous f64 buffer, plus a schema.

Everything the public API speaks is a Frame: a contiguous, row-major f64 buffer with named columns. It is what lets a linfa model and a smartcore DenseMatrix meet in one signature without your code ever naming their array versions — each backend converts Frame ⇄ its native type inside the adapter only. A Dataset pairs a frame with a target.

use millwright::prelude::*;

let x = Frame::from_rows(
    vec![vec![0.0, 0.1], vec![0.4, 0.2], vec![9.0, 9.1], vec![9.4, 8.7]],
    vec!["a".into(), "b".into()],
)?;
assert_eq!(x.shape(), (4, 2));        // (rows, cols)

let train = Dataset::new(x.clone(), vec![0.0, 0.0, 1.0, 1.0])?;
let _features = train.features();  // &Frame
let _target   = train.target();    // &[f64]

The task — classification vs. regression — is inferred from the target: an all-integral target is class labels, anything else is regression. A pure-numeric CSV loads directly with Frame::from_csv; typed data uses Table below.

Ingest & explore

Table reads it, Profile reports it.

Behind the eda feature, a polars-backed Table reads real CSV/Parquet — strings, categories, dates, booleans, nulls — and a Profile returns a typed analysis (not just an HTML blob) and drafts the preprocessing.

let table = Table::from_csv("customers.csv")?;   // or ::from_parquet(…)

// a typed profile — overview, per-column stats, missingness, correlations, alerts
let profile = Profile::of_with_target(&table, "churned")?;
println!("{}", profile.summary());
for alert in profile.alerts() {
    println!("{alert}");   // "[city] categorical (3 levels) → OneHotEncoder"
}
profile.to_html("eda_report.html")?;             // a shareable report

Because Millwright owns EDA and the pipeline, the profile drafts the starting preprocessing from its own findings — the loop scikit-learn can't close:

// lower the typed table to the numeric world
let train = table.into_dataset("churned")?;   // categoricals encoded, nulls → NaN

// EDA drafts the pipeline; you just add the model
let mut pipe = profile.suggest_pipeline()        // impute · encode · scale, from the alerts
    .estimator("rf", RandomForest::new());
pipe.fit(&train)?;

cargo run --example explore --features "eda smartcore-backend"

Dtype-aware

Types flow through the pipeline.

A Frame carries a per-column Dtype (defaulting to Numeric). Table marks the columns it knows are Categorical as it lowers — so preprocessing doesn't have to guess: scalers, Winsorize, and PowerTransform pass categorical columns through untouched, and OneHotEncoder encodes by dtype rather than a value heuristic.

// a genuinely-integer feature is NOT wrongly one-hot'd; the categorical one is
let f = Frame::from_rows(rows, cols)?
    .with_dtypes(vec![Dtype::Categorical, Dtype::Numeric])?;
let out = OneHotEncoder::infer().fit_transform(&f)?;   // expands only column 0

// or one-hot at the Table boundary, with real category names
let train = table.into_dataset_with("churned", CategoryEncoding::OneHot)?;

Nominal categories become "{col}={value}" indicator columns instead of ordinal codes — the correct representation for linear and tree models.

← prevHome