Beginner Workflow with DataKit¶

This notebook demonstrates how a beginner can use DataKit to perform a complete data analysis workflow with minimal code. We'll load a sample CSV, explore it, clean it, visualize it, and compare groups.

Let's start by importing DataKit and pandas.

In [ ]:

import datakit as dk
import pandas as pd

import datakit as dk import pandas as pd

1. Load the data¶

For this example, we'll create a simple sales dataset on the fly. In reality, you would use pd.read_csv('your_file.csv').

In [ ]:

# create sample sales data
import numpy as np
np.random.seed(42)

n = 200
df = pd.DataFrame({
    'Order Date': pd.date_range('2024-01-01', periods=n, freq='D'),
    'Region': np.random.choice(['North', 'South', 'East', 'West'], size=n),
    'Product': np.random.choice(['Laptop', 'Phone', 'Tablet'], size=n),
    'Revenue': np.random.normal(500, 150, size=n).round(2),
    'Units Sold': np.random.poisson(10, size=n),
    'Customer Satisfaction': np.random.uniform(1, 5, size=n).round(1)
})

# add some missing values to simulate real data
df.loc[10:20, 'Revenue'] = np.nan
df.loc[30:35, 'Customer Satisfaction'] = np.nan
df.loc[50, 'Region'] = np.nan

print("Data loaded. Shape:", df.shape)
df.head()

# create sample sales data import numpy as np np.random.seed(42) n = 200 df = pd.DataFrame({ 'Order Date': pd.date_range('2024-01-01', periods=n, freq='D'), 'Region': np.random.choice(['North', 'South', 'East', 'West'], size=n), 'Product': np.random.choice(['Laptop', 'Phone', 'Tablet'], size=n), 'Revenue': np.random.normal(500, 150, size=n).round(2), 'Units Sold': np.random.poisson(10, size=n), 'Customer Satisfaction': np.random.uniform(1, 5, size=n).round(1) }) # add some missing values to simulate real data df.loc[10:20, 'Revenue'] = np.nan df.loc[30:35, 'Customer Satisfaction'] = np.nan df.loc[50, 'Region'] = np.nan print("Data loaded. Shape:", df.shape) df.head()

2. Quick EDA (Exploratory Data Analysis)¶

One function call gives us a complete profile, distribution plots, and a correlation heatmap.

In [ ]:

dk.quick_eda(df, show_plots=True, export_profile=False)

dk.quick_eda(df, show_plots=True, export_profile=False)

3. Clean column names and data types¶

DataKit can fix messy column names and automatic type detection.

In [ ]:

df = dk.clean_columns(df, style='snake_case')
df = dk.fix_dtypes(df)
print("Cleaned column names:", df.columns.tolist())
print("\nData types:\n", df.dtypes)

df = dk.clean_columns(df, style='snake_case') df = dk.fix_dtypes(df) print("Cleaned column names:", df.columns.tolist()) print("\nData types:\n", df.dtypes)

4. Handle missing values automatically¶

auto_impute drops columns with too many nulls, fills numeric with median, categorical with mode, and datetime with forward fill.

In [ ]:

df_clean = dk.auto_impute(df, verbose=True)
print("\nMissing values after imputation:", df_clean.isnull().sum().sum())

df_clean = dk.auto_impute(df, verbose=True) print("\nMissing values after imputation:", df_clean.isnull().sum().sum())

5. Encode categorical variables¶

Automatically choose one-hot encoding for low cardinality, label encoding for high cardinality.

In [ ]:

df_encoded = dk.encode_categoricals(df_clean, method='auto', verbose=True)
df_encoded.head()

df_encoded = dk.encode_categoricals(df_clean, method='auto', verbose=True) df_encoded.head()

6. Visualise distributions of numeric columns¶

Histograms and boxplots for each numeric column.

In [ ]:

dk.plot_distributions(df_clean, cols=['revenue', 'units_sold', 'customer_satisfaction'], show=True)

dk.plot_distributions(df_clean, cols=['revenue', 'units_sold', 'customer_satisfaction'], show=True)

7. Correlation heatmap¶

Shows relationships between numeric variables.

In [ ]:

dk.plot_corr(df_clean, method='pearson', threshold=0.5, show=True)

dk.plot_corr(df_clean, method='pearson', threshold=0.5, show=True)

8. Compare two groups¶

Let's compare revenue between North and South regions. DataKit automatically checks normality, selects the right test, and reports effect size.

In [ ]:

north_revenue = df_clean[df_clean['region'] == 'North']['revenue'].dropna()
south_revenue = df_clean[df_clean['region'] == 'South']['revenue'].dropna()

result = dk.compare_groups(north_revenue, south_revenue, verbose=True)
print("\nInterpretation:", result['interpretation'])

north_revenue = df_clean[df_clean['region'] == 'North']['revenue'].dropna() south_revenue = df_clean[df_clean['region'] == 'South']['revenue'].dropna() result = dk.compare_groups(north_revenue, south_revenue, verbose=True) print("\nInterpretation:", result['interpretation'])

9. Group by and chart¶

Quickly create aggregate bar charts.

In [ ]:

dk.groupby_chart(df_clean, by='region', metric='revenue', agg='mean', chart_type='bar', annotate=True, title='Average Revenue by Region')

dk.groupby_chart(df_clean, by='region', metric='revenue', agg='mean', chart_type='bar', annotate=True, title='Average Revenue by Region')

10. Done!¶

With just a few lines of DataKit code, we have:

• Profiled the data
• Cleaned column names and types
• Handled missing values
• Encoded categoricals
• Visualised distributions and correlations
• Performed a statistical test
• Created an aggregate bar chart

This would have taken dozens of lines of boilerplate without DataKit.