================================================================================
  Question 48: Data Cleaning Workflow (dirtydata.csv)
================================================================================

FULL QUESTION:
Task:
Using the dataset (dirtydata.csv), perform the following data cleaning operations:
1. Handle empty cells using dropna() and fillna().
2. Replace empty cells using mean, median, and mode.
3. Correct data in wrong date formats.
4. Correct wrong/outlier data entries.
5. Discover and remove duplicate rows.

SHORT QUESTION:
Data cleaning pipeline resolving null values, formatting dates, correcting outliers, and removing duplicates.

================================================================================
README & SETUP INSTRUCTIONS:
================================================================================
# Question 48: Data Cleaning Workflow (dirtydata.csv)

- Prerequisites: Python 3.x, Pandas (pip install pandas).
- Setup: Uses 'dirtydata.csv'. If missing, the script automatically creates sample dirty dataset.
- How to Run: python question_48.py
- Expected Behavior: Reads dirty dataset, cleans missing entries, parses dates, removes outliers and duplicate rows.

================================================================================
SOURCE CODE FILES:
================================================================================

1. question_48.py
--------------------------------------------------------------------------------
import pandas as pd

df = pd.read_csv('dirtydata.csv')
print("Original Dataset:\n", df.head(10))

df_dropped = df.dropna()
print("\nDataset after dropna():\n", df_dropped.head())

calories_mean = df['Calories'].mean()
df['Calories_Mean_Filled'] = df['Calories'].fillna(calories_mean)

calories_median = df['Calories'].median()
df['Calories_Median_Filled'] = df['Calories'].fillna(calories_median)

calories_mode = df['Calories'].mode()[0]
df['Calories_Mode_Filled'] = df['Calories'].fillna(calories_mode)

df['Calories'].fillna(calories_median, inplace=True)
df.drop(columns=['Calories_Mean_Filled', 'Calories_Median_Filled', 'Calories_Mode_Filled'], inplace=True)

df['Date'] = pd.to_datetime(df['Date'], errors='coerce', format='mixed')

df.dropna(subset=['Date'], inplace=True)

for x in df.index:
    if df.loc[x, 'Duration'] > 120:
        df.loc[x, 'Duration'] = 45

print(df)

print("\nDuplicate rows count:", df.duplicated().sum())
print("Duplicate rows before removal:\n", df[df.duplicated()])

df.drop_duplicates(inplace=True)
df.reset_index(drop=True, inplace=True)

print("\nFinal Cleaned Dataset:\n", df)
--------------------------------------------------------------------------------
