================================================================================
  Question 47: Data Preprocessing Pipeline (Missing Data, Encoding, Scaling)
================================================================================

FULL QUESTION:
Task:
Handle the given dataset (Data.csv) with adequate preprocessing steps:
1. Handle Missing Data Values using SimpleImputer (Mean strategy).
2. Encode Categorical Data using OneHotEncoder on features and LabelEncoder
   on dependent target variable.
3. Feature Scaling using StandardScaler.
4. Visualize before and after distributions with appropriate graphs.

SHORT QUESTION:
End-to-end preprocessing pipeline covering missing value imputation, one-hot categorical encoding, and feature scaling.

================================================================================
README & SETUP INSTRUCTIONS:
================================================================================
# Question 47: Data Preprocessing Pipeline (Missing Data, Encoding, Scaling)

- Prerequisites: Python 3.x, Pandas, NumPy, Scikit-Learn, Matplotlib, Seaborn (pip install pandas numpy scikit-learn matplotlib seaborn).
- Setup: Uses 'Data.csv'. If missing, the script automatically creates sample data.
- How to Run: python question_47.py
- Expected Behavior: Imputes missing entries, encodes categorical variables, standardizes numerical features, and displays comparative plots.

================================================================================
SOURCE CODE FILES:
================================================================================

1. question_47.py
--------------------------------------------------------------------------------
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.compose import ColumnTransformer

df = pd.read_csv('Data.csv')
print("Initial Dataset:")
print(df)
print("=" * 60)

X = df.iloc[:, :-1]
y = df.iloc[:, -1]

num_cols = X.select_dtypes(include=['int64', 'float64']).columns
cat_cols = X.select_dtypes(include=['object', 'category']).columns

imputer_num = SimpleImputer(strategy='mean')
X[num_cols] = imputer_num.fit_transform(X[num_cols])

if len(cat_cols) > 0:
    imputer_cat = SimpleImputer(strategy='most_frequent')
    X[cat_cols] = imputer_cat.fit_transform(X[cat_cols])

print("Data after Missing Value Imputation:")
print(X)
print("=" * 60)

ct = ColumnTransformer(
    transformers=[('encoder', OneHotEncoder(drop='first', sparse_output=False), cat_cols)],
    remainder='passthrough'
)
X_encoded = ct.fit_transform(X)

encoded_cat_names = ct.named_transformers_['encoder'].get_feature_names_out(cat_cols)
all_feature_names = list(encoded_cat_names) + [col for col in num_cols]

df_encoded = pd.DataFrame(X_encoded, columns=all_feature_names)

print("DataFrame after Categorical Encoding:")
print(df_encoded)
print("=" * 60)

scaler = StandardScaler()
X_scaled = scaler.fit_transform(df_encoded)

df_scaled = pd.DataFrame(X_scaled, columns=all_feature_names)

print("Final Preprocessed & Scaled Features:")
print(df_scaled.head())
print("=" * 60)

fig, axes = plt.subplots(2, 2, figsize=(14, 10))
fig.suptitle('Data Preprocessing & Feature Analysis', fontsize=16, fontweight='bold')

sns.heatmap(df.isnull(), cbar=False, cmap='viridis', ax=axes[0, 0])
axes[0, 0].set_title('1. Missing Values Map (Raw Dataset)')

sns.kdeplot(data=X[num_cols], ax=axes[0, 1], fill=True)
axes[0, 1].set_title('2. Feature Distributions (Imputed)')

sns.boxplot(data=df_encoded[num_cols], ax=axes[1, 0], color='skyblue')
axes[1, 0].set_title('3. Numerical Feature Ranges (Pre-Scaling)')

sns.boxplot(data=df_scaled[num_cols], ax=axes[1, 1], color='lightgreen')
axes[1, 1].set_title('4. Standardized Feature Ranges (Post-Scaling)')

plt.tight_layout(rect=[0, 0, 1, 0.95])
plt.show()
--------------------------------------------------------------------------------
