Recap

Recap

(iClicker) Exercise 5.1

iClicker cloud join link: https://join.iclicker.com/YJHS

Take a guess: In your machine learning project, how much time will you typically spend on data preparation and transformation?

• 1. ~80% of the project time
• 2. ~20% of the project time
• 3. ~50% of the project time
• 4. None. Most of the time will be spent on model building

The question is adapted from here.

(iClicker) Exercise 5.2

iClicker cloud join link: https://join.iclicker.com/YJHS

Select all of the following statements which are TRUE.

• 1. StandardScaler ensures a fixed range (i.e., minimum and maximum values) for the features.
• 2. StandardScaler calculates mean and standard deviation for each feature separately.
• 3. In general, it’s a good idea to apply scaling on numeric features before training \(k\)-NN or SVM RBF models.
• 4. The transformed feature values might be hard to interpret for humans.
• 5. After applying SimpleImputer The transformed data has a different shape than the original data.

(iClicker) Exercise 5.3

iClicker cloud join link: https://join.iclicker.com/YJHS

Select all of the following statements which are TRUE.

• 1. You can have scaling of numeric features, one-hot encoding of categorical features, and scikit-learn estimator within a single pipeline.
• 2. Once you have a scikit-learn pipeline object with an estimator as the last step, you can call fit, predict, and score on it.
• 3. You can carry out data splitting within scikit-learn pipeline.
• 4. We have to be careful of the order we put each transformation and model in a pipeline.
• 5. If you call cross_validate with a pipeline object, it will call fit and transform on the training fold and only transform on the validation fold.

Break

Let’s take a break!

/

Preprocessing motivation: example

You’re trying to find a suitable date based on:

• Age (closer to yours is better).
• Number of Facebook Friends (closer to your social circle is ideal).

Preprocessing motivation: example

• You are 30 years old and have 250 Facebook friends.

Based on the distances, the two nearest neighbors (2-NN) are:

• Person D (Distance: 30.00)
• Person B (Distance: 50.09)

What’s the problem here?

Imputation: Fill the gaps! (🟩 🟧 🟦)

Fill in missing data using a chosen strategy:

• Mean: Replace missing values with the average of the available data.
• Median: Use the middle value.
• Most Frequent: Use the most common value (mode).
• KNN Imputation: Fill based on similar neighbors.

Example:

Fill in missing values like filling empty seats in a classroom with the average student.

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy='mean')
X_imputed = imputer.fit_transform(X)

Scaling: Everything to the same range! (📉 📈)

Ensure all features have a comparable range.

• StandardScaler: Mean = 0, Standard Deviation = 1.
• MinMaxScaler: Scales features to a [0, 1] range.
• RobustScaler: Scales features using median and quantiles.

Example:

Rescaling everyone’s height to make basketball players and gymnasts comparable.

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

One-Hot encoding: 🍎 → 1️⃣ 0️⃣ 0️⃣

Convert categorical features into binary columns.

• Creates new binary columns for each category.
• Useful for handling categorical data in machine learning models.

Example:

Turn “Apple, Banana, Orange” into binary columns:

from sklearn.preprocessing import OneHotEncoder
encoder = OneHotEncoder()
X_encoded = encoder.fit_transform(X)

Ordinal encoding: Ranking matters! (⭐️⭐️⭐️⭐️⭐️ → 1️⃣)

Convert categories into integer values that have a meaningful order.

• Assign integers based on order or rank.
• Useful when there is an inherent ranking in the data.

Example:

Turn “Poor, Average, Good” into 1, 2, 3:

from sklearn.preprocessing import OrdinalEncoder
encoder = OrdinalEncoder()
X_ordinal = encoder.fit_transform(X)

Transformers

• Are used to transform or preprocess data.
• Implement the fit and transform methods.
  • fit(X): Learns parameters from the data.
  • transform(X): Applies the learned transformation to the data.
• Examples:
  • Imputation (SimpleImputer): Fills missing values.
  • Scaling (StandardScaler): Standardizes features.

Estimators

• Used to make predictions.
• Implement fit and predict methods.
  • fit(X, y): Learns from labeled data.
  • predict(X): Makes predictions on new data.
• Examples: DecisionTreeClassifier, SVC, KNeighborsClassifier

from sklearn.tree import DecisionTreeClassifier
tree_clf = DecisionTreeClassifier()

The golden rule in feature transformations

• Never transform the entire dataset at once!
• Why? It leads to data leakage — using information from the test set in your training process, which can artificially inflate model performance.
• Fit transformers like scalers and imputers on the training set only.
• Apply the transformations to both the training and test sets separately.

Example:

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

sklearn Pipelines

• Pipeline is a way to chain multiple steps (e.g., preprocessing + model fitting) into a single workflow.
• Simplify the code and improves readability.
• Reduce the risk of data leakage by ensuring proper transformation of the training and test sets.
• Automatically apply transformations in sequence.

Example:

Chaining a StandardScaler with a KNeighborsClassifier model.

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier

pipeline = make_pipeline(StandardScaler(), KNeighborsClassifier())

pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)