Tag: Machine Learning

Questions I ask before a team spends £100k on an ML project

Is the problem worth solving?

Clarify the business problem we’re solving, how it maps to an ML problem and its solution, what is the expected effort and what the resulting improvement would actually be worth. If it’s a large engineering effort with marginal improvement, that’s important to know before we start.

Do we need ML at all?

Do we have a simpler baseline? Heuristics, rule sets, or off-the-shelf tools often offer the best value for money.

Do we have the data to support this?

“We have a lot of data” is very different from “The data is high quality, enough to support the specific task, labelled appropriately, and readily available to the right engineers/scientists”.

Can we get better data to support this?

Producing high quality annotations for supervised learning looks expensive and boring, but it often has a higher ROI than choosing an inferior approach simply because it works with the data we already have.

Are we measuring “success” correctly?

We’re optimising for NDCG (a ranking metric), while the business cares about revenue. How are the two connected?

What is the cost of a mistake?

We’re not going to have 100% accuracy. How do we deal with it? Do false positives and false negatives have the same cost? How does that compare with solving this problem manually?

What happens if it doesn’t work?

ML is iterative and experimental. We often don’t know if and what will work. Is the culture ready for experimentation? What if we discover the data isn’t good enough, the signal isn’t there, or the improvement isn’t worth the complexity? Do we have executive sponsorship for this type of uncertainty?

What happens if it works?

Who will use the model output, and how? What decisions will it affect? How does it get to production? What happens when the data changes? How are we monitoring quality in the long term?

These help us make sure we’re building the right thing, and that it gets used.

Feature Scaling – Machine Learning Notes

Feature Scaling, also known as Data Normalisation, is a data preprocessing technique used in Machine Learning to normalise the range of predictor variables (i.e. independent variables, or features).

This is done to ensure that all the input variables have values on a normalised range. Since ranges of values can be widely different, and many Machine Learning algorithms use some notion of distance between data points, features with broader ranges will have a stronger impact on the computation of such distance.

By scaling the features into a normalised range, their contribution to the final result will be about the same.

There are several methods to perform feature scaling, common examples include Data Standardisation and Min-Max Normalisation.

Data Standardisation

Each predictor variable is transformed by subtracting its mean and dividing by the standard deviation. The resulting distribution is centred in zero and has unit variance.

x^{\prime} = \frac{x - x_{mean}}{\sigma_{x}}

Min-Max Normalisation

Also called rescaling, the transformed values are in the [0, 1] range. Each predictor variable is transformed by subtracting its minimum value and dividing by the difference between maximum and minimum value.

x^{\prime} = \frac{x - x_{min}}{x_{max} - x_{min}}

The min-max normalisation approach can be generalised to produce transformed variables with values in any [a, b] range, using the following formula:

x^{\prime} = a + \frac{(x - x_{min})(b - a)}{x_{max} - x_{min}}

Do all Machine Learning algorithms need feature scaling?

Algorithms based on distance/similarity and curve fitting require scaling (kNN, SVM, Neural Networks, Linear/Logistic Regression).

Tree-based algorithms (Random Forest, XGBoost) and Naive Bayes don’t require scaling.

Scaling training/test data sets correctly

When scaling on a dataset that is going to be used for supervised learning using a train/test split, we need to re-use the training parameters to transform the test data set. By “training parameters” in this context we mean the relevant statistics like mean and standard deviation for data normalisation.

Why do we need to compute these statistics on the training set only? When using a trained model to make predictions, the test data should be “new and unseen”, i.e. not available at the time the model is built.

In Python/scikit-learn, this translates roughly to the following:

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

scaled_train_data = scaler.fit_transform(train_data)
scaled_test_data = scaler.transform(test_data)

The first function fit_transform() computes the mean and standard deviation on the training data, while the second function transform() re-uses those statistics and applies them to transform the test data.