What to Do When the Logit Decision Boundary Fails? | by Lukasz Gatarek | January, 2025

Apply engineering classification models using Bayesian Machine Learning

Logistic regression is a widely used machine learning model for binary classification datasets. The model is simple and based on a key assumption: the existence of a line decision boundary (a line or area in a high-dimensional feature space) that can separate categories of the target variable y based on model features.

Briefly, a decision boundary can be interpreted as a threshold at which the model assigns a data point to one class or another, conditional on the predicted probability of belonging to the class.

The figure below shows a schematic representation of the decision boundary that divides the target variable into two categories. In this case the model is based on a set of two factors (x1 and x2). The target variables can be clearly divided into two categories based on the characteristic values.

However, in your daily modeling activities, the situation may look like the figure below.