Auditing Model Bias with Balanced Data Sets with Mimesis

# Introduction

Whether they are established classifiers or modern large-scale models such as large-scale linguistic models (LLMs), building machine learning solutions often include a risk: algorithms may silently accept biases present in the historical training dataset on which they were trained. But in a critical situation or when the data is sensitive, how can we know investigate whether the model is biased without compromising real-world experience?

This hands-on article guides you through training a simple classification model for “loan approval” on biased data. Based on this, we will use Mimesisan open source library that can help produce a perfect balance, which is contrary to the facts The dataset. You will be able to test “fake” users with similar financial backgrounds but different demographics, thus determining whether or not the model discriminates against certain groups.

# Step by Step Guide

Start by installing the Mimesis library if you're new to using it, or working in a cloud notebook environment like Colab:

Before testing the model, we actually need to find it! In this example, we will collectively generate a dataset of 1,000 bank customers, with only two characteristics: gender and income. These features are categorical and numerical respectively. The creation of the data will be deliberately manipulated so that the gender attribute unduly influences a binary outcome: loan approval. Specifically, in the labeling of the dataset, we will consider the situation where men are allowed in general, while women are only allowed if they have a significantly higher income.

The procedure for creating this implicitly biased dataset and training a decision tree classifier on it is shown below:

import pandas as pd
import numpy as np
from sklearn.tree import DecisionTreeClassifier

# 1. Simulating biased historical data (1000 instances)
np.random.seed(42)
n_train = 1000
genders = np.random.choice(['Male', 'Female'], n_train)
incomes = np.random.randint(30000, 120000, n_train)

approvals = []
for gender, income in zip(genders, incomes):
    if gender == 'Male':
        # Historically, males are approved
        approvals.append(1)
    else:
        # Only females with high income are approved
        approvals.append(1 if income > 80000 else 0)

train_df = pd.DataFrame({'Gender': genders, 'Income': incomes, 'Approved': approvals})

# Converting categories to numbers for the machine learning model
train_df['Gender_Code'] = train_df['Gender'].map({'Male': 1, 'Female': 0})

# 2. Training a Decision Tree classifier
model = DecisionTreeClassifier(max_depth=3)
model.fit(train_df[['Gender_Code', 'Income']], train_df['Approved'])

The next step shows Mimesis in action. We will use this library to create a small set of test cases using the Generic class. This will be done by defining three basic financial profiles that contain random UUIDs (universally unique identifiers) and an average income between 40K and 70K. Note that these profiles will not have aggregated gender information yet:

from mimesis import Generic

generic = Generic('en')

# Generating 3 base financial profiles
base_profiles = []
for _ in range(3):
    profile = {
        'Applicant_ID': generic.cryptographic.uuid(),
        'Income': generic.random.randint(40000, 70000) # Moderate income
    }
    base_profiles.append(profile)

For example, three newly created profiles might look like this:

[{'Applicant_ID': '1f1721e1-19af-4bd1-8488-6abf01404ef9', 'Income': 44815},
 {'Applicant_ID': '5c862597-7f55-43f4-9d6e-ac9cc0b9083e', 'Income': 47436},
 {'Applicant_ID': '3479d4cf-0d9b-4f06-9c43-1c3b7e787830', 'Income': 58194}]

Let's finish building our set of fake examples, which are the core of our research process! From these three basic profiles, we will create two dummy models: one male and one female. For each customer who tests, their application ID and income will be exactly the same, so the only difference will be gender: any difference in the way our trained decision tree model treats them will undoubtedly be evidence of gender bias.

counterfactual_data = []

for profile in base_profiles:
    # Version A: Male Counterfactual
    counterfactual_data.append({
        'Applicant_ID': profile['Applicant_ID'], 
        'Gender': 'Male', 
        'Gender_Code': 1, 
        'Income': profile['Income']
    })
    
    # Version B: Female Counterfactual
    counterfactual_data.append({
        'Applicant_ID': profile['Applicant_ID'], 
        'Gender': 'Female', 
        'Gender_Code': 0, 
        'Income': profile['Income']
    })

audit_df = pd.DataFrame(counterfactual_data)

Here's what three pairs of clients might look like:

1f1721e1-19af-4bd1-8488-6abf01404ef9	Male	1	44815
1	1f1721e1-19af-4bd1-8488-6abf01404ef9	Female	0	44815
2	5c862597-7f55-43f4-9d6e-ac9cc0b9083e	Male	1	47436
3	5c862597-7f55-43f4-9d6e-ac9cc0b9083e	Female	0	47436
4	3479d4cf-0d9b-4f06-9c43-1c3b7e787830	Male	1	58194
5	3479d4cf-0d9b-4f06-9c43-1c3b7e787830	Female	0	58194

An important point to emphasize here: we recently used Mimesis to quickly create perfectly matched “clones” for applicants with the same income but different gender. This emphasizes the importance of the library in providing complete statistical control, separated by the protected attribute.

Now it's time to investigate the model and see what it reveals.

# Asking the model to predict approval for our counterfactuals
audit_df['Predicted_Approval'] = model.predict(audit_df[['Gender_Code', 'Income']])

# Formatting the output for readability (1 = Approved, 0 = Denied)
audit_df['Predicted_Approval'] = audit_df['Predicted_Approval'].map({1: 'Approved', 0: 'Denied'})

print("n--- Model Audit Results ---")
print(audit_df[['Applicant_ID', 'Gender', 'Income', 'Predicted_Approval']].sort_values('Applicant_ID'))

The decision-making results produced by our model couldn't be clearer:

--- Model Audit Results ---
                           Applicant_ID  Gender  Income Predicted_Approval
0  1f1721e1-19af-4bd1-8488-6abf01404ef9    Male   44815           Approved
1  1f1721e1-19af-4bd1-8488-6abf01404ef9  Female   44815             Denied
4  3479d4cf-0d9b-4f06-9c43-1c3b7e787830    Male   58194           Approved
5  3479d4cf-0d9b-4f06-9c43-1c3b7e787830  Female   58194             Denied
2  5c862597-7f55-43f4-9d6e-ac9cc0b9083e    Male   47436           Approved
3  5c862597-7f55-43f4-9d6e-ac9cc0b9083e  Female   47436             Denied

Note that it is exactly the same Applicant_ID again Incomemale clones are allowed to be borrowed. Meanwhile, female clones with such a low income are often rejected. The Mimesis function we used based on the profiles helped us to hold all other variables constant, thus effectively classifying and revealing the discriminative decisions of the model.

# Wrapping up

Throughout this practical article, we have shown how Mimesis can be used to generate limited, anecdotal data examples – without privacy or sensitive data barriers – that can help test the behavior of a model and identify whether the model is behaving in a biased way or not. The next steps you can take if your model is biased may include:

• Supplementing your training data with balanced profiles to correct for historical drift or bias.
• Depending on the type of model, model re-estimation techniques are used.
• Using open source tools for justice – for example, AI Fairness 360 – useful for reducing bias in machine learning pipelines.

Iván Palomares Carrascosa is a leader, author, speaker, and consultant in AI, machine learning, deep learning and LLMs. He trains and guides others in using AI in the real world.