Overcoming the Bias-Variance Tradeoff into a Double Descent Phenomenon | by Farzad Nobar | January, 2025
It's not how many times you get knocked down that counts, it's how many times you get up.
In terms of data scientist interviews, discussing the bias-variance tradeoff is one of the most common topics I've come across, both as an interviewee in the past and more recently as someone interviewing candidates or joining such interviews. Later in the post, we'll discuss what the bias-variance tradeoff is and why it works differently in deep learning experiments, but let me explain why I think this topic keeps coming up in determining the scope of machine learning for data science students both. entry and experienced levels.
As machine learning scientists, we spend a lot of time, energy, care and computing resources to train good machine learning models but we always know that our models will have an error rate as they generalize, also known as error of evaluation. Inexperienced data scientists tend to focus on learning new modeling techniques and algorithms, which I believe is a healthy exercise. However, more experienced data scientists are the ones who have learned over time how to better understand and manage the experimental error that is inevitably present in those trained models.
