I get asked all the time:
“Which project should I build?”
The question is full of great intent, but it's actually flawed.
over 100 applications and portfolios, and only a few times did someone's project impress me enough to advance to the interview stage.
So in this article, I'll give you the exact framework I created and followed to get your perfect ML project up and running.
Let's get into it!
Why Many ML Projects Fail
Let me tell you something that every hiring manager thinks at every company I've worked for.
When we see a model for predicting the price of a house or the survival class of the Titanic, we do not think of “solid foundations.” We think “next.”
I'm not even kidding.
These programs are done so many times that they don't tell me anything about the person.
It only tells me that they can follow a bog-standard course and replicate the results.
A project that gets you hired has four key elements:
- It's personal – He really cares about what he predicts.
- It is a novel — I have never seen it a hundred times before.
- It's worth it – Connects to the type of work you want to do.
- It's live — People can literally see it.
Get all four right, and your project makes the hiring manager remember you, and this comes from my own recruiting experience.
The problem is no one can give you a project like that. It has to come from you.
So instead of giving you an idea, I'm going to give you an outline to follow to develop a project like this.
I've also turned this outline into a 7-page PDF workbook that you can check out in the description below to work through and find the right project for you.
Project Construction Framework
Steal the EXACT project framework I'm using to get a $100k+ offer and join 8,000+ data job seekers today.project.egorhowell.com
An example of a project
Before I get into the outline, let me give you an example of a project from a candidate we hired.
At one of my previous companies, we were hiring a junior data scientist to work on troubleshooting and operational research issues.
The candidate we hired stood out for one main reason: they had a very important and deeply personal project closely aligned with the role.
They were interested in NFL kids football and wanted to improve the way they made their weekly team picks.
Therefore, they developed their own development engine to assign players efficiently within the constraints of the program.
It wasn't just the engine itself; they read academic papers on development techniques and learn how others deal with the same problem.
This project achieves all four points we mentioned at the beginning:
- It's personal– It was a personal problem they were interested in.
- It is a novel– It was different, and we had never seen anything like it before or since.
- It's worth it– showed their enthusiasm and interest in developing and researching the works, which is exactly what we were looking for.
- It's live– It was directly related to the job they were applying for.
Let me break down the exact framework you can follow to build a project like this.
Start with Your Interests
When people are looking for a project to build, they open a list of ML datasets, most likely on Kaggle, and try to find something interesting.
That's in the background.
Start with yourself and your interests.
Specifically, list five things that you really care about outside of work and outside of data and ML.
Focus on your hobbies, concerns, and anything else you can happily talk about for an hour without a problem.
When I did this, my list was something like this:
- Investing money
- Hockey
- Gym/fitness
- Movies
- YouTube
Why do we need to choose something we like?
Because a project you love and are passionate about is a project you will complete.
I can't stress enough how much easier it is to do a project that truly inspires you than one you “think” you should do.
Once you have five interests, please write down five questions for each interest that you really want answered.
For example, “Which soccer players are underrated this week?” question, and “football statistics” are not.
Don't overthink and just write things down.
Now you will have 25 project ideas, which may be completely different or at least not many people have seen.
Sort Your Top Picks
Now we need to cut that list down to our top picks.
The first step is to eliminate questions or ideas that are not clearly an ML problem. For example, “Why do I like movies?” good question, but it's not a machine learning project.
Machine learning at a very high and dirty level can be divided into 5 main areas:
- Predicting the number– retreat.
- Prediction phase– separation.
- Prediction over time– time series.
- To recommend things– recommendation programs.
- Putting things together– to combine.
Go through your 25 or less questions and find the ones that fit one of those 5 areas and eliminate the ones that don't.
This should leave us with about 10-15 concepts that we can solve using ML.
Now you need to choose one, and the way to do this is to evaluate these ideas by comparing them with the following methods:
- What kind of person is he?
- How is the novel?
- How relevant is it to the role I am going for?
- How hard is it to get data?
- How hard is it to build?
Rate each one out of 5, sum them all up, and the one with the highest score will build it.
Validate the Project
Before you dedicate weeks to building this project, I want you to do three quick tests.
First –Where does your data actually come from? Cite the original source — an API, a public dataset or any other unique source. If you can't say one word, finding data is your first task.
Second time— Can you get the first hard version running in about two months, taking an hour or two a day? If it's bigger than that, reduce it. The little project you finish actually outweighs the big one you leave – every time.
The third time— How common is it really? Nothing really original, but if this project is something you've seen a few times before, then maybe rethink it and choose your second choice.
Pass all three checks, and you're done.
Well done! You have a project that is yours, that the hiring manager hasn't seen a hundred times, and that you can actually complete.
Make It Live
Like most people, you will probably do the initial research and prototyping of this project in a Jupyter Notebook.
However, companies today are looking for people who can use their solutions to create business impact.
Even if your model is the best thing since a transformer, it's useless if it's stuck inside a manual.
Modeling is actually not as complicated as people think. I have trained several of my clients to build their first machine learning model with no prior experience using the following technology stack and process:
- Build a prototype solution in Jupyter Notebook.
- Split that Jupyter Notebook into individual Python files that follow production code standards, use features like typing, formats, and docstrings.
- Add your Python files to the git repo and create a nice README that describes the project.
- Add all important software engineering tools and concepts, including unit testing, poetry dependency management, Makefiles, and PyEnv.
- Create a Streamlit dashboard to display your results, and upload them to the Streamlit public cloud.
- Set your repo to run daily using GitHub Actions.
Bish. Bash. Boss.
You just shipped your model from end to end using industry standard tools, which I've been using as a machine learning engineer at top tech companies for years.
I realize it can seem overwhelming to a newbie at this, with no one by your side to guide you, so I've created a template repo with all the boilerplate code to set this up.
GitHub – egorhowell/ML-Project-Starter
Contribute to the development of egorhowell/ML-Project-Starter by creating an account on GitHub.github.com
One thing!
If you're interested in getting data or machine work, I've opened up a few spaces in mine Training program.
You will work personally with my team and myself for 12 months in this program specially designed to help you not only apply, but actually. the world your dream data/ML work.
.Apply and book your free call here..
Find Your Dream Data Career
Get the data science job of your dreams – and boost your salary up to $150kcoaching.egorhowell.com
