whether you're a manager, data scientist, engineer, or product owner, you've probably been in at least one meeting where the discussion focused on “putting the model into production.”
But seriously, what does productivity mean?
As you may know, I am an AI engineer. I started my first data science job in 2015, at a large French company in the energy sector. At that time, we were among the first actors to develop AI applications for energy management and production (nuclear, hydraulic, and renewable). And if there's one domain where putting AI into production is highly regulated, it's energy, especially nuclear. This is closely related to the nature of the data and the fact that you cannot easily push machine learning models into an existing environment.
Because of this experience, I learned early on that modeling in a notebook is just the tip of the iceberg. I started talking about productivity very quickly, not really knowing what it meant. For these reasons, I want to share with you a clearer idea that I have developed over the years when it comes to pushing machine learning projects into production.
But let's pause and think about our main question.
What does productivity mean?
Sometimes, what is behind the word “productivity,” can be difficult to read and understand. There are tons of YouTube videos and articles about it, but very few translate into something you can use in real projects.
If you try to answer it, our ideas will probably converge at the end of this article, even if the methods we use to achieve production may differ from one context to another.
Key Definition
In the context of machine learning, productivity means that the output of your model directly affects the user or product.
That influence can take many forms, such as teaching someone, helping them make a decision, or empowering them to do something they couldn't do before; it may also mean adding a feature to a shopping app's recommendation system.
Any program containing a machine learning algorithm used by an end user or other product or application may be considered a model in production.
Besides being influential, productivity also comes with a layer of accountability. What I mean is that if no one or no system is responsible for fixing the model if it is wrong, then your model may be used, but not in production.
There is a common perception that 87% of ML projects fail to reach the final stage of production. I don't know if that's really true, but my explanation is simple: most ML models don't reach the point where they impact the user or the product. And even when they do, there's usually no plan in place to make them reliable over time, so they're just used and accessible.
So if we agree that productivity means having an impactful and responsive ML project, how do we get there?
The Many Faces of Production
To answer that, we need to accept that production has many faces. A model is only one part within the main ETL pipeline.
This point is important.
We tend to think of a model as a black box, data comes in, mathematical magic happens, and a prediction comes out. In fact, that is a gross oversimplification. In manufacturing, models are often part of a wider data flow, often closer to data transformation than a stand-alone decision engine.
Also, not all “productions” look the same depending on how powerful the model is in the final system.
Sometimes a model supports a decision, such as a score, recommendation, alert, or dashboard.
Sometimes it makes a decision, such as automatic actions, real-time blocking, or triggering workflows.
The difference is very important. If your system is automated, the cost of error is not the same, and engineering requirements often increase very quickly.
In my experience, most production systems can be divided into:
→ A data storage system in production, this means that all data is stored in file systems or in a securely hosted database in production environments (cloud or on-premises).
→ Production of data acquisition component, this means having a system or workflow that connects to the production database and retrieves data to be used as input to the model. This workflow can contain data preparation steps.
→ Pushing the machine learning component into production, this is the interesting part. It means that the model is already trained, and we need a program that allows it to work in the same environment as other components.
These three parts clearly show us that ML in manufacturing is not about the machine learning model itself, it's about everything around it.
But let's focus only on part 3, “pushing ML into production,” because the other steps are often handled by different groups in the company.
4 step classification
If I had a young data scientist I needed to explain how to work on this part, I would break it down like this:
Step 1: Work
You start with a trained model. The first thing you need is a function, some code that loads the model, gets the input data, makes a prediction, and returns the output.
At this stage, everything works in place. It's exciting when you start seeing the predictions come out, but we don't want to stop there.
It's an important functional detail early on, don't just think “does it predict?”, and think “does it fail cleanly?” In production, your job will eventually accept unusual input, missing values, unexpected sections, corrupted files, or out-of-range signals. Your future will thank you for basic validation and clear error messages.
Step 2: Interface
To make this work for others (without asking them to use your code), you need an interface, usually an API.
Once invoked, this API receives configured requests containing input data, passes them to your prediction function, and returns the output. This is what allows other systems, applications, or users to interact with your model.
And here is the reality of production, the interface is not just a technical thing, it is a contract. If another system expects / predicts and you reveal something else, a conflict is guaranteed. The same applies if you change the schema every two weeks. When parties say “the model is in production,” most of the time what they mean is “we've built a contract that other people rely on.”
Step 3: Nature
Now we need portability. That means packaging the environment, code, API, and all dependencies so that the program can run elsewhere without modification.
If you've followed the steps so far, you've built a model, wrapped it in a function, and exposed it through an API. But none of this matters if everything stays closed in your area.
This is where things get more professional: reproducibility, versioning, and traceability. Not really, enough so that if you use v1.2 today, you can explain in three months what changed and why.
Step 4: Infrastructure
The last step is to host everything somewhere that users or applications can access.
In practice, this usually means the cloud, but it can also be a company's internal servers or edge infrastructure. The important point is that what you have created should be accessible, stable, and usable when needed.
And this is where many teams learn a hard lesson. In manufacturing, the “best model” is usually not the one with the best metric in the notebook. It's the one that fits the real constraints, delays, costs, security, regulations, monitoring, maintenance, and sometimes simply, “can we implement this with the team we have?”
Step 5: Monitoring
You can have a clean API and a great infrastructure, and still fail in production because you don't see problems early.
The model in unattended production is actually already broken, you just don't know it yet.
Monitoring doesn't have to be complicated. At a minimum, you want to know:
- is the service high and the delay tolerable?
- does the input still look “normal”?
- is the data output increasing?
- does the business impact still make sense?
With most real-world projects, performance doesn't fall with a huge crash. It rots quietly.
Having all these components in place is what turns a model into something useful and impactful. Based on experience, here are a few practical guidelines.
For Step 1 (Activity), stick to the tools you know (scikit-learn, PyTorch, TensorFlow), but think about portability early. Formats like ONNX can make future automation much easier. When developing your packages, you need to be sure, whether you are a manager or a data scientist, that the necessary software engineering or data engineering skills are available, because building internal libraries is a very different matter from using off-the-shelf tools.
For Step 2 (Interface), frameworks like FastAPI work very well, but always think about the consumer. If another system expects / predicts and you reveal something else, a conflict is guaranteed. You need to coordinate with the stakeholders, all the technical points about where the output of the machine learning goes should be very clear.
For Step 3 (Environment), this is where Docker comes in. You don't need to do everything immediately, but you should understand the basics. Think of Docker as putting everything in a box that can run almost anywhere. If you already have good data engineering skills, this should be fine. If not, you need to build your own or rely on someone in your team.
For Step 4 (Infrastructure), the parameters determine the choice. Lambda, microservices, edge devices, and GPUs. ML workloads often require specialized infrastructure, sometimes with managed services like SageMaker.
At every step, one rule saves lives: always keep the easy way back. Productivity isn't just about posting, it's also about recovery when reality hits.
Don't think of this step of your data science project as a single milestone. Sequence of steps and change of mind. In the company, we don't expect you to push the most complex model, we want you to build a model that answers business questions or adds a feature expected of a certain product. We need this model to reach the product or the user, and be monitored so that people can always trust and use it.
Understanding your environment is very important. The tools I mentioned earlier may be different for each group, but the method of operation is the same. I am sharing them to give you a concrete idea.
You can build a great model, but if no one uses it, it doesn't matter.
And when people use it, it becomes real, requiring ownership, supervision, boundaries, and a system around it.
Don't let your work sit at 87%.
Note: Some parts of this article were originally written in French and translated into English with the help of Gemini.
🤝 Stay Connected
If you enjoyed this article, feel free to follow me on LinkedIn for reliable information about AI, Data Science, and careers.
👉 LinkedIn: Sabrhas Bendimerad
👉 Average:
👉 Instagram:
