When Dee talks about “the human black box” with pre-trained patterns, I couldn't help but think how closely similar that is machine learning process. Just as people have many interconnected factors that influence their decisions, ML models have their own version of this complexity.
So, what is machine learning?
It is a subset of AI that enables machinery reading in past data (or historical data) then do predictions or decisions to new data records outside clearly arranged in all possible cases.
With this said, some of the most common “cases” of ML are:
- Prediction or Getting down (eg, forecasting house prices)
- Separation (eg, labeling pictures of cats and dogs)
- Integration (eg, finding customer groups by analyzing their buying habits)
- Which is confusing Adoption (eg, finding available items in your transaction for fraud analysis)
Or, to exemplify these conditions through our daily activities of the human mind, too predict (eg will it rain today?), separate (eg Is that a friend or a stranger?), and find out which is not understood (eg cheese that went bad in our fridge). The difference is in us process these activities and what input or data we be (eg the presence of clouds against a bright and clear sky).
Therefore, the data (and its quality) is always the core of generating quality model results from the above scenarios.
Data: Core “Input”
Similar to humans, who collect multiple sensory inputs from various sources (eg, videos from YouTube, music from the radio, blog posts from Medium, financial records from an Excel sheet, etc.), ML models rely on them. data that would be:
- It was built (like rows in a spreadsheet)
- It's built a little (JSON, XML files)
- Informal (images, PDF documents, free form text, audio, etc.)
Because the data fuels all the information produced by the ML model, we (data specialists) spend a lot of time preparing it – often cited 50-70% of the entire ML project effort.
This preparation phase gives ML models a taste of the “filtering and pre-processing” that humans do naturally.
We check for outliers, handle missing and duplicate values, remove part of the input (elements) unnecessary elements, or create new ones.
Besides the functions listed above, we can also “tune” the data input. – Remember how Dee talked about things being “larger” or “smaller”? – In ML, we achieve the same through use feature engineering again weight sharesalthough fully in mathematical form.
In short, we “organize” the data input so that the model “learns” from clean, high-quality data, yielding more reliable model results.
Modeling: Training and Evaluation
While humans can learn and adapt their “feature weights” through deliberate processes, as Dee explains, ML models have a similarly structured learning process.
Once our data is in good shape, we feed it into ML algorithms (such as neural networks, decision trees, or clustering methods).
In a typical supervised learning program, an algorithm recognizes examples labeled with the correct answers (such as a thousand pictures labeled “cat” or “dog”).
It then adjusts its internal weights – its version of “significant features” – to match (predict) those labels as accurately as possible. In other words, a trained model may provide a probabilistic result indicating how “cat” or “dog” each new image is, based on the learned patterns.
This is where ML is more “intelligent” than the human mind: the results of the model come from a defined process of summarizing the weighted inputs, while people shuffle through many factors – such as hormones, unconscious biases, or immediate physiological needs – that make our internal processes less transparent.
Therefore, the two main stages in building a model are:
- Training: The model is shown with labeled data. It “learns” patterns that link input (image features, for example) to output (the correct pet label).
- Testing: We test the model on new, unobserved data (new images of cats again dogs) to measure how well it converges. If it mislabels certain images, we may adjust the parameters or collect more training examples to improve the accuracy of the generated output.
Since it all comes back to the data, it is important to mention that there can be more in the modeling part, especially if we have “inequality data.”
Example: if the training set has 5,000 a dog but only 1,000 images cat images, the model may depend on the prediction dogs often – unless we apply special techniques deal with”inequality“. But this is a story that would require a whole new post.
The meaning of this is that the number of examples in the input dataset for each possible result (image “cat” or “dog”) influences the complexity of the model's training process and the accuracy of the output.
Continuous Improvement and the Human Factor
However, despite its straightforward appearance, the ML pipeline is not a “fire-and-forget”.
When the model predictions start to go wrong (maybe because the new data has changed the situation), we retrain again fine tuning system.
Againdata experts behind the scenes need to decide how to do it clean or enrich data again readjust the model parameters improving model performance metrics.
That's it”relearning” in machine learning.
This is important because biases and errors in data or models can flow into erroneous results as well it has real life consequences. For example, a credit scoring model trained on biased historical data may systematically lower scores for certain demographic groups, resulting in unfair denial of credit or financial opportunities.
In essence, people are still driving the feedback loop of training machine development, shaping how the ML/AI model “evolves” and “behaves”.
