Machine Learning
How to Use ModernBERT and Data for Robust Text Classification | by Eivind Kjosbakken | January, 2025
Learn how to fine-tune ModernBERT and create add-ons for text samples
Eivind Kjosbakken
Follow up
Published on
About Data Science
8 min read
Right now
—
In this article, I discuss how to use and optimize the new ModernBERT text model. In addition, I apply the model to a classic text classification task and show you how you can use artificial data to improve the performance of the model.
· Content
· Finding the dataset
· Using ModernBERT
· Finding errors
· Integrate data to improve model performance
· New effects after being added
· My thoughts and future work
· Conclusion
First, we need to get a dataset to do some text editing on it. To keep it simple, I found an open source dataset on HuggingFace where you predict the sentiment of a given text. The feeling can be predicted in classes:
- Bad (id 0)
- Neutrality (id 1)
- Good (id 2)
