When Irregularity is Free: Using Low Impact Points to Reduce Computing Costs

As data privacy concerns in machine learning grow, the ability to unread—or remove—certain data points from trained models becomes even more important. Although sophisticated unlearning methods have emerged as an answer, they often treat all points in the forgetting set equally. In this work, we challenge this approach by asking: do points that have a negative impact on model learning need to be removed? Through a comparative analysis of the influence tasks across language and perception tasks, we identify subsets of training data that have a negligible effect on the model's results. Using these insights, we propose an efficient delearning framework that reduces the size of datasets before stopping learning—resulting in significant computational savings (up to ~50%) for real-world examples.

• † Harvard University
• ‡ University College London (UCL)
• ** Work done while at Apple