Hi, I have a Question related to AI-Engineering Chapter 8 (Chip Huyen) - Dataset Engineering (Page No. : 703): Below is the excerpt: “You investigate and find that in the training data, there are several examples of annotations with unsolicited suggestions. You put in a request to remove these examples from the training data and another request to acquire new examples that demonstrate fact-checking without unsolicited rewriting.” What is being suggested is an unlearning process. Is there a way to make a neural network forget/remove certain trained data after it has been trained. According to me what the above section suggest is: retraining the model from a previous checkpoint (before the corrupted data was used to train) with the new data. Any idea?

I feel like I remember reading something about machine "unlearning" in the past in the way I think you're talking about as research, but not sure what the status is on the latest. If you have access to the checkpoints and training data it's probably more reliable to remove the data and train. But depending on the model and use cases you can fine-tune model on datasets to change behavior or predictions which might get you the results you need too. or even change the output layers.

Yes. Machine Unlearning is a research topic. Based on what i have read it is still work in progress. I think the book states the second. Just imagine if we are able to unlearn the biases, toxicity etc from the model, how amazing would that be. As the saying goes “Empty your cup !!!” even hold true for ASI/AGI