Or why you shouldn’t belief predict_proba strategies
In earlier articles, I identified the significance of realizing how positive a mannequin is about its predictions.
For classification issues, it isn’t useful to solely know the ultimate class. We’d like extra data to make well-informed selections in downstream processes. A classification mannequin that solely outputs the ultimate class covers necessary data. We have no idea how positive the mannequin is and the way a lot we are able to belief its prediction.
How can we obtain extra belief within the mannequin?
Two approaches can provide us extra perception into classification issues.
We may flip our level prediction right into a prediction set. The objective of the prediction set is to ensure that it incorporates the true class with a given likelihood. The scale of the prediction set then tells us how positive our mannequin is about its prediction. The less lessons the prediction set incorporates, the surer the mannequin is.
