Why is cross_val_predict so much slower than fit for KNeighborsClassifier?

Question

Running locally on a Jupyter notebook and using the MNIST dataset (28k entries, 28x28 pixels per image, the following takes 27 seconds.

from sk         


        

Answer 1

KNN is also called as lazy algorithm because during fitting it does nothing but saves the input data, specifically there is no learning at all.

During predict is the actual distance calculation happens for each test datapoint. Hence, you could understand that when using cross_val_predict, KNN has to predict on the validation data points, which makes the computation time higher!

Answer 2

cross_val_predict does a fit and a predict so it might take longer than just fitting, but I did not expect 64 times longer