Re-training classifier with new data produce massive class changes

Question

I\'m training every day XGBoost binary classifier on 150k of records. every day, approximately 500 records added to the train set. The test set (inference) 10M records witho