save and load keras.callbacks.History

Question

I\'m training a deep neural net using Keras and looking for a way to save and later load the history object which is of keras.callbacks.History type. Here\'s the se

Answer 1

You can create a class so you will have the same structure and you can access in both cases with the same code.

import pickle
class History_trained_model(object):
    def __init__(self, history, epoch, params):
        self.history = history
        self.epoch = epoch
        self.params = params

with open(savemodel_path+'/history', 'wb') as file:
    model_history= History_trained_model(history.history, history.epoch, history.params)
    pickle.dump(model_history, file, pickle.HIGHEST_PROTOCOL)

then to access it:

with open(savemodel_path+'/history', 'rb') as file:
    history=pickle.load(file)

print(history.history)

Answer 2

history_model_1 is a callback object. It contains all sorts of data and isn't serializable.

However, it contains a dictionnary with all the values that you actually want to save (cf your comment) :

import json
# Get the dictionary containing each metric and the loss for each epoch
history_dict = history_model_1.history
# Save it under the form of a json file
json.dump(history_dict, open(your_history_path, 'w'))

You can now access the value of the loss at the 50th epoch like this :

print(history_dict['loss'][49])

Reload it with

history_dict = json.load(open(your_history_path, 'r'))

I hope this helps.

Answer 3

You can use Pandas to save the history object as a CSV file.

import pandas as pd

pd.DataFrame.from_dict(history_model_1.history).to_csv('history.csv',index=False)

The JSON approach results in a TypeError: Object of type 'float32' is not JSON serializable. The reason for this is that the corresponding values in the history dictionary are NumPy arrays.