keras mnist.load_data() is superslow and throw an error after some times

问题

here is the whole code I use

import os
import numpy as np
import matplotlib.pyplot as plt
from tqdm import tqdm


from keras.layers import Input
from keras.models import Model, Sequential
from keras.layers.core import Dense, Dropout
from keras.layers.advanced_activations import LeakyReLU
from keras.datasets import mnist
from keras.optimizers import Adam
from keras import initializers

os.environ["KERAS_BACKEND"] = "tensorflow"

np.random.seed(10)

random_dim = 100

def load_mnist_data():
    # load the data
    (x_train, y_train), (x_test, y_test) = mnist.load_data()
    # normalize our inputs to be in the range[-1, 1]
    x_train = (x_train.astype(np.float32) - 127.5)/127.5
    print(x_train.shape)
    # convert x_train with a shape of (60000, 28, 28) to (60000, 784) so we have
    # 784 columns per row
    x_train = x_train.reshape(60000, 784)
    return (x_train, y_train, x_test, y_test)


load_mnist_data()

it takes around 30mins to get to half of the download, and after some times (around 5000000/11490434 and 6000000/11490434) it crash and throw this error:

ConnectionResetError: [WinError 10054] Une connexion existante a dû être fermée par l’hôte distant

(translation)

An existing connexion has been closed by the distant host

What is causing this? and is it normal that downloading the mnist data with keras takes like an hour and when I download it from tensorflow it takes only few mins?

回答1:

I wrote how I fixed the problem in a comment but I want to make it more clear.

The problem happened only when the program was lauched from idle AND if it was the first time you would import mnist data from keras.

The easy solution is to simply lauch your program from the command prompt, it seem like idle couldn't handle printing that many thing and printing something on idle actually stop your program for a short amount of time which could explain the absurd amount of time to download it.

TLDR: launch your program from the command prompt

来源：https://stackoverflow.com/questions/51109688/keras-mnist-load-data-is-superslow-and-throw-an-error-after-some-times

标签

python

keras

mnist