问题
I created a DataGenerator
with Sequence class.
import tensorflow.keras as keras
from skimage.io import imread
from skimage.transform import resize
import numpy as np
import math
from tensorflow.keras.utils import Sequence
Here, `x_set` is list of path to the images and `y_set` are the associated classes.
class DataGenerator(Sequence):
def __init__(self, x_set, y_set, batch_size):
self.x, self.y = x_set, y_set
self.batch_size = batch_size
def __len__(self):
return math.ceil(len(self.x) / self.batch_size)
def __getitem__(self, idx):
batch_x = self.x[idx * self.batch_size:(idx + 1) *
self.batch_size]
batch_y = self.y[idx * self.batch_size:(idx + 1) *
self.batch_size]
return np.array([
resize(imread(file_name), (224, 224))
for file_name in batch_x]), np.array(batch_y)
Then, I applied this to my training and validation data. X_train
is a list of strings which contains the image paths to the training data. y_train
are onehotencoded labels of the training data. The same for validation data.
I created the image paths using this code:
X_train = []
for name in train_FileName:
file_path = r"/content/gdrive/My Drive/data/2017-IWT4S-CarsReId_LP-dataset/" + name
X_train.append(file_path)
After that, I applied the DataGenerator
to the training and validation data:
training_generator = DataGenerator(X_train, y_train, batch_size=32)
validation_generator = DataGenerator(X_val, y_val, batch_size=32)
Afterwards I used the fit_generator
method to run a model:
model.fit_generator(generator=training_generator,
validation_data=validation_generator,
steps_per_epoch = num_train_samples // 32,
validation_steps = num_val_samples // 32,
epochs = 10,
use_multiprocessing=True,
workers=2)
On CPU it worked fine the first times, my model was initialized and the first epoch started. Then, I changed the runtime type in Google Colab to GPU and ran the model again.
And got the following error:
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-79-f43ade94ee10> in <module>()
5 epochs = 10,
6 use_multiprocessing=True,
----> 7 workers=2)
16 frames
/usr/local/lib/python3.6/dist-packages/imageio/core/request.py in _parse_uri(self, uri)
271 # Reading: check that the file exists (but is allowed a dir)
272 if not os.path.exists(fn):
--> 273 raise FileNotFoundError("No such file: '%s'" % fn)
274 else:
275 # Writing: check that the directory to write to does exist
FileNotFoundError: No such file: '/content/gdrive/My Drive/data/2017-IWT4S-CarsReId_LP-dataset/s01_l01/1_1.png'
Today, I got this error also when running the program without the usage of GPU. When running the program, Colab told me that there was Google Drive Time Out. So, is this error due to this timeout of Google Drive? And if yes, how can I solve this? Does anyone know what I should change in the program?
回答1:
You can write this code to avoid timeout in google colab in console
ConnectButton() {
console.log("Connect pushed");
document.querySelector("#top-toolbar > colab-connect-
button").shadowRoot.querySelector("#connect").click()
}
setInterval(ConnectButton,60000);
Source: How to prevent Google Colab from disconnecting?
回答2:
The problem seems to be the input. Your model cannot find the input file. If you change runtime then there's gonna be a factory reset. All your disk content in the session will be erased.
Run cells from the beginning if you change runtime in between.
来源:https://stackoverflow.com/questions/62468830/filenotfounderror-no-such-file-error-occuring-due-to-timeout-of-google-driv