Create Numpy array of images

Question

I have some (950) 150x150x3 .jpg image files that I want to read into an Numpy array.

Following is my code:

X_data = []
files = glob.glob (\"*.jpg\")
fo         


        

Answer 1

Appending images in a list and then converting it into a numpy array, is not working for me. I have a large dataset and RAM gets crashed every time I attempt it. Rather I append the numpy array, but this has its own cons. Appending into list and then converting into np array is space complex, but appending a numpy array is time complex. If you are patient enough, this will take care of RAM crasing problems.

def imagetensor(imagedir):
  for i, im in tqdm(enumerate(os.listdir(imagedir))):
    image= Image.open(im)
    image= image.convert('HSV')
    if i == 0:
      images= np.expand_dims(np.array(image, dtype= float)/255, axis= 0)
    else:
      image= np.expand_dims(np.array(image, dtype= float)/255, axis= 0)
      images= np.append(images, image, axis= 0)
  return images

I am looking for better implementations that can take care of both space and time. Please comment if someone has a better idea.

Answer 2

Your definition for the .JPG frame that will be put into a matrix of the same size should should be x, y, R, G, B, A. "A" is not used, but it does take up 8 bits at the end of each pixel.

Answer 3

Here is a solution for images that have certain special Unicode characters, or if we are working with PNGs with a transparency layer, which are two cases that I had to handle with my dataset. In addition, if there are any images that aren't of the desired resolution, they will not be added to the Numpy array. This uses the Pillow package instead of cv2.

resolution = 150

import glob
import numpy as np
from PIL import Image

X_data = []
files = glob.glob(r"D:\Pictures\*.png")
for my_file in files:
    print(my_file)
    
    image = Image.open(my_file).convert('RGB')
    image = np.array(image)
    if image is None or image.shape != (resolution, resolution, 3):
        print(f'This image is bad: {myFile} {image.shape if image is not None else "None"}')
    else:
        X_data.append(image)

print('X_data shape:', np.array(X_data).shape)
# If you have 950 150x150 images, this would print 'X_data shape: (950, 150, 150, 3)'

If you aren't using Python 3.6+, you can replace the r-string with a regular string (except with \\ instead of \, if you're using Windows), and the f-string with regular string interpolation.

Answer 4

I tested your code. It works fine for me with output

('X_data shape:', (4, 617, 1021, 3))

however, all images were exactly the same dimension.

When I add another image with different extents I have this output:

('X_data shape:', (5,))

So I'd recommend checking the sizes and the same number of channels (as in are really all images coloured images)? Also you should check if either all images (or none) have alpha channels (see @Gughan Ravikumar's comment)

If only the number of channels vary (i.e. some images are grey), then force loading all into the color format with:

image = cv2.imread (myFile, cv2.IMREAD_COLOR)

EDIT: I used the very code from the question, only replaced with a directory of mine (and "*.PNG"):

import cv2
import glob
import numpy as np

X_data = []
files = glob.glob ("C:/Users/xxx/Desktop/asdf/*.PNG")
for myFile in files:
    print(myFile)
    image = cv2.imread (myFile)
    X_data.append (image)

print('X_data shape:', np.array(X_data).shape)