dlib/cv2 Working with hundred thousand of pictures

问题

For my next university-project i will have to teach a Convoluted Neural Network how to denoise a picture of a face so i started digging the we for datasets of faces. I stumbled upon this dataset (CelebA) with 200k+ pictures of people and i found the first few problems: there are too many pictures to do basic computation on them.

I should:

1. Open each image and make a numpy array out of it (dlib.load_rgb_image is fine)
2. Find a face it, use the 5 point shape predictor to find the eyes and align them
3. Rotate the picture so that the eyes are in a straight horizontal line
4. Crop the face and resize it to 256x256 (i could choose 64x64 but its not a huge time saver)
5. Make a copy and add artificial noise to it
6. Save them both to two different folder

On a pc that the university gave me i could do about 40ish image each minute, around 57k images every 24hours.

To speedup thing i have tried threads; one thread for each pictures but the speedup is about 2-3 images more per-minute.

This is the code i'm running:

### Out of the threads, before running them ###
def img_crop(img, bounding_box):
    # some code using cv2.copyMakeBorder to crop the image

MODEL_5_LANDMARK = "5_point.dat"
shape_preditor = dlib.shape_predictor(MODEL_5_LANDMARK)
detector = dlib.get_frontal_face_detector()


### Inside each thread ###
img_in = dlib.load_rgb_image("img_in.jpg")
dets = detector(img_in, 1)
shape = shape_preditor(img_in, dets[0])

points = []
for i in range(0, shape.num_parts):
    point = shape.part(i)
    points.append((point.x, point.y))

eye_sx = points[1]
eye_dx = points[3]

dy = eye_dx[1] - eye_sx[1]
dx = eye_dx[0] - eye_sx[0]
angle = math.degrees(math.atan2(dy, dx))

center = (dets[0].center().x, dets[0].center().y)
h, w, _ = img_in.shape
M = cv2.getRotationMatrix2D(center, angle + 180, 1)
img_in = cv2.warpAffine(img_in, M, (w, h))

dets = detector(img_in, 1)
bbox = (dets[0].left(), dets[0].top(), dets[0].right(), dets[0].bottom())
img_out = cv2.resize(imcrop(img_in, bbox), (256, 256))
img_out = cv2.cvtColor(img_out, cv2.COLOR_BGR2RGB)

img_noisy = skimage.util.random_noise(img_out, ....)
cv2.imwrite('out.jpg', img_out)
cv2.imwrite('out_noise.jpg', img_noisy)

My programming language is Python3.6, how i can speedup things?

Another problem will be loading the whole 200k images into memory as numpy array, from my initial testing 12k images will take around 80seconds with a final shape of (12000, 256, 256, 3). Is there a faster way to achieve this?

回答1:

First of all, forgive me because I am familiar with c++ only. Please find below my suggestion to speed up dlib functions and convert to your python version if it is helpful.

1. Color does not matter to dlib. Hence, change input image to gray before proceeding to save time.

2. I saw you call the below function twice, what is the purpose? it could double the consuming time. If you need to get the new landmarks after alignment, try to rotate landmarks points directly instead of re-detecting. How to rotate points

   dets = detector(img_in, 1)
   

3. Because you just want to detect 1 face per image only. Try to set pyramid_down to 6 (by default it is 1 - room out the image to detect more face). You can test value from 1 - 6

   dets = detector(img_in, 6)
   

4. Turn on AVX instruction.

Note: more detail could be found here Dlib Github

来源：https://stackoverflow.com/questions/53563169/dlib-cv2-working-with-hundred-thousand-of-pictures