Android Q - Get depth map from portrait mode photos

Question

I\'m trying to build a sample Android App to extract the depth map of portrait mode photos taken with the google camera app. I know that it\'s saved along the blurred photos.

Answer 1

Pixel phone portrait mode photo is concatenated of 4 JFIF structure https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format. Each JFIF structure is an jpeg image.

A JFIF structure starts with marker 0xFFD8 and ends with marker 0xFFD9. Therefore, we can split a portrait mode image into 4 jpeg files.

The following python code prints the marker positions and splits PXL_20210107_114027740.PORTRAIT.jpg into,

1. pxl_out_0.jpg: display image
2. pxl_out_1.jpg: original image
3. pxl_out_2.jpg: depthmap with 256 grey level
4. pxl_out_3.jpg: dummy image filled with 255

with open('PXL_20210107_114027740.PORTRAIT.jpg', mode='rb') as infile:
    buffer = infile.read()

bufferlen = len(buffer)
pos = 0
pos_d8 = 0
n = 0
i = 0
while i < bufferlen:
    if buffer[i] == 0xff:
        pos = i
        i += 1
        if buffer[i] == 0xd8:
            print('ffd8: {0}'.format(pos))
            pos_d8 = pos
        elif buffer[i] == 0xd9:
            print('ffd9: {0} len: {1}'.format(pos, pos - pos_d8 + 2))
            with open('pxl_out_{0}.jpg'.format(n), mode='wb') as outfile:
                n += 1
                outfile.write(buffer[pos_d8: pos + 2])
    i += 1

Answer 2

I managed to extract the depth image using "exiftool". But I'm also trying to find a way to do that programmatically directly from the metadata.