Using OpenCV remap function crops image

Question

I am trying to warp an 640x360 image via the OpenCV remap function (in python 2.7). The steps executed are the following

1. Generate a curve and store its x and y coordinates in two seperate arrays, curve_x and curve_y.I am attaching the generated curve as an image(using pyplot):

   

2. Load image via the opencv imread function

   original = cv2.imread('C:\\Users\\User\\Desktop\\alaskan-landscaps3.jpg')
   

3. Execute a nested for loop so that each pixel is shifted upwards in proportion to the height of the curve at that point.For each pixel I calculate a warping factor by dividing the distance between the curve's y coordinate and the "ceiling" (360) by the height of the image. The factor is then multiplied with the distance between the pixel's y-coordinate and the "ceiling" in order to find the new distance that the pixel must have from the "ceiling" (it will be shorter since we have an upward shift). Finally I subtract this new distance from the "ceiling" to obtain the new y-coordinate for the pixel. I thought of this formula in order to ensure that all entries in the map_y array used in the remap function will be within the area of the original image.

   for i in range(0, y_size):
       for j in range(0,x_size):
           map_y[i][j]= y_size-((y_size - i) *  ((y_size - curve_y[j]) / y_size))
           map_x[i][j]=j`
   

4. Then using the remap function

   warped=cv2.remap(original,map_x,map_y,cv2.INTER_LINEAR)
   

The resulting image appears to be warped somewhat along the curve's path but it is cropped - I am attaching both the original and resulting image

I know I must be missing something but I can't figure out where the mistake is in my code - I don't understand why since all y-coordinates in map_y are between 0-360 the top third of the image has disappeared following the remapping

Any pointers or help will be appreciated. Thanks

[EDIT:] I have edited my function as follows:

#array to store previous y-coordinate, used as a counter during mapping process
floor_y=np.zeros((x_size),np.float32)
#for each row and column of picture
for i in range(0, y_size):
    for j in range(0,x_size): 
        #calculate distance between top of the curve at given x coordinate and top
        height_above_curve = (y_size-1) - curve_y_points[j]
        #calculated a mapping factor, using total height of picture and distance above curve
        mapping_factor = (y_size-1)/height_above_curve
        # if there was no curve at given x-coordinate then do not change the pixel coordinate
        if(curve_y_points[j]==0):
            map_y[i][j]=j
        #if this is the first time the column is traversed, save the curve y-coordinate
        elif (floor_y[j]==0):
            #the pixel is translated upwards according to the height of the curve at that point
            floor_y[j]=i+curve_y_points[j]
            map_y[i][j]=i+curve_y_points[j] # new coordinate saved
        # use a modulo operation to only translate each nth pixel where n is the mapping factor. 
        # the idea is that in order to fit all pixels from the original picture into a new smaller space
        #(because the curve squashes the picture upwards) a number of pixels must be removed 
        elif  ((math.floor(i % mapping_factor))==0):
            #increment the "floor" counter so that the next group of pixels from the original image 
            #are mapped 1 pixel higher up than the previous group in the new picture
            floor_y[j]=floor_y[j]+1
            map_y[i][j]=floor_y[j]
        else:
            #for pixels that must be skipped map them all to the last  pixel actually translated to the new image 
            map_y[i][j]=floor_y[j]
        #all x-coordinates remain unchanges as we only translate pixels upwards
        map_x[i][j] = j
#printout function to test mappings at x=383
for j in range(0, 360):
    print('At x=383,y='+str(j)+'for curve_y_points[383]='+str(curve_y_points[383])+' and floor_y[383]='+str(floor_y[383])+'  mapping is:'+str(map_y[j][383]))

The bottom line is that now the higher part of the image should not receive mappings from the lowest part so overwriting of pixels should not take place. Yet i am still getting a hugely exaggerated upwards warping effect in the picture which I cannot explain. (see new image below).The top of the curved part is at around y=140 in the original picture yet now is very close to the top i.e y around 300. There is also the question of why I am not getting a blank space at the bottom for the pixels below the curve.

I'm thinking that maybe there is also something going on with the order of rows and columns in the map_y array?

Answer 1

I don't think the image is being cropped. Rather, the values are "crowded" in the top-middle pixels, so that they get overwritten. Consider the following example with a simple function on a checkerboard.

import numpy as np
import cv2
import pickle

y_size=200
x_size=200

x=np.linspace(0,x_size,x_size+1)
y=(-(x-x_size/2)*(x-x_size/2))/x_size+x_size
plt.plot(x,y)

The function looks like this:

Then let's produce an image with a regular pattern.

test=np.zeros((x_size,y_size),dtype=np.float32)

for i in range(0, y_size):
    for j in range(0,x_size):
        if i%2 and j%2:
            test[i][j]=255
cv2.imwrite('checker.png',test)

Now let's apply your shift function to that pattern:

map_y=np.zeros((x_size,y_size),dtype=np.float32)
map_x=np.zeros((x_size,y_size),dtype=np.float32)

for i in range(0, y_size):
    for j in range(0,x_size):
        map_y[i][j]= y_size-((y_size - i) *  ((y_size - y[j]) / y_size))
        map_x[i][j]=j

warped=cv2.remap(test,map_x,map_y,cv2.INTER_LINEAR)

cv2.imwrite('warped.png',warped)

If you notice, because of the shift, more than one value corresponds to the top-middle areas, which makes it look like it is cropped. But if you check to the top left and right corners of the image, notice that the values are sparser, thus the "cropping" effect does not occur much. I hope the simple example helps better to understand what is going on.