Python/OpenCV — Matching Centroid Points of Bacteria in Two Images

Question

I am working on an algorithm to match the centroids of bacteria using computer vision.

As I\'m an undergraduate and beginner to computer vision, I do not have code specif

Answer 1

My approach works directly on the right image.

The code is shared below and explained with comments:

I am creating a function at the beginning which erodes and dilates the image with a circular kernel, specified number of times.

kernel = np.array([[0, 0, 1, 0, 0], 
                   [0, 1, 1, 1, 0], 
                   [1, 1, 1, 1, 1], 
                   [0, 1, 1, 1, 0], 
                   [0, 0, 1, 0, 0]], dtype=np.uint8)
def e_d(image, it):
    image = cv2.erode(image, kernel, iterations=it)
    image = cv2.dilate(image, kernel, iterations=it)
    return image

Note: Image on the right is read in the grayscale format in the variable "img".

# Applying adaptive mean thresholding
th = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV,11,2)
# Removing small noise
th = e_d(th.copy(), 1)

# Finding contours with RETR_EXTERNAL flag and removing undesired contours and 
# drawing them on a new image.
cnt, hie = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cntImg = th.copy()
for contour in cnt:
    x,y,w,h = cv2.boundingRect(contour)
    # Eliminating the contour if its width is more than half of image width
    # (bacteria will not be that big).
    if w > img.shape[1]/2:      
        continue
    cntImg = cv2.drawContours(cntImg, [cv2.convexHull(contour)], -1, 255, -1)

# Removing almost all the remaining noise. 
# (Some big circular noise will remain along with bacteria contours)
cntImg = e_d(cntImg, 5)


# Finding new filtered contours again
cnt2, hie2 = cv2.findContours(cntImg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

# Now eliminating circular type noise contours by comparing each contour's 
# extent of overlap with its enclosing circle.
finalContours = []      # This will contain the final bacteria contours
for contour in cnt2:
    # Finding minimum enclosing circle
    (x,y),radius = cv2.minEnclosingCircle(contour)
    center = (int(x),int(y))
    radius = int(radius)

    # creating a image with only this circle drawn on it(filled with white colour)
    circleImg = np.zeros(img.shape, dtype=np.uint8)
    circleImg = cv2.circle(circleImg, center, radius, 255, -1)

    # creating a image with only the contour drawn on it(filled with white colour)    
    contourImg = np.zeros(img.shape, dtype=np.uint8)
    contourImg = cv2.drawContours(contourImg, [contour], -1, 255, -1)

    # White pixels not common in both contour and circle will remain white 
    # else will become black.
    union_inter = cv2.bitwise_xor(circleImg, contourImg)
    
    # Finding ratio of the extent of overlap of contour to its enclosing circle. 
    # Smaller the ratio, more circular the contour.
    ratio = np.sum(union_inter == 255) / np.sum(circleImg == 255)
    
    # Storing only non circular contours(bacteria)
    if ratio > 0.55:
        finalContours.append(contour)

finalContours = np.asarray(finalContours)


# Finding center of bacteria and showing it.
bacteriaImg = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)

for bacteria in finalContours:
    M = cv2.moments(bacteria)
    cx = int(M['m10']/M['m00'])
    cy = int(M['m01']/M['m00'])

    bacteriaImg = cv2.circle(bacteriaImg, (cx, cy), 5, (0, 0, 255), -1)
    
cv2.imshow("bacteriaImg", bacteriaImg)
cv2.waitKey(0)

NOTE: I am only taking the image on the right and my image's size is (221, 828). If your input image is smaller or bigger than this, adjust the value of the number of iterations of erosion and dilation for removing noise accordingly to get good results.

Here are the output images:

Also, as you can see in the third image, the leftmost bacteria, its center is marked not exactly at the center. This is happening because, in the code, I have used convex hull of the contours in one place. You can solve this by keeping a track of all the contours and then at the end, take the center of the initial contour.

I am sure that this code can also be modified and made better but this is what I could think of right now. Any suggestions are most welcomed.

Answer 2

This seems to be an easy calibration problem.

Find two corresponding points left and right (i.e. the same points in the real world). If your setup is fixed, you can do that "manually" and once for all. You may have to add markers for this purpose (or use two distant bacteria centers that you match visually). If the setup is not fixed, add the markers anyway and design them so that they are easy to locate by image processing.

Now you have a simple linear relation between the left and right coordinates by solving

XR = a XL + b

for the two points. Then using one of the points to find c,

YR = a YL + c

holds.

Now knowing a, b, c, every point on the left can be mapped to the right. From your sample images, I have determined that

a ~ 1.128
b ~ 773
c ~ -16

very grossly.

---

Do not attempt any kind of matching of the shapes, rely on the geometric transformation of the coordinates.