Having difficulties detecting small objects in noisy background. Any ways to fix this?

Question

I am trying to make a computer vision program in which it would detect litter and random trash in a noisy background such as the beach (noisy due to sand).

Original Image:

Canny Edge detection without any image processing:

I realize that a certain combination of image processing technique will help me accomplish my goal of ignoring the noisy sandy background and detect all trash and objects on the ground.

I tried to preform median blurring, play around and tune the parameters, and it gave me this:

It preforms well in terms of ignoring the sandy background, but it fails to detect some of the other many objects on the ground, possibly because it is blurred out (not too sure).

Is there any way of improving my algorithm or image processing techniques that will ignore the noisy sandy background while allowing canny edge detection to find all objects and have the program detect and draw contours on all objects.

Code:

from pyimagesearch.transform import four_point_transform
from matplotlib import pyplot as plt
import numpy as np
import cv2
import imutils

im = cv2.imread('images/beach_trash_3.jpg')


#cv2.imshow('Original', im)

# Histogram equalization to improve contrast




###
#im = np.fliplr(im)

im = imutils.resize(im, height = 500)

imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)

# Contour detection
#ret,thresh = cv2.threshold(imgray,127,255,0)

#imgray = cv2.GaussianBlur(imgray, (5, 5), 200)
imgray = cv2.medianBlur(imgray, 11)

cv2.imshow('Blurred', imgray)

'''
hist,bins = np.histogram(imgray.flatten(),256,[0,256])
plt_one = plt.figure(1)
cdf = hist.cumsum()
cdf_normalized = cdf * hist.max()/ cdf.max()

cdf_m = np.ma.masked_equal(cdf,0)
cdf_m = (cdf_m - cdf_m.min())*255/(cdf_m.max()-cdf_m.min())
cdf = np.ma.filled(cdf_m,0).astype('uint8')
imgray = cdf[imgray]

cv2.imshow('Histogram Normalization', imgray)
'''
'''
imgray = cv2.adaptiveThreshold(imgray,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,\
            cv2.THRESH_BINARY,11,2)
'''

thresh = imgray

#imgray = cv2.medianBlur(imgray,5)
#imgray = cv2.Canny(imgray,10,500)
thresh = cv2.Canny(imgray,75,200)
#thresh = imgray
cv2.imshow('Canny', thresh)


contours, hierarchy = cv2.findContours(thresh.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)

cnts = sorted(contours, key = cv2.contourArea, reverse = True)[:5]

test = im.copy()
cv2.drawContours(test, cnts, -1,(0,255,0),2)
cv2.imshow('All contours', test)

print '---------------------------------------------'
#####  Code to show each contour #####
main = np.array([[]])
for c in cnts:
    epsilon = 0.02*cv2.arcLength(c,True)
    approx = cv2.approxPolyDP(c,epsilon,True)

    test = im.copy()
    cv2.drawContours(test, [approx], -1,(0,255,0),2)
    #print 'Contours: ', contours
    if len(approx) == 4:
        print 'Found rectangle'
        print 'Approx.shape: ', approx.shape
        print 'Test.shape: ', test.shape

        # frame_f = frame_f[y: y+h, x: x+w]
        frame_f = test[approx[0,0,1]:approx[2,0,1], approx[0,0,0]:approx[2,0,0]]

        print 'frame_f.shape: ', frame_f.shape
        main = np.append(main, approx[None,:][None,:])
        print 'main: ', main


    # Uncomment in order to show all rectangles in image
    #cv2.imshow('Show Ya', test)


    #print 'Approx: ', approx.shape
    #cv2.imshow('Show Ya', frame_f)
    cv2.waitKey()
print '---------------------------------------------'
cv2.drawContours(im, cnts, -1,(0,255,0),2)
print main.shape
print main
cv2.imshow('contour-test', im)
cv2.waitKey()

Answer 1

what i am understanding from your problem is: you want to segment out the foreground objects from a background which is variable in nature(sand gray level is depending on many other conditions).

there are various ways to approach this kind of problem:

Approach 1:

From your image one thing is clear that, background color pixels will always much more in numbers than foreground, simplest method to start initial segmentation is:

1. Convert the image into gray.
2. Create its histogram.
3. Find the peak index of the histogram, i.e. index which have maximum pixels.

above three steps give you an idea of background BUT the game is not ends here, now you can put this index value in the center and take a range of values around it like 25 above and below, for example: if your peak index is 207 (as in your case) choose a range of gray level from 75 to 225 and threshold image, As according to nature of your background above method can be used for foreground object detection, after segmentation you have to perform some post processing steps like morphological analysis to segment out different objects after extraction of objects you can apply some classification stuff for finer level of segmentation to remove false positive.

Approach 2:

Play with some statistics of the image pixels, like make a small data set of gray values and

1. Label them class 1 and 2, for example 1 for sand and 2 for foreground,
2. Find out mean and variance(std deviation) of pixels from both the classes, and also calculate probability for both the class ( num_pix_per_class/total_num_pix), now store these stats for later use,
3. Now come back to image and take every pixel one by one and apply a gaussian pdf: 1/2*pisigma(exp(-(pix - mean)/2*sigma)); at the place of mean put the mean calculated earlier and at the sigma put std deviation calculated earlier.
4. after applying stage 3 you will get two probability value for each pixel for two classes, just choose the class which have higher probability.

Approach 3:

Approach 3 is more complex than above two: you can use some texture based operation to segment out sand type texture, but for applying texture based method i will recommend supervised classification than unsupervised(like k-means). Different texture feature which you can use are:

Basic:

1. Range of gray levels in a defined neighborhood.
2. local mean and variance or entropy.
3. Gray Level Co-occurrence Matrices (GLCM).

Advanced:

1. Local Binary Patterns.
2. Wavelet Transform.
3. Gabor Transform. etc.

PS: In my opinion you should give a try to approach 1 and 2. it can solve lot of work. :)

Answer 2

For better results you should apply many algorithms. The OpenCV-tutorials focus always on one feature of OpenCV. The real CV-applications should use as many as possible techniques and algorithms.

I've used to detect biological cells in noisy pictures and I gained very good results applying some contextual information:

• Expected size of cells
• The fact that all cells have similar size
• Expected number of cells So I changed many parameters and tried to detect what I'm looking for.

If using edge detection, the sand would give rather random shapes. Try to change the canny parameters and detect lines, rects, circles, ets. - any shapes more probable for litter. Remember the positions of detected objects for each parameters-set and at the and give the priority to those positions (areas) where the shapes were detected most times.

Use color-separation. The peaks in color-histogram could be the hints to the litter, as the distribution of sand-colors should be more even.

For some often appearing, small objects like cigarette-stubs you can apply object matching.

P.S: Cool application! Jus out of curiosity, are yoou going to scan the beach with a quadcopter?

Answer 3

If you want to detect objects on such uniform background, you should start by detecting the main color in the image. Like that you will detect all the sand, and the objects will be in the remaining parts. You can take a look to papers published by Arnaud LeTrotter and Ludovic Llucia who both used this type of "main color detection".