问题
My question is that: I have ROI's for the bounding boxes around the objects in an image. The ROI's are obtained by the Faster R-CNN. Now what I want is to apply the thresholding to get the object accurately contained within the bounding box. The ROI of this image was got by the Faster RCNN.
So, After getting the ROI's, I only selected ROI from the image and pasted on the black image of the same size and dimension which result in the following image.let say
As you can see that boxes are rectangular so in some places it covers some background area along with spikes. So, how can I apply thresholding to get only the spikes and other pixels turn to black?
EDIT: I've added the link to the ROI text file of the first image in the question
ROI file for first image
回答1:
Color thresholding using cv2.inRange() should work here. I'm assuming you want to isolate the green area
Here's the main idea
- Convert image to HSV format since it is easier to represent color than RBG
- Perform color segmentation with a lower/upper threshold
You could also perform morphological operations to smooth or remove noise after obtaining the mask
import numpy as np
import cv2
image = cv2.imread('1.jpg')
result = image.copy()
image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
lower = np.array([18, 0, 0])
upper = np.array([179, 255, 255])
mask = cv2.inRange(image, lower, upper)
result = cv2.bitwise_and(result,result, mask=mask)
cv2.imshow('result', result)
cv2.imwrite('result.png', result)
cv2.waitKey()
You can use a HSV color thresholder script to isolate the desired color range
import cv2
import sys
import numpy as np
def nothing(x):
pass
# Create a window
cv2.namedWindow('image')
# create trackbars for color change
cv2.createTrackbar('HMin','image',0,179,nothing) # Hue is from 0-179 for Opencv
cv2.createTrackbar('SMin','image',0,255,nothing)
cv2.createTrackbar('VMin','image',0,255,nothing)
cv2.createTrackbar('HMax','image',0,179,nothing)
cv2.createTrackbar('SMax','image',0,255,nothing)
cv2.createTrackbar('VMax','image',0,255,nothing)
# Set default value for MAX HSV trackbars.
cv2.setTrackbarPos('HMax', 'image', 179)
cv2.setTrackbarPos('SMax', 'image', 255)
cv2.setTrackbarPos('VMax', 'image', 255)
# Initialize to check if HSV min/max value changes
hMin = sMin = vMin = hMax = sMax = vMax = 0
phMin = psMin = pvMin = phMax = psMax = pvMax = 0
img = cv2.imread('1.jpg')
output = img
waitTime = 33
while(1):
# get current positions of all trackbars
hMin = cv2.getTrackbarPos('HMin','image')
sMin = cv2.getTrackbarPos('SMin','image')
vMin = cv2.getTrackbarPos('VMin','image')
hMax = cv2.getTrackbarPos('HMax','image')
sMax = cv2.getTrackbarPos('SMax','image')
vMax = cv2.getTrackbarPos('VMax','image')
# Set minimum and max HSV values to display
lower = np.array([hMin, sMin, vMin])
upper = np.array([hMax, sMax, vMax])
# Create HSV Image and threshold into a range.
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, lower, upper)
output = cv2.bitwise_and(img,img, mask= mask)
# Print if there is a change in HSV value
if( (phMin != hMin) | (psMin != sMin) | (pvMin != vMin) | (phMax != hMax) | (psMax != sMax) | (pvMax != vMax) ):
print("(hMin = %d , sMin = %d, vMin = %d), (hMax = %d , sMax = %d, vMax = %d)" % (hMin , sMin , vMin, hMax, sMax , vMax))
phMin = hMin
psMin = sMin
pvMin = vMin
phMax = hMax
psMax = sMax
pvMax = vMax
# Display output image
cv2.imshow('image',output)
# Wait longer to prevent freeze for videos.
if cv2.waitKey(waitTime) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()
Here's the result on the original image
回答2:
In your TensorFlow detection, the output dictionary you get after you run the prediction has a field, "detection_scores".
output_dict = sess.run(tensor_dict,feed_dict={image_tensor: image})
Set a threshold on that,
indexes=np.where(output_dict['detection_scores']>0.5)
Use the boxes, i.e. output_dict['detection_boxes'] only on those specific indexes which you filtered in the previous step.
[EDIT] Adding more code after the discussion in comments
#convert the image to hsv
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
#tune the numbers below accordingly
lower_green = np.array([60, 100, 50])
upper_green = np.array([60 , 255, 255])
mask = cv2.inRange(hsv, lower_green, upper_green)
res = cv2.bitwise_and(frame,frame, mask= mask)
#res has the output masked image
[EDIT] editing with the actual image given in the question
img=cv2.imread("idJyc.jpg")
lower_green = np.array([0, 10, 0])
upper_green = np.array([255 , 100, 255])
mask = cv2.inRange(img, lower_green, upper_green)
mask = np.abs(255-mask)
res = cv2.bitwise_and(img,img, mask=mask)
cv2.imshow("a",res)
cv2.waitKey(0)
Adding the output image for your reference.
回答3:
If you re familiar with applying neural networks and you re having enough data. This task is perfectly suited for segmentation.
I recommend U-Net, since it works with a small amount of labelled data for training. It is also fast, with few operations for this task with comparably low complexity. And has shown good performance on various tasks.
I also found a full code pipeline, in this case for heart segmentation in zebrafish, but in my opinion they did a good job explaining how to prepare the data (proposing labeling tools, etc.) and train the model.
Also taking a step back, you could also think of interpreting your task as a segmentation task from the beginning. Especially for the U-Net it shouldn't be a problem to segment multiple instances in the same image.
来源:https://stackoverflow.com/questions/57585944/how-to-apply-threshold-within-multiple-rectangular-bounding-boxes-in-an-image