silhouette extraction from depth

Question

Hello I have a depth image, I want to extract the person(human) silhouette from that. I used pixel thresholding like this:

for i=1:240 
  for j=1:320 
    if b(i,j)>2400 || b(i,j)<1900 
      c(i,j)=5000; 
    else 
      c(i,j)=b(i,j); 
    end 
  end
end

but there is some part left. Is there any way to remove that?

Original_image:

Extracted_silhouette:

Answer 1

According to this thread depth map boundaries can be found based on the direction of estimated surface normals.
To estimate the direction of the surface normals, you can

[dzx dzy] = gradient( depth_map ); %// horizontal and vertical derivatives of depth map
n = cat( 3, dzx, dzy, ones(size(dzx)) );
n = bsxfun( @rdivide, n, sqrt( sum( n.^2, 3 ) ) ); %// normalize to unit length

A simple way would be to threshold

e = abs( n(:,:,3) ) < 1e-2;

Resulting with

A more sophisticated method of deriving silhouette from boundary can be found in this answer.

Answer 2

This is hard to do by thresholding, because the couch is at the same depth as the person's upper body. Do you need to segment out the entire person, or would it be sufficient to segment out the upper body? In the latter case, you can try using vision.CascadeObjectDetector is the Computer Vision System Toolbox to detect the person's upper body in the RGB image.

Answer 3

Building on the work of @Shai above, you could take the output of the thresholding and then apply a boundary to THAT image. Below is an an example that you can feed in [YOUR_IMAGE] from the output from the previous steps, and then modify the value of [ADJUST] until only the person is selected.

This code is looking for boundaries based on size and will not select anything bigger than the value you enter. Simply, but it works for me. Hope this helps.

boundaries = bwboundaries([YOUR_IMAGE]);    
 numberOfBoundaries = size(boundaries)
   if (size(boundaries) < [ADJUST])
      for k = 1 : numberOfBoundaries
         thisBoundary = boundaries2{k}
         plot(thisBoundary(:,2), thisBoundary(:,1), 'b', 'LineWidth', 2);   
      end
   end