thats my first post, so please be kind.
I have a matrix with 3~10 coordinates and I want to connect these points to become a polygone with maximum size.
I tried
x1 = rand(1,10)
y1 = rand(1,10)
vi = convhull(x1,y1)
polyarea(x1(vi),y1(vi))
fill ( x1(vi), y1(vi), 'r' );
hold on
plot(x1,y1,'.')
hold off
What is happening here is that CONVHULL is telling us which verticies (vi) are on the convex hull (the smallest polygon that encloses all the points). Knowing which ones are on the convex hull, we ask MATLAB for the area with POLYAREA.
Finally, we use your FILL command to draw the polygon, then PLOT to place the points on there for confirmation.