given a convex polgyon and a number N, how do I find the smallest polygon that
You need to define the notion of "smallest" in your question. Whatever your definition, this question has been heavily studied in the computational geometry literature. The key search phrase is minimal enclosing k-gon:
The general algorithms are not simple (although algorithms for min area triangles or rectangles are simple). Depending on your goals, you might have to abandon any mathematical notion of "smallest" and head for a heuristic.
While number of edges > N do
remove the shortest edge by replacing its endpoints
with the intersection point of the adjacent edges