I\'m trying to secure access to an internal static website.
Everyone in the company is using a VPN to access our Amazon VPC so I would like to limit access to that site
Actually, @Michael - sqlbot was right until until May 15, 2015. What you do is correct. You found the documentation (again, correctly) that allows you to set up S3 bucket within VPC (probably with no access from outside world), the same way you set up your EC2 machines. Therefore,
On my S3 bucket, I verified that I could access index.html both from the regular Web and from the VPN.
is a problem. If you didn't make mistakes, you shouldn't be able to access the bucket from regular Web. Everything that you did afterwards is irrelevant - because you didn't create S3 bucket inside your VPN-connected VPC.
You don't give much details as to what you did in your very first step; the easiest is probably to delete this bucket and start from the beginning. With the need to set up route tables and what not it is easy to make a mistake. This is a simper set of instructions - but it doesn't cover as much ground as the document that you followed.
But any links that predate this capability (that is, any links before May 2015) are irrelevant.