The zip you downloaded should contain a file named mysql-connector-java-5.0.8-bin.jar
or similar. Version might be different. Check that it is present and check that it is directly in your classpath (not in a subdirectory etc.).
If that's alright, check that your classpath variable is not overwritten by something else. If you execute your program manually do it like this: java -cp "C:\Program Files\Java\jre6\lib\ext\mysql-connector-java-5.0.8-bin.jar:." YourClass
If you are using an IDE such as eclipse, check your build-path settings there. (In eclipse you would right click your project -> Properties -> Java Build Path -> Add external JARs and then supply the path of your connector...jar.