See Jason's answer. It works nicely.
The problem is that the USPS doesn't allow bulk downloads of their zip-code lists unless you pay for it. Google's API, which is used in the gem mentioned by Splashlin, no longer seems to support the city and state, instead it now returns the area code:
require 'open-uri'
require 'json'
json = JSON::parse(open('http://maps.google.com/maps/geo?q=852581').read)
puts json
# >> {"name"=>"852581", "Status"=>{"code"=>602, "request"=>"geocode"}}
This page shows some ways you could roll your own. The sources of the data might not be current though:
http://www.ruby-forum.com/topic/48815