Is there a way to pre-load images just using html/css?
Yes: http://perishablepress.com/press/2008/04/15/pure-css-better-image-preloading-without-javascript/
Yes you can. There are many tutorials available on this topic - here is one of them.
Or even easier: <link rel="prefetch" href="path to image"> (Firefox).
<link rel="prerender" href="http://example.org/index.html"> (Chrome)
If you include an url in your CSS, the image will be downloaded as the .css file is loaded into the document.
Yes, there is. There's a technique called CSS sprites.
It basically works by loading one big image and then only displaying certain sections of the image as needed. Obviously, this technique isn't practical for every solution but it's good for many.
A List Apart has a great tutorial on this: A List Apart: Articles: CSS Sprites: Image Slicing's Kiss of Death