Is there a way to pre-load images just using html/css?
Yes, there is. There's a technique called CSS sprites.
It basically works by loading one big image and then only displaying certain sections of the image as needed. Obviously, this technique isn't practical for every solution but it's good for many.
A List Apart has a great tutorial on this: A List Apart: Articles: CSS Sprites: Image Slicing's Kiss of Death