I've found Chrome's Developer Tools (Ctrl+Shift+I) to be absolutely indispensable when debugging HTML, CSS, and JavaScript.
Fiddler is a fantastic tool to spy on (and fiddle with) the traffic your browsers (and other HTTP-connecting applications) are sending.
Aptana is great as a HTML/CSS/JavaScript IDE, complete with red sqigglies for syntax errors and intellisense/autocomplete. (You'll never use Notepad again.)
jQAPI is a really zippy version of the jQuery documentation.