WARNING: A little shameless self promotion here:
If you use TextMate I maintain a bundle specifically to help make working with WordPress easier. Check it out: http://top-frog.com/projects/wordpress-textmate-bundle/
If you're using BBEdit I've also got a simple function, action and filter clipping set: http://top-frog.com/projects/wordpress-clippings-for-bbedit/
On top of that, like others have said, check out the WordPress codex for getting started. If you go straight for WP 3.0 it'll be out of beta by the time you go live. Skip the download links and check it out straight over SVN while its still being developed: http://core.svn.wordpress.org/trunk
I've got my site running on 3.0b2 and my work site is on the same thing. Its a nice improvement and the custom-post-types feature goes a long way to making WordPress much nicer to work with as a CMS.
If you're looking for a theme framework check out Carrington: http://carringtontheme.com/ - starting with Carrington JAM (short for Just Add Markup) you get a really nice bare bones theme to start building on. Much better than Sandbox, IMHO. Carrington takes a little while to wrap your head around but once you do its quite nice and easy to extend. Very handy for complex sites.
Also keep an eye on:
http://markjaquith.wordpress.com/
http://josephscott.org/
