I vote for the Nintendo DS:
- Nice hardware : 2 CPUs, 2 screens, touchscreen, mic, speakers, wireless, 2D and 3D acceleration
- No OS to speak of
- Freedom to talk to the bare metal without restriction
- Well-documented
- Very active dev community
- Enthusiastic audience for anything cool you create
- Cheap (shockingly so if you go for 1st-gen units)
All-in-all it's really excellent fun to play with.
To get started:
- Get a DS
- Get a SLOT1 flash-cart (I've got a DS-X, but there are plenty of others)
- Get devkitpro
- Go here for help or advice