Build a time machine and go back to 1987? Ho ho.
Ok, no more snarky comments.
How do I figure out what the configuration settings (e.g. baud rate) should be...
Read the datasheet? Ok, ok. Seriously, last one. If you don't know the baud rate of the device you are trying to communicate with, you have two choices. Start guessing, or possibly bust out an o-scope. If you need a good starting point, let me suggest 9600-8-N-1. My suspicion is you can get there with brute force relatively quickly. There's a third option of having an old-school ninja who can tell just by the LOOK of the garbled characters at some standard baud rate what actual baud rate is. An impressive party trick to be sure.
Hopefully though you have access to this information. In unix/linux, you can get ahold of minicom to play with the serial port directly. This should make it fairly quick to get the configuration figured out.
one of the major Unix shells
In Unix the serial port(s) is/are file-mapped into the /dev/ subdir. ttyS0, for example. If you setup the correct baud rate and whatnot using minicom, you can even cat stuff to that file to send stuff out there.
On to the meat of the question, you can access it programmatically through the POSIX headers. termios.h is the big one.
See: http://www.easysw.com/~mike/serial/serial.html#3_1
(NOT AVAILABLE ANYMORE)
but I also have some interest in serial programming using Windows/Hyperterminal.
Hyperterminal and minicom are basically the same program. As for how Windows let's you get access to the serial port, I'll leave that question for someone else. I haven't done that in Windows since the Win95 days.