I just did my first install of PostgreSQL 9.1 on Ubuntu 10.04.
note: I have done it a few times on Windows with an installer without issues.
After a bit of effor
The answer provided by Erwin Brandstetter, was helpful, but for whatever reason, it didn't work for me. The reason is that I couldn't initdb to ever run. I kept getting a "bash: command not found" error when trying to run it with the locale. What I ended up doing was:
changing the locale of the OS. For me, this was:
$ update-locale LANG=en_US.UTF-8
Note: then I had to reboot the server. To confirm that it worked, just run:
$ locale
with that set, I stopped and dropped the cluster:
$ su postgres
$ pg_dropcluster --stop 9.1 main
note: main is the default cluster that got created for me (your cluster name might be different)
$ pg_createcluster --start 9.1 main
Again, I just recreated the cluster with the same name (main).
Note: because I picked the same cluster name (main), I had to go back and update my .conf files. For me, this specifically was postgres.conf and pg_hba.conf to re-enable remote access to the box. I'm not going to include how to do that here as there is lots of documentation on the web for this. But, if someone wants to edit this answer later to include it, that would be fine! :)