I am merging a CVS branch and one of the larger changes is the replacement wherever it occurs of a Singleton pattern with abstract classes that have a static initialisation bloc
If my original post was the correct understanding and the discussion from Sun that was linked to is accurate (which I think it might be), then I think you have to make a trade off between clarity and performance.
Ask yourself these questions:
From a strict runtime performance point of view, the difference is really negligible. The main difference between the two lies down in the fact that the "static" lifecycle is linked to the classloader, whereas for the singleton it's a regular instance lifecycle. Usually it's better to stay away from the ClassLoader business, you avoid some tricky problems, especially when you try to reload the web application.
Write some code to measure the performance. The answer is going to be dependent on the JVM(Sun's JDK might perform differently than JRockit) and the VM flags your application uses.
I would use a singleton if it needed to store any state, and static classes otherwise. There's no point in instantiating something, even a single instance, unless it needs to store something.
Static is bad for extensibility since static methods and fields cannot be extended or overridden by subclasses.
It's also bad for unit tests. Within a unit test you cannot keep the side effects of different tests from spilling over since you cannot control the classloader. Static fields initialized in one unit test will be visible in another, or worse, running tests concurrently will yield unpredictable results.
Singleton is generally an ok pattern when used sparingly. I prefer to use a DI framework and let that manage my instances for me (possibly within different scopes, as in Guice).
From my experience, the only thing that matters is which one is easier to mock in unit tests. I always felt Singleton is easier and natural to mock out. If your organization lets you use JMockit, it doesn't matter since you can overcome these concerns.