I agree that using an interface is not the way to go. Avoiding this pattern even has its own item (#18) in Bloch's Effective Java.
An argument Bloch makes against the constant interface pattern is that use of constants is an implementation detail, but implementing an interface to use them exposes that implementation detail in your exported API.
The public|private static final TYPE NAME = VALUE; pattern is a good way of declaring a constant. Personally, I think it's better to avoid making a separate class to house all of your constants, but I've never seen a reason not to do this, other than personal preference and style.
If your constants can be well-modeled as an enumeration, consider the enum structure available in 1.5 or later.
If you're using a version earlier than 1.5, you can still pull off typesafe enumerations by using normal Java classes. (See this site for more on that).
