In my current project we have to develop a good looking desktop based application in java.
Currently, we have a very basic desktop application written in swing and team
Check out pushing-pixels. Kirill is the author of Substance which provides a ton of new look and feel options for Swing. Here's a blog entry which shows a little bit of what is possible with Swing. There used to be a link to his 2007 presentation at Java One, but unfortunately the link no longer works.
Swing is very flexible with regards to look-and-feel and you can certainly make very good looking applications with Swing.
I recommend the book Filthy Rich Clients by Chet Haase and Romain Guy if you want to learn how to make nice Swing GUI applications.
You can certainly create great looking UI's in Swing - applications such as NetBeans and IntelliJ IDEA have proved this amongst many others.
The only alternative Swing has is SWT(Eclipses toolkit), however it's not portable as Swing and not as flexible as well. It offers, however, faster performance and the use of native components, which might be what some people actually want.
There are Java bindings for most existing GUI toolkits as well - for example Java for Gnome and someone might find them interesting.
I however favour Swing's extreme portability, MVC adoption, pluggable look and feels, availability of lots of third party libraries with additional components(SwingX, JIDE, ...) and ease of use.
Btw for optimal results in Swing, choosing a good layout manager is essential - I recommend you to have a look at the almighty MiG layout.
You can make it look just about however you want. Especially if you write up your own look and feel. I've done it myself with semi-transparent panels and simplistic buttons.
Have a look at the screen shots at:
http://www.taranfx.com/best-java-swing-look-and-feel-themes-professional-casual-top-10
Nimbus is a new look and feel which is gaining popularity, and it looks very professional imo:
Don't touch JavaFX 1 with a bargepole.
JavaFX 2 however looks very promising, with Oracle saying they're going to integrate it tightly to Java 8. It's got a very nice API behind it, it's graphically accelerated, you can skin it pretty much how you please using CSS3 and by default it looks pretty nice too.
Only caveat is that it's not available for Linux yet at all - but if you're looking to develop an application that's going to be released in the next couple of years or so, or you're not too fussed about waiting for cross platform support, it's a good option to consider.
From you answer its not very clear what your problem is. A simple, but often sufficent solution is to change the look and feel for you application to the Systems default look an feel (so Swing looks just like a native application). All you need to do is insert the following code somewhere before any of your UI is created/shown (if you don't know where, place it as the first in your main-method):
try {
UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
} catch (Exception e) {
// exit application, log or ignore exception
}