I tried a few things thanks to your suggestions, and here is my observation :
If you are using Java 6, and not Java 7, you are f***ed.
If you are using Java 7, you should do as BalusC tells : change the setting in Region and Language Settings. After that, Locale.getDefault() will by default return the display language, meaning English if you set it to English. To make sure of it, you can create you locale by calling Locale.getDefault(Locale.Category.DISPLAY)