Java SE contains the standard Java libraries: all the basics, containers, network, GUI, and so on. It's meant mostly for desktop apps and browser applets.
Java EE adds a server framework. When you talk about Java in the server, this is what you're referring to. It's a huge library, with lots and lots of pre-established 'best practices'. Back when JITs were still young, the heavily scalable design of Java EE made it very well-received in the server, besides the (then significant) overhead of JVM.
I haven't really used Java EE in anything serious; but I think it's better to learn Java SE first, and then Java EE. It takes more time; but lets you better understand what is the language and what is the framework.
