what is the difference between socket programming, rmi and Servlets. When to use what?
Use sockets to implement a specific TCP/IP protocol, whether an existing standard or your own custom protocol. You have complete control over all aspects of network communication.
Servlets support request/reply semantics in the general sense, but it far more likely you will be using HTTPServlets which support, as expected, the HTTP request/reply semantics. For example, a web-server, or a RESTful HTTP based endpoint.
Use RMI for distributed Java Objects. RMI is itself implemented using Sockets (see above) and implements the Java Wire Protocol.
The Socket APIs are the low-level (transport level) abstraction by which a Java application interacts with the network, and by extension with remote clients and services. Socket and related APIs support reliable byte stream and unreliable messaging services. They are typically used for TCP/IP and UDP/IP, though other networking protocol stacks can (at least in theory) be supported.
RMI is a framework and protocol family for implementing application-level networking between Java applications. It models network interactions as Java method calls made against objects that live in other applications. This model requires a mechanism (typically a name server) that allows one application to "publish" objects so that another application can refer to them. This (and the fact that RMI ports are typically blocked by default) means that there is a non-trivial amount of configuration effort in setting up RMI-based applications.
Servlets are a collection of APIs that are primarily designed for implementing the server side of HTTP communications; i.e. for building webservers in Java. They (or more accurately the web container in which they run) take care of the details of the HTTP protocol, so that the programmer (in theory) only needs to deal with "application" concerns.
In practice, the servlet developer and/or deployer has to deal with other things such as mapping URLs to servlets to objects, security and authentication. In addition, Servlets only deal with the server side of an HTTP interaction ... the client side must be handled by different APIs. (You could also argue that Servlets by themselves do not do enough, as evidenced by the proliferation of web application frameworks that are built on top of Servlets.)
In brief:
Sockets -- Few simple calls which directly interface with TCP/IP. Very simple but you to implment your own buffer handling and deal with incomplete responses and timeouts in yourself. No authentication or security provided.
rmi -- handles all of the above, <personal opinion>its one of the worse APIs to have contaminated the java standards </personal opinion>, fairly simple to program, handles basic network errors, authentication and security issues. Difficult to configure and deploy.
Servlets -- lovely simple API, all network issues handled for you, security and authentication via plugins. No deployment issues, simple configuration.