I have been reading about about Restful webservices being Stateless. I can also see that most of Soap based webservices are also stateless and can be made stateful if needed
Web service, whether it is Rest or SOAP, is by default, designed stateless. But there can be use cases where it is required to be stateful. Web Services Resource Framework (WSRF) provided by OASIS, can be used to make a SOAP web service, stateful. Add a ResourceProperties attribute to the endpoint and add a ResourceProperty operation to the WSDL and pass it across web services.
Rest engages in state transfer and to make them stateful, we can use client side or db persisted session state, and transfer them across web service invocations as an attribute in either the header or a method parameter.
The statelessness of REST is designed to ensure applications scale well. You can add state, but it's at a trade-off in scalability.
One of the most common reasons to add state to REST is for authentication. Once a secure connection is established, a secure cookie can be sent to the client. This cookie is then added by the client to all requests for the session. The server maintains state and then loads that state in with every request based on the cookie.
Consider a simple web page. If you are not maintaining state, you can put up a reverse proxy, cache the page in memory by URL, and distribute that resource across many servers for load. If you now add the name of the currently logged in user to that web page, you can no longer cache anything (at least at the most basic HTTP level). The response can now be cached only with a combination of the authentication cookie and the URL.
As long as I know, we(.net developers) can use WS-binding in wcf for stateful Web services.