I installed Jenkins on my vserver. When I had a look at htop Jenkins was running with 30 thread, each was allowed to allocate 247MB memory and up to 1181MB virtual memory.
You can also cut down the number of executors you have in your node. This may or may not help - it may be that Jenkins is smart enough to kill the thread when it isn't using an executor. Still, some more information would be useful: How many jobs do you have? What plugins are installed? With more details I could give better advice.
http://winstone.sourceforge.net/#commandLine is the official command-line reference - but as I mention in the comment --handlerCountMax
(or --handlerCountStartup
, for that matter) did not seem to work for me. Try it yourself (here's how to run Jenkins in stand-alone mode).
You may want to try to reduce the number of executors as well as disabling plugins you do not need and see what happens.
Please keep in mind, however, that if you plan to continue using Jenkins seriously, you should plan for more resources, not less: as the number of your jobs grows, so will the resource utilization.