Change the thread count of test plan in JMeter, at run time

Question

I want to change the number of threads for a JMeter test plan at runtime.

I have Googled my problem and found a proposed solution to use JMeter plugins. But in this

Answer 1

you can change it based on a variable which you set in a startup thread. See below.

In Jmeter how do I set a variable number of threads using a beanshell sampler variable?

However once the thread group has started you can not modify it. To the guy who said this feature would not be useful I disagree. There are many types of load tests and they do not all have the same number of users running for the duration. Here are just 2 example types of enterprise load tests we conduct at the bank where I work:

• duration test - same number of users run the entire time (possibly with a short ramp-up period)
• Break point test - ramp up the number of users incrementally till the application breaks
• Spike test - run with a constant number of users but sporadically
  throw in a large number of users

A break point test ramps up the number of users until the application breaks (the point being to see how high your app can scale). You can sort of do this using the thread groups "ramp up period" property. If you set ramp up time to 1000 and the number of threads to 100 it will add 1 thread every 10 seconds.

Spike tests are like duration tests but at some intervals a large number of users log in. This is used to guage the applications response time during peak hours or how it will respond if you all of a sudden get a large number of users (a very real scenario).

I find that Jmeter does not handle all load test scenarios that are needed in enterprise load testing. One work around Im considering is to just start all the threads but find a way to make some of them sleep. So you could set the number of threads to 1000 but somehow make 980 of them sleep or do nothing. Then maybe when the time_in_seconds%5==0 (every 5 minutes) you allow the other threads to run - simulating a spike test. The idea is you can hard code the threads to 1000 and will always have 1000 threads running - but they don't all have to be doing something at all times.

(in other words you can probably find a way but you have to get creative)

Update: I just found this plugin which allows different types of testing. Have not tried it yet but looks promising: http://jmeter-plugins.org/wiki/ThroughputShapingTimer/

Answer 2

IMHO that's just a fancy feature that has no real benefit when doing proper performance testing. In order to generate relevant test output (report), you need repeatability, and clearly defined test methodology and scenarios. In order to compare impact of any application/server/infrastructure changes, you need repeatability.

What do you mean by

We can't predict the user of our site

That's why we do performance testing at the first place. To find out what is our application/infrastructure limit.
I.e. the most significant metric you can produce is how your application response time changes when number of parallel users change. But not change erratically, in run time.

With jMeter plugins' Ultimate thread group you can cover any imaginable scenario.

Answer 3

This feature is indeed useful, and surprisingly difficult to implement even with commercial tools such as Loadrunner. I would compare it to finding a loudspeakers maximum volume. You would manually turn the volume up until it started to crackle, then turn it back down slightly to maintain that maximum volume. In the same way, to find the peak capacity of an application, you want control to 'turn the volume up' until errors are seen, then back it down slightly to see if it stabilizes. You can then maintain that load to find where the bottleneck is.

Anyway, to answer the question, what I have done in the past is use an external influence, such as a file name or similar. Then combine that with the thread unique reference you can control which threads run and which are held (by pausing or similar).

For example, if you start with 100 threads, then create a file called '5.txt' in a specific location, you can add code such that if the threads sees that it's own reference is equal to or lower than the number then it can run. If not then it drops into a pause. At the start of this example 5 threads would run, and 95 would pause. You can then rename the file to '25.txt', and threads 6 to 25 would start running. It would work the other way too, changing it to '20.txt' would mean threads 21-25 pause again.

The key is to start enough threads to exceed your expected peak.

Answer 4

You can set/change the number of threads at runtime using command line option...

you can use function calls, or variable references to User Parameters (which in turn could be functions), or variable references to variables set up by functions earlier in the test. There's more than one way to do it.

Suppose you want to be able to vary the number of threads in a test plan. Choose a suitable property name, say group1.threads. Replace the thread count in the GUI (or the JMX, if you're feeling brave!) with the following function call:

Please set below property in JMeter thread group as below ${__property(group1.threads)}

Then, when starting JMeter, define the property on the command line:

jmeter -Jgroup1.threads=10

Answer 5

By enabling the BeanShell server you can vary properties at runtime. Just enable it and telnet on port 9001 (warning: not secure!) Based on a test I did, unfortnately, it appears that the thread count it's not applied at runtime. However you can still manipulate the load of the test by other means, for example apply a costant throughput timer parametrized with a property named "throughput" and vary it at runtime like this:

setprop("throughput","2000");

It's well explained in the guide.

Answer 6

We can't predict the user of our site.

Sure you can. This is what the HTTP logs of your existing site are for. You can also use logs from tools like Omniture or your CDN logs. If you look at the combination of Actual user IP address, request and referer tags in the logs you will be able to build a traversal map of ever single user on your site. You will be able to profile the high hit unique leaf node pages of a given business process to understand how many times a particular business process happens an hour. You will be able to examine abandonment by taking a look at the funnel in tools such as Omniture. If you need tools for this analysis I recommend Splunk. It's easy to install and configure. Time to value is very fast.

The more log data you have which you are using to profile the closer you can come to actual for what users do during a day/week/month/spot sale/end of quarter/end of year/etc....You need to combine actual at a point in time with actual from earlier points in time to project growth over time since you will need to allow for growth in your performance testing model.

If you don't get the values right then the value of your test as a predictor of what will/can happen in production will be quite low. This is not a failure of any given tool, but a failure in process on the planning front for the actual load model used as part of the test requirements. If you cannot build these models then you need to pull someone into your team who can.

This ability to produce a valid load model independent of tool is the difference between tests which reduce risk and throwing load.