Skip to main content

Should you be using Distributed Testing for Performance Testing ?

Having posted elaborately on how to carry out JMeter distributed and using Jmeter Backend listener to plot live graph of running test. I have been disappointed with entire approach. And my disappointment is not limited to just JMeter as any other performance testing tool would suffer from similar factors.
Distributed test itself results in lower throughput than what you can achieve from test running from independent machine. This is described on http://www.http503.com/2012/jmeter-ec2/ (unfortunately web site is down for quite some time). I am pasting the reasoning here -

  1. If you want to use your local machine to control the test you have to navigate the joyous process of getting RMI to work over multiple subnets and firewalls to allow your local machine to control multiple remote (EC2) slaves – you can do this by tunnelling RMI communication and patching JMeter, it’s messy though. To workaround this issue you have to use a remote Master as well as remote Slaves.
  2. Even then, in master/slave mode when running high throughput tests JMeter will eventually reach an IO or network bottleneck that will affect the results (even using Batch mode). Too many processes trying to write to a single file at the same time inevitably start to queue. To be certain of avoiding this issue you have to not use Distributed mode and instead run multiple independent tests over n hosts that do not use the GUI and do not write results to a single master but instead run at the command line and keep results local. Then, after the test is complete, you have to collate everything. This approach has several annoying problems: a) excessive terminal windows, b) limited visibility on test progress as it happens, c) to preserve throughput the jmx file must be adjusted for the number of hosts in use and d) too much time is spent on repetitive tasks that could be automated.
I myself encountered weird issue of sudden drop in application throughput when using JMeter Distributed test on aws. But when I executed test independently on various machines then the issue never surfaced.

Not just this, using backend listener on distributed mode causes more drop in application throughput than what you can achieve with just distributed mode.
Given these reasons I decided to gave up on using distributed mode and backend listener entirely and began to use JMeter-EC2 project. Since the original project is not compatible with latest JMeter version I ended up forking it and using my own version. You could try it and it is compatible with JMeter 2.13

Now this leaves us on how we can yet use snazy grafana dashboard. The way to approach this would be to directly push results to influx db either from summary report or from jtl files. Something have not yet explored, have you?
Labels:

Popular posts from this blog

Verify email confirmation using Selenium WebDriver

Note: If you are new to java and selenium then start with selenium java training videos .   How to Verify Email Confirmation Using Selenium 4 and JavaMail (2026 Guide) Email confirmation is a critical part of most registration flows — account activation, password reset, multi-factor authentication, and onboarding. Every automation engineer eventually faces the same challenge: How do you verify an email confirmation link inside a Selenium test without making it slow and flaky? The wrong instinct is to automate Gmail's UI with Selenium. It's fragile, slow, and breaks constantly. The right approach: Use Selenium for browser automation Use JavaMail (IMAP) to read the email directly Extract the confirmation link Continue the test in Selenium Why Not Automate Gmail UI With Selenium? Automating the Gmail UI means logging in, searching, clicking a message, and parsing content from a third-party interface that changes frequently. This leads to: Flaky...
13 comments
Read more

Using xPath to reach parent of an element

Note: If you are new to java and selenium then start with selenium java training videos .   I prefer css locator over xPath but there are times when css locators don't fit requirement. One such requirement is when you want to navigate to parent element of an element and may be parent of parent and even more. Unfortunately css locators don't provide any mechanism to navigate to parent of an element. See this for more. Of late I came across a scenario when I wanted to click on a link depending upon the text in a text box. Herein parent of text box and parent of link were at the same location. More over there could have been many such combinations in application. Fortunately I just need to pick first such instance and Web Driver any way considers only first instance when multiple locators are found matching an element. Element in question is in following html - Here I need to click on highlighted anchor on the basis of input element (which is also hig...
Read more

Real Time JMeter Result Using Backend Listener

Since JMeter 2.13 Backend Listener has been available to create real time graph of JMeter Test. Following tutorial explain the entire process in detail. At the end of this tutorial you would be able to create JMeter Live Test Result dashboard similar to following - This tutorial borrows information from many sources and my own experiments with JMeter live reporting dashboard. I have added source of information wherever applicable But before we can build such a snazzy JMeter Live Reporting dashboard we need to understand two more components - influxDB (a time series database) and Grafana Dashboard This is a big tutorial, so take deep breath :-) and follow on. Once you complete set up specified in this tutorial then you can watch JMeter Training Video Tutorial to watch this in action. What is Time Series Database? A time series is a sequence of data points , typically consisting of successive measurements made over a time interval . Examples of time ...
Read more