Skip to main content

Real Time JMeter Result Using Backend Listener

Since JMeter 2.13 Backend Listener has been available to create real time graph of JMeter Test. Following tutorial explain the entire process in detail. At the end of this tutorial you would be able to create JMeter Live Test Result dashboard similar to following -

Grafana   S2S Load Test.png

This tutorial borrows information from many sources and my own experiments with JMeter live reporting dashboard. I have added source of information wherever applicable

But before we can build such a snazzy JMeter Live Reporting dashboard we need to understand two more components -

influxDB (a time series database) and
Grafana Dashboard
This is a big tutorial, so take deep breath :-) and follow on.
Once you complete set up specified in this tutorial then you can watch JMeter Training Video Tutorial to watch this in action.

What is Time Series Database?

A time series is a sequence of data points, typically consisting of successive measurements made over a time interval. Examples of time series are ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
And we are interested in load test which also generates time series data points.

source -

A time series database handles time series data, ex - influxDB

influxDB -

InfluxDB is a time series, metrics, and analytics database. It’s written in Go and has no external dependencies. That means once you install it there’s nothing else to manage (such as Redis, ZooKeeper, Cassandra, HBase, or anything else). InfluxDB is targeted at use cases for DevOps, metrics, sensor data, and real-time analytics.

source -

InfluxDB Installation Details

To access influx db from command line interface (CLI) and use influx query language -

How is data stored in influxDB?

Data in InfluxDB is organized by time series, which contain a measured value, like “cpu_load” or “temperature”. Time series have zero to many points, one for each discrete sample of the metric. Points consist of time (a timestamp), a measurement(“cpu_load”), at least one key-value field (the measured value itself, e.g. “value=0.64” or “15min=0.78”), and zero to many key-value tags containing metadata (e.g. “host=server01”, “region=EMEA”, “dc=Frankfurt”). Conceptually you can think of a measurement as an SQL table, with rows where the primary index is always time. tags and fields are effectively columns in the table. tags are indexed, fields are not. The difference is that with InfluxDB you can have millions of measurements, you don’t have to define schemas up front, and null values aren’t stored.
source -

Screenshot from 2015-12-29 14:28:14.png

This is how JMeter max active threads data is stored in influxDB. We will see later how JMeter metrics end up in influxDB

Authentication and Authorization in influxDB -

Create at least one admin user. See the authorization section for how to create an admin user -

create a non admin user as -
Screenshot from 2015-12-09 16:28:22.png

We will use these credentials later when setting up data source in Grafana>

By default, authentication is disabled in the configuration file. Enable authentication by setting the auth-enabled option to true in the [http] section of the configuration file (/etc/opt/influxdb/influxdb.conf) -
enabled = true  
bind-address = ":8086"  
auth-enabled = true
log-enabled = true  
write-tracing = false  
pprof-enabled = false  
https-enabled = false  
https-certificate = "/etc/ssl/influxdb.pem"
Restart the process -
sudo service influxdb start

or specifying the config file when starting influxDB -

cd /opt/influxdb$
./influxd -config /etc/opt/influxdb/influxdb.conf

Enable Graphite listener in influxDB -
Add Graphite data to influxDB config file - (/etc/opt/influxdb/influxdb.conf)
Search for section [[graphite]] and modify it as following -

 enabled = true
 bind-address = ":2003"
 database = jmeter

now restart influxDB as -

./influxd -config /etc/opt/influxdb/influxdb.conf

This will create JMeter database in influxdb -

Screenshot from 2015-12-03 17:15:43.png

The InfluxDB HTTP API runs on port 8086 by default.

The built in user interface runs on port 8083 by default.
Access influx DB at - http://localhost:8083/# and select database jmeter from dopdown as described below -

Screenshot from 2015-12-03 16:52:30.png

JMeter measurements in influxDB -

At the time of writing this doc following JMeter metrics are posted to influxDB -

Screenshot from 2015-12-29 14:43:21.png
Screenshot from 2015-12-29 14:43:32.png
Source -

Notice that these measurements would be available only after adding BackenListener and running test at least once. Hence add Backend Listener to test plan as -

Screenshot from 2015-12-29 14:45:58.png

You can specify following properties in Backend Listener -
Screenshot from 2015-12-04 11:27:35.png
And run the test.

In all of the metric i.e. jmeter.all.a.count, jmeter.all.a.max etc word “jmeter” is rootMetricPrefix and is specified in Backend Listener

By default JMeter sends only metrics for all samplers using "all" as sampler Name so it looks like this in influxDB -

Screenshot from 2015-12-03 17:09:14.png

To query a measurement in influxDB you can run select query on required measurement as -

Screenshot from 2015-12-03 17:11:02.png
And to drop the database (you won’t do this very often ;-)) execute drop query as -
drop database <dbname>

When you need values for specific samples then specify sampler name in Backend listener and run test. (Multiple sample names as comma separated values) -

Screenshot from 2015-12-04 11:43:34.png


Grafana is a leading open source application for visualizing large-scale measurement data. It provides a powerful and elegant way to create, share, and explore data and dashboards from your disparate metric databases, either with your team or the world.
Grafana is most commonly used for Internet infrastructure and application analytics, but many use it in other domains including industrial sensors, home automation, weather, and process control.
Grafana features pluggable panels and data sources allowing easy extensibility. There is currently rich support for Graphite, InfluxDB and OpenTSDB.

source -

Grafana Installation -

Refer following doc for grafana installation -

Once done, then you can start grafana-server by executing
$ sudo service grafana-server start

This will start the grafana-server process as the grafana user, which was created during the package installation. The default HTTP port is 3000 and default user and password is admin.
Launch http://localhost:3000/ and login as admin/admin

Grafana config file is located under following directory -
cat defaults.ini

Grafana Date Source -

Grafana supports many storage backend for time series data. Query Language and capabilities of each data source are different. Influxdb is one such supported data source. You can read more on Grafana Data Source here -

Adding a data source -  
Login as admin and navigate to http://localhost:3000/datasources/new
Select Type as influxdb and specify other values -

Screenshot from 2015-12-04 13:14:10.png
Screenshot from 2015-12-04 12:47:29.png
add user and password which you have set for database (root and root)

Creating Grafana dashboard -

mvn test > will generate GenerateDashboard.jar
Follow instructions at ReadME file and generate Grafana dashboard.

Now you can import dashboard json file to Grafana -
Screenshot from 2015-12-04 14:26:02.png

Screenshot from 2015-12-04 14:26:35.png

You can view json of each panel by clicking on head -
Screenshot from 2015-12-04 14:32:05.png

And now click on “Panel Json” -

Screenshot from 2015-12-04 14:33:06.png

You can modify existing row by clicking on green symbol on LHS of each row
You can also a new row using “Add Row” button -

Screenshot from 2015-12-04 14:38:22.png

Or add a new graph to existing row using Add Panel > Graph from green bar on LHS -
Screenshot from 2015-12-04 14:39:33.png

Or edit an existing graph > Click top header of a graph and click Edit on panel -

Screenshot from 2015-12-04 14:43:53.png

Screenshot from 2015-12-04 15:07:27.png

You can switch to Metrics mode which might be easy for writing queries -

Screenshot from 2015-12-10 09:49:31.png

Make sure that you have data source influxDB selected as -

Screenshot from 2015-12-29 15:05:48.png

And if you DataSource connection is really successful ;-) then you would see available measurements in FROM dropdown as following -
Screenshot from 2015-12-10 09:47:43.png
Metrics define series data and sources to render. Each data source provides different Metrics option and we are using influxDB here.

One example query to get last value of max active threads -

Screenshot from 2015-12-10 17:12:04.png

Along with graph, you can also add Single stat display on a row -

Screenshot from 2015-12-14 15:29:48.png

You can set height of Single stat using <Set height> option

Let’s consider an example - To display throughput for a successful request, you could query sum of <samepleName>.ok.count and group it against unit of time for ex Sec as following -

Screenshot from 2015-12-14 13:18:23.png

Now select <Value> as <current> in Options tab and you would display current value of req/sec -
Screenshot from 2015-12-14 15:07:23.png

You can also choose another value to display (Min, Max, Avg), Postfix, Spark lines, Coloring etc as -

Screenshot from 2015-12-14 15:01:04.png

For ex, I set up following values to display current active threads -

Screenshot from 2015-12-14 15:07:23.png

If you want to superimpose one graph on another then you can click the +Query button and this would let to combine multiple queries -

Screenshot from 2015-12-11 12:01:55.png

More info on Metric options -

You can modify the settings of dashboard using “Settings” button -

Screenshot from 2015-12-04 15:18:41.png

You can modify the data collection period from top section -

Screenshot from 2015-12-10 10:18:22.png

And then you can also set the refresh rate -

Screenshot from 2015-12-10 10:18:36.png

To see the value on graph when hovering over it; select Tooltip under Display Styles -

Screenshot from 2015-12-10 14:21:52.png

And now you would see values on graph when hovering on it -

Screenshot from 2015-12-10 14:31:05.png

If you want current value of metric to appear the select “Current” under “Axes & Grid” -

Screenshot from 2015-12-11 11:10:29.png

And then you would see Current value for graph -

Screenshot from 2015-12-11 11:15:16.png

You can set header of row using Row Editor -

Screenshot from 2015-12-14 10:53:25.png

Screenshot from 2015-12-14 10:55:13.png

And now you would see row title on dashboard -

Screenshot from 2015-12-14 10:56:36.png

And in the end one of my load test dashboard looked as -

Grafana   S2S Load Test.png

More detail on graph here -

Did you succeed creating your own graph? Do share your experience in comment section below -

Popular posts from this blog

Where is my defect ID?

Don't you feel ecstatic when your automated tests find bug? After all tests finding bugs give us a sense of accomplishment, is not it? And this is followed by usual cycle of defect reporting, retesting and hopefully closure of defect. But at times defects are deferred to next or future releases. Which causes test method to fail for subsequent releases. And if you are dealing with a test suite having 100s of tests then it may become difficult to remember if there was a defect reported for a failing test? How do you deal with such situation. How about adding defect-id to @description tag of TestNG test. Hence it is reported on automated test report and we would know if defect exists for a failing test - How do you track defect-id of a failing test?

Return only first or last element from webelements collection

We often come across situation when there are multiple elements on a page and we probably like to exercise only a few of them using selenium webdriver. May be just first and last element. For example on a search result page we may like to click on only first and last link and not all. This is when Iterables API comes handy. (By the way I am assuming that you have already completed watching selenium training videos :)). Once we have collection of web element then we can use Iterables to get only first or last element as following - Consider that we fetch collection of element as - List< WebElement > webElements = getDriver().findElements(By. id ( "htmlID" ));   Now we can get the first web element from this collection as -  WebElement firstElement = Iterables. getFirst (webElements,  getDriver().findElement(By. id ( "defaultElement" )));   Herein second argument -   (getDriver().findElement(By. id ( "defaultElement" )))    in the me

Selenium Tutorial: Pattern Mathing using Selenium

I must confess I have never been admirer of Regular Expression but then there are times you can not escape from it, especially while working on a website which has dynamic contents appeared in static text and you want to validate it. like - "Validate that this text appears and there is 123 here and 456 here" And the test condition is 123 and 456 could be any three digits but number if digits should not be more than three. In a crude way we can at least test this - Assert.assertTrue(selenium.getText("elementLocator").contains("Validate that this text appears and there is")); but what if text goes wrong after "and there is"... what if more than 3 digits appear in text. This is where pattern matching/regular expression comes for our rescue and we can use matches method of String class to achieve same. So the assertion would be - String text = selenium.getText("elementLocator"); Assert.assertTrue(text.matches("Validate