Skip to main content

Real Time JMeter Result Using Backend Listener

Since JMeter 2.13 Backend Listener has been available to create real time graph of JMeter Test. Following tutorial explain the entire process in detail. At the end of this tutorial you would be able to create JMeter Live Test Result dashboard similar to following -


Grafana   S2S Load Test.png


This tutorial borrows information from many sources and my own experiments with JMeter live reporting dashboard. I have added source of information wherever applicable


But before we can build such a snazzy JMeter Live Reporting dashboard we need to understand two more components -


influxDB (a time series database) and
Grafana Dashboard
This is a big tutorial, so take deep breath :-) and follow on.
Once you complete set up specified in this tutorial then you can watch JMeter Training Video Tutorial to watch this in action.


What is Time Series Database?


A time series is a sequence of data points, typically consisting of successive measurements made over a time interval. Examples of time series are ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.
And we are interested in load test which also generates time series data points.


source -


A time series database handles time series data, ex - influxDB


influxDB -


InfluxDB is a time series, metrics, and analytics database. It’s written in Go and has no external dependencies. That means once you install it there’s nothing else to manage (such as Redis, ZooKeeper, Cassandra, HBase, or anything else). InfluxDB is targeted at use cases for DevOps, metrics, sensor data, and real-time analytics.


source -


InfluxDB Installation Details


To access influx db from command line interface (CLI) and use influx query language - https://influxdb.com/docs/v0.9/introduction/getting_started.html


How is data stored in influxDB?


Data in InfluxDB is organized by time series, which contain a measured value, like “cpu_load” or “temperature”. Time series have zero to many points, one for each discrete sample of the metric. Points consist of time (a timestamp), a measurement(“cpu_load”), at least one key-value field (the measured value itself, e.g. “value=0.64” or “15min=0.78”), and zero to many key-value tags containing metadata (e.g. “host=server01”, “region=EMEA”, “dc=Frankfurt”). Conceptually you can think of a measurement as an SQL table, with rows where the primary index is always time. tags and fields are effectively columns in the table. tags are indexed, fields are not. The difference is that with InfluxDB you can have millions of measurements, you don’t have to define schemas up front, and null values aren’t stored.
source -


Ex:
Screenshot from 2015-12-29 14:28:14.png


This is how JMeter max active threads data is stored in influxDB. We will see later how JMeter metrics end up in influxDB


Authentication and Authorization in influxDB -


Create at least one admin user. See the authorization section for how to create an admin user - https://influxdb.com/docs/v0.9/administration/authentication_and_authorization.html

create a non admin user as -
Screenshot from 2015-12-09 16:28:22.png
ex - CREATE USER root WITH PASSWORD 'root’


We will use these credentials later when setting up data source in Grafana>

By default, authentication is disabled in the configuration file. Enable authentication by setting the auth-enabled option to true in the [http] section of the configuration file (/etc/opt/influxdb/influxdb.conf) -
[http]  
enabled = true  
bind-address = ":8086"  
auth-enabled = true
log-enabled = true  
write-tracing = false  
pprof-enabled = false  
https-enabled = false  
https-certificate = "/etc/ssl/influxdb.pem"
Restart the process -
sudo service influxdb start


or specifying the config file when starting influxDB -


cd /opt/influxdb$
./influxd -config /etc/opt/influxdb/influxdb.conf


Enable Graphite listener in influxDB -
Add Graphite data to influxDB config file - (/etc/opt/influxdb/influxdb.conf)
Search for section [[graphite]] and modify it as following -


 [[graphite]]
 enabled = true
 bind-address = ":2003"
 database = jmeter


now restart influxDB as -


./influxd -config /etc/opt/influxdb/influxdb.conf


This will create JMeter database in influxdb -


Screenshot from 2015-12-03 17:15:43.png


The InfluxDB HTTP API runs on port 8086 by default.


The built in user interface runs on port 8083 by default.
Access influx DB at - http://localhost:8083/# and select database jmeter from dopdown as described below -


Screenshot from 2015-12-03 16:52:30.png


JMeter measurements in influxDB -


At the time of writing this doc following JMeter metrics are posted to influxDB -


Screenshot from 2015-12-29 14:43:21.png
Screenshot from 2015-12-29 14:43:32.png
Source -


Notice that these measurements would be available only after adding BackenListener and running test at least once. Hence add Backend Listener to test plan as -


Screenshot from 2015-12-29 14:45:58.png


You can specify following properties in Backend Listener -
Screenshot from 2015-12-04 11:27:35.png
And run the test.


In all of the metric i.e. jmeter.all.a.count, jmeter.all.a.max etc word “jmeter” is rootMetricPrefix and is specified in Backend Listener


By default JMeter sends only metrics for all samplers using "all" as sampler Name so it looks like this in influxDB -


Screenshot from 2015-12-03 17:09:14.png


To query a measurement in influxDB you can run select query on required measurement as -


Screenshot from 2015-12-03 17:11:02.png
And to drop the database (you won’t do this very often ;-)) execute drop query as -
drop database <dbname>


When you need values for specific samples then specify sampler name in Backend listener and run test. (Multiple sample names as comma separated values) -


Screenshot from 2015-12-04 11:43:34.png




Grafana:


Grafana is a leading open source application for visualizing large-scale measurement data. It provides a powerful and elegant way to create, share, and explore data and dashboards from your disparate metric databases, either with your team or the world.
Grafana is most commonly used for Internet infrastructure and application analytics, but many use it in other domains including industrial sensors, home automation, weather, and process control.
Grafana features pluggable panels and data sources allowing easy extensibility. There is currently rich support for Graphite, InfluxDB and OpenTSDB.


source -


Grafana Installation -


Refer following doc for grafana installation - http://docs.grafana.org/installation/


Once done, then you can start grafana-server by executing
$ sudo service grafana-server start


This will start the grafana-server process as the grafana user, which was created during the package installation. The default HTTP port is 3000 and default user and password is admin.
Launch http://localhost:3000/ and login as admin/admin


Grafana config file is located under following directory -
/usr/share/grafana/conf
cat defaults.ini


Grafana Date Source -


Grafana supports many storage backend for time series data. Query Language and capabilities of each data source are different. Influxdb is one such supported data source. You can read more on Grafana Data Source here - http://docs.grafana.org/datasources/overview/


Adding a data source -  
Login as admin and navigate to http://localhost:3000/datasources/new
Select Type as influxdb and specify other values -



Screenshot from 2015-12-04 13:14:10.png
Screenshot from 2015-12-04 12:47:29.png
add user and password which you have set for database (root and root)




Creating Grafana dashboard -


mvn test > will generate GenerateDashboard.jar
Follow instructions at ReadME file and generate Grafana dashboard.


Now you can import dashboard json file to Grafana -
Screenshot from 2015-12-04 14:26:02.png


Screenshot from 2015-12-04 14:26:35.png


You can view json of each panel by clicking on head -
Screenshot from 2015-12-04 14:32:05.png


And now click on “Panel Json” -


Screenshot from 2015-12-04 14:33:06.png


You can modify existing row by clicking on green symbol on LHS of each row
You can also a new row using “Add Row” button -

Screenshot from 2015-12-04 14:38:22.png


Or add a new graph to existing row using Add Panel > Graph from green bar on LHS -
Screenshot from 2015-12-04 14:39:33.png


Or edit an existing graph > Click top header of a graph and click Edit on panel -


Screenshot from 2015-12-04 14:43:53.png


Screenshot from 2015-12-04 15:07:27.png


You can switch to Metrics mode which might be easy for writing queries -


Screenshot from 2015-12-10 09:49:31.png


Make sure that you have data source influxDB selected as -


Screenshot from 2015-12-29 15:05:48.png


And if you DataSource connection is really successful ;-) then you would see available measurements in FROM dropdown as following -
Screenshot from 2015-12-10 09:47:43.png
Metrics define series data and sources to render. Each data source provides different Metrics option and we are using influxDB here.


One example query to get last value of max active threads -



Screenshot from 2015-12-10 17:12:04.png


Along with graph, you can also add Single stat display on a row -


Screenshot from 2015-12-14 15:29:48.png


You can set height of Single stat using <Set height> option


Let’s consider an example - To display throughput for a successful request, you could query sum of <samepleName>.ok.count and group it against unit of time for ex Sec as following -


Screenshot from 2015-12-14 13:18:23.png


Now select <Value> as <current> in Options tab and you would display current value of req/sec -
Screenshot from 2015-12-14 15:07:23.png


You can also choose another value to display (Min, Max, Avg), Postfix, Spark lines, Coloring etc as -


Screenshot from 2015-12-14 15:01:04.png


For ex, I set up following values to display current active threads -


Screenshot from 2015-12-14 15:07:23.png


If you want to superimpose one graph on another then you can click the +Query button and this would let to combine multiple queries -


Screenshot from 2015-12-11 12:01:55.png


More info on Metric options - http://docs.grafana.org/datasources/influxdb/


You can modify the settings of dashboard using “Settings” button -


Screenshot from 2015-12-04 15:18:41.png


You can modify the data collection period from top section -


Screenshot from 2015-12-10 10:18:22.png


And then you can also set the refresh rate -


Screenshot from 2015-12-10 10:18:36.png


To see the value on graph when hovering over it; select Tooltip under Display Styles -


Screenshot from 2015-12-10 14:21:52.png


And now you would see values on graph when hovering on it -


Screenshot from 2015-12-10 14:31:05.png


If you want current value of metric to appear the select “Current” under “Axes & Grid” -


Screenshot from 2015-12-11 11:10:29.png


And then you would see Current value for graph -


Screenshot from 2015-12-11 11:15:16.png


You can set header of row using Row Editor -


Screenshot from 2015-12-14 10:53:25.png


Screenshot from 2015-12-14 10:55:13.png


And now you would see row title on dashboard -


Screenshot from 2015-12-14 10:56:36.png


And in the end one of my load test dashboard looked as -



Grafana   S2S Load Test.png


More detail on graph here - http://docs.grafana.org/reference/graph/

Did you succeed creating your own graph? Do share your experience in comment section below -

Popular posts from this blog

Appium and android mobile app automation

Next appium and Android mobile app automation video tutoria l is live. If you are new to appium then please check - appium-tutorial This video tutorial covers - Start vysor (Just for this session and not mobile automation :)) Start appium and start appium inspector Desired Capabilities platformName - Android deviceName - L2N0219828001013 (as seen on "adb devices") Saved Capability Sets Start Session Scan app elements using appium inspector Get appPackage and appActivity using "APK info" app Install "APK info" app and open app whose appPackage and appActivity are required i.e. calculator Check top section of app icon com.android.calculator2 is app package com.android.calculator2.Calculator is app activity testng.xml file settings for running Android app tests Test details com.seleniumtests.tests.mobile.AndroidAppTest and CalculatorScreen class View beautiful STF test report  

Return only first or last element from webelements collection

Note: If you are new to java and selenium then start with selenium java training videos .     We often come across situation when there are multiple elements on a page and we probably like to exercise only a few of them using selenium webdriver. May be just first and last element. For example on a search result page we may like to click on only first and last link and not all. This is when Iterables API comes handy. (By the way I am assuming that you have already completed watching selenium training videos :)). Once we have collection of web element then we can use Iterables to get only first or last element as following - Consider that we fetch collection of element as - List< WebElement > webElements = getDriver().findElements(By. id ( "htmlID" ));   Now we can get the first web element from this collection as -  WebElement firstElement = Iterables. getFirst (webElements,  getDriver().findElement(By. id ( "defaultElement" )));   Herein second

Using chrome console to test xPath and css selectors

Note: If you are new to java and selenium then start with selenium java training videos .       Since the advent of selenium there have been many plugin to test xPath / css selectors but you don’t need any of them if you have chrome browser. Using Chrome console you can test both xPath and css selectors. Launch website to be tested in chrome browser and hit F-12 and you would see chrome console opened in lower pane of application - Hit escape key and console would open another pane to write element locators - And now you can start writing xPath or css selectors in chrome console and test them - The syntax for writing css id - $$(“ ”) And hit the enter key. If your expression is right then html snippet of the application element corresponding to the css selector would be displayed - If you mouse over the html snippet in chrome console then it would highlight the corresponding element in application - If you want to clean console of previously wri