Please post your Web Driver questions in official Web Driver forum

Thursday, July 23, 2015

Performance Measurement commands for Unix

If you have been following my ongoing video tutorials on jmeter performance testing then you may have wondered what server side statistics should be gathered during performance testing. We have not yet looked in to specific statistics which we should gather for server side monitoring. In this post we would have a look at useful linux commands which can be used to gather several important performance statistics. We would use these commands in future performance testing tutorials to gather server side performance statistics. Most of the description of these commands is from command "man"ual. So lets begin -  

netstat command and options

netstat (network statistics) - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

Type the following command:
$netstat
Listening on all ports
    $netstat -a
OPTIONS
-l, --listening
      Show only listening sockets.  (These are omitted by default.)
    $netstat -i

tcpdump command and options

tcpdump - dump traffic on a network. Tcpdump  prints  out a description of the contents of packets on a network interface that match the boolean expression.

Type the following command:
$tcpdump

tcpdump.png

You need to interrupt the command else it would keep scrolling


OPTIONS
-i interface
--interface=interface Listen on interface.  If unspecified, tcpdump searches the system interface list for the lowest numbered,  configured  up  interface (excluding loopback), which may turn out to be, for example, ``eth0''.
    $tcpdump -i eth0
-c count
             Exit after receiving count packets.
$tcpdump -c 30 -i eth0
        Screenshot from 2015-07-23 10:39:04.png
-D
--list-interfaces, prints  the list of the network interfaces available on the system and on which tcpdump can capture packets.  For each network interface, a number and an interface name, possibly followed by a text description of the interface, is printed.  The interface  name  or the number can be supplied to the -i flag to specify an interface on which to capture.
    $tcpdump -D
tcpdump.png
-w file
Write the raw packets to file rather than parsing and printing them out.  They can later be printed with the -r option.
    $tcpdump -w dump01.pcap -c 10 -i eth0
tcpdump.png
and now read the dumped file -
    $tcpdump -r dump01.pcap
tcpdump.png

-n Don't convert addresses (i.e., host addresses, port numbers, etc.) to names.   
    $tcpdump -n

To capture packets from specific port, specify port number -
    $tcpdump -i eth0 port 22
To capture packets from specific source -
    $tcpdump -i eth0 src 192.168.1.1

lsof command and options

lsof - list open files. An  open file may be a regular file, a directory, a block special file, a character special file, an executing text reference, a library, a stream or a network file (Internet socket, NFS file or UNIX domain socket.)  A specific file or all the files in a file system may be selected by path. Given that this command identifies which files are being used, this information can be used when disk can not be unmounted due to files being used or opened.
Type the following command:
$lsof
    lsof.png
Herein -
FD – stands for File descriptor and may have some of the values as:
  1. cwd current working directory
  2. rtd root directory
  3. txt program text (code and data)
  4. mem memory-mapped file
Also in FD column numbers like 41u is actual file descriptor and followed by u,r,w of it’s mode as:
  1. r for read access.
  2. w for write access.
  3. u for read and write access.
TYPE – of files and it’s identification.
  1. DIR – Directory
  2. REG – Regular file
  3. CHR – Character special file.
  4. FIFO – First In First Out
  5. IPv4
  6. IPv6
OPTIONS
-u s selects the listing of files for the user whose login names or user ID numbers are in the comma-separated set s -  e.g.,  ``abe'', or ``548,root''.  (There should be no spaces in the set.)
    $lsof -u tarun
-i [i] selects the listing of files any of whose Internet address matches the address specified in i. If no address is specified, this option selects the listing of all Internet and x.25 (HP-UX) network files. Following command lists all network connections listening and established -
$lsof -i
Screenshot from 2015-07-22 14:35:31.png
An Internet address is specified in the form (Items in square brackets are optional.):
[46][protocol][@hostname|hostaddr][:service|port]
    $lsof -i TCP:80
Screenshot from 2015-07-22 14:13:46.png
List running processes of open files of TCP port range from 1-800
    $lsof -i TCP:1-800
Screenshot from 2015-07-22 14:25:27.png
If -i4 or -i6 is specified with no following address, only files of the indicated IP version, IPv4 or IPv6,  are  displayed.
    $lsof -i4
    Screenshot from 2015-07-22 14:22:02.png
    $lsof -i6
Screenshot from 2015-07-22 14:22:30.png

-p s excludes or selects the listing of files for the processes  whose  optional  process  IDentification  (PID)  numbers  are  in  the comma-separated set s - e.g., ``123'' or ``123,^456''.  (There should be no spaces in the set.) PID numbers that begin with `^' (negation) represent exclusions.

        $lsof -p 2499

-t   specifies  that  lsof  should  produce  terse output with process identifiers only and no header - e.g., so that the output may be piped to kill(1)

        $lsof -p 2499

        $kill -9 `lspf -t -u tarun`

iostat command and options

iostat - Report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions. It is used for monitoring system input/output device loading by observing the time the devices are active in relation to their average transfer rates.The iostat command generates reports that can be used to change system configuration to better balance the input/output load between physical disks.
The first report generated by the iostat command provides statistics concerning the time since the system was booted, unless the -y  option is used (in this case, this first report is omitted).  Each subsequent report covers the time since the previous report.
Type the following command:
$iostat
Screenshot from 2015-07-22 11:13:36.png
REPORTS
The iostat command generates two types of reports, the CPU Utilization report and the Device Utilization report.
CPU Utilization Report - The first report generated by the iostat command is the CPU Utilization Report. For multiprocessor systems, the CPU values are global averages among all processors. The report has the following format:
%user shows the percentage of CPU utilization that occurred while executing at the user level (application).
%nice shows the percentage of CPU utilization that occurred while executing at the user level with nice priority.
%system shows the percentage of CPU utilization that occurred while executing at the system level (kernel).
%iowait shows the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.
%steal    shows the  percentage of time spent in involuntary wait by the virtual CPU or CPUs while the hypervisor was servicing another virtual processor.
%idle shows the percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.
Device Utilization Report - The second report generated by the iostat command is the Device Utilization Report. The device report provides statistics on a per physical device or partition basis. Block devices and partitions for which statistics are to be displayed may be entered on the command line. If no device nor partition is entered, then statistics are displayed for every device used by the system. The report may show the following fields, depending on the flags used:

More on device file here - http://www.cyberciti.biz/faq/understanding-unix-linux-bsd-device-files/ 
Device: This column gives the device (or partition) name as listed in the /dev directory. As you can see from snapshot above that I have only one device.
tps indicates the number of transfers per second that were issued to the device. A transfer is an I/O request to the device. Multiple logical requests can be combined into a single I/O request to the device.
Blk_read/s (kB_read/s, MB_read/s) indicates the amount of data read from the device expressed in a number of blocks (kilobytes, megabytes) per second. Blocks are equivalent to sectors and therefore have a size of 512 bytes.
Blk_wrtn/s (kB_wrtn/s, MB_wrtn/s) Indicate the amount of data written to the device expressed in a number of blocks (kilobytes, megabytes) per second.
Blk_read (kB_read, MB_read) The total number of blocks (kilobytes, megabytes) read.
OPTIONS
  -c     Display the CPU utilization report.
$ iostat -c
Screenshot from 2015-07-22 11:44:52.png
  -d     Display the device utilization report.
    $ iostat -d
Screenshot from 2015-07-22 11:46:27.png

free command and options


Free command is frontend to /proc/meminfo file. It provide more human readable output to show you the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel:
$ free -m

vmstat command

===========================================

The vmstat, sar and iostat commands are the collection of package included in sysstat – the system monitoring tools. vmstat - Report virtual memory statistics. It reports information about processes, memory, paging, block IO, traps, disks and cpu activity. The  first report produced gives averages since the last reboot.  Additional reports give information on a sampling period of length delay.
Type the following command:
$vmstat
vmstat.png
FIELD DESCRIPTION FOR VM MODE
  Procs
      r: The number of runnable processes (running or waiting for run time).
      b: The number of processes in uninterruptible sleep.
  Memory
      swpd: the amount of virtual memory used.
      free: the amount of idle memory.
      buff: the amount of memory used as buffers.
      cache: the amount of memory used as cache.
      inact: the amount of inactive memory.  (-a option)
      active: the amount of active memory.  (-a option)
  Swap
      si: Amount of memory swapped in from disk (/s).
      so: Amount of memory swapped to disk (/s).
  IO
      bi: Blocks received from a block device (blocks/s).
      bo: Blocks sent to a block device (blocks/s).
  System
      in: The number of interrupts per second, including the clock.
      cs: The number of context switches per second.
Two important fields are free under memory and si, so under swap column
  1. Free – Amount of free/idle memory spaces.
  2. si – Swapped in every second from disk in Kilo Bytes.
  3. so – Swapped out every second to disk in Kilo Bytes.
OPTIONS
delay  The delay between updates in seconds.  If no delay is specified, only one report is printed with the average values since boot.
count  Number of updates.  In absence of count, when delay is defined, default is infinite.

Execute vmstat ‘X’ seconds and (‘N’ number of times)

$ vmstat 3 5

With this command, vmstat execute every 3 seconds and stop after executing 5 intervals.
Screenshot from 2015-07-22 10:30:41.png

-s, --stats Displays a table of various event counters and memory statistics.  This display does not repeat.
$ vmstat -s
Screenshot from 2015-07-22 10:32:12.png
-S, --unit character Switches outputs between 1000 (k), 1024 (K), 1000000 (m), or 1048576 (M) bytes.  Note this does not change the swap (si/so) or block
             (bi/bo) fields.
The vmstat displays in Megabytes with parameters -S and m. By default vmstat displays statistics in kilobytes.
$ vmstat -S m
Screenshot from 2015-07-22 10:36:29.png
    -d, --disk Report disk statistics
        $ vmstat -d
        Screenshot from 2015-07-22 10:42:07.png

top command and options

Back to performance measurement commands

The top program provides a dynamic real-time view of a running system.  It can display system summary information as well as a list of processes or threads currently being managed by the Linux kernel.

Type the following command:

$ top
top.png

The top command used to display all the running and active real-time processes in ordered list and updates it regularly. Press ‘c‘ option in running top command, it will display absolute path of running process.
If you want to kill a running process then use kill command as - {kill -9 <process-id>}

Press (Shift+W) to save the running top command results under /root/.toprc.

OPTIONS

-u | -U  :User-filter-mode as:  -u | -U number or name
Display only processes with a user id or user name matching that given.  The '-u' option matches on  effective user whereas  the  '-U' option matches on any user (real, effective, saved, or filesystem).

Prepending  an  exclamation point ('!') to the user id or name instructs top to display only processes with users not matching the one provided.

$ top -u tarun

replace tarun with your username ;-)

top.png


-c  :Command-line/Program-name toggle Starts top with the last remembered 'c' state reversed. Thus, if top was displaying command lines, now that field will show program        names, and visa versa. See the 'c' interactive command for additional information. You can use this command to display absolute path of running process.


        $ top -u tarun

        top.png

-d  :Delay-time interval as: -d ss.t (secs.tenths) Specifies the delay between screen updates, and overrides the corresponding value in one's personal configuration file or the startup default.

        $ top -d 1

        No screen capture, try it yourself ;-)

Friday, July 10, 2015

STF and Android Test Automation

SeleniumTestsFramework (aka STF) has been available publicly for quite some time now.
Given the importance of mobile market, test automation for mobile sites and apps can not be ignored any more. So here is the version 2.0 of STF which supports android testing, both m-site as well as app. STF internally uses appium APIs to control android automation hence you don't need to switch to a new framework if you have already been using appium.
Use the following configuration in your testng.xml file to run test on android browser - 




Make sure you set android emulator or real device first as described in appium tutorial
You can use sample test project which uses latest version of STF and is configured to run test on android browser.
Let us know your feedback :-)

Tuesday, June 16, 2015

STF (SeleniumTestsFramework) on maven repository

Since the commencement of STF it required installation of STF jar locally. This was a tedious process since for each new update of STF, the jar file should be installed locally. But not any more :-)

I am happy to announce the release of STF to maven repository. To be able use STF you would need to add following dependency to you pom file - 

(Please check POM file to find latest version of STF jar)

<dependency>
    <groupId>com.seleniumtests</groupId>
    <artifactId>seleniumtestsframework</artifactId>
    <version>2.0</version>
</dependency>


that's all, and maven will download STF for you.

You can also download Sample Selenium Project which uses and illustrates STF features.

In case you are a Selenium Beginners, then you should first begin with Selenium Training Videos

Have a question on STF? Is there a feature missing? don't forget to post on Testing Forum :-) 

Fork me on GitHub