Please post your Web Driver questions in official Web Driver forum

Monday, December 8, 2008

Conducting Test using Remote Agent.


SP Agent is the machine which is used to generate virtual user load. We have
load test being carried in our project which is to be simulated from client
environment This is to avoid the network delays as users of this app would be
in client environment load should be simulated form client environment. (In fact
we observed difference in response time as high as 10 sec when we compared
two test run. One using offshore Agent and other using onsite Agent.)

For this we got a machine in Client and we would connect to this machine
through vpn. Hence this would be used as Agent while we would control
execution of test from offshore. To be able to do this we got two ports 19200
and 19202 opened on this machine from client environment . Opening of these
two ports got us going with performance test.

How to avoid think time / wait time from being accumulated in Custom Timers.

In Silk Performer custom Timers are used to generate response time around
transactions. It is usually inserted in the script generated by SP recorder. In
a typical script it looks as following -

==========================================================
MeasureStart("1020_ RANK_PREFERENCE"); // Custom Timer - Start for Rank
Prefpage

WebUrl("https://test-qa.test.edu/testtesting/images/arrow.png");
**** Few more functions***
**************************
**** Few more functions***

MeasureStop("1020_ RANK_PREFERENCE"); // Custom Timer - stop for
Rank Pref page
==========================================================

With the inclusion of Think Timer (wait time kept deliberately to simulate
real user behaviour as a typical user waits on a page before performing
any action) script would look as following -

===========================================================
MeasureStart("1020_ RANK_PREFERENCE"); // Custom Timer - Start for Rank
Pref page

WebUrl("https://test-qa.test.edu/testtesting/images/arrow.png");

ThinkTime(154.56); // Think timer inserted in middle of Measure Start and
Stop timers.

**** Few more functions***
**************************
**** Few more functions***

MeasureStop("1020_ RANK_PREFERENCE"); // Custom Timer - stop for Rank
Pref page
============================================================

Having think time in middle of Start and Stop timers would cause think time to
be added to the overall response time which eventually would give a false
impression about the response time of page.
To avoid think time from being added to custom timer one needs to stop timing
measurement before Think Time and must resume it after it. This is as following
-

==============================================================
MeasureStart("1020_ RANK_PREFERENCE"); // Custom Timer - Start for Rank Pref
page

WebUrl("https://test-qa.test.edu/testtesting/images/arrow.png");

MeasurePause("1020_ RANK_PREFERENCE"); // Pause the time measurement
ThinkTime(154.56); // Think Time
MeasureResume("1020_ RANK_PREFERENCE"); // Resume the time measurement

**** Few more functions***
**************************
**** Few more functions***

MeasureStop("1020_ RANK_PREFERENCE"); // Custom Timer - stop for Rank
Pref page
=============================================================

This how think time can be stooped from getting added to custom timers.

fwait-Parameter:
When browser-level Web traffic is recorded using the Recorder, a wait time is
included at the end of recorded functions to tell the replay engine how long to
wait before executing the next function in the script. In WebURL function it
look as -

==============================================================
WebUrl("http://www.shopit.com/ad.gif", 17.2); // Here 17.2 is fwait parameter.
==============================================================
The wait time times simulate:

* The time the browser needs to parse out links of embedded documents (e.g.
images, etc).
* A users thinking time.

This wait time is ignored in Stress Test mode.

But again this wait gets added to custom timers and results in overall high
response time. If WebUrl function is just a call to image then it can be
manually removed from the script, But if its need is indispensable then it can
be replaced with equivalent Think Timer as -

===========================================================
WebUrl("http://www.shopit.com/ad.gif"); // WebURL function with out fwait.
ThinkTime(17.2); // Think time to simulate the wait period
===========================================================

Here again time measurement should be stopped before Think Time and should be
resumed just after it as illustrated above.

Wednesday, September 17, 2008

QTP vs Seleium vs Other tools

I had a chance to look at this comparison and do not agree with few claims.
(Not because I am dedicated to selenium but since I have found few of these working my self.)



They are just mentioned as mere Yes or No with out any associated reasoning. I have tried to contradict few of these claims below -

  • Ability to identify objects using multiple parameters: Selenium does indeed support multiple ways for identifying object like - id, name, Xpath, css, dom etc.
  • Ability to run multiple scripts consistantly in a Batch mode: This can be easily achieved using frameworks like JUnit, TestNG etc,
  • Object Oriented Scripting Support: Selenium has a long list of supported client drivers and if Java, C# are not object oriented then which language is object oriented?
  • Integration with External libraries: What are libraries like TestNG, Fest, Ant which I have integrated with Selenium for my test effort
  • Object parametrization: Can be achieved using TestNG
  • Integrated Data Driven Test framework support: Cab be achieved using TestNG
Though these features are supported from external libraries but there is no harm in plugging in external libraries in existing tool and getting the things done.

Tuesday, September 16, 2008

Testing HTML object using DOM and selenium

DOM is w3c standard for navigating through the objects of a structured doc,
Same can be used for exercising HTML/XML docs.

In HTML, each element is referred as an object hence text box, dropdown, radio
button etc are objects. These objects have properties (like max length in case
of text box etc) and methods. Herein properties of objects can be collected in
selenium through evaluation of java script and then these can be compared
against expected values. For example one can collect max length and size
properties of text box by evaluating java script through getEval function as
following -

selenium.getEval("var property = new Array(); property[0] =
window.document.getElementsByName('empNumber')[0].maxLength;" +

"property[1] = window.document.getElementsByName('empNumber')[0].size;
property.toString();");

/**** I have explained evaluation of js in issue - 9211 ****/

Now individual properties can be collected in an string array as following -

String[] Textproperties = collectTextProperty.split(",");

And then this can be compared against expected values.

Below I have listed the HTML Element object and their properties which are
useful for testing. Please add on to this if I have missed any.

/*****************************************************************************/
Anchor Object - href
Button Object – disabled
Form Object - elements[]
Frame Object – marginHeight, marginWidth
IFrame Object – height, marginHeight, marginWidth, width
Image Object – height, hspace, vspace, width, src
Checkbox Object - checked, disabled
FileUpload Object – disabled, value
Password Object – defaultValue, disabled, maxLength, size, value
Radio Object – checked, disabled, type, value
Submit Object – disabled, value
Text Object – defaultValue, disabled, maxLength, readOnly, size, value
Link Object – disabled, href
Meta Object – content
Option Object - defaultSelected, disabled, label, selected, text, value
Select Object - options[],disabled, length, multiple, size
/*****************************************************************************/

For details on these properties look here -

http://www.w3schools.com/htmldom/

Selenium also has prebuilt methods which can be used to test the position of
elements in a web page. These methods are as following -

getElementHeight(java.lang.String locator)
getElementPositionLeft(java.lang.String locator)
getElementPositionTop(java.lang.String locator)
getElementWidth(java.lang.String locator)

Using these methods one can build UI tests regarding position of elements.

Execute java script from selenium

getEval method can be used to evaluate java script from selenium.
Remember to use window object in case of dom expressions as by default selenium
window is referred and not the test window. Hence a sample code will look as
following -

selenium.getEval("window.document.images.length;");

This will return total number of images in a web page. Same result can be
retrieved using only "document.images.length;" string IF selenium is run in
proxy injection mode. (As selenium runner will not appear in the test window).
BUT To be on safer side irrespective of mode selenium is run always use
'window' object of java script.

If evaluated expression is to return multiple values then use it as following -


This will return all the image links available on a page. This can be collected
in a string and individual links can be found by using split method.

Now to execute java script using Selenium IDE. lets type a random number on a text box -


    type
    objectLocator
    javascript{Math.random()}



or in java -

selenium.type("objectLocator", selenium.getEval("Math.random()"));



Disadvantage of using isTextPresent method and possible work around

Default Selenium class has a method called 'isTextPresent' which can be used to
verify presence of text on a page. Disadvantage of using this method is -

It checks for presence of text in entire page, hence even if text is present at
a different location test would pass and no error would be reported.

A better approach would be to find text on the basis of it's position in page.

If text which is to be validated is part of table then 'getTable' method can be
used. 'getText' method can also be used BUT element locator should be taken as
dom so that presence of text is validated at a certain position.

Ex -

Consider presence of text "Employee Number1 :*" is to validated on a page. It
can be done as following using getText method -

verifyEquals("Asserstion Failed for getText.", "Employee Number1 :*",
selenium.getText("document.getElementsByTagName('table')[0].
getElementsByTagName('tr')[0].getElementsByTagName('td')[0]"));

verifyEquals method takes three string -

1. Error message which would be displayed if objects do not match.
2. Expected String object from app.
3. Original String Object in app. (Here getText method is used which returns
string object from the specified element locator.)

getTable method can be used here if text resides in a table. This would be as
following -

verifyEquals("Assertion Failed for getTable", "Employee Number1 :*",
selenium.getTable("document.getElementsByTagName('table')[0].0.0"));

Here Cell Address syntax is - tableLocator.row.column, where row and column
start at 0

I have used verify methods here as I didn’t want test to be aborted in case
of failure but wanted these exceptions to be raised in reports. Don't forget to
call checkForVerificationErrors as the last statement of method call or better
make it part of SelTestCase class as following -

@AfterTest(alwaysRun = true, description = "This methods checks verification
errors of testmethods.")
public void checkAfterTest() throws Exception {
checkForVerificationErrors();
}

This would avoid code duplication which would be introduced if calls are made
to checkForVerificationErrors in each test method.

Mouse over DOM

Mouse Over Dom can be used to analyze element in a web page.

This would be useful for the applications which are to be automated but do not
support Firefox Browser (Hence Selenium IDE or Firebug can not be used to
analyze elements). Browser which are currently supported are -

Firefox, Mozilla, Netscape 8, Opera 7.5+ and MSIE6+

http://slayeroffice.com/tools/modi/v2.0/modi_help.html

Selenium Tutorial: Capture screenshot on selenium test failure using TestNG and Fest

There is a method in DefaultSelenium class called CaptureScreenshot()which can be used to capture screen shot in general. But this is not effective when screen shot is to be taken only during failures.
To achieve capturing screen shot on failure FEST can be used (http://fest.easytesting.org/)

Regardless of how the FEST-Swing takes a screenshot, it needs to know which tests should be considered "GUI tests." In order to do so, one needs to add the annotation @GUITest. This annotation can be placed at class or method level, and it is inherited by subclasses of annotated classes. Hence in case of selenium it should be added on the class which is extended by other test classes
Once downloaded Fest, add these jar files in library of Intellij -

fest-swing-1.2.jar
fest-swing-testng-1.2.jar
fest-util-1.1.2.jar
fest-reflect-1.2.jar
Same should be added in the library folder of project and property of ANT so that ANT could access these. Now import the GUI test in SelTestCase class and make add @GUITest annotation at the SelTestCase class as following -

import org.fest.swing.annotation.GUITest;
@GUITest
public class SelTestCase extends SeleneseTestCase {

************  Set Up Code ***************
}

Since all test classes would inherit SelTestCase class hence all classes would be able to capture screen in case of failures. Add the following lister in Build.xml file of ANT. listeners="org.fest.swing.testng.ScreenshotOnFailureListener" Hence whole TestNG task of ANT looks as
following -
(I had to keep the image as angle brackets of xml don't let this text appear on blog) Make sure that png files which are created while taking the screen shot are DELETED before running new set of test. I do this by specifying the delete task in clean target as following - NOTICE THAT IF THESE SCREEN SHOT FILES ARE NOT DELETED BEFORE EACH TEST RUN THEN FEST WILL THROW EXCEPTION AND EVEN HINDER THE GENERATION OF NORMAL TEST REPORT.
I have observed that for a failure in a method screen shot is taken only once. This looks fine for hard assertion but may not be sufficient for soft assertion as there should be multiple errors captured in case of soft assertion. BUT I still recommend it as it is better to have one screen shot in case of failures than none.

Setting up selenium with .NET Framework

Prequisites -

1. Visual Studio Team System - VSTS (preferably 2005)
2. Selenium .NET Client Driver
{http://archiva.openqa.org/repository/releases/org/openqa/selenium
/selenium-remote-control/1.0-beta-1/selenium-remote-control-1.0-beta-1-dist.zip}
On unzipping the package one folder dotnet will be found.)
3. NUnit (2.4.7){http://www.nunit.org/download.html > download
NUnit-2.4.7-net-1.1.msi}

Steps -
1. Launch VSTS > File > New > Project.
2. Select Visual C# > Test > Test Project > Click Ok
3. This will open AuthorisingTest.txt and UnitTest1.cs files.
4. Click on UnitTest1.cs file and replace the existing code with following -




****************************************************************
This is a simple test over google where Search is performed over
"Selenium OpenQA" string and search results are verified.
****************************************************************

5. On the RHS under Solution Explorer right click on References > Add
References > Browser to the dotnet directory which was 2nd prerequisite. Select
following dll files - nmock.dll, nunit.core.dll, nunit.framework.dll,
ThoughtWorks.Selenium.Core.dll, ThoughtWorks.Selenium.IntegrationTests.dll,
ThoughtWorks.Selenium.UnitTests.dll and click on Ok button

6. Under Build menu click on Build Solution > If compilation goes fine then you
should see this message -

***************************************************
Compile complete -- 0 errors, 0 warnings
TestProject1 -> D:\Test\TestProject1\TestProject1\bin\Debug\TestProject1.dll
========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========
***************************************************

Hence this will create a dll - TestProject1.dll where TestProject1 is the name
of the project given during step 1

7. Launch NUnit as Start > Programs > NUnit 2.4.7 > Click on File > Open
Project.
8. Browser to the TestProject1.dll which was created in step 6 and click on
Open button.
9. Before running the Tests we need  start selenium server. Launch command
prompt and browse to the director where selenium-server folder is installed.
For me it is under C drive -

*************************************************
C:\Program Files\Selenium and other
files\selenium-remote-control-0.9.0\server>j
ava -Dhttp.proxyHost=192.168.1.14 -Dhttp.proxyPort=80 -jar selenium-server.jar
-
proxyinjectionmode
*************************************************

This should get the selenium server running with following info -

*************************************************
The selenium server will execute in proxyInjection mode.
Jul 22, 2008 4:43:26 PM org.mortbay.http.HttpServer doStart
INFO: Version Jetty/0.9.0
Jul 22, 2008 4:43:26 PM org.mortbay.util.Container start
INFO: Started HttpContext[/,/]
Jul 22, 2008 4:43:26 PM org.mortbay.util.Container start
INFO: Started HttpContext[/selenium-server,/selenium-server]
Jul 22, 2008 4:43:26 PM org.mortbay.util.Container start
INFO: Started HttpContext[/selenium-server/driver,/selenium-server/driver]
Jul 22, 2008 4:43:26 PM org.mortbay.http.SocketListener start
INFO: Started SocketListener on 0.0.0.0:4444
Jul 22, 2008 4:43:26 PM org.mortbay.util.Container start
INFO: Started org.mortbay.jetty.Server@a62fc3
*************************************************

Notice: I had to explicitly declare our n/w proxy as google is out side Iteamic
n/w and to be able to access we should go through n/w proxy.

10. Now click on Run button on NUnit window and enjoy the tests being launched
in IE :-)

To Summerize -

Build the Project in VSTS.
Run Selenium Server.
Take the dll file generated after building the project file and click on Run
button.

In absense  of VSTS same can be achieved Visual Studio 2005 as
well. Follow these steps instead of one described in Description below. -

1. Launch MS Visual Studio 2005 > File > New > Project
2. Select Visual C# > Class Library > Name your project > Click OK
3. A Class (.cs) is already created. Rename it as appropriate

This should get you going.

Implementing soft assertion with selenium

Soft Assertion is a check which doesn't abort tests if assertion fails. (Same
way there is Hard Assertion which aborts tests if assertion fails.)

Selenium API has verify.* methods for this but their are draw backs of these
methods -

1. Verification methods do not take string arg hence if multiple verifications
fail in a method then there is no way to point as to which assertion failed.

2. To check verification errors checkForVerificationErrors method is called. But
if there is any verification error this method doesn't clear the verification
errors after having caught it. Hence no subsequent verification errors would be
checked.

These limitation are described here -
http://osdir.com/ml/web.selenium.devel/2007-09/msg00022.html

These methods are part of Java-Client-Driver of selenium.
I have modified checkForVerificationErrors method like this -

Here Reporter is the class in TestNG using which I get the message printed in
TestNG Report. If Selenium is to be used with JUnit then I believe it should be
replaced with System.out.println("error message");

You could use these methods in you tests as following -

Steps to bypass the NT authentication in windows

Below is the excerpt from my friend Poornima over the steps to be carried
out to stop NT log in window to appear.

STEP 1:

Start regedit.

Go to:
HKEY_LOCAL_MACHINE\Software\Microsoft\Internet
Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME_PASSWORD_DISABLE
to re-enable it for the entire machine.

Or go to:
HKEY_CURRENT_USER\Software\Microsoft\Internet
Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME_PASSWORD_DISABLE
to re-enable it for the logged in user.

STEP 2:

Now create iexplore.exe and explorer.exe DWORD values and set their value data
to 0.

STEP 3:

Now the URL should be used as – http://@www.

How to over come "Failed to start: SocketListener0@0.0.0.0:4444"

This is the most common error which is encountered while starting the
selenium server. The origin of this error is the fact that selenium server always
tries to listen at port 4444 by default so when following is written at command
prompt -

java -jar selenium-server.jar

The server starts running on port 4444 but if this post is already occupied
then server will fail to listen at this port and following error will be
encountered -

Failed to start: SocketListener0@0.0.0.0:4444

To over come this one can force server to listen at port any other port while
starting the selenium server as following -

java -jar selenium-server.jar -port 5555

and this will start server as following -

10:55:14.440 INFO - Java: Sun Microsystems Inc. 1.5.0_05-b05
10:55:14.440 INFO - OS: Windows XP 5.1 x86
10:55:14.456 INFO - v1.0-SNAPSHOT [1123], with Core v1.0-SNAPSHOT [2101]
10:55:14.549 INFO - Version Jetty/5.1.x
10:55:14.549 INFO - Started HttpContext[/,/]
10:55:14.549 INFO - Started HttpContext[/selenium-server,/selenium-server]
10:55:14.549 INFO - Started
HttpContext[/selenium-server/driver,/selenium-server
/driver]
10:55:14.565 INFO - Started SocketListener on 0.0.0.0:5555
10:55:14.565 INFO - Started org.mortbay.jetty.Server@e0e1c6
If same error is encountered again then keep changing port till server gets
started on a port.

You could also shut down Selenium Server using following URL in your browser -

http://localhost:4444/selenium-server/driver/?cmd=shutDownSeleniumServer

Monday, September 15, 2008

Selenium or Badboy

This is my first blog and first thing which comes on my mind is test automation. So far I have worked with two test automation tools Badboy and Selenium. Out of these Badboy was the first tool I used and then migrated to Selenium for various reasons like -
  • One can code in multiple languages in Selenium (i.e. Java, C#, Perl, Python etc)
  • Though there is no out of box support for data base testing but it can be achieved using client language like - java, c#
There are many more such reasons which prompted me to use Selenium.
I will describe them in detail when at leisure.
With this I come to end for my first blog.
Fork me on GitHub