Please post your Web Driver questions in official Web Driver forum

Monday, September 7, 2009

HTML Parsing and Selenium

HTML parsing is always been a burning requirement with selenium. Though Selenium doesn’t have built in API which could do HTML parsing,
given its high integrability it could be integrated with HTMP parser to achieve the same. I have experimented on HTML parsing using Jericho which is java library. To begin HTML parsing the only demand Jericho makes is about HTML Source and this could be obtained using Selenium API - getHtmlSource(). Herein I have listed functions which I have developed using Jericho -

Count number of tables on a page –

// Get Source object for HTML Tables.
Source source = new Source(selenium.getHtmlSource());
List table = source.getAllElements(HTMLElementName.TABLE);

Reporter.log("Number of Tables are: " +table.size());

***Reporter is TestNG API***
Retrieve Table Data-

// Retrieve table data from a specific table.Source tableSource = new Source(table.get(3).toString());

Reporter.log("Table data is:" +HTMLTableParser.getTableData(tableSource, false)); Reporter.log("True Table data is:" +HTMLTableParser.getTableData(tableSource, true));

Definition of ***getTableData*** is as following –


* Returns the Segment or content of HTML table

* available between Start and End tag


* @param tableSource

* @param rawHTMLData


* @return HTML Table data


public static List getTableData(Source tableSource, Boolean rawHTMLData) {

// Table data to be returned

List tableData = new ArrayList ();

// Collect table rows

List tableRows = tableSource.getAllElements(HTMLElementName.TR);

// Loop through table rows

for (int tableRowIndex=0; tableRowIndex data = tableRow.getAllElements(HTMLElementName.TD);

// Loop through table columns

for(int tableColummnIndex=0; tableColummnIndex tableRows = tableSource.getAllElements(HTMLElementName.TR);

return tableRows.size();


Count Number of columns in a individual rows –
Map rowAndCoumnCount = HTMLTableParser.countTableColumnsInRows(tableSource);

for(Map.Entry rowAndColumnData : rowAndCoumnCount.entrySet()){

Reporter.log("Number of columns at row: " +rowAndColumnData.getKey()

+" are: " +rowAndColumnData.getValue());


// Get data from individual columns.

Reporter.log("Column specific table data is:" +HTMLTableParser.getTableDataForColumn(tableSource, false, 0, 1));

Reporter.log("Column specific raw table data is:" +HTMLTableParser.getTableDataForColumn(tableSource, true, 0, 1));

Definition of ***countTableColumnsInRows*** is as following –


* Retrieves table data for specific columns beginning from specific row

* To return data from beginning of row pass rowNumber as *0


* @param tableSource

* @param rawHTMLData

* @param rowNumber

* @param columnNumber

* @return Table Data


public static List getTableDataForColumn(Source tableSource,

Boolean rawHTMLData, int rowNumber, int columnNumber) {

// Table data to be returned

List tableData = new ArrayList ();

// Collect table rows

List tableRows = tableSource.getAllElements(HTMLElementName.TR);

// Loop through table rows

for (int tableRowIndex=rowNumber; tableRowIndex data = tableRow.getAllElements(HTMLElementName.TD);

// If supplied index is with in size of table data

// This check is useful when retrieving data from uneven html table

if (columnNumber < rawhtmldata ="="">

Retrieving dynanic HTML Objects from selenium

We often come across situations of dynamic html objects where in HTML id/name of a page is not constant and some time we don't even get to have any constant part if HTML identifier. Of late I came across a page which had lots of check boxes but no part oh HTML id/name was constant. To over come this we created custom functions for selenium using js evaluation capabilities of selenium. Function objectives are:

****** Retrieves all dropdown objects in a web page
****** Retrieves all multiline text box objects in a web page
****** Retrieves all radio button objects in a web page
****** Retrieves all check box objects in a web page
****** Retrieves all text box objects in a web page

Depending on the availability of HTML id or name these functions would retrieve either HTML id or name. These methods are as following -

Tuesday, August 18, 2009

How selenium recorder compares with other tools?

I had chance to work on QTP and to sneak in to Test Complete in my Organization.
I am bit depressed as to how irrationally QTP and Test Complete record web application and how much script one should tweak manually to make it compact. Let me cite example of Google Search for this. A selenium test for Google Search would be like this -"");
selenium.type("q", "I love My Company");"btnG");

This certainly gives impression of app being opened, some thing being typed (though "q" might not be very clear here) and some button being clicked (again "bthG" is not very clear here)

Same recorded test in QTP be as following -

Browser("Google").Page("Google").WebEdit("q").Set "I Love My Company"
Browser("Google").Page("Google").WebButton("google search").Click


With Browser and Page objects embedded every where it looks over kill me (Though this can be minimized through manual tweaking)

Now comes the Test Complete -

Sub Test1
Dim page
Dim cell
Dim textbox
Set page = Aliases.firefox.pageGoogle1
Call page.ToURL("")
Set cell = page.formF.table.cell
Set textbox = cell.textboxQ
Call textbox.Keys("I Love My Company")
End Sub

First of all this is too much of script for simple search in Google and would intimidate a novice test automation engineer. More over Google search text box and search button are associated with page > form > table and cell. I tweaked and associated text box and button with page and got through. This also required me to change NameMapping

So As far as script generated through recorder is concerned selenium looks better to me,
Any opinion on this ?

p.s. It took me half an hour to records Google test using Test Complete. Tool looks heavy on memory usage.
Fork me on GitHub