Please post your Web Driver questions in official Web Driver forum

Monday, September 7, 2009

HTML Parsing and Selenium

HTML parsing is always been a burning requirement with selenium. Though Selenium doesn’t have built in API which could do HTML parsing,
given its high integrability it could be integrated with HTMP parser to achieve the same. I have experimented on HTML parsing using Jericho which is java library. To begin HTML parsing the only demand Jericho makes is about HTML Source and this could be obtained using Selenium API - getHtmlSource(). Herein I have listed functions which I have developed using Jericho -

Count number of tables on a page –

// Get Source object for HTML Tables.
Source source = new Source(selenium.getHtmlSource());
List table = source.getAllElements(HTMLElementName.TABLE);

Reporter.log("Number of Tables are: " +table.size());

***Reporter is TestNG API***
Retrieve Table Data-

// Retrieve table data from a specific table.Source tableSource = new Source(table.get(3).toString());

Reporter.log("Table data is:" +HTMLTableParser.getTableData(tableSource, false)); Reporter.log("True Table data is:" +HTMLTableParser.getTableData(tableSource, true));


Definition of ***getTableData*** is as following –


/**

* Returns the Segment or content of HTML table

* available between Start and End tag

*

* @param tableSource

* @param rawHTMLData

*

* @return HTML Table data

*/

public static List getTableData(Source tableSource, Boolean rawHTMLData) {

// Table data to be returned

List tableData = new ArrayList ();

// Collect table rows

List tableRows = tableSource.getAllElements(HTMLElementName.TR);

// Loop through table rows

for (int tableRowIndex=0; tableRowIndex data = tableRow.getAllElements(HTMLElementName.TD);

// Loop through table columns

for(int tableColummnIndex=0; tableColummnIndex tableRows = tableSource.getAllElements(HTMLElementName.TR);

return tableRows.size();

}


Count Number of columns in a individual rows –
Map rowAndCoumnCount = HTMLTableParser.countTableColumnsInRows(tableSource);

for(Map.Entry rowAndColumnData : rowAndCoumnCount.entrySet()){

Reporter.log("Number of columns at row: " +rowAndColumnData.getKey()

+" are: " +rowAndColumnData.getValue());

}

// Get data from individual columns.

Reporter.log("Column specific table data is:" +HTMLTableParser.getTableDataForColumn(tableSource, false, 0, 1));

Reporter.log("Column specific raw table data is:" +HTMLTableParser.getTableDataForColumn(tableSource, true, 0, 1));

Definition of ***countTableColumnsInRows*** is as following –
/**

*

* Retrieves table data for specific columns beginning from specific row

* To return data from beginning of row pass rowNumber as *0

*

* @param tableSource

* @param rawHTMLData

* @param rowNumber

* @param columnNumber

* @return Table Data

*/

public static List getTableDataForColumn(Source tableSource,

Boolean rawHTMLData, int rowNumber, int columnNumber) {

// Table data to be returned

List tableData = new ArrayList ();

// Collect table rows

List tableRows = tableSource.getAllElements(HTMLElementName.TR);

// Loop through table rows

for (int tableRowIndex=rowNumber; tableRowIndex data = tableRow.getAllElements(HTMLElementName.TD);

// If supplied index is with in size of table data

// This check is useful when retrieving data from uneven html table

if (columnNumber < rawhtmldata ="="">

Retrieving dynanic HTML Objects from selenium

We often come across situations of dynamic html objects where in HTML id/name of a page is not constant and some time we don't even get to have any constant part if HTML identifier. Of late I came across a page which had lots of check boxes but no part oh HTML id/name was constant. To over come this we created custom functions for selenium using js evaluation capabilities of selenium. Function objectives are:

****** Retrieves all dropdown objects in a web page
****** Retrieves all multiline text box objects in a web page
****** Retrieves all radio button objects in a web page
****** Retrieves all check box objects in a web page
****** Retrieves all text box objects in a web page

Depending on the availability of HTML id or name these functions would retrieve either HTML id or name. These methods are as following -



Fork me on GitHub