Please post your Web Driver questions in official Web Driver forum

Wednesday, February 8, 2012

Is it sane to compare Selenium with QTP?


I have seen many articles which compare Selenium (1 or 2) with QTP (or other commercial tools). But these articles miss very basic point that Selenium is a library available in variety of language while QTP has entire gamut of tools with it. So if you were to really compare QTP with Selenium then comparison should be based on the library of languages which is offered either of these tools, which in my opinion Selenium wins outright over QTP.
My only intention of writing this post is not claim Selenium victory of QTP but to demonstrate how badly Selenium is compared with QTP at times. There might be things QTP would be good at doing (I left QTP long ago hence can not name any) but to disregard Selenium on the factors I listed herein is highly biased.
  • Object spying - Sure, you have QTP recorder which does it for you but don't forget that you can test object locator using Selenium IDE and it is not limited to using only Selenium IDE. Firebug with Firefox can be used for same while Chrome has built in development tool bar to test element locator. In case you don't know you can use $$("cssLocator") to test css locator and $x("xpathLocator") to test xPath in both FF and chrome to test application objects. Can you use QTP to test object locators in FF and Chrome?
  • Debugging code - So QTP comes in with built-in editor and Selenium is defeated in this aspect. Is it? Given the variety of languages Selenium can be used with, you have far greater language editor options with Selenium than with QTP. You have umpteen options with editors like Intellij,  Eclipse, Visual Studio than the one available with QTP. When I was working with QTP I could not even conveniently rename test methods spanning across multiple files. Such re factoring is child's play with professional editors you use with Selenium
  • Recovery Scenarios - This is one of the biggest cheat point of QTP. Add recovery scenarios and tool takes care of working around the update windows. First of all if you add recovery scenario then it slows down the speed of test execution as there would be one listener always looking for such events, which means more recovery scenario and be ready for more slower test execution. Why should you bother about those automatic update windows in your test environment? Should not you proof your test environment against such update windows? If it is windows or FF then disable automatic updates. And best if you can not manage your test environment then out source it to some else like - Sauce Labs or Testing Bot
  • Testing non browser applications - Well, when did Selenium claim that it can tests window based applications. Selenium is browser automation framework. Comparing Selenium and QTP on this front is like comparing apples with oranges.
  • Great object Repository of QTP - So QTP can store objects in an external location, all you need to do is to update object repo when your object location changes. And then you find that the easiest way to port QTP scripts from one machine to another is to use Descriptive Programming which indeed means that you should do away with your beloved object repository of QTP. Now coming to absence of object repository mechanism in Selenium. Have you heard of properties files or Page Factory and guess what, you you don't have to do away with any thing to be able to port your Selenium tests from one machine to another.
  • No built-in Report with Selenium - QTP generates nice test reports and with Selenium - none. Really? When you pick up a language for Selenium you would be using a framework in that language and most probably the framework will provide you the reporting capabilities. For example there is ant task to generate test report in JUnit while TestNG has buit-in reporting mechanism. Google it and you would find similar options in language of your choice.
  • QTP and QC integration- You bought QTP for test automation now buy QC for test management. What with Selenium Test Management tools. None. There are indeed options - xStudio is one good option for small Selenium teams looking for test management with out spending any extra money. And their commercial options are cheaper than  QC. If you are looking to port your test results to Test Management tool then TestLink might be worth a try. In this case you don't have to pay at all.

Fork me on GitHub