Skip to main content

Is it sane to compare Selenium with QTP?


I have seen many articles which compare Selenium (1 or 2) with QTP (or other commercial tools). But these articles miss very basic point that Selenium is a library available in variety of language while QTP has entire gamut of tools with it. So if you were to really compare QTP with Selenium then comparison should be based on the library of languages which is offered either of these tools, which in my opinion Selenium wins outright over QTP.
My only intention of writing this post is not claim Selenium victory of QTP but to demonstrate how badly Selenium is compared with QTP at times. There might be things QTP would be good at doing (I left QTP long ago hence can not name any) but to disregard Selenium on the factors I listed herein is highly biased.
  • Object spying - Sure, you have QTP recorder which does it for you but don't forget that you can test object locator using Selenium IDE and it is not limited to using only Selenium IDE. Firebug with Firefox can be used for same while Chrome has built in development tool bar to test element locator. In case you don't know you can use $$("cssLocator") to test css locator and $x("xpathLocator") to test xPath in both FF and chrome to test application objects. Can you use QTP to test object locators in FF and Chrome?
  • Debugging code - So QTP comes in with built-in editor and Selenium is defeated in this aspect. Is it? Given the variety of languages Selenium can be used with, you have far greater language editor options with Selenium than with QTP. You have umpteen options with editors like Intellij,  Eclipse, Visual Studio than the one available with QTP. When I was working with QTP I could not even conveniently rename test methods spanning across multiple files. Such re factoring is child's play with professional editors you use with Selenium
  • Recovery Scenarios - This is one of the biggest cheat point of QTP. Add recovery scenarios and tool takes care of working around the update windows. First of all if you add recovery scenario then it slows down the speed of test execution as there would be one listener always looking for such events, which means more recovery scenario and be ready for more slower test execution. Why should you bother about those automatic update windows in your test environment? Should not you proof your test environment against such update windows? If it is windows or FF then disable automatic updates. And best if you can not manage your test environment then out source it to some else like - Sauce Labs or Testing Bot
  • Testing non browser applications - Well, when did Selenium claim that it can tests window based applications. Selenium is browser automation framework. Comparing Selenium and QTP on this front is like comparing apples with oranges.
  • Great object Repository of QTP - So QTP can store objects in an external location, all you need to do is to update object repo when your object location changes. And then you find that the easiest way to port QTP scripts from one machine to another is to use Descriptive Programming which indeed means that you should do away with your beloved object repository of QTP. Now coming to absence of object repository mechanism in Selenium. Have you heard of properties files or Page Factory and guess what, you you don't have to do away with any thing to be able to port your Selenium tests from one machine to another.
  • No built-in Report with Selenium - QTP generates nice test reports and with Selenium - none. Really? When you pick up a language for Selenium you would be using a framework in that language and most probably the framework will provide you the reporting capabilities. For example there is ant task to generate test report in JUnit while TestNG has buit-in reporting mechanism. Google it and you would find similar options in language of your choice.
  • QTP and QC integration- You bought QTP for test automation now buy QC for test management. What with Selenium Test Management tools. None. There are indeed options - xStudio is one good option for small Selenium teams looking for test management with out spending any extra money. And their commercial options are cheaper than  QC. If you are looking to port your test results to Test Management tool then TestLink might be worth a try. In this case you don't have to pay at all.

Comments

  1. i'm working on qtp i taught qtp to be the only good testing tool but after reading this post i would like to say qtp is one of the testing tool.

    ReplyDelete
  2. Good write up, Many aspects to consider when selecting an auto tool. Selenium has many strong features as mentioned and best of all its free.

    ReplyDelete
  3. I find this post very slanted and inaccurate. For example, your first question: Can you test object locators in FF and chrome in QTP. Well, yes you can. And since both CSS and Xpath are available, in addition to all the other recognition properties in QTP why was that made a central point?

    And your point about the object repository is down right bizarre? Why on earth would you need to use descriptive programming to move your test from one machine to another? We have ours running on a Jenkins grid, with our tests stored in Git without the need for resorting to descriptive programming. Some simple regex's is probably all you needed for a property or two.

    One last point I want to make. Given the report in Qtp which will show you a screenshot or video, expected result, and where the test was when it failed vs a unit test Xml report, I find it hard to believe you think Selenium is on par with QuickTest.

    If you are going to do a review at least make it fair and do your homework about what you are reviewing. Both tools have their strong points and are good for a specific type of job.

    ReplyDelete
    Replies
    1. Let me answer your observations-

      I suppose you are referring to testing object locator in FF and Chrome using QTP by running the test, is not it? With Selenium you don't have to. Use FF and Chrome console to test your locator. Not really a selenium feature and so the point, why compare object spying of QTP with Selenium.

      About object repo, keeping descriptive programming aside, how slow it behaves when it begins to grow in size. Using object repo and you are committing binary files to your git. If there is a conflict in object repo committed by two team members and here your git goes for toss resolving conflicts in binary file.

      About Test Reporting, if built in reporting of TestNG is not enough then Use SauceLab or Testing bot and you have videos of your Selenium test execution on cloud.

      Talking about review, you got it wrong here. It is not review, it is about how Selenium should not be compared with QTP and it is has to be then it could be on some thing else but not on what I mentioned in post. There is no comparison. Having worked on both QTP and Selenium. I prefer Selenium may be because of its language support, may be because it is open source.

      b/w I appreciate when I see comments with identity of poster. Else it is difficult to see if comments are from a Test Engineer or from a QTP marketing guy. Any way thanks for your comments.

      Delete
  4. There is one big difference between Selenium and QTP.QTP has made all essential test automation features simpler for the user while an average Selenium user is on his journey to make automation easier for him.

    ReplyDelete

Post a Comment

No spam only genuine comments :)

Popular posts from this blog

Where is my defect ID?

Don't you feel ecstatic when your automated tests find bug? After all tests finding bugs give us a sense of accomplishment, is not it? And this is followed by usual cycle of defect reporting, retesting and hopefully closure of defect. But at times defects are deferred to next or future releases. Which causes test method to fail for subsequent releases. And if you are dealing with a test suite having 100s of tests then it may become difficult to remember if there was a defect reported for a failing test? How do you deal with such situation. How about adding defect-id to @description tag of TestNG test. Hence it is reported on automated test report and we would know if defect exists for a failing test - How do you track defect-id of a failing test?

Return only first or last element from webelements collection

We often come across situation when there are multiple elements on a page and we probably like to exercise only a few of them using selenium webdriver. May be just first and last element. For example on a search result page we may like to click on only first and last link and not all. This is when Iterables API comes handy. (By the way I am assuming that you have already completed watching selenium training videos :)). Once we have collection of web element then we can use Iterables to get only first or last element as following - Consider that we fetch collection of element as - List< WebElement > webElements = getDriver().findElements(By. id ( "htmlID" ));   Now we can get the first web element from this collection as -  WebElement firstElement = Iterables. getFirst (webElements,  getDriver().findElement(By. id ( "defaultElement" )));   Herein second argument -   (getDriver().findElement(By. id ( "defaultElement" )))    in the me

Using chrome console to test xPath and css selectors

Since the advent of selenium there have been many plugin to test xPath / css selectors but you don’t need any of them if you have chrome browser. Using Chrome console you can test both xPath and css selectors. Launch website to be tested in chrome browser and hit F-12 and you would see chrome console opened in lower pane of application - Hit escape key and console would open another pane to write element locators - And now you can start writing xPath or css selectors in chrome console and test them - The syntax for writing css id - $$(“ ”) And hit the enter key. If your expression is right then html snippet of the application element corresponding to the css selector would be displayed - If you mouse over the html snippet in chrome console then it would highlight the corresponding element in application - If you want to clean console of previously written element selectors then just hit ctrl+L keys and chrome console would be empty again. Pro