Skip to main content

Should selenium tests be always independent? The endless debate !!

Though unit tests have always recommended to be independent of each other, advent of selenium+TestNG has given birth to dependent integration/end to end tests. TestNG Author C├ędric Beust had written about it few years ago. When I had begun writing selenium tests I used to launch browser once in one test class and test methods were usually dependent on one another. Hence 2nd test method would begin test run on same browser + application state where 1st test method stopped. Launching just one browser for five or more test methods in a class is big time saving is not it? Well not really, for the reasons I am going to describe in this post. Since then I have moved on to launching browser per test method but yet give in “charm” of dependent test methods at times. to So here are the few pros and cons for dependent tests and -

Pros -

  • Similar steps of execution don’t have to be repeated in multiple tests when steps constitutes a test method in themselves

  • If login fails then only login test should fail and not other tests which are dependent on login feature. Tests which are dependent on login feature should be should be skipped and no time should be spent on execution of these test. This is a big saving on time when running end to end test.

There is a compelling example of dependent tests provided on the post I linked earlier. The example goes as -

Say I want to test a JDBC driver I have written by running 100 queries and verifying the results. Say also that for whatever reason, establishing a database connection and ensuring the sample data is there takes a fairly long time (say 30 seconds) relative to the query time (0.1 ms).
Under the JUnit philosophy, I have two choices
1) Establish the DB connection in the setUp() method and have a test* method for each of the 100 queries. GOOD: fine-grained test cases BAD: the tests take nearly an hour to run.
2) Do everything in a single test* method. GOOD: the test takes less than a minute to run. BAD: coarse-grained test cases (just one)
Are you really prepared to argue that I must choose between these approaches simply to get a warm fuzzy feeling that my test methods are independent? As Mr. Boyens quite rightly points out above, this does nothing more than allow you to be sloppy in writing your tests. Fortunately, though, I have other options:
3) Store the DB connection in a static variable in my test class and initialize it only during the first test case. GOOD: fine-grained tests AND they run in 60 seconds BAD: it’s a gross hack but JUnit gives me no choice.
4) Use TestNG and do the DB setup in a separate test case on which all the other tests depend. GOOD: even more fine-grained test cases that run in seconds and without a hack. BAD: I still find myself arguing with people who inexplicably insist on the inherent evil of dependent test methods.
The bottom line is that when you’re testing real world systems, you simply have to be able to use the postconditions of some tests as preconditions for others. It is naive to insist that every single test start from scratch in building up the state of the system under test. And you’re a lot better off if your test framework realizes this and has formal mechanisms to help you cope with it.

This approach seems good to me but not the implementation. Instead of creating a dependent method I would have created a setup method applicable for only this test class and have it run only once before first test method in class. I described a similar approach later in the article though in that example setup method is run before each test method and not just once.

Cons -

  • Dependent tests make debugging difficult as you can see from following examples -
    • Test 4 dependent on Test 3 dependent on Test 2 dependent on Test 1
Now it is not easy to read one test and understand what it is doing, what interactions it is testing and what internal states it verifies

  • What if you just want to run one/many failed tests. Here in dependent tests would be run firsts and assertion carried out in the dependent tests would slow down the overall test execution

  • One test failure would hide another test failures. Consider that your login test asserts that after login user sees >. What if element locator of > is changed hence login test fails. If you are using TestNG then all the tests dependents on login test would be skipped. This does not seem right. The failure in login test is not because of application bug but changes which are to be fixed on element locator. But herein dependent test would also stop further test execution and may hide more defects (if any) down the test execution line.
    • Now you can argue what if element locator of login textbox changed, now login test as well as any other test which is dependent in login feature would fail. Do  you see my point how this is an endless argument.

So how do I not repeat the same steps / page object methods in multiple test method? How about moving them to set up method like following -

Screenshot from 2016-05-30 11:47:00.png

This approach also gives an indication of cohesiveness of test methods. If test methods have different setup methods then probably they don’t belong to same test class? Or am I wrong in my assumption?

Whether you go ahead with dependent test would eventually boil down to if you and your team sees value in it. If you any way decide to use dependent tests then my suggestion would be to not use more than one level of dependency

What do you think of dependent tests? Would you write dependent tests?

Popular posts from this blog

Appium and android mobile app automation

Next appium and Android mobile app automation video tutoria l is live. If you are new to appium then please check - appium-tutorial This video tutorial covers - Start vysor (Just for this session and not mobile automation :)) Start appium and start appium inspector Desired Capabilities platformName - Android deviceName - L2N0219828001013 (as seen on "adb devices") Saved Capability Sets Start Session Scan app elements using appium inspector Get appPackage and appActivity using "APK info" app Install "APK info" app and open app whose appPackage and appActivity are required i.e. calculator Check top section of app icon is app package is app activity testng.xml file settings for running Android app tests Test details and CalculatorScreen class View beautiful STF test report  

Return only first or last element from webelements collection

Note: If you are new to java and selenium then start with selenium java training videos .     We often come across situation when there are multiple elements on a page and we probably like to exercise only a few of them using selenium webdriver. May be just first and last element. For example on a search result page we may like to click on only first and last link and not all. This is when Iterables API comes handy. (By the way I am assuming that you have already completed watching selenium training videos :)). Once we have collection of web element then we can use Iterables to get only first or last element as following - Consider that we fetch collection of element as - List< WebElement > webElements = getDriver().findElements(By. id ( "htmlID" ));   Now we can get the first web element from this collection as -  WebElement firstElement = Iterables. getFirst (webElements,  getDriver().findElement(By. id ( "defaultElement" )));   Herein second

Using chrome console to test xPath and css selectors

Note: If you are new to java and selenium then start with selenium java training videos .       Since the advent of selenium there have been many plugin to test xPath / css selectors but you don’t need any of them if you have chrome browser. Using Chrome console you can test both xPath and css selectors. Launch website to be tested in chrome browser and hit F-12 and you would see chrome console opened in lower pane of application - Hit escape key and console would open another pane to write element locators - And now you can start writing xPath or css selectors in chrome console and test them - The syntax for writing css id - $$(“ ”) And hit the enter key. If your expression is right then html snippet of the application element corresponding to the css selector would be displayed - If you mouse over the html snippet in chrome console then it would highlight the corresponding element in application - If you want to clean console of previously wri