Skip to main content

Should selenium tests be always independent? The endless debate !!

Though unit tests have always recommended to be independent of each other, advent of selenium+TestNG has given birth to dependent integration/end to end tests. TestNG Author C├ędric Beust had written about it few years ago. When I had begun writing selenium tests I used to launch browser once in one test class and test methods were usually dependent on one another. Hence 2nd test method would begin test run on same browser + application state where 1st test method stopped. Launching just one browser for five or more test methods in a class is big time saving is not it? Well not really, for the reasons I am going to describe in this post. Since then I have moved on to launching browser per test method but yet give in “charm” of dependent test methods at times. to So here are the few pros and cons for dependent tests and -

Pros -

  • Similar steps of execution don’t have to be repeated in multiple tests when steps constitutes a test method in themselves

  • If login fails then only login test should fail and not other tests which are dependent on login feature. Tests which are dependent on login feature should be should be skipped and no time should be spent on execution of these test. This is a big saving on time when running end to end test.

There is a compelling example of dependent tests provided on the post I linked earlier. The example goes as -

Say I want to test a JDBC driver I have written by running 100 queries and verifying the results. Say also that for whatever reason, establishing a database connection and ensuring the sample data is there takes a fairly long time (say 30 seconds) relative to the query time (0.1 ms).
Under the JUnit philosophy, I have two choices
1) Establish the DB connection in the setUp() method and have a test* method for each of the 100 queries. GOOD: fine-grained test cases BAD: the tests take nearly an hour to run.
2) Do everything in a single test* method. GOOD: the test takes less than a minute to run. BAD: coarse-grained test cases (just one)
Are you really prepared to argue that I must choose between these approaches simply to get a warm fuzzy feeling that my test methods are independent? As Mr. Boyens quite rightly points out above, this does nothing more than allow you to be sloppy in writing your tests. Fortunately, though, I have other options:
3) Store the DB connection in a static variable in my test class and initialize it only during the first test case. GOOD: fine-grained tests AND they run in 60 seconds BAD: it’s a gross hack but JUnit gives me no choice.
4) Use TestNG and do the DB setup in a separate test case on which all the other tests depend. GOOD: even more fine-grained test cases that run in seconds and without a hack. BAD: I still find myself arguing with people who inexplicably insist on the inherent evil of dependent test methods.
The bottom line is that when you’re testing real world systems, you simply have to be able to use the postconditions of some tests as preconditions for others. It is naive to insist that every single test start from scratch in building up the state of the system under test. And you’re a lot better off if your test framework realizes this and has formal mechanisms to help you cope with it.

This approach seems good to me but not the implementation. Instead of creating a dependent method I would have created a setup method applicable for only this test class and have it run only once before first test method in class. I described a similar approach later in the article though in that example setup method is run before each test method and not just once.

Cons -

  • Dependent tests make debugging difficult as you can see from following examples -
    • Test 4 dependent on Test 3 dependent on Test 2 dependent on Test 1
Now it is not easy to read one test and understand what it is doing, what interactions it is testing and what internal states it verifies

  • What if you just want to run one/many failed tests. Here in dependent tests would be run firsts and assertion carried out in the dependent tests would slow down the overall test execution

  • One test failure would hide another test failures. Consider that your login test asserts that after login user sees >. What if element locator of > is changed hence login test fails. If you are using TestNG then all the tests dependents on login test would be skipped. This does not seem right. The failure in login test is not because of application bug but changes which are to be fixed on element locator. But herein dependent test would also stop further test execution and may hide more defects (if any) down the test execution line.
    • Now you can argue what if element locator of login textbox changed, now login test as well as any other test which is dependent in login feature would fail. Do  you see my point how this is an endless argument.

So how do I not repeat the same steps / page object methods in multiple test method? How about moving them to set up method like following -

Screenshot from 2016-05-30 11:47:00.png

This approach also gives an indication of cohesiveness of test methods. If test methods have different setup methods then probably they don’t belong to same test class? Or am I wrong in my assumption?

Whether you go ahead with dependent test would eventually boil down to if you and your team sees value in it. If you any way decide to use dependent tests then my suggestion would be to not use more than one level of dependency

What do you think of dependent tests? Would you write dependent tests?

Popular posts from this blog

How can you save resources when instantiaing driver?

I asked on my previous post about what was wrong in instantiating driver on set up method? And here is the solution video on my YouTube channel (After 5 years of gap I finally added new video tutorial :)) The solution described on video tutorial uses following set up - public class BaseClassOnDemandDriverSetup { private WebDriver driver ; @BeforeMethod public void setupTest () { // Any other set up goes here } @AfterMethod public void teardown () { if ( driver != null ) { driver .quit() ; } } public WebDriver getDriver () { if ( driver == null ) { WebDriverManager. chromedriver ().setup() ; driver = new ChromeDriver() ; } return driver ; } }

Using chrome console to test xPath and css selectors

Since the advent of selenium there have been many plugin to test xPath / css selectors but you don’t need any of them if you have chrome browser. Using Chrome console you can test both xPath and css selectors. Launch website to be tested in chrome browser and hit F-12 and you would see chrome console opened in lower pane of application - Hit escape key and console would open another pane to write element locators - And now you can start writing xPath or css selectors in chrome console and test them - The syntax for writing css id - $$(“ ”) And hit the enter key. If your expression is right then html snippet of the application element corresponding to the css selector would be displayed - If you mouse over the html snippet in chrome console then it would highlight the corresponding element in application - If you want to clean console of previously written element selectors then just hit ctrl+L keys and chrome console would be empty again. Pro

Using xPath to reach parent of an element

Well, I am big fan of css locator my self there are times when css locators don't fit requirement. One such requirement is when you want to navigate to parent element of an element and may be parent of parent and even more. Unfortunately css locators don't provide any mechanism to navigate to parent of an element. See this for more. Of late I came across a scenario when I wanted to click on a link depending upon the text in a text box. Herein parent of text box and parent of link were at the same location. More over there could have been many such combinations in application. Fortunately I just need to pick first such instance and Web Driver any way considers only first instance when multiple locators are found matching an element. Element in question is in following html - Here I need to click on highlighted anchor on the basis of input element (which is also highlighted in image) Herein first I need to reach div parent (class = 'left couponmainarea