Skip to main content

Capture network traffic using WebDriver

We often come across testing requirements when we need to analyze the network traffic to find -
    • HTTP status of page
    • Analyze header information to find if right information is passed
    • Validating parameters related to ajax requests etc
Selenium 1 has had a way to capture n/w traffic but the feature does not always work as expected. At times Selenium 1 does not capture all n/w traffic, And given that Selenium 1 APIs are almost dead it is
worth using solutions available for Selenium 2 (WebDriver). I have come across two solution of capturing n/w traffic with WebDriver -
  • Browser Mob Proxy (BMP) - which acts as proxy server b/w WebDriver and your application, captures all the transactions b/w browser and application and creates a HAR file. Sounds interesting? But I came across a few issues using BMP - 


    1. Test execution goes sluggish because of using proxy server
    2. Test steps go abrupt at times, for example you may have to click same button twice/thrice to perform operation
    3. BMP does not support capturing n/w traffic against application running on localhost (at least not at the time of writing this post.) 
If you get to work with BMP then bigger advantage is its capability of capturing network traffic across all browsers. But if your test require you to use only Firefox then there is more elegant solution available, that is to use 
Firebug is as popular tool among selenium users as it is among developers. NetExport is add on to firebug which captures network traffic in form of HAR files. HAR file is json based file, refer HAR specification for more. NetExport can export all the n/w traffic in HAR format as detailed in image here












Manual operations of exporting HAR file can be automated using WebDriver.
WebDriver can easily configure firefox plug-in and step through the application to get the required HAR file generated. Refer this excellent example of configuring Firefox driver with firebug and netexport add ons. Once you get the HAR file then HarLib can be used to fetch the required data from HAR file.

Comments

  1. Hi, I tried this and it is working fine for the normal sites like facebook and so but when i am running this for my company site walmart.com then its not generating .har file.

    Can you please help me with this? I am stuck here.

    ReplyDelete
    Replies
    1. Is there some ajax driven operation for which you are gathering data? If so then ajax has been pain to capture n/w traffic. If it is a normal page then should have worked. In either case, I would ask you to add more details.

      Delete
  2. Hi Tarun,

    After adding some more preferences like
    profile.setPreference(domain + "currentVersion", "2.0");
    profile.setPreference(domain + "allPagesActivation", "on");
    profile.setPreference(domain + "defaultPanelName", "net");
    profile.setPreference(domain + "net.enableSites", true);

    profile.setPreference(domain + "addonBarOpened", true);
    profile.setPreference(domain + "console.enableSites", true);
    profile.setPreference(domain + "script.enableSites", true);
    profile.setPreference(domain + "previousPlacement", 1);
    profile.setPreference(domain + "allPagesActivation", "on");
    profile.setPreference(domain + "onByDefault", true);

    Now i am able to get har file of www.walmart.com but still for websites like www-e4.walmart.com(which is internal test env, replica of production site ), not able generate har file.

    Architecture and design wise both are same, so could it be some security related issue.

    ReplyDelete
    Replies
    1. Does not seem any thing explicitly wrong. Could you post this question here - https://groups.google.com/forum/#!forum/firebug
      This is official forum for firebug.

      Delete

Post a Comment

No spam only genuine comments :)

Popular posts from this blog

How can you save resources when instantiaing driver?

I asked on my previous post about what was wrong in instantiating driver on set up method? And here is the solution video on my YouTube channel (After 5 years of gap I finally added new video tutorial :)) The solution described on video tutorial uses following set up - public class BaseClassOnDemandDriverSetup { private WebDriver driver ; @BeforeMethod public void setupTest () { // Any other set up goes here } @AfterMethod public void teardown () { if ( driver != null ) { driver .quit() ; } } public WebDriver getDriver () { if ( driver == null ) { WebDriverManager. chromedriver ().setup() ; driver = new ChromeDriver() ; } return driver ; } }

Using chrome console to test xPath and css selectors

Since the advent of selenium there have been many plugin to test xPath / css selectors but you don’t need any of them if you have chrome browser. Using Chrome console you can test both xPath and css selectors. Launch website to be tested in chrome browser and hit F-12 and you would see chrome console opened in lower pane of application - Hit escape key and console would open another pane to write element locators - And now you can start writing xPath or css selectors in chrome console and test them - The syntax for writing css id - $$(“ ”) And hit the enter key. If your expression is right then html snippet of the application element corresponding to the css selector would be displayed - If you mouse over the html snippet in chrome console then it would highlight the corresponding element in application - If you want to clean console of previously written element selectors then just hit ctrl+L keys and chrome console would be empty again. Pro

Using xPath to reach parent of an element

Well, I am big fan of css locator my self there are times when css locators don't fit requirement. One such requirement is when you want to navigate to parent element of an element and may be parent of parent and even more. Unfortunately css locators don't provide any mechanism to navigate to parent of an element. See this for more. Of late I came across a scenario when I wanted to click on a link depending upon the text in a text box. Herein parent of text box and parent of link were at the same location. More over there could have been many such combinations in application. Fortunately I just need to pick first such instance and Web Driver any way considers only first instance when multiple locators are found matching an element. Element in question is in following html - Here I need to click on highlighted anchor on the basis of input element (which is also highlighted in image) Herein first I need to reach div parent (class = 'left couponmainarea