Please post your Web Driver questions in official Web Driver forum

Thursday, May 28, 2015

Troubleshooting

Update chrome driver binaries -
At times chrome browser on real device may not work and can be fixed by updating chrome driver binaries which are downloaded by appium. download latest chrome driver binaries from -
And replace them in your appium download location, The location looks as following on my Ubuntu - /home/tarun/node-v0.12.2-linux-x64/lib/node_modules/appium/build/chromedriver/linux
Using Chrome browser on non rooted devices -
If you have a non rooted device then chrome would work for only version higher than 33, so if you using a version of chrome lower than 33.0 then upgrade it to higher version. This may be far easier than rooting the device :-) Moreover since chrome browser update automatically on Android hence you would like to test on latest version of chrome browser.
Killing a running instance of adb -
If you come across following error when executing tests -
org.openqa.selenium.SessionNotCreatedException: A new session could not be created. (Original error: Command failed: /bin/sh -c /home/tarun/Documents/mine/lib/android-sdk-linux/platform-tools/adb -s adb server is out of date.  killing... wait-for-device ADB server didn't ACK * could not start server *)
then you may already have adb server up and running and you would have kill it. kill adb from command line as following -
killall -9 adb

Executing appium tests on mobile web

Having set up appium, lets run our first tests. But before that start the appium and android device /  emulator as following -

navigate to appium installation, which is following on my Ubuntu -
/home/tarun/node-v0.12.2-linux-x64/bin
and start appium
./appium
Open another command prompt and execute -
adb devices
and you should see list of devices attached as -
List of devices attached
emulator-5554    device
If you see any error message then re-verify all the steps mentioned earlier to make sure none of the step is missed

Sample JUnit project to run tests on android emulator
  • Clone the following appium project -
git@github.com:tarun3kumar/appium-tests.git
  • Import project in IDEA (or your favorite IDE) and Open AndroidEmulatorWebTest class -
AndroidTools.png

  • Right click on method testMobileWebOnEmulator and click Run. This would execute test on android emulator -
AndroidTools.png

Sample JUnit project to run tests on android emulator

Use the same project but AndroidDeviceWebTest class to run test on mobile device. Of course you need to have device setup as explained in previous steps

Set up an android device to run appium tests

While emulators are cheap solutions to run automated tests on mobile device. They can not emulate behavior of real user. if your requirement is to run tests on a mobile chrome browser then you are better off using android device since chrome browser can not be installed on emulator.

Following configurations are required to be able to run tests on mobile device -

  • Download appropriate version of chrome browser on your device
  • Enable to Developer Mode on device. For ex, to enable developer mode on moto-4.4.4 do following -

Tap  moto-g-apps
Tap Settings
Tap  About phone
Tap the Build number field 7 times
You will begin seeing a message as you approach the 7 touches
moto-g-developer-options
Tap the back arrow once complete, and Developer options will now appear under Settings. Once the developer options are enabled under settings in Moto G, it will be there permanently. Hence it is not necessary to repeat these procedures again.You can also disable the developers options if you want.  
Once you tap developer options under settings, you can drag it to left to disable the options.
  • Disable/uncheck “Verify Apps” settings, else it can prevent some of Appium’s helper apps from launching and doing their job correctly. To disable “Verify Apps” settings select  HOME > Security – PERSONAL > and uncheck the Verify app checkbox

To connect device to adb -

Under Developer Option select “USB Debugging checkbox”
Note: When you connect a device running Android 4.2.2 or higher to your computer, the system shows a dialog asking whether to accept an RSA key that allows debugging through this computer. This security mechanism protects user devices because it ensures that USB debugging and other adb commands cannot be executed unless you're able to unlock the device and acknowledge the dialog. This requires that you have adb version 1.0.31 (available with SDK Platform-tools r16.0.1 and higher) in order to debug on a device running Android 4.2.2 or higher. (if you don’t see this option and after enabling “USB debugging”, and command “adb devices” list your device(s) then check or uncheck “USB debugging” a few time, this how it worked for me :-))
When plugged in over USB, you can verify that your device is connected by executing adb devices from your SDK platform-tools/ directory. If connected, you'll see the device name listed as a "device."

Appium and Android setup

Android Requirements
  • Android SDK API >= 17 (Additional features require 18/19)
  • Appium supports Android on OS X, Linux and Windows. Follow these directions for setting up your environment properly for testing on different OSes:
Android setup does not differ greatly on different operating system. Let’s look at detailed setup instructions for Ubuntu  -

Install node.js (v0.10 or greater)

Setup with Ubuntu: curl -sL https://deb.nodesource.com/setup | sudo bash -
Then install with Ubuntu: sudo apt-get install -y nodejs


DON’T install node js using above method, it would result in following error when starting appium -
error: Appium will not work if used or installed with sudo. Please rerun/install as a non-root user. If you had to install Appium using `sudo npm install -g appium`, the solution is to reinstall Node using a method (Homebrew, for example) that doesn't require sudo to install global npm packages.

To fix it -

  • Download latest nodejs linux binaries (or windows, mac if you are not on linux) form http://nodejs.org/download/
  • Extract into a folder that doesn't need sudo rights to access, for example your home folder.
tar -xvf <downloaded_binary_tar.gz>
{extracted file looks as following in my home directory -
/home/tarun/node-v0.12.2-linux-x64}
  • Add the following line to your ~/.bashrc file.
export PATH=$PATH:<full_path_of_the_extracted_node_folder>/bin
refresh bashrc so you don’t have to logout and login -
source ~/.bashrc
  • Open a now terminal and execute
npm install -g appium
    {appium installation looks as following on my ubuntu -
/home/tarun/node-v0.12.2-linux-x64/bin}
  • Now start appium as -
cd /home/tarun/node-v0.12.2-linux-x64/bin
./appium
If everything goes right then you would have appium server up and running as following  
info: Welcome to Appium v1.3.7 (REV 72fbfaa116d3d9f6a862600ee99cf02f6d0e2182)
info: Appium REST http interface listener started on 0.0.0.0:4723
info: Console LogLevel: debug

Install Android SDK

  • Install android sdk from -

Unpack the .zip file you've downloaded. The SDK files are downloaded separately to a user-specified directory

By default, Android SDK does not include everything required to start development. The SDK separates tools, platforms, and other components into packages that can be downloaded as needed using the Android SDK Manager. Hence we must first download a few packages and add to Android SDK.
  • Downloading additional packages and latest SDK tools. launch the Android SDK Manager.
Open a terminal and navigate to the tools/ directory in the location where the Android SDK was installed, then execute android sdk.
For ex it looks as following on my ubuntu -
cd /Documents/mine/lib/android-sdk-linux/tools$
./android &

  • As a minimum when setting up the Android SDK, you should download the latest tools and Android platform:
    1. Open the Tools directory and select:
      • Android SDK Tools
      • Android SDK Platform-tools
      • Android SDK Build-tools (highest version)
AndroidTools.png
    1. Open the first Android X.X folder (the latest version) and select:
      • SDK Platform
      • A system image for the emulator, such as ARM EABI v7a System Image
  1. AndroidTools.png
3. Get the support library for additional APIs
The Android Support Library provides an extended set of APIs that are compatible with most versions of Android.
Open the Extras directory and select:
    • Android Support Repository
    • Android Support Library
  • AndroidTools.png
  • Once you've selected all the desired packages, continue to install:

  1. Click Install X packages.
  2. In the next window, double-click each package name on the left to accept the license agreement for each.
  3. Click Install.

The download progress is shown at the bottom of the SDK Manager window. Do not exit the SDK Manager or it will cancel the download.

Creating new emulator
Android emulator can be created from command line or using AVD (Android Virtual Device manager). AVD can be launched from Android SDK Manager as following -
Android SDK Manager > Tools > Manage AVDs  - launches Android Virtual Device (AVD) manager
You can create a new device or select any of the existing Device Definitions, for example select Nexus 5 by Google and click Create AVD button

AndroidTools.png
Select Target, CPU/ABI, Skin and click ok to set up new virtual device. When you click Ok then device would be listed under “Android Virtual Devices” tab -
AndroidTools.png

Select device > click Start button and Launch the device. This would launch Android emulator -
AndroidTools.png

NOTE: if Android emulator opens a blank screen then select Emulation Options > Use Host GPU and start emulator again. This would also increase the emulator performance drastically.

Setup ANDROID_HOME variable

Finally, set $ANDROID_HOME to be your Android SDK path. This looks as following on my ubuntu -
  • export ANDROID_HOME=/home/tarun/Documents/mine/lib/android-sdk-linux
  • export PATH=$PATH:/home/tarun/Documents/mine/lib/android-sdk-linux/tools
  • and then execute source ~/.bashrc
Install Android Debug Bridge (adb) -
Execute following command to install adb -
sudo apt-get install android-tools-adb
Android Debug Bridge (adb) is a command line tool that lets you communicate with an emulator or android device. It is a client-server program and includes three components:
  • A client, which runs on development machine. You can invoke a client from a shell by issuing an adb command. Other Android tools such as the ADT plugin and DDMS also create adb clients. For example if you want to see list of emulator/devices running then execute following command -
adb devices
  • A server, which runs as a background process on development machine. The server manages communication between the client and the adb daemon running on an emulator or device.
  • A daemon, which runs as a background process on each emulator or device instance.
You can find the adb tool in <sdk>/platform-tools/ directory. It is available on following directory on my linux - /home/tarun/Documents/mine/lib/android-sdk-linux/platform-tools

Appium Concepts

Client/Server Architecture
Appium (to be more precise Appium Server) is a web server which exposes a REST API. It performs following operations -
  • Receives connections from a client,
  • Listens for commands,
  • Executes those commands on a mobile device,
  • Responds with an HTTP response representing the result of the command execution.
Since Appium uses client server architecture, we can write our test code in any language that has a http client API, though it is easier to use one of the Appium client libraries. Appium server can be on a different machine than the one running test.
Session
Automation is performed in the context of a session. Clients initiate a session with a server specific to each library, but they all eventually send a POST /session request to the server, with a JSON object known as 'desired capabilities' object. At this point the server will start up the automation session and respond with a session ID which is used for sending further commands.


Desired Capabilities
Desired capabilities are a set of keys and values (like a map) sent to the Appium server. Desired capabilities tell server what kind of automation session should be started. Capabilities doc lists the capabilities available for Appium.
Appium Server
Appium is a server written in Node.js. It can be built and installed from source or installed directly from NPM.
Appium Clients
Appium client libraries support Appium's extensions to the WebDriver protocol. When using Appium, Appium client libraries should be used instead of regular WebDriver client.
Appium.app and Appium.exe are GUI wrappers around the Appium server. Appium.app is for mac and Appium.exe is for window. These are bundled with everything required to run the Appium server, hence you need not worry about node. They also have element inspection capabilities, which lets you check the hierarchy of app. This facilitates writing tests. Unfortunately .app is not available for Ubuntu

Appium Philosophy

from http://appium.io/ Appium philosophy has four tenets -

  1. Application under test should not have to be modified to be able to run automated tests
  2. Automated tests should not be tied to one framework or programming language
  3. A mobile automation framework shouldn't reinvent the wheel when it comes to automation APIs.
  4. A mobile automation framework should be open source, in spirit and practice as well as in name!

Appium meets its philosophy in following ways -
  1. Appium satisfies its first Philosophy by using vendor-provided automation frameworks to control device. Hence Appium-specific or third-party code or frameworks do not have to be compiled with app under test. Appium describes it as - you're testing the same app you're shipping. The vendor-provided frameworks Appium uses are:

  1. Appium satisfies its second philosophy by wrapping the vendor-provided frameworks in one API, the WebDriver API. WebDriver specifies JSON Wire Protocol which is a client-server protocol. Hence a client written in any language can be used to send the appropriate HTTP requests to the server. Webdriver supports clients written in many popular programming languages. Hence you are free to use whichever test framework you want. for example JUnit or Testng with java, NUnit with C#, PHP unit with PHP etc; The client libraries are HTTP clients and can be mixed into your code any way you want. Hence like, WebDriver, Appium is also not a “test frameworks" but rather an "automation libraries". And it is left to you how you want to write tests.

  1. Appium satisfies third philosophy by using WebDriver. Web driver has become the de facto standard for web browsers automation and is a W3C Working Draft. Appium  extended the protocol with extra API methods useful for mobile automation.

  1. Appium satisfies its fourth philosophy by being open source :-)

Appium Architecture

Appium is an HTTP server which is written in Node.js that creates and handles multiple WebDriver sessions. Appium listens for commands from Appium server and starts tests on the device. It is same as the Selenium server, where in selenium server gets HTTP requests from Selenium client libraries.
Like selenium tests, appium tests are also written once and can be run on multiple devices (well almost). This primarily entails four entities -

  • Tests, written in one of the appium supported languages
  • Appium server (also known as test server), which interacts with device (or emulator) running the test
  • Device (or emulator) to run tests on
  • Machine from where tests are run

This could be visualized from following Appium Architecture Image
Herein one machine can be used to run both appium server and test. In Appium video tutorial we would be running appium server on the same machine on which tests are executed. Test code performs HTTP calls to server which then translates those commands into device specific actions sent to device.  These commands are user interactions to the device for ex - clicking on button, swiping the screen, scrolling in to area etc

Appium Supported Languages

Appium tests could be written in following languages. These are the extensions of WbDriver’s client binding and you would have to install the binding of your favorite language. (or use maven dependency in java which we would see later)

C# - https://github.com/appium/appium-dotnet-driver

Appium Requirements

The requirements for running appium tests for iOS are Mac OSX 10.7+, XCode 4.5+ w/ Command Line Tools and requirement for for android are Mac OSX 10.7+ or Windows 7+ or Linux, Android SDK ≥ 16 (SDK < 16 Selendroid mode is used in appium to execute tests)
and of course iOS and android emulators. And of course you also need -

  • Appium Latest version
  • Xcode latest version
  • Eclipse / IDEA (or any other java IDE you prefer)

Is there selenium for mobile devices? Yes that is called Appium :-)

If you have been working on selenium from then you may have come across AndroidWebDriver and Selendroid. Though both of them are limited to only Android devices. AndroidWebDriver has been deprecated in favor of Selendoid. But what if there were a library which can be used for both android and iOS devices or emulators. And this is what appium does.

Appium is an open-source library which can be used to automate native, mobile web, and hybrid applications for iOS and Android platforms. Native apps are created using iOS or Android SDKs. Mobile web apps are web apps accessed on a mobile browser. Hybrid apps are hosted inside a native application that utilizes a mobile platform’s WebView.
Appium supports Safari on iOS phone and  and Chrome or the built-in 'Browser' on Android phone. Tests can also be run on Android and iOS emulator, but well emulators are emulators can not replace real device. So if you have real device at your disposal then avoid emulators.

Saturday, May 16, 2015

Regular Expression extractor and JMeter

When using JMeter for performance testing of RESTful services, you may come across situations when you have to extract response from one request and pass it on to another request. This can be accomplished using regular expression extractor. You may like to watch JMeter training video on regular expression extractor if you have not already :-)

This post illustrates an example of extracting parameter from json expression without using any JMeter plugin. The drawback of using plugin is that you would have to make it available on each JMeter installation where you want to run the test.

Let's consider following JSON expression - 


   "access_token":"18b8b379-b875-467a-9f22-37b60024c1b9",
   "uid":"test user",
   "grant_type":"password",
   "scope": 
      "uid",
      "cn"
   ],
   "realm":"employees",
   "cn":"",
   "token_type":"Bearer",
   "expires_in":3599
}

Let's assume that we want to fetch value of access_token from this expression.
Then following regular expression can be used - 

"access_token":"(.+?)"

Let's look at these special characters one by one -

1 opening and closing parenthesis () enclose the portion of matched string to be returned
2 . means match any character
3 + means one or more times
4 ? means stop when first match occurs

without ?, the .+ would continue beyond first quotation " until it finds last quotation " and indeed this is not what we want.

For this specific example it is more efficient to use following expression - 
"access_token":"([^"]+)"
Herein [^"] means match everything except "
And matcher stops as soon as it finds first quotation "

Did you know that you can also test regular expression from with in JMeter? Run your test with 1 user and open results in ViewResultsInTree listener. ViewResultsInTree listener has a drop down to show results in different views. Select RegExpTester, paste your regular expression and hit Test button :-)



You may also like to see JMeter guide on Regular expression

9th JMeter Training Video - HTTP Cache Header

This is 9th JMeter training video Session. You may like to watch all previous JMeter training video sessions before continuing with this video. This session covers -

HTTP Cache Headers -

Response Headers -
Cache-control  
Expires
Last-Modified
ETag

Request Headers - 
            If-Modified-Since 
          If-None-Match


9th JMeter training video session can be watched online.

ShareThis

Fork me on GitHub