QTP VS Selenium Webdriver and Cucumber

My company was using QTP to do the test automation for the past 3-4 years. But we are now moving to cucumber and selenium Webdriver. This blog is about why we are moving away from QTP in my company.

One of the main reason that we move from QTP is that it is a reactive approach of writing test automation. The product is finished and the tester starts writing the automation test. Only the tester writes the automation. With cucumber and selenium , writing automation test is a joint effort between the developers and the testers. The testers write the test cases in cucumber scenario format, then the developers write the steps definition. This way, everyone is contributing and everyone is reviewing the test cases.

In our case, our QTP automation tester left the company and the QTP vbs code is hard to maintain and to read compared to cucumber and ruby code. I have tried to read QTP code to try to convert the test to cucumber and it is just taking quite a lot of time to just understand what the scripts do. With cucumber, selenium webdriver reading what the tests do only take 1-2 minutes as cucumber scenarios describe the behaviour of the system. With cucumber and selenium, everyone is contributing to the automation code and therefore knowledge is being transferred all the time.


QTP is very costly while cucumber, selenium webdriver is free.

Is there any version control in QTP? In our case, we put the automation test code (cucumber, selenium webdriver) as part of the system source code so the automation code is sourced controled.

We can develop the automation code in Windows, Ubuntu, Mac or in any other platforms that you like. This is another topic, but in our case everyone is moving away from windows as well especially the developers which is why QTP is abit useless as you can use Windows only.

Continous Integration is also an ease. We use hudson to automatically run the automation test whenever there is a new build so the developers can get an instant feedback of their changes.

Selenium webdriver can do alot more tests than the previous version (selenium rc) including java script requests and ajax test.

Getting a support answer is very fast in the open source community. Also, you have access to the source code. Since it is a joint effort to write the automation test, we are able to solve alot of issues together.

Other IT departments in our company are already ditching QTP in favor of cucumber and selenium and we never want to go back to QTP after this.

I believe QTP have other benefits as well that I do not know. However, in our case using cucumber and selenium webdriver is more relevant and better than using QTP.

How to change .RubyMine config default directory location

I am using a JetBrains RubyMine ide to write and develop the test automation. RubyMine has provided alot of useful features for developing Cucumber in Ruby such as providing the function to easily find the steps definition from the cucumber scenario.

My work computer is using a desktop profile space which only allow me to store 2mb of data. Unfortunately, RubyMine by default stores all of the RubyMine config ".RubyMine20" in the default directory which is usually under your own folder. This RubyMine default directory happens to be in my desktop profile and I don't have the privilege to change/remove the desktop profile. This causes a lot of inconveniences for me since I have to delete the .RubyMine20 folder every time I shutdown my computer. It is very annoying.

I finally found out that you can change the .RubyMine20 default directory. You would need to change the "idea.properties" file which is usually located under your RubyMine\bin\idea.properties folder. In my case, it is located in "C:\JetBrains\RubyMine\bin\idea.properties".

Once you open the idea.properties file, you need to change the following variables:

idea.config.path="D:\your_new_directory\.RubyMine20\config"
idea.system.path="D:\your_new_directory\.RubyMine20\system"
idea.plugins.path="D:\your_new_directory\.RubyMine20\config\plugins"

WebDriver Error - Element is no longer attached to the DOM (Selenium::WebDriver::Error::ObsoleteElementError)

I have been getting this error message for a while now and it is very annoying. The reason it is very annoying is because the error is not always reproducible. The ObsoleteElementError only happens about 1 out of 8-10 times I run my test.

After tracing down the errors I found out the code that causes the error:

find(:css, "#travel_control").click
find(:xpath, ".//*[@id='field_container']/a[#TRAVEL_MODE_INDEX[travel_mode]}]").click

It seems that Web driver could not find the #travel_control element and therefore it throws the ObsoleteElement Error. The #travel_control element in my test is actually an ajax call on a popup box so probably WebDriver still has some issue in handling elements with ajax request and javascript.

Nevertheless, I found some help from google groups regarding this issue. There does not seem to be a solution in that thread but there is one comment regarding a temporary fix for this issue which is to ignore the error and execute the same code again.

So I tried it and changed my code into:

begin
   find(:css, "#travel_control").click
   find(:xpath, ".//*[@id='field_container']/
     a[#{TRAVEL_MODE_INDEX[travel_mode]}]").click
rescue Selenium::WebDriver::Error::ObsoleteElementError 
   find(:css, "#travel_control").click
   find(:xpath, ".//*[@id='field_container']/
     a[#{TRAVEL_MODE_INDEX[travel_mode]}]").click
end 

However, This does not work for me and instead I get this error


Element is not currently visible and so may not be clicked (Selenium::WebDriver::Error::ElementNotDisplayedError)


It turns out because when I get the ObsoleteElementError, WebDriver actually is able to click the "#travel_control element. so what I need to change is to not click the same element again. Here is how I fix my test:

begin
  find(:css, "#travel_control").click
  find(:xpath, ".//*[@id='field_container']/a[#{TRAVEL_MODE_INDEX[travel_mode]}]").click
rescue Selenium::WebDriver::Error::ObsoleteElementError 
  #just execute the next element
  find(:xpath, ".//*[@id='field_container']/a[#{TRAVEL_MODE_INDEX[travel_mode]}]").click
end 

Error When installing Nokogiri gem - 'no such file to load -- mkmf (LoadError)'

So basically when I try to install nokogiri gem using 'gem install nokogiri', I get the following error:

root@ubuntu:/home# gem install nokogiri
Building native extensions.  This could take a while...
ERROR:  Error installing nokogiri:
 ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb
extconf.rb:5:in 
 `require': no such file to load -- mkmf (LoadError)
    from extconf.rb:5


Gem files will remain installed in 
/usr/lib/ruby/gems/1.8/
  gems/nokogiri for inspection.
Results logged to 
/usr/lib/ruby/gems/1.8/
  gems/nokogiri/ext/nokogiri/gem_make.out

I was able to successfully install nokogiri on Windows and Mac so I was confused as to why it is failing in Ubuntu. I found out that you need to install the ruby1.8-dev to make it work.

sudo apt-get install ruby1.8-dev

If that still does not work try to install the following as well:

sudo apt-get install libxml2 libxml2-dev libxslt1-dev
sudo gem install nokogiri

Then just run the gem install nokogiri:

root@ubuntu:/home# gem install nokogiri -v 1.4.3.1
Building native extensions.  This could take a while...
Successfully installed nokogiri-1.4.3.1
1 gem installed
Installing ri documentation for nokogiri-1.4.3.1...
r
No definition for parse_memory

No definition for parse_file

No definition for parse_with

No definition for get_options

No definition for set_options
Installing RDoc documentation for nokogiri-1.4.3.1...

No definition for parse_memory

No definition for parse_file

No definition for parse_with

No definition for get_options

This above solution will also work if you get the following error message:

Could not open library 'xml2' : xml2: 
cannot open shared object file: No such file or directory. 
Could not open library 'libxml2.so' :
 libxml2.so: cannot open shared object file: No such file or directory (LoadError)

as mentioned on this blog

Upgrading to Capybara 0.4.0

So Capybara 0.4.0 is out now and if you do a 'gem install capybara', you will get Capybara 0.4.0 by default. This has byfar the most changes and not backward compatible with the previous version.

After upgrading to 0.4.0, my test started to fail with a very weird error message ("superclass mismatch for class Node (TypeError)"). It took me one whole day to familiarize myself with the new version which include looking at Capybara code to be able to modify my current test.

Here are what I have to do to upgrade to Capybara 0.4.0 from 0.3.9 in my project

1. Changed "class Node < Capybara::Node" to "class Capybara::Element < Capybara::Node"
See How to send keystrokes to an element in the browser
2. Changed the use of variable "node" to "native"
3. #locate is now deprecated, changed all of my use of "locate" into "find"
4. Changed the way we setup the capybara config

Before

Capybara.default_driver = :selenium
Capybara.run_server = false
Capybara.default_selector = :css
Capybara.default_wait_time = 30

After

Capybara.configure do |config|
  config.default_driver = :selenium
  config.default_driver = :selenium
  config.run_server = false
  config.default_selector = :css
  config.default_wait_time = 30
end

5. Changed the way we register the browser driver

Before

class Capybara::Driver::Selenium
  def self.driver
    unless @driver

      profile = Selenium::WebDriver::Firefox::Profile.new
      profile["network.proxy.type"] = 1
      profile["network.proxy.http"] = "proxy setting"
      profile["network.proxy.http_port"] = 8080
      @driver = Selenium::WebDriver.for(:firefox, :profile => profile)

      at_exit do
        @driver.quit
      end
    end
    @driver
  end
end

After

Capybara.register_driver :selenium do |app|
  profile = Selenium::WebDriver::Firefox::Profile.new
  profile["network.proxy.type"] = 1
  profile["network.proxy.http"] = "proxy setting"
  profile["network.proxy.http_port"] = 8080
  Capybara::Driver::Selenium.new(app, :browser => :firefox, :profile => profile)
end

How to send keystrokes to an element in the browser

From time to time you might need to test sending a keystroke to the browser. For example, for a map based application such as google map, you might want to test the arrow keys on the keyboard to test that pressing the arrow key will move the maps.

Fortunately, webdriver already implements the send-keys class and if you use capybara you can use this extensions by Mark Gandolfo. Please note that as of this writing, the current send-keys version in github does not support capybara 0.4.0 It only supports version < 0.4.0.

However, I have modified the code to work with capybara version 0.4.0 below:

Cucumber Scenario

  And I send arrow_left to google map
  And I send arrow_up to google map
  And I send arrow_right to google map
  And I send arrow_down to google map

Steps Definitions - send_keys_steps.rb
put this into your step_definitions directory

And /^I send (.*) to google map$/ do |key|
  find(:css,"#page").send(key)
end

Class definition - send_keys.rb
put this into your support/patches/send_keys.rb. If you use capybara version < 0.4.0, then change the "Class Capybara::Element < Capybara::Node" into "Node < Capybara::Node" and change the "native.send_keys(send_key)" to "node.send_keys(send_key)".

If you don't, then you will get this error:
"superclass mismatch for class Node (TypeError)"

class Capybara::Driver::Selenium < Capybara::Driver::Base
  class Capybara::Element < Capybara::Node
    def allowed_keys
      @allowed_keys ||= %q(option, null cancel help backspace 
tab clear return enter shift left_shift control left_control
alt left_alt pause escape space page_up page_down end home
left arrow_left uparrow_up right arrow_rightdown arrow_down
insert delete semicolon equals numpad0 numpad1 numpad2 numpad3
numpad4 numpad5 numpad6 numpad7 numpad8 numpad9 multiplyadd
separator subtract decimal divide f1 f2 f3 f4 f5 f6 f7 f8
f9 f10 f11 f12)
    end

    def send(key)
      send_key = []

      if key.match(/\[.*\]/i)
        key.gsub!(/[\[\]]/,'')
        key = key.split(',')
      else
        key = [key]
      end

      key.each do |k|
        if k.match(/(\'|\")/i)
          send_key << k.gsub(/(\"|\')/, '')
        elsif allowed_keys.include?(k)
          send_key << k.to_sym
        else
          send_key << "#{k}"
        end
      end

      native.send_keys(send_key)
    end
  end
end

Modify support/env.rb to include send_keys

 require 'features/support/patches/send_keys'

How to get Colour in JRuby for window

I have to use JRuby to run my cucumber automation from ruby recently since the developers are using ant to run the build. So I was able to setup the automation code to use JRuby. However, I had issue in running my test using JRuby since it does not give a nice cucumber colour onto my command prompt screen.

After looking around, I found out that I need to install WAC.exe, Windows ANSI Color, on my Windows machine.

All I need to do is to download wac.exe and put it in my PATH variables.

Before

D:\code\> ant cucumber -Dtest.env=production 

     [java]     ←[32mGiven I am on the google homepage←[90m
0m
     [java]     ←[31mAnd I search for "←[31m←[1m222 selenium webdriver←[0m←[0m←[31m" ←[90m
main_steps.rb:79←[0m←[0m
     .
     .
     .
     .

After: WAC.exe is able to interpret the ansi color

D:\code\> ant cucumber -Dtest.env=production | wac

   [java]     Given I am on the google homepage
   [java]     And I search for "Selenium Webdriver"
   [java]     When I click on the go button
    .
    .
    .
    .

How to get JSON responses using HTTP GET and POST request in ruby

JSON (JavaScript Object Notation) is a java script object notation that is used by computer to parse and generate data. It is a format that is quite widely used in some API implementation to exchange data. Last week I had to create an automation test to verify the JSON response result from an API. I eventually use 'net/http' gem to send the json request to the server and use json.parse to parse the JSON response data.

You would need to install "net/http" gem and "json" gem to use it. I created a HTTPResponse class to initialize and send the json request to the server.

class HTTPResponse
  def self.get url
    uri = URI.parse(URI::escape(url))
    http = Net::HTTP.new(uri.host,uri.port)
    headers={'content-type'=>'applications/json'}
    http.get(uri.request_uri, headers)
  end

  def self.post url, payload
    uri = URI.parse(URI::escape(url))
    headers={'content-type'=>'applications/json'}
    request = Net::HTTP::Post.new(uri.request_uri, header)
    request.body = payload
    Net::HTTP.new(uri.host,uri.port).start{|http| http.request(request)}    
  end
end

Then in your step definition you can just call the Get and Post method

GET

  payload = {
        "address" => customer_address,
        "cust_id" => customer_id"
      }.to_json
 
  url = 'http://www.example.com/web/json/newcust=#{payload}'
  @response, @body = HTTPResponse.post(url)

  p @response #will return httpok status
  p JSON.parse(@body) #will return a hash table which you can parse the value easily

POST

  url = 'http://www.example.com/web/json/newcust'


  payload = {
        "address" => customer_address,
        "cust_id" => customer_id"
      }.to_json
 

  @response, @body = HTTPResponse.post(url, payload)

  p @response #will return httpok status
  p JSON.parse(@body) #will return a hash table which you can parse the value easily

Debug your Web Test with ruby-debug

Ruby-debug has been giving me alot of help in debugging my cucumber web test. I use ruby-debug to pause my web testing so that I can check the variables values and to see what causes the test to fail on the websites.

To install ruby-debug gems, please visit this page

I created a simple steps definitions to invoke ruby-debug:

When /^I go into debug mode$/ do
  require 'ruby-debug'
  debugger
  1
end

Then, I can simply use the steps definition in any cucumber scenario.

Scenario: A simple google search scenario
  Given I am on the main google search
  And I search for "site:qastuffs.blogspot.com"
  And I click on the search button
  And I go into debug mode
  And I click on the first result
  Then I should land on the qastuffs blog

The above cucumber scenario will run the browser to google.com to do a search on site:qastuffs.blogspot.com. Then it will hit the debug mode and it will pause. You will see the following text "(rdb:1) on your console where you can print your test variables.

To see all of commands that you can run in ruby-debug, simply type in "h" and press enter

(rdb:1) h
ruby-debug help v0.10.3
Type 'help ' for help on a specific command

Available commands:
backtrace  delete   enable  help    next  quit     show       up
break      disable  eval    info    p     reload   step       var
catch      display  exit    irb     pp    restart  thread     where
condition  down     finish  list    ps    save     trace
continue   edit     frame   method  putl  set      undisplay

Ruby Devkit for windows native extensions

I receive the above error when trying to install ruby-debug in Windows.

gem install ruby-debug


C:\>gem install ruby-debug
Building native extensions.  This could take a while...
ERROR:  Error installing ruby-debug:
ERROR: Failed to build gem native extension.

C:/Ruby187/bin/ruby.exe extconf.rb
creating Makefile

make
'make' is not recognized as an internal or external command,
operable program or batch file.


Gem files will remain installed in 
C:/Ruby187/lib/ruby/gems/1.8/gems/linecache-0.43 for inspection.
Results logged to C:/Ruby187/lib/ruby/gems/1.8/gems/linecache-0.43/ext/gem_make.out

To fix it:
1. Download Devkit in the ruby website. Download the one under the "Development Kit"
2. Extract the file
3. run "ruby dk.rb init"
4. run "ruby dk.rb install"

Result:

D:\DevKit>gem install ruby-debug
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
Building native extensions.  This could take a while...
Successfully installed linecache-0.43
Successfully installed ruby-debug-base-0.10.3
Successfully installed ruby-debug-0.10.3
3 gems installed
Installing ri documentation for linecache-0.43...
Installing ri documentation for ruby-debug-base-0.10.3...
Installing ri documentation for ruby-debug-0.10.3...
Installing RDoc documentation for linecache-0.43...
Installing RDoc documentation for ruby-debug-base-0.10.3...
Installing RDoc documentation for ruby-debug-0.10.3...

For more information about dev-kit please visit the github page

Cucumber, Steps Definitions and Page Models

There are 3 different things involve in writing the automation test. The first one is about writing a cucumber scenario which is where the tester/BA writes the feature or acceptances test in plain text.

For more information about cucumber please visit the official sites of Cucumber which has heaps of information about cucumber. To learn how to write cucumber scenarios effectively please visit the Cucumber's Wiki page.

The second one is writing the steps definitions. This is where we define the code for our cucumber scenarios.

However, alot of times our steps definitions becomes cluttered with all of the xpath or css selections codes. Furthermore, if the web page changes the design, then we have to change all of our steps definition code to work with the new designs. This is where we need a page model design where we defined a particular web page design in a page model file. Gizmo is a really neat rubygems to help us write the page models. So instead of specifying the css or xpath selectors in the steps definitions, we just need to call the css or xpath selectors from the page models.

To see the implementation of page models, please see the "Setting up Cucumber" page where I use cucumber, steps definitions and page models in the project.

New Capybara version 0.4 coming soon

Apparently Capybara version 0.4 could be out this weekend. I have been using capybara version 0.3.9 and so far found no issues for my projects. I will try the new version soon and hopefully my project can run with no issues. One improvement in version 0.4 that might be beneficial for my project is that the CSS selectors can now support multiple selectors such as "h1, h2", previously it only recognises one selector.

So if you have the following code:

< div id='test_1' class='field_1 field_2' >.inner_text

Then you can only select @document.css(".field_1").inner_text in version 0.3.9. However, in version 0.4 I assume I should be able to do something like @document.css(".field_1, .field_2").inner_text.

The other bug fix that might make the test more stable is the fix for some obscure selenium bugs by changing the default from localhost to 127.0.0.1.

To see the complete changes to version 0.4.0.rc, please visit Capybara site.

How to setup Cucumber YAML Configuration File

Application configurations should be placed in one location and we can do this by using an external YAML file called config.yaml. On my blog on Setting up a cucumber project, I have not covered on how to create a config.yml file. This file is where you normally put your BASE_URL address and other configuration files for you to use in your projects.

For examples, you might want to run your test against a test, staging or production environments and for that you need to tell your code which url you want to test. So, here is how you write a config.yml file and how to use it.

I assume that you have the same project structure mentioned in here

1. Create a new config.yml file under features\support\config.yml

  google_australia:
    base_url: http://www.google.com.au

  google_singpaore:
    base_url: http://www.google.com.sg

2. Create a new "features\support\lib\configuration.rb" files to load the config.yml

require 'yaml'

class Configuration
  def self.[] key
    @@config[key]
  end

  def self.load name
    @@config = nil
    io = File.open( File.dirname(__FILE__) + "/../config.yml" )
    YAML::load_documents(io) { |doc| @@config = doc[name] }
    raise "Could not locate a configuration named \"#{name}\"" unless @@config
  end

  def self.[]= key, value
    @@config[key] = value
  end

end

raise "Please set the TEST_ENV environment variable" unless ENV['TEST_ENV']
Configuration.load(ENV['TEST_ENV'])

3. Add the following lines in your "features\support\env.rb" file

require File.dirname(__FILE__) + '/../lib/configuration';
BASE_URL = Configuration["base_url"]

4. Now modify "features\step_definitions\search_steps.rb"

Change

Given /^I am on the main google search$/ do
  visit "http://www.google.com"
end

to

Given /^I am on the main google search$/ do
  visit "#{BASE_URL}"
end

5. Now before you run the project you just need to set the TEST_ENV

set test_env=google_australia

This will run your test againts www.google.com.au. If you set the test_env=google_singapore then
your test will run againts www.google.com.sg

undefined method `camelize' for "default_search_form":String (NoMethodError)

I encountered that camelize error a few weeks ago. It was quite frustrating since I have not modified my code and the error was quite unfamiliar for me. After doing a bit more research and trial and error I found out the error is caused by activesupport 3.0.

undefined method `camelize' for "default_search_form":String (NoMethodError)

I used a ruby gem called Gizmo which is a simple page model testing framework. At that time, I was using Gizmo version 0.1.0 which was using activesupport 3.0. Apparently, activesupport 3.0 is not compatible with Gizmo or some other gems that I am using in my code.

To fix the issue I simply downgraded my activesupport gem to be 2.3.3. However, Gizmo gem has now been updated to 0.1.1 which uses the correct version of activesupport.

Testing Pop up windows using Selenium webdriver

There are times when we need to test a pop up windows within the current browser. For example of this is when you try to test a pop up box for the user to fill in the user details or a print pop up box.

The problem is that the webdriver will still focus on the current browser and not the pop up windows and therefore any interaction will only go to the browser and not to the new pop up browsers.

To solve this simply switch the browser to point to the pop up.

Here is the implementation for selenium webdriver in ruby

def switch_to_new_pop_up  
  response.driver.browser.switch_to.window
   (response.driver.browser.window_handles.last)
end

def close_active_window
 response.driver.browser.close  
 response.driver.browser.switch_to.window
   (response.driver.browser.window_handles[0])
end

Apt-get error in Ubuntu - 'Something wicked happened resolving...'

The Apt-get command in Ubuntu is a powerful tool to install application into the Ubuntu system. However, you may get the following error when trying to use the apt-get install command.

>sudo apt-get install ruby
Err http://us.archive.ubuntu.com/ubuntu/ lucid/main ruby 4.2
Something wicked happened resolving 'us.archive.ubuntu.com:http' 
(-5 - No address associated with hostname) 
Failed to fetch
http://us.archive.ubuntu.com/ubuntu/pool/main/r/ruby1.8/ruby1.8_1.8.7.249-2_i386.deb

This error is caused because you have not setup the correct proxy setting in your apt.conf.

To fix this issue simply add the following line into your /etc/apt/apt.conf file as setting up the proxy in the export http_proxy env variable won't solve this issue.

> sudo /etc/apt/apt.conf
Then add the following line into
>Acquire::http::Proxy "http://username:password@proxy:8080";

Changing User-agent profile in firefox through Selenium Webdriver

Firefox has an add-on called user-agent switcher which is used to switch the browser user agent so that the server thinks the query comes from a mobile devices.

There are times when we need to be able to switch user agent automatically from the cucumber test. This is when we need to test that the apache config can redirect the site correctly to the mobile URL.

We can do this by overriding the default firefox user agent profile to use the required user agent. Firefox has an attribute called general.useragent.override which we can put any user agent profile that we want.

Below is the code on how to do it.

class Capybara::Driver::Selenium
  def self.driver
    unless @driver
      profile = Selenium::WebDriver::Firefox::Profile.new

      profile["general.useragent.override"] 
              = "Mozilla/5.0 (iPhone; U; CPU iPhone" +
              " OS 3_0 like Mac OS X; en-us) AppleWebKit/528.18" + 
              "(KHTML, like Gecko) Version/4.0 Mobile/7A341 Safari/528.16"
    
      @driver = Selenium::WebDriver.for(:firefox, :profile => profile)
   
      at_exit do
        @driver.quit
      end
    end
    @driver
  end
end

If you want to see all of the availables firefox profile that we can override, you can type the following command on the firefox URL text field "about:config".

Please see below for user agent settings for iPhone, iPad, Nokia, Android:

iPad_UA=Mozilla/5.0 (iPad; U; CPU OS 3_2 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Version/4.0.4 Mobile/7B334b Safari/531.21.10

iPhone_UA=Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7A341 Safari/528.16

N95_UA=Mozilla/5.0 (SymbianOS/9.2; U; Series60/3.1 NokiaN95/10.0.010; Profile/MIDP-2.0 Configuration/CLDC-1.1 ) AppleWebKit/413 (KHTML, like Gecko) Safari/413

LG-G850=LG-G850 V100 UP.Browser/6.2.2 (GUI) MMP/1.0 Profile/MIDP-1.0 Configuration/CLDC-1.0

ANDROD_EMULATOR=Mozilla/5.0 (Linux; U; Android 1.0; en-us; generic) AppleWebKit/525.10+ (KHTML, like Gecko) Version/3.0.4 Mobile Safari/523.12.2

X10_UA="Mozilla/5.0 (Linux; U; Android 1.6; en-gb; SonyEricssonX10i Build/R1FA016) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1"

Adding Screenshot in cucumber html report

Cucumber has provided us a nice way to generate an HTML test report. We can simply run cucumber using the "-f html -out filename" to output the test result into the nicely formatted html report. It would be nice, however, to also automatically embed the failed scenario screenshot into the report as well. This is very useful if you want to run cucumber test in Hudson where you don't monitor how the test run and the log report is sometime not enough to determine whether the error is from the code or from a server error.

In the previous post about screenshot capture using capybara and webdriver, I have put the code on how to capture the screenshot by utilising the "After" hook method in cucumber. This time we will use the after hook as well as the at_exit method.

After do |scenario|

#need to write as a byte instead of IP#puts
if scenario.failed?
  name =  scenario.name.gsub!(' ', '').gsub!(/[\?\.\&=\-\/\\\(\),\'\"\|]/, '')
  File.open("./build/publish/screenshots/#{name}.jpeg", 'wb') do |f|
    f.write(Base64.decode64(page.driver.browser.screenshot_as(:base64)))
  end
  raise "lessthanimg src='screenshots/#{name}.jpeg' /greaterthan"
end
end

Notice that I have to deliberately raise an error in the code so that the img src code is thrown into the html report. Using the usual ruby "puts" will only print it to the console and not the report. I use the scenario.name for the file name and I have to replace the special characters into empty string to create a valid file name.

The other issues that I encountered is that cucumber actually converts "<" and ">" into "&lt" and "&gt". Therefore I have to replace it with "lessthanimg" and "/greaterthan".

That is why I have the "at_exit" code which is to replace the "lessthanimg" and "greaterthan" tags into "<" and ">".

at_exit do
ARGV.each do |a|
 if a =~ /\.htm(l)?/
   file = File.open(a, 'r')
   new_file = ""
   while (line = file.gets)
     if line.match(/\lessthan/) and line.match(/greaterthan\<\/pre\>/)
      new_file = new_file + line.gsub!(/\lessthan/, "<").gsub!(/greaterthan\<\/pre\>/, ">")
     else
      new_file = new_file + line
     end
   end
   file.close

   File.open(a, "w") do |f|
     f.write(new_file)
   end
 end
end
end

I am sure that there should be an easier way to do this. But for now, I can only find this solution. Please feel free to add comments if someone has found an easier and more elegant way for this.

Firefox Proxy configuration using Selenium Webdriver and Capybara

Quite often we have to run the automation test within the proxy setting. The default browser profile setup that Capybara use is the anonymous profile which does not use proxy setting by default. This will cause problem as the browser will not be able to access the webpage.

To see all of the available configuration in Firefox simply type "about:config" on the firefox address bar. You will then see the following screen:

One solution is to set the browser profile to use a proxy setting. The code can be placed inside the env.rb files to override the anonymous browser profile that Capybara uses.

# Need to override the driver to setup our proxy profile
class Capybara::Driver::Selenium
def self.driver
unless @driver
profile = Selenium::WebDriver::Firefox::Profile.new
profile["network.proxy.type"] = 1
profile["network.proxy.http"] = "proxy setting"
profile["network.proxy.http_port"] = 8080
@driver = Selenium::WebDriver.for(:firefox, :profile => profile)

at_exit do
@driver.quit
end
end
@driver
end
end

If you need to use the Automatic proxy configuration (pac file) URL then you would need to add the following line:

profile["network.proxy.autoconfig_url"]="proxy.pac"

And change the network.proxy.type into 2

profile["network.proxy.type"] = 1

If you need Capybara to use a different browser other than Firefox you can change the webdriver browser's profile. To change the default profile for other browsers you just need to change the @driver to use the browser that you want.

@driver = Selenium::WebDriver.for :ie
or
@driver = Selenium::WebDriver.for :chrome

Please note that webdriver currently only supports Firefox, IE and chrome.

Screenshots capture using capybara and webdriver

Recently, I stumble upon a task on how to automatically capture a screenshot when one of my cucumber scenario fails. I use capybara and webdriver for my automation test.

I found several solutions which include to use a program called scrot in ubuntu and to use another gem http://github.com/mocoso/cucumber-screenshot. However, I run my automation test in both Windows and Ubuntu and therefore I need a way to be able to capture screenshot in both OS.

Webdriver fortunately already has this module implemented in the driver class. Although this only supports firefox browser, it is enough for me as I run the automation in Hudson with firefox and I would want to embed the screenshot in the cucumber html report.

After do |scenario| 
 if scenario.failed? 
   File.open("./screenshots/#{scenario.name}.jpeg",'wb') do |f|    
      f.write(Base64.decode64(page.driver.browser.screenshot_as(:base64)))
   end
 end
end

The webdriver class already has the save_screenshot methods, but it does not work in Windows and therefore you need to write the file as byte.

The above code will automatically capture a screenshot whenever the scenario.failed? is true. This is very useful for error reporting because if you run your test in Hudson, you would not be able to see your test running.