follow me icons

Friday, September 30, 2011

Use '--expand' to expand Scenario outline tables in Cucumber output

The default cucumber options is to not expand the scenario outline table in the reporting. So when you run bundle exec features, it will print out the report like the one below:

Scenario Outline: Add two numbers
   Given I have entered "<num_1>" into the calculator
   And I press the add button
   And I have entered "<num_2>" into the calculator
   And I press the equal button
   Then the result should be "<result>" on the screen
Examples:
  |num_1 | num_2 | result |
  | 1        |   1       |   2    |
  | 2        |   2       |   4    |
  | 3        |   5       |   8    |

In one way this is good because the report is concise and easily readable. However, one issue that I have is when there is an error in the scenario. Let say the error is on the "And I press the equal button", then you would not know it because it will not show which one is the error.

One way to overcome this is to use the "--expand" options from cucumber --help which will expand the scenario outline. I found this very useful when you have thousands of scripts running. This --expand option will give a much more verbose reporting.

Here is the result of running bundle exec --expand cucumber features:
As you can see, if the error occurs in the "And I press the add button" then cucumber will highlight it with red.


 Scenario Outline: Add two numbers                  
   Given I have entered "<num_1>" into the calculator
   And I press the add button                      
   And I have entered "<num_2>" into the calculator
   And I press the equal button                    
   Then the result should be "<result>" on the screen

   Examples:

     Scenario: | 1 | 1 | 2 |                  
       Given I have entered "1" into the calculator
       And I press the add button                
       And I have entered "1" into the calculator
       And I press the equal button              
       Then the result should be "2" on the screen

     Scenario: | 2 | 2 | 4 |                  
       Given I have entered "2" into the calculator
       And I press the add button                
       And I have entered "2" into the calculator
       And I press the equal button              
       Then the result should be "4" on the screen

     Scenario: | 3 | 5 | 8 |                  
       Given I have entered "3" into the calculator
       And I press the add button                
       And I have entered "5" into the calculator
       And I press the equal button              
       Then the result should be "8" on the screen

Reference:

>bundle exec cucumber --help

Usage: cucumber [options] [ [FILE|DIR|URL][:LINE[:LINE]*] ]+

Examples:
cucumber examples/i18n/en/features
cucumber @rerun.txt (See --format rerun)
cucumber examples/i18n/it/features/somma.feature:6:98:113
cucumber -s -i http://rubyurl.com/eeCl

    -r, --require LIBRARY|DIR        Require files before executing the features. If this
                                     option is not specified, all *.rb files that are
                                     siblings or below the features will be loaded auto-
                                     matically. Automatic loading is disabled when this
                                     option is specified, and all loading becomes explicit.
                                     Files under directories named "support" are always
                                     loaded first.
                                     This option can be specified multiple times.
        --i18n LANG                  List keywords for in a particular language
                                     Run with "--i18n help" to see all languages
    -f, --format FORMAT              How to format features (Default: pretty). Available formats:
                                       debug       : For developing formatters - prints the calls made to the listeners.
                                       html        : Generates a nice looking HTML report.
                                       json        : Prints the feature as JSON
                                       json_pretty : Prints the feature as prettified JSON
                                       junit       : Generates a report similar to Ant+JUnit.
                                       pdf         : Generates a PDF report. You need to have the
                                                     prawn gem installed. Will pick up logo from
                                                     features/support/logo.png or
                                                     features/support/logo.jpg if present.
                                       pretty      : Prints the feature as is - in colours.
                                       progress    : Prints one character per scenario.
                                       rerun       : Prints failing files with line numbers.
                                       stepdefs    : Prints All step definitions with their locations. Same as
                                                     the usage formatter, except that steps are not printed.
                                       usage       : Prints where step definitions are used.
                                                     The slowest step definitions (with duration) are
                                                     listed first. If --dry-run is used the duration
                                                     is not shown, and step definitions are sorted by
                                                     filename instead.
                                     Use --format rerun --out features.txt to write out failing
                                     features. You can rerun them with cucumber @rerun.txt.
                                     FORMAT can also be the fully qualified class name of
                                     your own custom formatter. If the class isn't loaded,
                                     Cucumber will attempt to require a file with a relative
                                     file name that is the underscore name of the class name.
                                     Example: --format Foo::BarZap -> Cucumber will look for
                                     foo/bar_zap.rb. You can place the file with this relative
                                     path underneath your features/support directory or anywhere
                                     on Ruby's LOAD_PATH, for example in a Ruby gem.
    -o, --out [FILE|DIR]             Write output to a file/directory instead of STDOUT. This option
                                     applies to the previously specified --format, or the
                                     default format if no format is specified. Check the specific
                                     formatter's docs to see whether to pass a file or a dir.
    -t, --tags TAG_EXPRESSION        Only execute the features or scenarios with tags matching TAG_EXPRESSION.
                                     Scenarios inherit tags declared on the Feature level. The simplest
                                     TAG_EXPRESSION is simply a tag. Example: --tags @dev. When a tag in a tag
                                     expression starts with a ~, this represents boolean NOT. Example: --tags ~@dev.
                                     A tag expression can have several tags separated by a comma, which represents
                                     logical OR. Example: --tags @dev,@wip. The --tags option can be specified
                                     several times, and this represents logical AND. Example: --tags @foo,~@bar --tags @zap.
                                     This represents the boolean expression (@foo || !@bar) && @zap.

                                     Beware that if you want to use several negative tags to exclude several tags
                                     you have to use logical AND: --tags ~@fixme --tags ~@buggy.

                                     Positive tags can be given a threshold to limit the number of occurrences.
                                     Example: --tags @qa:3 will fail if there are more than 3 occurrences of the @qa tag.
                                     This can be practical if you are practicing Kanban or CONWIP.
    -n, --name NAME                  Only execute the feature elements which match part of the given name.
                                     If this option is given more than once, it will match against all the
                                     given names.
    -e, --exclude PATTERN            Don't run feature files or require ruby files matching PATTERN
    -p, --profile PROFILE            Pull commandline arguments from cucumber.yml which can be defined as
                                     strings or arrays.  When a 'default' profile is defined and no profile
                                     is specified it is always used. (Unless disabled, see -P below.)
                                     When feature files are defined in a profile and on the command line
                                     then only the ones from the command line are used.
    -P, --no-profile                 Disables all profile loading to avoid using the 'default' profile.
    -c, --[no-]color                 Whether or not to use ANSI color in the output. Cucumber decides
                                     based on your platform and the output destination if not specified.
    -d, --dry-run                    Invokes formatters without executing the steps.
                                     This also omits the loading of your support/env.rb file if it exists.
    -a, --autoformat DIR             Reformats (pretty prints) feature files and write them to DIRECTORY.
                                     Be careful if you choose to overwrite the originals.
                                     Implies --dry-run --formatter pretty.
    -m, --no-multiline               Don't print multiline strings and tables under steps.
    -s, --no-source                  Don't print the file and line of the step definition with the steps.
    -i, --no-snippets                Don't print snippets for pending steps.
    -q, --quiet                      Alias for --no-snippets --no-source.
    -b, --backtrace                  Show full backtrace for all errors.
    -S, --strict                     Fail if there are any undefined or pending steps.
    -w, --wip                        Fail if there are any passing scenarios.
    -v, --verbose                    Show the files and features loaded.
    -g, --guess                      Guess best match for Ambiguous steps.
    -l, --lines LINES                Run given line numbers. Equivalent to FILE:LINE syntax
    -x, --expand                     Expand Scenario Outline Tables in output.
        --drb                        Run features against a DRb server. (i.e. with the spork gem)
        --port PORT                  Specify DRb port.  Ignored without --drb
        --version                    Show version.
    -h, --help                       You're looking at it.