Using Robot Framework for ATDD

Question

I would like to hear other people\'s experience with Robot Framework for automated acceptance testing.

What are its major strengths and weaknesses as well as any compar

Answer 1

We have been using Robot Framework at my place of work for several over a year now with moderate success. Like the poster, we also do C++ work. We took some time to evaluate Robot against Fitnesse/Slim and, at the time, both solutions were good, but the deciding factors were (as of ~2009):

• It was clearer how Robot and its reports would scale to large projects
• It wasn't obvious how to version control Fitnesse artifacts

From a technical perspective, we have been using SWIG to bridge between Robot and C++. We wrap our test fixtures in SWIG and link it with the production code under test - giving us a python module that can be imported by Robot.

We use the .txt format for Robot input almost exclusively - we've found that this version controls better, it's easier to read, and we simply weren't using the advanced features of HTML (which is where we started). In addition, we are using the "BDD Style" Robot syntax as well. We use GoogleMock with some wrappers to help us set expectations which are checked during the teardown of each Robot test.

As far as organizing the tests, we have settled on the following approach (with inspiration from Dale Emery's approach given here):

• Major functional hierarchy is represented by a folder structure.
• A feature-ish sized thing is described in a Robot test file name.
• A description of each part of that feature is used at the Robot test case name.
• An example is given as the steps in the test case.
• The example text is broken down into steps using Robot "keywords".
• The test fixture drives the production code.

For example, a phone might have something like this:

//  PhoneProject/MakingCalls/DialAPhoneNumber.txt

*** Test Case ***
A user can dial a US number with an area code, up to 10 digits
   Given a phone without any numbers dialed
   Expect the attempted phone number to be 123-456-7890
   When a user enters the numbers 1234567890


// A separate MakingCallsKeywords.txt or something similar
*** Keyword ***
Given a phone without any numbers dialed                CreateNewDialer
Expect the attempted phone number to be ${phoneNumber}  ExpectDialedNumber  ${phoneNumber}
When a user enters the numbers ${numbersEntered}        DialNumbers ${numbersEntered}

// MakingCallsFixture.cpp  (which gets wrapped by SWIG)

std::wstring MakingCallsFixture::DialNumbers(const std::wstring& numbersEntered)
{
   ... Drive production code ...
}

// CreateNewDialer, ExpectDialedNumber also go here

We would then pair this up with unit tests that cover the corner conditions (e.g. ensure no more than 10 digits are allowed) - or maybe that would be another acceptance test (executable spec) depending on who is reading the tests and their familiarity with the domain.

We create a draft of these specs and review with domain experts and the team prior to beginning work. This has helped flush out a number of misunderstandings early on.