Michael Dowling's Whispers

This is the first part of many on how to use Spectacular, an open-source acceptance test / behavior driven development tool for development teams. This installment will focus on how to use Spectacular to create Executable Use Cases, and how those use cases can be executed using Selenese to drive a Selenium browser!

Part of why I wanted to write a tool like Spectacular is that I wanted to explore the possibility of using simplified Use Cases in the context of Epics in order to describe the overall requirement, in addition to using that Use Case to assist in creating stories during a story workshop. Of course, the one thing bugging me about the whole subject is that I would also want those Use Cases to be completely testable. Enter Executable Use Cases.

The idea works like this: Let’s say that during a story mapping workshop, an agile team identified an Epic/Activity for logging into a system as an existing user. They collaborated on a simple diagram showing how this would work:

Using this web-flow diagram as a guide, the team can now start to write a simplified use case in order to weed out some details and describe the user flow and interactions. In Spectacular, the requirement for a Use Case is to create a table with at least 2 columns (though 3 or more is typical, to describe ancillary details). The first row should be the title of the use case, with the word “Flow” in there (i.e. “Primary Flow: Log In” or “Alternate Flow: Remember Me Log In”). The next row are the column headers (“User Action”, “Expectation), followed by the use case itself. Additional columns are ignored by Spectacular:

Now that the team has written a use case, its time to make it executable. Many of the use cases I run into with my Agile teams are user-interaction based, which is what inspired me to giving user the ability to express a use case as a series of Selenese commands (in addition to writing the fixtures in Java, Ruby, and Groovy). Once the team has agreed to the use case details (and brainstormed user stories from this i.e. bad login, already logged in, etc.), the team opens up a text editor or IDE and writes the fixture in Selenese:

Flow: User navigates to system home page.
    open "/"
    waitForPageToLoad 5000

Expectation: User sees system home page with a header of "Home Page"
    verifyText "xpath://h1[class='mainHeader']" "Home Page"
    verifyText "id=loginLink" "Login"

Flow: User clicks login link
    click "id=loginLink"
    waitForPageToLoad 5000

Expectation: User sees a login form requesting credentials
    verifyElementPresent "id=usernameLogin"
    verifyElementPresent "id=passwordLogin"
    verifyElementPresent "xpath://input[@class='loginButton']"

Flow: User enters credentials and submits form
    type "id=usernameLogin" "myusername"
    type "id=passwordLogin" "mypassword"
    click "xpath://input[@class='loginButton']"
    waitForPageToLoad 5000

Expectation: User sees a landing page with a list of submitted articles
    verifyTitle "Welcome Michael!"
    verifyElementPresent "xpath://div[@class='article']"

Make sense? The Selenese code above in a text file in effect becomes the fixture code for our executable use case. Executing Spectacular would look like (assuming we named the Selenese file “login.selenese” and the spec as “SystemSpec.html”):

> Spectacular -fixtures ./login.selenese -specLocation ./SystemSpec.html
 -seleniumRCHost localhost -seleniumRCPort 4444
-seleniumRCInitialUrl http://system-uat/ -seleniumRCStartCommand *safari

(don’t worry, these commands can be stored in a project xml file so that running Spectacular doesn’t require such a long command line )

By default, results are output in the current directory as “TestResults.html”, which will be the same document the team wrote, but with annotated test results in the use case table. In our case, we haven’t written any implementation code yet, so it would look like this:

This is the essence of writing Executable Use Cases with Spectacular. For more details about how to write them or how to write the fixtures using other methods, visit the Spectacular EUC documentation.

Next in the “Using Spectacular” series, I’ll focus on how to document the executable use cases along with FIT and Gherkin/BDD tests and what they look like…

I haven’t updated my blog in a while…  sorry for that.  Besides my day job, I’ve been busily working on a new open-source project that has been in my head for quite a while now and I finally decided to just go for it.  I’m almost close to “announcing” it more publicly, but for right now I am gathering some feedback from colleagues before I unleash my horrifying code for the world to consume.

More later…

Here I thought I was going to have to bite the bullet and learn more than a little bit of Ruby by way of Behavior Driven Development.  I thought I was going to have to actually put my head down and start active coding in Ruby in order to use the Cucumber BDD framework, specifically to parse the Given/When/Then clauses.

I thought about writing my own framework to do just that in Java, and actually started to.  Then I went to the magical Google and – wouldn’t you know it? – I found JBehave.  Of course!  I knew Dan wouldn’t have let me down by making me go the way of Ruby   (not that there’s anything wrong with Ruby, I just like to take my sweet time drinking kool-aid )

I love how it is just as easy to write JBehave tests as it is to write Cucumber tests:

Given a user with the email address “someone@something.com” has not already registered with the system
When a user tries to register in the system with the email of “someone@something.com”
And a name of “Michael Dowling”
And a date of birth of “1/1/1985″ (yeah right)
Then register the user with an access token of “Registered”

In JBehave, being clearly well-designed in an IoC manner, asks you write the Steps to execute the above spec:

package minderupt.scenarios.steps;
 
import org.jbehave.scenario.annotations.Given;
import org.jbehave.scenario.annotations.Then;
import org.jbehave.scenario.annotations.When;
import org.jbehave.scenario.steps.Steps;
 
public class CreateUserSteps extends Steps {
 
	private String email;
 
	@Given("a user with the email address \"$email\" has not already registered with the system")
	public void checkForRegisteredUser(String email) {
		// code to check for pre-existing email
	}
 
	@When("a user tries to register in the system with the email of \"$email\"")
	public void registerUserWithEmail(String email) {
		// code to register with email
	}
 
	@When("a name of \"$name\"")
	public void registerUserWithFullName(String name) {
		// code to register with name
	}
 
	@When("a date of birth of \"$dob\"")
	public void registerUserWithDateOfBirth(String dob) {
		// code to register with DOB
	}
 
	@Then("register the user with an access token of \"$token\"")
	public void checkForCreatedUser(String tokenExpected) {
		// register user and confirm returned token, blah blah
	}
 
}

…and then write the actual test case, specifying configuration options and the actual steps to execute during the test:

package minderupt.scenarios;
 
import org.jbehave.scenario.JUnitScenario;
import org.jbehave.scenario.Scenario;
 
import minderupt.scenarios.steps.CreateUserSteps;
 
public class CreateUser extends Scenario {
 
	public CreateUser() {
		super(new CreateUserSteps());
	}
 
}

I like this approach for multiple reasons, not the least because I can re-use the steps for different spec documents utilizing the same language, but also, I can change the test case configuration so that on my dev machine (local build) I can run all the tests and simply outputthe results to my console, but in my CI system, the output goes out to some kind of standard reporting system in pretty HTML.  Yay.

Like I mentioned in a previous post, one of the things I want to do is be able to fully spec out a requirement in a document (in Word, HTML, Wiki, etc) which will have a mix of business logic rules, FIT tests, and BDD tests.  Then as part of a build, split out the FIT tests and run FIT against those, parse out the BDD tests and run those, and then output a report back into the combined document for the entire team to see.  Can’t be too hard, just a matter of parsing Word/HTML, recognizing the type of tests, and running them.  Integration project, really.

But I think it would be immensely useful, especially for organizations that (still) communicate over such types of documentation;  a behavior that I feel isn’t going away anytime soon.  Might as well make the transition a little bit easier, I think.

I’ve always been a huge believer in using Code Coverage tools (such as Clover, CodeCover, EMMA, gcov, etc) as a way to report on how in-depth your unit tests are.  Generally, I use them with Quality Assurance/Quality Control groups to determine where to focus any additional testing efforts during a Sprint.  While we tend to benchmark required coverage at 85%, this is always a general rule with the understanding that every project (and, therefore, every project’s code) is going to be different, with different needs.

An interesting post at InfoQ talks a bit about the need to be careful about using a required coverage metric during TDD:

Out of the gates, Gruber’s primary statement is that TDD proponents do not suggest code coverage as “a one true metric”, that it is useful but only to degree and taken in context with other sources of feedback. He denounces Pang’s claim that “(Pang) 100% code coverage has long been the ultimate goal of testing fanatics”, stating that “(Gruber) high code coverage is a desired attribute of a well tested system, but the goal is to have a fully and sufficiently tested system”.

I completely agree with this statement.  Code coverage is, again, a tool for determining where additional testing time can be spent, as well as identifying any risks of releasing code into production that is not currently covered by unit tests.

The following quote sums it up nicely:

code coverage [in the context of TDD] is a great way to notice that you screwed up and missed something, but nothing else

More information can be found on InfoQ’s post.