Introduction to Testing with BDD and the Spock Framework

I'd always use JUnit as the testing framework, a good library and pretty much the standard for testing in Java. However, sometimes you can feel something is missing.

In fact, this feeling is well described in the article Introducing BDD by Dan North (considered the father of BDD):

There's no doubt that TDD is a useful tool in software development. But more than testing, TDD is about improving the design process.

Of course, writing tests gives you the confidence to know that the system works correctly. However, more important than that, by writing tests before code the behavior of the system is well defined and documented.

This is not always seen, so the aim of Behaviour-Driven Development (BDD) is to make us realize the importance of defining behavior rather than testing per se by using a specific vocabulary to minimize misunderstandings and encourage the collaboration between business analysts, developers, testers, and stakeholders. For this reason, BDD can be considered an extension of TDD.

In BDD, testing and development are based on that common language and the basic unit to describe a feature of the system is a story. Acceptance criteria (the base of tests) are part of the story.

In his article What's in a story?, Dan North suggests the following template for a story:

For example:

Based on this template, Dan North developed JBehave, a Java framework that supports BDD and maps the steps of a scenario to Java methods via annotations.

JBehave inspired other frameworks like RSpec, which in turn, inspired the Spock framework, that in my opinion, brings testing to another level. You're going to see why.

Spock is a testing framework writing in Groovy, which means that runs on the Java Virtual Machine (JVM) and let you test code written in any of the languages the JVM supports.

In this page you'll read ten reasons why you might use Spock. Here, I just want to highlight three of them:

But enough talk. Let get to the code.

For this tutorial, we are going to use Eclipse and Maven to test a class in Java using Groovy and Spock.

Why not use IntelliJ IDEA and Gradle? Because setting up Eclipse and Maven is a little more complicated, and at least where I come from, more folks are using these tools.

For Eclipse, first, we need to install the Groovy plugin and some dependencies. Then, we'll create a Maven project, configure all the required dependencies and plugins in the pom.xml file and add a directory where the testing classes will live.

Let's start. I'll assume you're using Eclipse Mars.2 (version 4.5.2, but I shouldn't be very different if you're using a different version), so open Eclipse and to Help -> Install new software.

Click the Add... button

Enter the following information and press OK (if you're using another version of Eclipse, you can find the correct link to the update site here):

Name: Groovy plugin (or any name you want)

Location: http://dist.springsource.org/snapshot/GRECLIPSE/e4.5/

Wait for it to fetch the information and then select the Groovy-Eclipse and Extra Groovy Compilers options and click Next.

Review the items to install and click Next.

Accept the terms of the license and click Finish.

Wait for a while and Eclipse will install the Groovy plugin. You will need to restart Eclipse.

Now let's create the project. Go to File -> New -> Other... and choose Maven Project (notice that thanks to the Groovy plugin, you now have the options to create a Groovy class and a Groovy project)

In the next window, check the option Create a simple project (skip archetype selection) and click Next.

In this window, enter the project information, For instance:

Group ID: com.example

Artifact ID: spock

Version: 0.0.1-SNAPSHOT

Packaging: jar

This will create a project with a pom.xml file and the following directories:

However, since we're going to use Groovy, the right thing to do is create an src/test/groovy directory. To do so, right-click on the name of the project and choose the option Build Path -> Configure Build Path and then the Source tab.

Click Add Folder... and create the new folder (relative to src/test you should only enter groovy) without inclusion or exclusion patterns.

Next, select the new directory's option Output Folder and click Edit.

To specify the output directory for tests (target/test-classes) and click OK.

Regarding the pom.xml file, it should look like this:

In the plugins section, we're adding the gmavenplus-plugin, required to compile groovy code. Optionally, we can also add the maven-surefire-plugin to generate test reports.

In the dependencies section, we add the required Spock library (and explicitly specify the groovy version to make sure it matches the Spock version). Optionally, we can also add CGLIB and Objenesis to enable mocking of classes.

Save the file and if the Maven dependencies are not added automatically to the Eclipse project, right-click the name of the project and choose the option Maven -> Update Project....

To test your environment, create HelloWorld.groovy (with New -> Other -> Groovy class) in the spock package inside the src/test/groovy directory:

And run it like a JUnit test with Run As -> Junit Test. The output should be the famous green bar.

Groovy is a language that it's compiled to bytecode that is executed by the JVM, and it's very similar to Java, but with features comparable to Ruby, Python, or Javascript.

For example, everything is treated as an object and semicolons are optional:

You can use the def keyword instead of a type to declare a variable, Groovy will determine the type of the variable at runtime, which gives us some flexibility:

The def keyword can also be used to define methods (which makes them public). Methods can accept arguments without types, and the value of the last expression is returned by default (so a return statement is optional):

The parentheses are optional when calling a method with at least one argument:

And you can use a full string as a method name:

Groovy also supports string interpolation, which allows us to substitute any expression into a string:

In Java, we have the boolean (or Boolean) type to evaluate true/false expressions. But Groovy treats all variables as true unless the variable is:

This way, in Groovy, statements like these are valid:

An important consideration is that in Groovy, the == operator it's like calling the equals() method of the object, it doesn't test for identity like in Java:

Finally, we have closures. A closure is an anonymous block of code (like a function) which can be assigned to variables or passed as an argument to a function and has access to data of the context where it was defined. It can be as simple as:

By default, closures have one parameter named it:

If you don't pass the parameter, it will be null. Of course, you can define your own parameters (you can omit the type to pass anything):

As you can see, Groovy is not as bulky as Java. Of course, there's a lot more to learn about Groovy, but this should be enough to start coding some tests with Spock. That is what we will do next.

In Spock, you write specifications. Every test class must extend from spock.lang.Specification:

This class can have many methods, where each of them is considered a scenario (a test case):

In BDD, these scenarios are described in a given-when-then format. Spock supports this style using labels with an optional description. Notice that the assert keyword is optional:

If we change one of the conditions so the test fails:

We can see how descriptive Spock presents the failure trace:

Blocks divide a method into sections. They start with a label and extend either to the beginning of the next block or the end of the method. We already use the given, when and then, but there're six blocks actually:

In the setup: or given: (may people prefer this because it's more readable) block you create the objects and do any setup work for the test. Actually, this block is optional can be implicit, anything between the beginning of the method and the first (different) block represents the setup block:

If any, there can be only one setup: or given: block.

The when: and then: blocks always go together. They represent a cause (called stimulus) and an effect (or response). For that reason, the when: block can contain any code but then: can only contain conditions, definitions, or interactions. We can have multiple pairs of when-then blocks:

Or multiple then blocks after a when (which are evaluated separately):

The expect: block is the same as then:, it can only contain conditions or variable definitions, but it doesn't require a then: block:

The optional cleanup: block is used to free resources and is run even if the code before the block has thrown an exception. It's usually the last block in the method, but it may only be followed by a where: block:

where: is also an optional block that must be the last one because it lets you define values to perform parameterized tests, also known as data-driven tests. It's commonly used with expect::

Spock will run as many tests as rows in the where block. In the example, Spock will execute three tests, and the vars x, y, and z will be initialized to the following values:

The minimum number of columns for this to work is two, so if you only have one variable, you can do something like this:

The data can be formatted horizontally also. This way, the previous tests can be written as:

A data pipe (represented by the left-shift operator, <<) connects a variable to a data provider. In the example, the values of each variable are provided by a list, but the data provider can be any iterable object, like a Collection, a String, or other implementations of Iterable. Also, the data can be fetched from external sources, like a database:

Values can be ignored with an underscore (_):

Variables can be assigned directly also:

And of course, you can use all of these methods at the same time:

Although it's not considered a block, there's also an and: label that can be added to any block to describe and individual part of it:

In a data-driven test, we can test many data sets. If one or more fail, for example:

Something like this is the result:

But if we want to be more specific about the failing datasets, we can use the spock.lang.Unroll annotation:

This way, each dataset will be reported separately:

And there's more. We can have custom names with placeholders that will be replaced with the values of the variables defined inside the where: block. Either on the @Unroll annotation:

Or at the method name directly:

The result:

You also can annotate the whole class to avoid annotating each method:

Fixture methods execute code before or after tests. There are four of them.

setup() and cleanup() execute before and after each test case respectively. For example:

The output will be:

As you can see, instance objects are created before each test feature, like if they were defined at the beginning of the setup() method.

The other two fixture methods, setupSpec() and cleanupSpec(), are executed once for the whole class, before and after the test methods. For example:

The output will be:

You can think of setupSpec() and cleanupSpec() as static methods, and as such, they can only use static variables or the ones annotated with spock.lang.Shared, otherwise, a compiler error will be generated.

As a side note, cleanup() and cleanupSpec() are executed even when the test failed, or an exception is thrown by the test.

Sometimes we need to test if an exception is thrown. In Spock we do it like this:

Using the thrown(Class<Throwable> type) method to indicate we expect an exception. We can even save a reference to the exception to access its properties:

When the exception is not thrown, an error like the following will be generated:

Certainly, there's a method to indicate that an exception should not be thrown:

Other times, you'll want not to run (ignore) some tests- Spock can provide a way to do this through extensions that are activated by annotations called directives. There are four extensions related to ignoring tests:

A method annotated with @Ignore will not be executed by Spock no matter what:

A method annotated with @IgnoreIf(predicate) is ignored only if the given predicate evaluates to true. For the predicate part, we can use:

For example:

A method annotated with @IgnoreRest will be executed by Spock, but all the other tests of the class will be ignored. For example:

A method annotated with @Requires(predicate) is the opposite of @IgnoreIf, it will be executed only if the given predicate evaluates to true. For example:

@Ignore, @IgnoreIf, and @Requires can annotate the whole class.

Suppose we have a DAO (Data Access Object) interface and a service in Java to interact with a database:

Sometimes we want to test the interaction between classes, but don't want to set up a connection to a database or create a complex object just for a unit test. One option is to create a dummy implementation to simulate results. Such implementation is called a mock object.

To ensure compatibility with Java, Spock uses JDK dynamic proxies when mocking interfaces and CGLIB proxies when mocking classes to generate these dummy objects at runtime.

Here's a simple example of how to create a mock implementation of Dao:

We can see that to create a mock object, we just have to call the Mock() method and define the behavior of the methods on the mock object (since initially, mock objects have no behavior). In the example, we are using the >> operator to define the return value of the method. Also, _ as String means any object of type String (you can pass any type here) and you can catch the parameter passed inside the closure.

We can also specify the return values of a mock method for more than one call:

In the example, we are define that for the first invocation of UserDao.delete(int), this method must return true, and for the second and subsequent invocations, false.

In the then: or expect: blocks, we can even specify how many times a method may be called with a number or a range:

For example:

This has been an introduction to the Spock framework and BDD. I hope you can see how readable and accessible is to test with the help of Groovy and this framework.

Of course, there is a lot of functionality that we didn't cover, but this should be enough to start programming test for most of your needs. You can learn more at https://github.com/spockframework/spock or with the excellent book Java Testing with Spock.