Introduction To JUnit Theories

Have you ever read a mathematical theory?

It typically reads something like this:

For all a, b > 0  the following is true: a+b > a and a+b > b

Just typically the statements are more difficult to understand.

There is something interesting about this kind of statement: It holds for EVERY element (or combination of elements) of a rather large (infinite in this case) set.

Compare that to the statement a typical test makes:


@Test
public void a_plus_b_is_greater_than_a_and_greater_than_b(){
int a = 2;
int b = 3;
assertTrue(a + b > a);
assertTrue(a + b > b);
}

This is just a statement about a single element of the large set we talked about. Not very impressive. Of course we can fix that somewhat by looping over the test (or using parameterized tests):


@Test
public void a_plus_b_is_greater_than_a_and_greater_than_b_multiple_values() {
List<Integer> values = Arrays.asList(1, 2, 300, 400000);
for (Integer a : values)
for (Integer b : values) {
assertTrue(a + b > a);
assertTrue(a + b > b);
}
}

Of course this still only tests a few values, but it also became pretty ugly. We are using 9 lines of code to  test what a mathematician writes in a single line! And the main point that this relation ship should hold for any value a,b is completely lost in translation.

But there is hope: JUnit Theories. Let’s see how the test looks like with that nifty tool:

import org.junit.experimental.theories.DataPoints;
import org.junit.experimental.theories.Theories;
import org.junit.experimental.theories.Theory;
import org.junit.runner.RunWith;

import static org.junit.Assert.assertTrue;

@RunWith(Theories.class)
public class AdditionWithTheoriesTest {

@DataPoints
public static int[] positiveIntegers() {
return new int[]{
1, 10, 1234567};
}

@Theory
public void a_plus_b_is_greater_than_a_and_greater_than_b(Integer a, Integer b) {
assertTrue(a + b > a);
assertTrue(a + b > b);
}
}

With JUnit Theories the test gets split in two separate parts: a method providing data points i.e. values to be used for tests, and the theory itself. The theory looks almost like a test, but it has a different annotation (@Theory) and it takes parameters. The theories in a class get executed with every possible combination of data points.

This means that if we have more then one theory about our test subject we only have to declare the data points once. So let’s add the following theory, which should be true for addition: a + b = b + a So we add the following theory to our class
@Theory public void addition_is_commutative(Integer a, Integer b) { assertTrue(a + b == b + a); }
This works like a charm and one can start to see that this actually saves some code as well, because we don’t duplicate the data points. But we only test with positive integers, while the commutative property should hold for all integers! Of course our first theory still only holds for positive numbers

There is a solution for this as well: Assume. With assume you can check precondition for your theory. If it isn’t true for a given parameter set, the theory gets skipped for that parameter set. So our test now looks like this:


@RunWith(Theories.class)
public class AdditionWithTheoriesTest {

@DataPoints
public static int[] integers() {
return new int[]{
-1, -10, -1234567,1, 10, 1234567};
}

@Theory
public void a_plus_b_is_greater_than_a_and_greater_than_b(Integer a, Integer b) {
Assume.assumeTrue(a >0 && b > 0 );
assertTrue(a + b > a);
assertTrue(a + b > b);
}

@Theory
public void addition_is_commutative(Integer a, Integer b) {
assertTrue(a + b == b + a);
}
}

This makes the tests nicely expressive.

The separation of test data from test/theory implementation can have another positive effect apart from brevity: You might start to think about you test data independent of the actual stuff to test.

Lets do just that. If you want to test a method that takes an integer argument, what integers would be likely to cause problems? This is my proposal:


@DataPoints
public static int[] integers() {
return new int[]{
0, -1, -10, -1234567,1, 10, 1234567, Integer.MAX_VALUE, Integer.MIN_VALUE};}

Which of course causes a test failure in our example. If you add a positive integer to Integer.MAX_VALUE you get an overflow! So we just learned that our theory in its current form is wrong! Yeah I know this is obvious, but have a look at the tests in your current project. Do all the tests that use Integers test with MIN_VALUE, MAX_VALUE, 0, a positive and a negative value? Yeah, thought so.

What about more complex objects? Strings? Dates? Collections? Or domain objects? With JUnit Theories you can setup test data generators once that create all the scenarios that are prone to create problems and then reuse those in all your tests using theories. It will make your tests more expressive and improve the probability of finding bugs.

Run, JUnit! Run!!!

JUnit together with JavaScript and SVN are some of the technologies that programmers often start using without even reading a single blog post let alone a book.  Maybe this is a good thing since they look simple enough and understandable so we can use them right away without any manuals, but this also means that they are also underused. In this article we will go through some features of JUnit that I consider very useful.

Parameterized tests 

Sometimes we need to run the same method or functionality with many different inputs and different expected results. One way to do this would be to create separate tests for each of the cases, or you

can use loop but that it would be harder to track down the origin of a possible test failure.

For example if we have the following value object representing rational numbers:


public class RationalNumber {

private final long numerator;
private final long denominator;

public RationalNumber(long numerator, long denominator) {
this.numerator = numerator;
this.denominator = denominator;
}

public long getNumerator() {
return numerator;
}

public long getDenominator() {
return denominator;
}

@Override
public String toString() {
return String.format("%d/%d", numerator, denominator);
}
}

And we have a service class called App with a method convert that divides the number to a rounded value of 5 decimal :

public class App {

/**
* THE Logic
*
* @param number some rational number
* @return BigDecimal rounded to 5 decimal points
*/
public static BigDecimal convert(RationalNumber number) {
BigDecimal numerator = new BigDecimal(number.getNumerator()).
setScale(5, RoundingMode.HALF_UP);

BigDecimal result = numerator.divide(
new BigDecimal(number.getDenominator()),
RoundingMode.HALF_UP);

return result;
}
}

And for the actual AppTest class we have

@RunWith(Parameterized.class)
public class AppTest {

private RationalNumber input;
private BigDecimal expected;

public AppTest(RationalNumber input, BigDecimal expected) {
this.input = input;
this.expected = expected;
}

@Parameterized.Parameters(name = "{index}: number[{0}]= {1}")
public static Collection<Object> data() {
return Arrays.asList(new Object[][]{
{new RationalNumber(1, 2), new BigDecimal("0.50000")},
{new RationalNumber(1, 1), new BigDecimal("1.00000")},
{new RationalNumber(1, 3), new BigDecimal("0.33333")},
{new RationalNumber(1, 5), new BigDecimal("0.20000")},
{new RationalNumber(10000, 3), new BigDecimal("3333.33333")}
});
}

@Test
public void testApp() {
//given the test data
//when
BigDecimal out = App.convert(input);
//then
Assert.assertThat(out, is(equalTo(expected)));
}

}

The Parametrized runner or  @RunWith(Parameterized.class)  enables the “parametrization” or in other words the injection of the collection of values annotated with  @Parameterized.Parameters into the Test constructor where each of the sublist is an parameter list.  This means that each of the RationalNumber objects in the data() method will be injected into the input variable and each of the BigDecimal values would be the expected value, so in our example we have 5 tests.

There is also an optional custom naming of the generated test added in the annotation, so “{index}: number[{0}]= {1}” will be replaced with the appropriate parameters defined in the data() method and the “{index}” placeholder will be the test case index, like in the following image

Running the parametrized tests in IntelliJ Idea 

JUnit rules

The simplest definition of JUnit rules is that they are in a sense an interceptors  and very similar to the Spring aspect oriented programming or Java EE interceptors API. Basically you can do useful things before and after the test execution.
OK so let’s start with some of the built in test rules. One of them is ExternalResource  where the idea is that we setup an external resource and after the teardown garteet the resource was freed up. A classic example of such test is a creation of file, so for that purpose we have a built in class TemporaryFolder but we can also create our own ones for other resources :


public class TheRuleTest {
@Rule
public TemporaryFolder folder = new TemporaryFolder();

@Test
public void someTest() throws IOException {
//given
final File tempFile = folder.newFile("thefile.txt");
//when
tempFile.setExecutable(true) ;
//then
assertThat(tempFile.canExecute(), is(true));
}
}

We could have done this in a @Before and @After blocks and use java temp files but it is easy to forget something and leave some of the files unclosed in some of the scenarios where a test fails.

For example there is also a Timeout rule for methods where if the the execution is not finished in given time limit the test will fail with a Timeout exception. For example to limit the running for 20 milliseconds :
   


@Rule
public MethodRule globalTimeout = new Timeout(20);


We can implement our own rules that can do a policy enforcement or various project specific changes. The only thing that needs to be done is for us to implement the TestRule interface.
A simple scenario to explain the behaviour is to add a rule that prints someting before and after test.


import org.junit.rules.TestRule;

import org.junit.runner.Description;
import org.junit.runners.model.Statement;

public class MyTestRule implements TestRule {

public class MyStatement extends Statement {

private final Statement statement;

public MyStatement(Statement statement) {
this.statement = statement;
}

@Override
public void evaluate() throws Throwable {
System.out.println("before statement" );
statement.evaluate();
System.out.println("after statement");
}

}

@Override
public Statement apply(Statement statement,
Description description) {

System.out.println("apply rule");
return new MyStatement(statement);
}

}


So now that we have our rule we can use it in tests, were the tests will just print out different values :


public class SomeTest {

@Rule
public MyTestRule folder = new MyTestRule();

@Test
public void testA() {
System.out.println("A");
}

@Test
public void testB() {
System.out.println("B");
}
}

When we run a test the following output will be created on the  console output :


apply rule
before statement
A
after statement
apply rule
before statement
B
after statement

From the built in one there is one called ExpectedException that can very useful when trying out testing errors. Additionally there is an option to chain the rules that can be useful in many scenarios.

To sum up

If you wanna say that Spock or TestNG or some library build on top of JUnit have more features than JUnit, than that is probably true.
But you know what? We don’t always have those on our class path and chances are that JUnit is there and already used all over the place. Than why not use it’s full potential ?

Useful links

Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!

Using Matchers in Tests

Gone are the days when we were forced to write way too many assertion lines in our testing code. There is a new sheriff in town: assertThat and his deputy: the matchers. Well, not that new, but anyway I’d like to present to you shortly how matchers are used and after that an extension to matchers concept that I found to be very useful when developing unit tests for my code.

First of all I’ll present the basic use of the matchers. Of course you can have a complete presentation of hamcrest matchers capabilities directly from its authors: https://code.google.com/p/hamcrest/wiki/Tutorial.

Basically a matcher is an object that defines when two objects match. The first question usually is why wouldn’t you use equals? Well, sometimes you don’t want to match two objects on all their fields, just on some of them and if you work with legacy code you’ll find that the equals implementation is not present or is not as you would’ve expected. Another reason is the fact that using assertThat gives you a more consistent way of “asserting the assertions” and arguably a more readable code. So, for example, instead of writing:


int expected, actual;
assertEquals(expected, actual);

you will write

assertThat(expected, is(actual));

where “is” is the statically imported org.hamcrest.core.Is.is
Not that much of a difference… yet. But Hamcrest offers you a lot of very useful matchers:
  • For arrays and maps : hasItem, hasKey, hasValue
  • Numbers: closeTo – a way to specify equality with an error of margin, greaterThan, lessThan…
  • Objects: nullValue, sameInstance
Now we’re making progress… still the power of Hamcrest matchers is that you have the possibility to write your own matchers for your objects. You just have to extend BaseMatcher<T> class. Here is an example of a simple custom matcher:

public class OrderMatcher extends BaseMatcher<Order> {
private final Order expected;
private final StringBuilder errors = new StringBuilder();

private OrderMatcher(Order expected) {
this.expected = expected;
}

@Override
public boolean matches(Object item) {
if (!(item instanceof Order)) {
errors.append("received item is not of Order type");
return false;
}
Order actual = (Order) item;
if (actual.getQuantity() != (expected.getQuantity())) {
errors.append("received item had quantity ").append(actual.getQuantity()).append(". Expected ").append(expected.getQuantity());
return false;
}
return true;
}

@Override
public void describeTo(Description description) {
description.appendText(errors.toString());
}

@Factory
public static OrderMatcher isOrder(Order expected) {
return new OrderMatcher(expected);
}
}

This is a completely new league compared to the old assertion methods.

So this is in a nutshell the usage of the Hamcrest’s matchers.
But, when I started using it in real life, especially when working with legacy code, I realized that there is more to the story. Here are some issues that I’ve encountered when using matchers:
  1. Matcher construction can be very repetitive and boring. I needed a way to apply DRY principle to matcher code.
  2. I needed an unified way to access the matchers. The correct matcher should be chosen by the framework by default.
  3. I needed to compare objects that had reference to another objects which should have been compared with matchers (the object referencing can go as deep as you want)
  4. I needed to check a collection of objects  using matchers without iterating that collection (doable also with the array matchers… but I wanted more J)
  5. I needed to have a more flexible matcher. For example, for the same object I needed to check one set of fields, but in another case another one. The out-of-box solution is to have a matcher for each case. Didn’t like that.
I’ve overcome these issues using a matcher hierarchy that some conventions and which knew which matcher to apply and which field to compare or ignore. At the root of this hierarchy is the RootMatcher<T> that extends BaseMatcher<T>.

To deal with the #1 issue (repetitive code), the RootMatcher class contains the common code for all the matchers like methods for checking if the actual is null, or it has the same type with the expected object, or even if they are the same instance:


public boolean checkIdentityType(Object received) {
if (received == expected) {
return true;
}
if (received == null || expected == null) {
return false;
}
if (!checkType(received)){
return false;
}
return true;
}
private boolean checkType(Object received) {
if (checkType && !getClass(received).equals(getClass(expected))) {
error.append("Expected ").append(expected.getClass()).append(" Received : ").append(received.getClass());
return false;
}
return true;
}

This will simplify the way the matchers are written, I don’t have to take into account null or identity corner cases; it’s all been taken care of in the root class.

Also the expected object and the errors reside in the root class:

public abstract class RootMatcher extends BaseMatcher {
protected T expected;
protected StringBuilder error = new StringBuilder("[Matcher : " + this.getClass().getName() + "] ");

This allows you to get to the matches method implementation as soon as you extend RootMatcher and for errors you just put the messages in the StringBuilder; RootMatcher will handle sending them to the JUnit framework to be presented to the user.

For issue #2 (automatic matcher finding) the solution was in its factory method:

@Factory
public static Matcher is(Object expected) {
return getMatcher(expected, true);
}
public static RootMatcher getMatcher(Object expected, boolean checkType) {
try {
Class matcherClass = Class.forName(expected.getClass().getName() + "Matcher");
Constructor constructor = matcherClass.getConstructor(expected.getClass());
return (RootMatcher) constructor.newInstance(expected);
} catch (ClassNotFoundException | NoSuchMethodException | InvocationTargetException | InstantiationException | IllegalAccessException e) {
}
return (RootMatcher) new EqualMatcher(expected);
}

As you can see the factory method tries to find out which matcher should it return by using two conventions
  1. The matcher for an object has the name of the object + the string Matcher
  2. The matcher is in the same package as the object to be matched (recommendable to be in the same package, but in the test directory)

Using this strategy I succeeded in using a single matcher: RootMatcher.is that will  provide me the exact matcher that I need

And to solve the recursive nature of the object relations (issue #3), when checking object fields I used the method from RootManager to check equality that will use matchers:

public boolean checkEquality(Object expected, Object received) {
String result = checkEqualityAndReturnError(expected, received);
return result == null || result.trim().isEmpty();
}

public String checkEqualityAndReturnError(Object expected, Object received) {
if (isIgnoreObject(expected)) {
return null;
}
if (expected == null && received == null) {
return null;
}
if (expected == null || received == null) {
return "Expected or received is null and the other is not: expected " + expected + " received " + received;
}
RootMatcher matcher = getMatcher(expected);
boolean result = matcher.matches(received);
if (result) {
return null;
} else {
StringBuilder sb = new StringBuilder();
matcher.describeTo(sb);
return sb.toString();
}
}

But how about collections (issue #4). To solve that, all you have to do is to implement matchers for collections that extend RootMatcher.

So the only remaining issue is #5: to make the matcher more flexible, to be able to tell the matcher which field should it ignore and which should it take into account. For this I introduced the concept of “ignoreObject”. This is an object that the matcher will ignore when it will find a reference to it in a template (expected object). How does it work? First of all, in RootMatcher I offer methods to return the ignore object for any Java type:

private final static Map ignorable = new HashMap();

static {
ignorable.put(String.class, "%%%%IGNORE_ME%%%%");
ignorable.put(Integer.class, new Integer(Integer.MAX_VALUE - 1));
ignorable.put(Long.class, new Long(Long.MAX_VALUE - 1));
ignorable.put(Float.class, new Float(Float.MAX_VALUE - 1));
}

/**
* we will ignore mock objects in matchers
*/
private boolean isIgnoreObject(Object object) {
if (object == null) {
return false;
}
Object ignObject = ignorable.get(object.getClass());
if (ignObject != null) {
return ignObject.equals(object);
}
return Mockito.mockingDetails(object).isMock();
}

@SuppressWarnings("unchecked")
public static M getIgnoreObject(Class clazz) {
Object obj = ignorable.get(clazz);
if (obj != null) {
return (M) obj;
}
return (M) Mockito.mock(clazz);
}

@SuppressWarnings("unchecked")
public static M getIgnoreObject(Object obj) {
return (M) getIgnoreObject(obj.getClass());
}

As you can see, the ignored object will be the one which is mocked. But for classes that cannot be mocked (final classes) I provided some arbitrary fixed values that are very improbable to appear(this part can be improved J). For this to work the developer has to use the equality methods provided in RootMatcher: checkEqualityAndReturnError which will check for ignored objects. Using this strategy and the builder pattern which I presented last year (http://www.javaadvent.com/2012/12/using-builder-pattern-in-junit-tests.html) I can easily make my assertions for a complex object:

import static […]RootMatcher.is;
Order expected = OrderBuilder.anOrder().withQuantity(2)
.withTimestamp(RootManager.getIgnoredObject(Long.class))
.withDescription(“specific description”).build()
assertThat(order, is(expected);

As you can see I could easily specify that the timestamp should be ignored and which allowed me to use the same matcher with a completely different set of fields to be verified.

Indeed, this strategy requires pretty much preparation, making all the builders and the matchers. But if we want to have a code that is tested, if we want to make testing a job that has the primary focus on the test flow that should be covered, we need such a foundation and these tools that help us easily establish our precondition and build our expected state.

Of course that the implementation can be improved using annotation, but the core concepts still remain.

I hope this article helps you improve your testing style, and if there’s enough interest I will do my best to put the complete code on a public repository.
Thank you.

Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!

Using Builder Pattern in JUnit tests

This is not intended to be a heavily technical post. The goal of this post is to give you some guidelines to make your JUnit testing life more easy, to enable you to write complex scenarios for tests in minutes with the bonus of having extremely readable tests.

There are two major parts in a Unit tests that require writing a lot of bootstrap code:

  • the setup part: constructing your initial state requires building the initial objects that will be fed to your SUT (system under test) 
  • the assertion part: constructing the desired image of your output objects and making assertions only on the needed data.

In order to reduce the complexity of building objects for tests I suggest using the Builder pattern in the following interpretation:

Here is the domain object:


public class Employee {
private int id;
private String name;
private Department department;

//setters, getters, hashCode, equals, toString methods

The builder for this domain object will look like this:


public class EmployeeBuilder {
private Employee employee;

public EmployeeBuilder() {
employee = new Employee();
}

public static EmployeeBuilder defaultValues() {
return new EmployeeBuilder();
}

public static EmployeeBuilder clone(Employee toClone) {
EmployeeBuilder builder = defaultValues();
builder.setId(toClone.getId());
builder.setName(toClone.getName());
builder.setDepartment(toClone.getDepartment());
return builder;
}

public static EmployeeBuilder random() {
EmployeeBuilder builder = defaultValues();
builder.setId(getRandomInteger(0, 1000));
builder.setName(getRandomString(20));
builder.setDepartment(Department.values()[getRandomInteger(0, Department.values().length - 1)]);
return builder;
}

public EmployeeBuilder setId(int id) {
employee.setId(id);
return this;
}

public EmployeeBuilder setName(String name) {
employee.setName(name);
return this;
}

public EmployeeBuilder setDepartment(Department dept) {
employee.setDepartment(dept);
return this;
}

public Employee build() {
return employee;
}
}

As you can see we have some factory methods:


public static EmployeeBuilder defaultValues()
public static EmployeeBuilder clone(Employee toClone)
public static EmployeeBuilder random()

These methods return different builders:

  • defaultValues : some hardcoded values for each fields ( or the Java defaults – current implementation)
  • clone : will take all the values from the initial object, and give you the possibility to change just some
  • random : will generate random values for each field. This is very useful when you have a lot of fields that you don’t specifically need in your test, but you need them to be initialized. getRandom* methods are defined statically in another class.

 You can add other methods that will initialized your builder accordingly to your needs.

Also the builder can handle building some objects that are not so easily constructed and changed. For example let’s change a little bit the Employee object and make it immutable:


public class Employee {
private final int id;
private final String name;
private final Department department;
...
}

Now we lost the possibility to change the fields as we wish. But using the builder in the following form we can regain this possibility when constructing the object:


public class ImmutableEmployeeBuilder {
private int id;
private String name;
private Department department;

public ImmutableEmployeeBuilder() {
}

public static ImmutableEmployeeBuilder defaultValues() {
return new ImmutableEmployeeBuilder();
}

public static ImmutableEmployeeBuilder clone(Employee toClone) {
ImmutableEmployeeBuilder builder = defaultValues();
builder.setId(toClone.getId());
builder.setName(toClone.getName());
builder.setDepartment(toClone.getDepartment());
return builder;
}

public static ImmutableEmployeeBuilder random() {
ImmutableEmployeeBuilder builder = defaultValues();
builder.setId(getRandomInteger(0, 1000));
builder.setName(getRandomString(20));
builder.setDepartment(Department.values()[getRandomInteger(0, Department.values().length - 1)]);
return builder;
}

public ImmutableEmployeeBuilder setId(int id) {
this.id = id;
return this;
}

public ImmutableEmployeeBuilder setName(String name) {
this.name = name;
return this;
}

public ImmutableEmployeeBuilder setDepartment(Department dept) {
this.department = dept;
return this;
}

public ImmutableEmployee build() {
return new ImmutableEmployee(id, name, department);
}
}

This is very useful when we have hard to construct objects, or we need to change fields that are final.

An here its the final result:

Without builders:


@Test
public void changeRoleTestWithoutBuilders() {
// building the initial state
Employee employee = new Employee();
employee.setId(1);
employee.setDepartment(Department.DEVELOPEMENT);
employee.setName("John Johnny");

// testing the SUT
EmployeeManager employeeManager = new EmployeeManager();
employeeManager.changeRole(employee, Department.MANAGEMENT);

// building the expectations
Employee expectedEmployee = new Employee();
expectedEmployee.setId(employee.getId());
expectedEmployee.setDepartment(Department.MANAGEMENT);
expectedEmployee.setName(employee.getName());

// assertions
assertThat(employee, is(expectedEmployee));
}

With builders:


@Test
public void changeRoleTestWithBuilders() {
// building the initial state
Employee employee = EmployeeBuilder.defaultValues().setId(1).setName("John Johnny").setDepartment(Department.DEVELOPEMENT).build();

// building the expectations
Employee expectedEmployee = EmployeeBuilder.clone(employee).setDepartment(Department.MANAGEMENT).build();

// testing the SUT
EmployeeManager employeeManager = new EmployeeManager();
employeeManager.changeRole(employee, Department.MANAGEMENT);

// assertions
assertThat(employee, is(expectedEmployee));
}

As you can see, the size of the test is much smaller, and the construction of objects became much simpler (and nicer if you have a better code format). The difference is greater if you have a more complex domain object (which is more likely in real-life applications and especially in legacy code).

Have fun!

Meta: this post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on! Want to write for the blog? We are looking for contributors to fill all 24 slot and would love to have your contribution! Contact Attila Balazs to contribute!