Here at Tyro we are an agile development team following XP methodologies. One of the practices we follow is a test driven development (TDD) while pair programming. We write the tests first to help drive the design of the class under test, as well as build a suite of tests for regression testing. If done correctly, it should be easy.

Recently there has been a lot of discussion amongst our team on the value of tests. This is not a discussion around whether we should test, what the best way to test is, or even how to test. There are lots of resources on this already. What I’m discussing here is whether TDD is worth it and if we should continue with unit testing.

Personally I don’t think you can have enough tests. But  then there is the law of diminishing returns where testing every possible combination of things becomes unwieldy. That’s why it’s important use pragmatism about where to stop.

Why use mock testing?

A couple of the frameworks we use are EasyMock and PowerMock. If you’ve used these before, then right now you’re either cringing in disgust or wondering why we use mocks at all. It’s not always necessary to use mocks, I prefer to use real objects where I can, but I think EasyMock and PowerMock frameworks are useful in the right place. They are especially useful if testing using real objects requires a lot of test overhead to set up, eg. database calls. I can test database methods separately and then mock them out in the service layer.

Why we code to exceptions

One of the nuances of mock testing are exceptions. Sometimes trying to get a real object to throw an exception can be difficult to set up and even harder to invoke. Most of the time I will use mocks if I want to test what happens when an exception is thrown. Yes, I know you should not code to exceptions – but at Tyro, we deal with money. We need to make sure we don’t lose transactions because that is ultimately money for our merchants and card holders.This is why we need to deal with exceptions.

How to use mocks to test exceptions

One of the things I have seen a few times now, and has caught out a number of our developers is the way to use mocks to test exceptions. The following code will pass as a unit test:

[java]
@Test(expected = IllegalArgumentException.class)
public void testingExceptionWithAnnotation() throws Exception {
expect(collaborator1.methodCall()).andReturn(new SomeObject("Collaborator1"));
expect(collaborator2.methodCall()).andThrow(new IllegalArgumentException("some exception"));
finalCollaborator.sendSomeEmails();

replayAll();
collaboratorCoordinator.aMethodCall();
verifyAll();
}

[/java]

This unit test passes and as a developer you think: “right, I can move on.” If you were looking at the test code only, you would assume that there is some try/catch clause around the collaborator2 method call and that this calls finalCollaborator to send some emails. The actual code being tested is:

[java]
public void aMethodCall() throws Exception {
collaborator1.methodCall();
    collaborator2.methodCall();
    collaborator3.methodCall();
    finalCollaborator.sendSomeEmails();
}

[/java]

If you are like me right now you’re starting to wonder what sort of Jedi mind trick is going on here.

After a little bit of digging, I found out that once the exception is thrown, the verifyAll() does not check any further mocks and their calls. I could have any extra number of calls in the test after the exception was thrown and the test would still pass. Example:

[java]
@Test(expected = IllegalArgumentException.class)
public void testingExceptionWithAnnotationAndExtraCalls() throws Exception {
    expect(collaborator1.methodCall()).andReturn(new SomeObject("Collaborator1"));
    expect(collaborator2.methodCall()).andThrow(new IllegalArgumentException("some exception"));
    expect(collaborator3.methodCall()).andReturn(new SomeObject("Collaborator3"));
    finalCollaborator.sendSomeEmails();

    replayAll();
    collaboratorCoordinator.aMethodCall();
    verifyAll();
}
[/java]

In fact the test itself could throw an exception and it would still pass.

I like to write tests first and ensure that they fail. So when something like this passes when it shouldn’t, it worries me. In fact there are strict mocks that you can use if you want to ensure that mocks are called in a specific order. I tend to prefer these strict mocks, but my colleagues think this is an unnecessary overhead. Most of the time they are correct, and the pragmatist in me will try to avoid these unless absolutely necessary.

But I digress. You probably just want to know how to fix the problem at hand and write a test that fails first so you can make sure the code is doing what you expect it to do.

Checking for exceptions

This is where I have taken the approach of testing for specific exceptions. Then I check for them in unit tests. As a rule of thumb (and almost in every instance), when I need to write a test that is checking exceptions, I will write the test with try/catch blocks so I can ensure that the test does what I expect it will do. In this test I expect nothing to happen after the exception is thrown in collaborater2.

[java]
@Test()
public void testingExceptionWithTryCatch() throws Exception {
    expect(collaborator1.methodCall()).andReturn(new SomeObject("Collaborator1"));
    expect(collaborator2.methodCall()).andThrow(new IllegalArgumentException("some exception"));

    replayAll();
    try {
        collaboratorCoordinator.aMethodCall();
        fail("Should not get here");
    } catch (IllegalArgumentException e) {
}
    verifyAll();
}
[/java]

I have now tested that the correct behaviour is being observed in the test, and the code it is testing is doing the right thing after an exception has been thrown. This test will fail at the line fail (“Should not get here”) if the code does not throw any sort of exception. In fact, if the wrong exception is thrown by the collaborator the test will still fail. This is a good way to test exceptions because it means you are testing that the correct exception is thrown and it’s thrown from the line you expect. If you use the @Test(expected = …) it’s possible for an exception to be thrown from a different mock object or the test code itself. In this situation the test would still pass even though the test is incorrect.

Try/catch

I must note that if you do change the implementation and surround it with a try/catch, some other tests will probably fail. I guess this comes down to the chicken or the egg problem. Do you want a failing test first, or do you want to change the code and then fix the tests? This is another discussion that is often had in our team, and at the moment for me the jury is still out. This is where the pragmatist in me takes over, and I like to evaluate each test on a case by case basis.

Happy testing…