I’ll probably be posting quite a bit about unit testing in this blog. NOT because I’m an expert in the topic – far from it. I’m also not particularly interested in discussing the efficacy of unit testing in general – that’s something that has been well covered many other places. Rather I’m going to assume that you, the reader, are already familiar with unit testing and are probably already using it. My interest here lies in exploring the difficult situations that may be encountered while attempting to test code.
I would call myself a “cautious believer” when it comes to unit testing. I’m convinced that it’s an invaluable technique to help ensure quality code. The cautious part comes about because while frameworks of the xUnit family make it easy to create tests, testing effectively can be difficult. There can be a lot of little things you can test for quickly and easily – basic getter / setter functionality, parameter checking, etc. Using code coverage metrics it may even appear that you’re covering the majority of your code based on raw line counts. So you reach what seems to be 70 – 80 – 90% coverage and you think you’re well covered, so you quit adding tests. But often that isn’t the case. It may just be that you’ve tested the “low hanging fruit” but have avoided the really difficult to test code. Unfortunately that usually means that what really needs to be tested the most, isn’t. These trivial tests may make you feel good, and certainly serve a purpose, but they may be masking the really tough testing that must be done.
What are some of the things that can make unit testing difficult? Some of the ones I’ve encountered are:
- Objects which require a large amount of “setup” work to get the object into a predictable, testable state.
- Objects which require some kind of external resources (e.g. DB server)
- Methods which don’t act in a fully predictable manner – some amount of randomness is part of the code being tested. (e.g. a current timestamp is being generated and needs to be checked.)
- Interactions with other libraries that make it difficult to set up necessary test conditions.
- The volume and / or complexity of the data that must be checked to ensure things are working.
- The necessity to mock up interfaces that the code under test relies on.
I’m sure there are other potential problem areas. I’m going to be discussing some of these in more detail in future entries and talk about some ideas that may make things a little easier, and I hope to get ideas that are better than mine in return!
The most useful technique I’ve found to help deal with your bullet items is to write the test first, then write your code. Of course, it only works when you’re writing new code. But it FORCES you to design your code to be testable. When adding tests to existing code, I can only recommend “Working with Legacy Code” by Michael Feathers.
But… those are technical problems that can be overcome by mechanical techniques. What I think is most interesting is this point: ‘you’ve tested the “low hanging fruit” but have avoided the really difficult to test code’. Even when you are working on code that’s easy to test, how do you know when your tests sufficienly exercise the code? Now there’s a question for discussion.