Cuban Car: Martin Harris
Well perhaps not Nirvana then, but at least having a suitable level of test coverage. 
I wanted to write an article around the uptake of test driven development. Scrum and agile are hard to do well. If you break these down, you often find that the components are pretty challenging too. TDD is difficult but it has gained widespread recognition at the intellectual level. On the ground though the practice can be patchy. Why?
Before writing this article I had a look around to see what else had been written. This article on Geiger’s Counterpoint sums up my thoughts exactly. Its such a good article that I hardly have anything left to say. In fact neatly I can just provide some bullet points!
Improving the uptake of Test Driven Development
Programmers need help to adopt test driven practice. So how can you go about that?
- Lead from the front: If your senior developers are not checking in fully tested code it will destroy the practice. Its highly demoralising to be producing well tested code only to have a senior ruin it by checking in code with partial coverage. I can not stress hard enough that the practice must come from the top.
- Pair code and mentor the practice: I have yet to meet a developer that took to TDD right from the off. Most people struggle to produce good code, and TDD is far from intuitive. The best way to improve is to see good code and work with people that write good code. So mentor and pair code to disseminate the practice.
- Use coverage reporting: Use tools like clover, cobertura and emma to provide reports so that you can see what lines of code are covered by tests. Make this reporting part of your build. Encourage developers to use coverage reports to check the quality of their testing practice. Understand though that coverage is not everything you need to concern yourself with, sensible assertions are important too.
- Encourage discussion: Next time you find something un-tested in your codebase, create a discussion about it. Sometimes there are reasons, its badly written making it difficult to test. Perhaps its an integration point. Through discussion you will encourage the team to find a solution and write some tests.
- Set a bar: Find a way to set a bar. i.e find the current coverage levels and don’t allow the numbers to fall. I worked on a project where a custom build component was added. It parsed the clover reports, found the current coverage level and broke the build if it fell after checkin.
Summary
Most project suffer lower levels of test coverage than ideal. Encourage test driven development and find ways to teach how it can be done. Create dialog and get the teams to self organise around the goal of improving test quality and coverage. Use tools and processes to support the team.
An excellent post. I would also add – code review tests, and put the effort into your build system to deal elegantly with fast unit and slow integration tests.
Also rewarding those who succeed; although how you do that if you’re not in charge is tricky other than relying on TDD to produce a better product in a shorter time.
Thanks.
Agreed.
It would be interesting to build in reward systems that reward teams who achieve results through Agile systems. I am more in favor of rewarding teams than individuals…but that is another story.
Martin,
My thoughts on the subject of reward:
To my experience, TDD and code quality can actually bring their own rewards. A codebase with good coverage and good assertions is harder for others to subsequently break if they try to expand your work for reuse or attempt optimisations (premature or legitimate). Failure cases can obviously be environmental or as a result of integration, but subtle problems introduced at the unit level have a way of snowballing into show-stoppers for an entire application. Good unit and integration tests can trap this at build time.
The reward? Fewer late nights spent trying to deduce how a regression got in or weekend phone calls when someone else’s change takes out the app. Free time is its own reward. More time with the wife/kids/television or alternatively fresh new tasks on a project, so you can be an innovator on the team rather than a maintenance man. I’d rather be consistently working on new challenging pieces and leaving work at a reasonable hour rather than any quantifiable reward. Well maybe not ANY reward…
The problems:
1 – project managers don’t always appreciate the value of TDD: it doesn’t give burn down. A false economy, since the team will just spend far more having to go back and fix simple bugs that could’ve been trapped more quickly earlier.
2 – other people can change your tests as well as your implementation. Unless your tests were bad in the first place, you should not accept any excuse from @Ignore writers. They could even be asked what their legitimate case is to modify the behaviour of an existing piece of code – are they changing a contract? Their unwillingness to spend 5 minutes extending a test is potentially going to cost you an evening or a weekend.
3 – transient staff. If people are only in teams for very short periods, the benefits from TDD are less apparent to them. Holding people accountable is hard enough if they move functional area, forget about it if they move teams or leave the company. This is why code reviews MUST include test cases.
I am with you on the project manager point, although I meet more and more that get it, but don’t know how to go about checking coverage or test quality.
Point 2. I think tests should evolve and improve with the system and should be treated in the same light as production code. I.e they should be well written and as you say, reviewed.
Point 3. Hey I have been transient staff!
As such left high coverage behind because the test can answer questions I might not be around for.
There is something that can help. Scrum has this concept of “Definition of Done.” Its a contract for a task. I think an example says more than a description so for a simple service method you might have something like this:
Method written.
Method code reviewed, test coverage, findbugs output checked. Eclipse warnings cleaned or suppressed.
Appropriate unit tests checked.
Appropriate integration and performance tests checked.
Release note updated.
And so on. The idea is that the task can not be in the sprint review until all tasks are done.
The team gets to decide what is an appropriate definition of done, but the scrum master and or product owner can be involved in the discussion.