1. Always stop to add that simple unit test. You will be amazed how often this discovers a bug or unexplored corner case.
2. Never develop from a main method, find a way to turn it into a test. Main driven development is horrible, they are not part of the continuous development cycle so once finished they get forgotten.
3. Make all tests run in the continuous integration environment. A test that does not run is a dark test.  Tests that are not continually run are probably broken.  Same goes for main method development.
4. Expend effort getting the time from commit to all project tests passing tests as small as possible. If I could check in, and see the results 1 second later there is little chance of me holding someone else up.  Aim for less than 15 mins with current technology.  Parallelism is the key.  Separate the build, many small suits that run in parallel.  Why not build a cloud just for running parallel test suits in?
5. Do as much testing as possible using simple unit tests. Unit tests are cheap to write, run fast and are easier to maintain. Test as much as possible with them, resort to higher level tests only to test integration.  Good unit level coverage will also mean the higher levels have less to do.

If you just did the above you would be doing very well indeed.

Some other test related thoughts I had today:

Why would you schedule your test runs on an hourly schedule?  Always hook unit tests into the commit.  If your unit coverage is high it might be possible to consider the unit tests a pass or fail threshold for a commit.

If you can’t fix a break quickly revert before the breaks pile up.  Choose a source control system that makes this easy to do.  The threat of a revert means people will go out of their way to test the build before checking in.

Integration tests should also be continuously run.  There may be challenges, they are often slower, and more fragile.  Break them into batches but not just by functional area.  Consider batches split by test categorisation, with buckets like:

Slow running: Schedule infrequent runs.

Fragile but fast: Trigger on commit.

Performance: Run when system is not under load.

The general rule is be inventive, find ways to get them run as frequently as possible.

Finally ask yourself this next time you realise the need for another test, but decide not to write it.  Why do I feel its ok not to write this test?

Too hard: Perhaps its time to find better ways to crack the problem.  An opportunity to be creative.

Too trivial: It should not take too long to write it then.

Your under pressure to deliver: Local optimization, delivering untested code ultimately slows down delivery.

Happy testing!

                          

Craig Larman in Agile & Iterative Development, A managers guide describes it thus:

Sprint Review: At the end of each iteration, there is a review meeting. (maximum of four hours) hosted by the Scrum Master.  The team, Product Owner, and other stakeholders attend. There is a demo of the product.  Goals include informing stakeholders of the system functions, design, strengths, weaknesses, effort of the team, and future trouble spots.

Feedback and brainstorming on future directions is encouraged, but no commitments are made during the meeting.  Later at the next Sprint Planning meeting, stakeholders and the team make commitments.

“Power Point” presentations are forbidden.  Preparation emphasis is on showing the product.

What is a Demo anyway?  What should we Demo?

Demo of a Balinese fire dance, Bali, Indonesia: Martin Harris

A demo has to be fully working software.  Although the team considers it ready for production and has fully tested it, its acceptable that the demo environment is scaled down in terms of moving parts and data sets.  Often there are manual test stages that follow user acceptance and such, but during this meeting we expect to get business agreement and sign-off so the environment has to be good enough to inspire confidence in the product.

Get the product owner to drive the software.  No cheating The product owner will get a better feel for it if they drive.  I have seen situations were very good software refuses to behave only when the product owner takes control!  Make sure that your demo room has a spare PC just in cases someone from the team needs to login, check logs, or re-start a service etc.  Its important not to stall on something trivial or unrelated to the demo.

Record any issues or problems.  These have two outcomes.  Either they are bad enough that the product owner will not accept the stories as finished, or if trivial the product owner may sign off a story if they are fixed in the next iteration.  The ideal is obviously to sail into the next iteration with no hangover whatsoever.  Don’t let issues slow the meeting down.  The product owner should make a decision and move on.  The scrum master documents any such issues and the outcome.  Note that the teams testers are incredibly useful in a demo, often providing usability and product usage advice.

If possible everything in an iteration should be included in the software demonstration.  If you get it right it should be possible to demonstrate every story.  If the team is in the habit of producing a definition of done for all tasks and done includes things like, documented, fully tested, deployable then preparation for the demonstration should just be a matter of updating the demonstration environment.

What a demo is not?

A demo is always actual running software, never mocked aside from test or development data.  If you can get production data then all well and good.  The product owner is always present, demonstrating to the development teams has little value.  This is a key opportunity for developers and the business to come together and feel part of the same team.  The product owner is the only person who can sign off a story.  If they are unavailable postpone.

How much does a demo cost?

Very little when measured against the return.  In an ideal project the demo environment is created off the back of a successful build possibly automated.  Its not just used for the demo, but provides a place to test.  In an integration situation its often useful to have somewhere that is guaranteed to be stable.  Its possible to share the environment over several teams as long as its stable during the demo.

What are the Benefits?

The main benefit is the team is able to demonstrate that a set of stories for an iteration is finished.  The burn down is complete.  The product owner is able to report that a key section is complete.  In some projects release to production follows soon after.  You are able to see exactly where you are after updating the backlog.

This process like no other Builds Confidence.  The confidence the team has in itself and the confidence the business has in the team to deliver.  It can not be overstated how important this is.

Sometimes this is the first time that the Team members see the product working end to end, when your working on the small parts is easy to forget where your heading.  The demo brings it all together.

Team becomes more focused on delivery.  I will paint you a picture.  Its Thursday morning, the demo is tomorrow, there is still quite a bit to do before the demo.  Nobody on the team wants to be the one, who’s key piece meant something was not ready for the demo.  By the afternoon you can feel the tension as the team readies the environment and tests everything one last time.  Teams with no demo do not have such focus.  A key part of scrum is something called cadence.  Iterations are always the same length, people get used to that, and they are marked by events.  I would say Sprint Review is the full stop marking the end of a sprint.

Its very addictive to the business.  The product owner will really start to get involved when at the end of each sprint demo after demo builds more working product.  Its a very important part of forging the teams relationship with the business.

Problems you may have with demonstrations.

Problem: We find that at the end of a sprint we have nothing that can be demonstrated!
Experiment with: Longer iterations.  Find ways to make it possible to demonstrate, look for blocks, missing environment and work hard to plug the gaps.  Its vital to demo never de-prioritize this.

Problem: We have difficulty getting the product owner to attend.
Experiment with: Shorter iterations.  Less to demo should mean shorter meetings that are easier to schedule.  Always do the demo at the same time and day.  This is also easier to plan for.

Problem: The team has no time to do demonstrations, they are busy building new functionality.
Experiment with: Plan less and schedule less stories.  Its vital to know you have actually finished an iteration.  A common scenario for teams who do not demo, is one where the project is just collecting more and more partially finished stories because of one block or another.  This can even be hidden as with no demo you have not the assurance that what you build is correct.  So you only think your complete.  There will be a price to pay later in re-work during test or worse after go-live.

Problem: We tried sprint reviews but they broke down after several chaotic attempts.
Experiment with: Create a story for Demo preparation. Have it in mind that your telling a story. Lead the product owner through the stories in a logical order that builds. See links below for more on how a review should be conducted.

Conclusion

Always hold a Sprint review with a demo.  It has hidden benefits and builds confidence in the team.  Don’t underestimate its power to motivate.  Business confidence and reporting accuracy will also improve.  Greater understanding of the product and business are other side effects.  So plan to demo now and enjoy, it can be the best part of the sprint.

Further Reading

Simple Scrum Sprint Review Note the amount of time allocated to demo in this example!

Successful Sprint Reviews Excellent work on how to make them work.

                          

Pushing the cart, Matheran, India: Martin Harris

I went along to the London Scrum User Group yesterday evening.  For a change it was a quiet night.  Christmas is around the corner so we had less attendees.  Nigel Baker of AgileBear kicked off and suggested putting together 15 tips for good scrum.  After some discussion, we came up with 6 good ones, and in true Agile style, we decided that if you did these 6 well, you would be in front of the pack.  So we stopped there and got on with eating the snacks and drinking the beer.  The night was sponsored by Rally Software, cheers for the food guys.  So here is what the group came up with, look at your team and ask yourself if your doing these, if not, perhaps its time for a scrum experiment?

The London Scrum Groups 6 Good Scrum Tips

• Love your product owner. The group agreed that the product owner should be part of the team. Include them in the meetings and get them involved. Its possibly the most important thing you can do for success in scrum. A fully integrated product owner will spot early on if the stories do not match their expectations. They negotiate the definition of done for a story. They are on hand to answer questions during the iteration removing waste and improving understanding of the stories. The product owner decides if the team has finished stories at the demo. Working closely with the product owner can avoid going adrift and missing your goals, saves a lot of stress when things hit a rough patch as they get to see the problems first hand. We agreed that this point can not be understated, if you do nothing else do this.
• Run Retrospectives. Its very important to take actions away from a retrospective.  Be realistic though, your never going to solve them all, so ask the team to priorities them.  If your doing the retrospective right your product owner will be there to help with prioritisation.  If you find something very big lands at the top, split it down into stories.  Otherwise pick one or two that the team feel strongly about and turn them into stories.  Make sure these stories are included in the next game planning sessions and make it into the iterations.  If you have adjustments to the process you can implement these straight away, but experiment with it, and try to measure the impact of changes, you might not get the process right first time.  Commit to doing them and then deliver.
• Ask your team to Pair Code. The XP technique of two programmers working on the same task.  It was agreed that there are different kinds of pair coding and that they all have a place, but the one we are talking about here is where two equal programmers work together to improve quality and throughput.  Don’t be dogmatic, let the team decide how much work should be pair coded.
• Setup Self Directed teams. Self directed teams have been proven to be more efficient.  We discussed the role of a scrum master in a self directed team.  Its very important that the scrum master does not tell the team how to work, or how to go about completing the tasks.  The scrum master does not plan or allocate tasks.  The empowered team needs to work out what the tasks are and find out how to finish the stories.  The scrum master should spend his efforts removing blocks for the team, checking quality.  Its important for the team and scrum master to spot if someone is not completing their work for whatever reason, but a strong team will sort out those kinds of issues if truly self directed.  We also decided that to be empowered you need to make the team multi discipline.  Include testers, user interface designers etc to remove hand off waste and increase team knowledge.  With role diversity comes better decision making.
• Deliver what you commit to. Another gem, it sounds obvious but is so often ignored.  Delivering builds trust in the team and the process.  Classic ways to miss delivery include: Failing to produce a strong definition of done.  The definition should include the programming, integration, testing and setup tasks.  In fact everything required to get that task ready for delivery.  Another way to miss delivers is to fail to demonstrate at the end of the iteration.  You may think your done, but when the product owner sees the work for the first time they may request refinement.  If you have kept your product owner close then the demo is likely to be painless.  No power-point slides please, only real working software in the demo!  So commit and then deliver what you commit to.
• Co-locate your team. The group defined co-location quite tightly.  Co-location is not putting everyone in the same office.  Its putting the team members next to each other in the same space.  Intra team communication does not happen with the team scattered around an office.  You should be able to turn around and join the stand up meeting.  This closeness, speeds up the myriad of tiny important messages that pass around the team.  Some of which is non verbal.

After this rich discussion we gave up on the 15 tips idea.  If your doing these, then your doing very well indeed, and are likely to get better over time.

So another excellent meeting, a few more beers and I headed home enlightenend, well fed and slighlty merry.  Why not come along to the next one, we could use your input.