The Inovis Blog

Read Part 1 of this post by Linda Eason>>

Disclaimer: The opinions expressed in my blogs are based on my personal experience with testing in an agile development environment – warts and all.  What works for one person, product, or company may not work as well for another.  My hope is these blogs will give the reader some insight into agile testing that they can then use to help them inspect and adapt their own QA process.

Continuous Quality

It is common knowledge that the later you discover a defect the more expensive it is to fix.  This is repeatedly stated in almost every development and QA book on the market.  And yet; most development environments today ignore quality until the very end of the process when it will be the most difficult and expensive to address.  Why?  There are a variety of reasons but in my experience they all boil down to habit, ineffective management and a lack of commitment to quality.

Quality takes focus and effort.  It is the responsibility of the entire team not just one part of the team.  Most of all quality takes commitment – because to do it right takes a tremendous amount of continuous effort.  You have to spend time working on quality. This is where a lot of development environments drop the ball.  They epitomize the old saying “You don’t have time to do it right, but you have time to do it over” and I would add “at a later date when it will be the most expensive to do.”

Continuous Quality occurs when quality is built into all phases of the development cycle.  You don’t wait until the end to start measuring quality.  You take quality measurements during each step of the development process and – this is the important part – you stop the process until quality is where it needs to be before you go to the next step.   You don’t allow the code to regress as you create new code.

Implementing continuous quality is not a trivial task.  It takes time, a certain amount of grunt work and a tremendous amount of commitment from team leaders/management.  You also have to make sure you don’t lose ground when the inevitable push back happens.  Human nature being what it is – everyone will want to start taking short cuts as soon as any delay happens due to the continuous quality measures.  And trust me – delays will happen.  You’d be amazed at how many issues continuous quality measures uncover before QA can even begin testing the new code.

Continuous Quality in Agile

Maintaining a high level quality of code is important in any development environment, but for an agile development environment it is essential.  Why?

It keeps your code releasable

Each sprint needs to produce releasable code. If you don’t maintain a high standard of quality – your code is not going to be releasable.  Poor quality code is extremely fragile.  Once you start checking code into a fragile code base you risk breaking a lot of other code which means at the end of a sprint more things are broken then fixed.

It keeps the application testable

You have to be able to test the code in the sprint and if the code is of poor quality before the sprint begins your testing will be blocked by all the defects and miscellaneous issues you will discover as you try to test the new code.

It keeps velocity constant

If the developers have to constantly work on issues in other parts of the code base, they cannot focus on their sprint tasks.  If they found serious issues in the code they are working in, they will have to create new user stories/defects to fix that code and push the planned sprint work to a later sprint.  This makes it very difficult to accurately plan iterations.

Continuous Quality in Actionable Intelligence

• All unit tests are automated.
• Each code check-in kicks of a new build (Continuous Integration) and all unit tests run on that build.
• All unit tests have coverage targets:
  • New code must be covered at 80% or better
  • Existing code must be covered (minimum) at the same level that it was before the new code was check in (over time this increases the coverage).
• If a unit test fails, the build fails.
• If the unit test does not meet the coverage requirements, the build fails.  This means that a build will fail if the coverage target is 80% but the coverage is actually only 79%.  Dev must bump up the coverage before we can get a build.

The above measures are the heart of our continuous quality measures.  They give us immediate feedback into the quality of the build as new code is introduced.  They are also designed to prevent regression of quality.  It is important to note that it is the combination of the above that provides continuous quality.  Automating unit tests in not enough – you have to run them on every build.  Running them on every build is not enough – you have to fail the build if the tests fail, etc.  In my opinion the step that really pulls everything together is setting coverage targets and then failing the build if those targets are not met.  That one step is the most important and most controversial of our continuous quality measures.  I will explain the controversial part in my next blog

More AI continuous quality measures:

• Provide immediate access to the build.  As soon as a build is complete (passes all unit and coverage tests) anyone can install the build for testing.  If needed, we can install more than one build in a given day.
• QA install builds the same way our customers do.  We test the install every time we test a build.
• We fix defects when we find them.  I will discuss defect management in an upcoming blog.
• Continuously run automated functional tests on the builds.   We have not implemented this yet.  More on this effort in a later blog.

AI Technical Summary

Unit Tests: JUnit

Coverage Targets: Emma

Continuous Integration System: BuildForge

Version Control: Subversion

Functional Test Automation: Selenium

What is the number one payoff for implementing continuous quality?

You can release more reliably and more often.

In these tough economic times software companies must be responsive to their customers needs.  Time is money and the faster you can get code out the door the more money your customers can make.   The Actionable Intelligence SaaS product releases into production every two weeks. Yes you read that correctly, we release into production every 2 weeks.  It would be impossible for us to release so often if we did not have continuous quality embedded into our development process.  It is the foundation on which we base everything else.

In my next blog I will talk about the actual process we followed to get our continuous quality measures where they are today and the challenges that we still face.