Holy Smokes, Batman - QA found a bug!

Even for those of you who did not grow up with the dynamic duo, the caped crusaders probably need no introduction. I am pretty certain that neither of them would ever actually write software in their crime fighting regalia; though I can certainly see them as part of an agile development team whilst in their street clothes!


Exploiting their personalities, they have an interesting dynamic and interplay that can help to underline the thoughts behind a common question in our space:

What do we do when QA (or anyone else for that matter) finds a defect?

In order to answer it, we will break it down into two parts and answer them separately:

1) What do we do when we find a defect with a work package during the iteration it was initially developed in?

Well, we would prefer to find all problems immediately, not solely because we want to move forward with maximum velocity, but also because as soon as the developers switch mental context to another work package, it is harder for them to resolve the issue. So, the recommended approach is to follow these steps:

a) Have QA develop acceptance criteria with the Customer prior to development starts – this will ensure the developers know what it means to be “done”.
b) Have the Customer review and validate the solution prior to verification testing – this tends to maximize throughput of the system, as long as this review is kept informal.
c) Once the solution is validated, QA will verify that it meets the acceptance test in a formal build (say, after the next cycle of Cruise Control).
d) If the solution can not be verified, the changes made are still very much in the team’s mind and finding the resolution will be most expeditious. No defect needs to be raised, unless the team is tracking metrics at such a level that these details are pertinent (CMMI level 4 and 5 organizations only).
e) Once the resolution is found, as long as the solution has not changed substantially from the original validation review, then it can be re-integrated and re-verified.
f) Receiving a clean bill of health from QA allows the team member to move on to a new story with no hangover and a clear conscience.


2) What do we do when we find a defect with a work package after the end of the iteration it was developed in?

Sometimes, defects show up well after the initial integration has been completed. Commonly, these are associated with architectural issues or defects with previous acceptance criteria. Regardless, though they can be trivial (typos or unsafe internationalization), more often they are reasonable arduous to resolve. Either way, the accepted procedure is:

a) Once found, raise a defect in the tracking system and request the Customer prioritize the resolution
b) If the priority is Immediate, then the issue should be assigned to someone to assess the scope of the resolution.

• If it is small, the fix should be made and the integration re-verified.
• If it is large, the Customer should be asked to confirm that the resolution takes precedence over planned work.
• A yes here initiates a change in the Iteration plan: an equal number of points are taken out of the iteration and this new work is substituted.
• A no puts the resolution into the backlog as a new story.
  

c) If the priority is not immediate, then the resolution is posted as a new story in the backlog and assessed once the Customer decided to play that story card (this could be in the next iteration or one way down the line, there are no rules here!).

In both scenarios, it is assumed that the team follows the agreed upon workflow for defects in the tracking system. We enter them into the system specifically so that we can capture as much context as possible while the issue is current in our minds just in case it is determined that we will not address it for some time. Logging of the issue and the failure modes will give those tasked with resolution background upon which to build a new context so that they can address and correct the issue.

The real key is to always evaluate the work through its value proposition to the Customer. They may be bothered by undesirable formatting of text in an automated report, but there may never be enough value to the company to have the team change it. On the other hand, if QA finds that through some quirk of the database manipulation, the system is arbitrarily over-writing data, it could be anticipated that an immediate correction must be made.