Help me understand how QA works in Scrum [closed]

Answer 1

We solved this problem as follows: - Every item in the product backlog must have fit criteria or acceptance criteria, without those, we don't start a sprint - A tester is part of our team, for every product backlog item, he creates test tasks (1 or more, based on the acceptance criteria) together with an estimation, and a link to the item to test - During the daily scrum, all tasks that are finished are placed in a 'To Test' column - We never do tasks that take longer than 16 hours; tasks that are estimated longer, are split up

Answer 2

Speaking as a QA who has worked on Agile projects for 2.5 years this is a really difficult issue and I still don't have all the answers.

I work as part of a "triplet" (two developers who pair program + one QA) and I am involved in tasking out stories and estimating in planning meetings at the beginning of two week iterations. As adrianh mentioned above it is essential for QAs to get their voice heard in the initial sprint planning. This can be difficult especially if you are working with Developers with very strong personalities however QAs must be assertive in the true sense of the word (i.e. not aggressive or forceful but respectfully seeking to understand the Truth/PO and Developers/technical experts whilst making themselves understood). I advocate producing QA tasks first during planning to encourage a test driven mentality - the QA may have to literally put themselves forward to get this adopted. It is opposite to how many people think software development works but pays dividends for several reasons;

1. QA is heard and not relegated to being asked "so how are you going to test that?" after Devs have said their piece (waterfall mentality).

2. It allows QA to propose ideas for testing which at the same time checks the testability of the acceptance criteria while the Truth/PO is present (I did say it is essential for them to be present in the planning meeting didn't I?!) to fill in any gaps in understanding.

3. It provides the basis for a test driven approach - after the test approach has been enunciated and tasked the Devs can think about how they will produce code to pass those tests.

4. If steps 1 - 3 are your only TDD activity for the rest of the iteration you are still doing a million times better than the scenario postulated by Steve in the first post; "Developers thrash around trying to accomplish their tasks. Generally the tasks take most of the sprint to complete. QA pesters Dev to release something they can test, Dev finally throws some buggy code out to QA a day or two before the sprint ends and spends the rest of the time fixing bugs that QA is finding"

Needless to say this comes with some caveats for the QA;

1. They must be prepared to have their ideas for testing challenged by Devs and Truth/PO and to reach a compromise; the "QA police" attitude won't wash in an Agile team.

2. QA tasks must strike a difficult balance to be neither too detailed nor too generic (tasks can be written on a card to go on a "radiator board" and discussed at daily stand up meetings - they need to be moved from "in progress" to "completed" DURING the iteration).

3. QAs need to prepare for planning/estimation meetings. Don't expect to be able to just turn up and produce a test approach off the top of your head for unseen user stories! Devs do seem to be able to do this because their tasks are often far more clear cut - e.g. "change x module to interface with z component" or "refactor y method". As a QA you need to be familiar with the functionality being introduced/changed BEFORE planning so that you know the scope of testing and what test design techniques you might apply.

4. It is almost essential to automate your tests and have these written and "failing" within the first two or three days of an iteration or at least to co-incide with when the Devs have the code ready. You can then run the test/s and see if they pass as expected (proper QA TDD). This is how you avoid a mini waterfall at the end of iterations. You should really demo the test to the Devs before or as they start coding so they know what to aim for.

5. I say 4 is "almost essential" because the same can sometimes be successfully achieved with manual checklists (dare I say scripts!) of expected behaviour - the key is to share this with Devs ahead of time; keep talking to them!

With regards to point 2 above on the subject of the tasks, I have tried creating tasks as granular as 1/2 hour to 2 hours in size each corresponding to a demonstrable piece of work e.g. "Add checks for incorrect password to auto test - 2 hrs". While this helps me organise my work it has been criticised by other team members for being too detailed and has the effect at stand ups of me either moving multiple tasks across to complete from the day before or not being able to move any tasks at all because I have not got onto them yet. People really want to see a sense of steady progress at daily stand ups so it is more helpful to create tasks in 1/2 day or 1 day blocks (but you might keep your own list of "micro-tasks" to do towards to completion of the bigger tasks that you use for COMMUNICATING overall progress at the stand-up).

With regards to points 4 and 5 above; the automated tests or manual checklists you prepare early should really cover just the happy paths or key acceptance criteria. Once these pass you can have planned an additional task for a final round of "Exploratory testing" towards the end of the iteration to check the edge cases. What the Devs do during that time is problematic because as far as they are concerned they are "code complete" unless and until you find a bug. Some Agile practitioners advocate going for the edge cases first although this can also be problematic because if you run out of time you may not have assured that the acceptance criteria have been delivered. This is one of those finely balanced decisions that depends on the context of the user story and your experience as a QA!

As I said at the beginning I still don't have all the answers but hope the above provide some pointers born out of hard experience!

Answer 3

It seems to me that there is a resource allocation problem in scenarios requiring QA functional testing in order for a given feature to be 'done' within a sprint. No one seems to address this in any QA-related scrum discussion I've found so far, and the original question here is almost the same (at least related), so I wanted to offer a partial answer and extend the question a bit.

As to the specific original question about development tasks taking the full sprint - it seems that the general advice of easing up on these tasks makes sense if functional testing by QA is part of your definition of 'done'. Given lets say a 4 week sprint, if it takes about a week to test multiple features from multiple developers, then it seems like development tasks taking about 3 weeks, followed by a lag week of testing tasks taking about 1 week is the answer. QA would of course start as soon as possible be we recognize that from the last set of delivered features, there will be about a week lag. I realize that we want to get features to QA asap so you don't have this waterfall-like scenario in a sprint, but the reality is that development usually can't get real, worthwhile delivered functionality to QA until 1 to 3 weeks into the sprint. Sure there are bits and pieces here and there, but the bulk of the work is 2-3 weeks development, then about a week's testing leftover.

So here is the resource allocation problem, and my extension to the question - in the above scenario QA has time to test the planned features of a sprint (3 weeks worth of development tasks, leaving the last week for testing the features delivered last). Also let's assume QA starts to get some testable features after 1 week of development - but what about week #1 for QA, and what about week #4 for development?

If QA functional testing is part of the definition of 'done' for a feature in a sprint, then it seems this inefficiency is unavoidable. QA will be largely idle during week #1 and development will be largely idle during week #4. Of course there are some things that fill in this time naturally, like bug fix and verification, design/plan, etc., but we are essentially scheudling our resources at 75% capacity.

The obvious answer seems to be overlapping sprints for development and QA since the reality is that QA always lags beind development to some degree. Demonstrations to product owners and others would follow the QA sprint since we want features to be tested before being shown. This seems to allow more efficient use of both develoment and QA since we don't have as much wasted time. Assuming we want to keep developers developing and tester testing, I can't see a better practical solution. Perhaps I have missed something, and I hope someone can shed some light on this for me - otherwise it seems this rigid approach to scrum is flawed. Thanks.

Answer 4

My opinion is that you have an estimation problem. It seems that the time to test each feature is missing, and only the building part is being considered when planning the sprint.

I'm not saying it is an easy problem to solve, because it is more common than anything. But things that could help are:

• Consider QA as members of the dev team, and include them in the sprint planning and estimating more closely.

• 'Releasable Dev tasks' should not take up most of the sprint. Complete working features should. Try to gather metrics about dev time vs QA time for each kind of task and use those metrics when estimating future sprints.

• You might need to review your backlog to see if you have very coarse grained features. Try to divide them in smaller tasks that could be easily estimated and tested.

In summary, it seems that your team hasn't found what its real velocity is because there are tasks that are not being considered when doing the estimation and planning for the sprint.

But in the end, estimation inaccuracy is a tough project management issue that you find in agile-based or waterfall-based projects. Good luck.

Answer 5

Split the tasks into smaller tasks.

Also, QA can create test cases for Dev to test against.

Answer 6

A little late to the party here but here's my take based on what you wrote.

Now, Scrum is a project management methodology, not a development one. But it is key, in my opinion, to have development process in place. Without one, you spend the majority of your time reacting rather than building.

I'm a test-first guy. In my development process I build tests first to enforce the requirements and the design decisions. How is your team enforcing those? The point I'm trying to make here is that you simply can't "throw stuff over the fence" and expect anything but failure to occur. That failure is either going to be by the test team (by not testing very well and thus letting problems slip by) or by the developers (by not building the product that solves the problem). I'm not saying you must write tests first - I'm not a militant or a test-first evangelist - but I'm saying you must have a process in place to produce quality, tested, ready-for-production code when you reach an iteration's end.

I've been right where you are in this development methodology that I call the Death Spiral Method. I built software for the government (US) for years in such a model. It doesn't work well, it costs a LOT of money, it produces late code, poor code, and does nothing for morale. You can't make any headway when you spend all your time fixing bugs you could have avoided making in the first place. I was absolutely beaten down by the affair.

You don't want QA finding your problems. You want to put them out of work, really. My goal is to make QA flabbergasted because everything just works. Granted, that is a goal. In practice, they'll find stuff. I'm not super-human. I make mistakes.

Back to scheduling...

At my current job we do Scrum, we just don't call it that. We aren't into labels here but we are into producing quality code on time. Everyone is on-board. We tell QA what we'll have ready to test and when. If they come a-knocking two weeks early for it, they can talk to the hand. Everyone knows the schedule, everyone knows what will be in the release and everyone knows that the product has to work as advertised before it goes to QA. So what does that mean? You tell QA "don't bother testing XYZ - it is broken and won't be fixed until release C" and if they go testing that, you point them back at that statement and tell them not to waste your time. Harsh, perhaps, but sometimes necessary. I'm not about being rude, but everyone needs to know "the rules" and what should be tested and what is a 'known issue'.

Your management has to be on board. If they aren't you are going to have troubles. QA can't run the show and the dev group can't completely run it either. All the groups (even if those groups are just one person per group or a guy that wears several hats) need to be on the same page: the customer, the test team, the developers, management, and anyone else. More than half the battle is communication, typically.

Perhaps you are biting off more than can be accomplished during a sprint. That might be the case. Why are you doing that? To meet a schedule? If so, that is where management needs to step in and resolve the issue. If you are giving QA buggy code, expect them to toss it back. Better to give them 3 things that work than 8 things that are unfinished. The goal is to produce some set of functionality that is completely implemented on each iteration, not to throw together a bunch of half-done stuff.

I hope this is received as it is intended to be - as an encouragement not a rant. Like I mentioned, I've been where you are and it isn't fun. But there is hope. You can get things turned around in a sprint, maybe two. Perhaps you don't add any new functionality in the next sprint and simply fix what is broken. You'll have to decide that as a team.

One more small plug for writing test code: I've found myself far more relaxed and far more confident in my product since adopting a 'write the tests first' approach. When all my tests pass, I have a level of confidence that I simply couldn't have without them.

Best of luck!