This post was triggered by Alan Page's post here about iterations. Like keep iterating often. Isn't this kinda like evolving something? Well I thought so, what did I first think of?..I think as Alan did....Test Plans and strategies
Why did I think of test plans and strategies? Well I just think that a test plans/strategies should be an evolving thing, have iterations. It is all too often that I see a Test Plan/Strategy that relates to a document. A document that gets written 'before' testing begins and left. It becomes out of date doesn't it? I have seen this a lot, I don't particularly like it that much, because most of the projects I have been on generally evolve, you gain more knowledge as you progress..focus, may change, risks may change, environments change, time changes, stakeholders change? etc etc
I know it's not always possible but I prefer an evolving test plan in my head that I try and transfer somewhere via /notes/scribbles or a whiteboard (Or does everyone just have that? and does everyone have that even if a document may be out of date?) Why? I have seen it all to often that a test plan/strategy document has been created and followed to the 'letter' and lots of tests get run and pass - All good right? or is it?
What happens when things change? The plan/strategy has to change doesn't it? and that affects the way we test right?
What I like to do is have a thin spread of sessions, across percieved high risk areas and see what happens. (these might not necessarily be sessions of test execution either, they could be meetings with a developer or stakeholder) , Are there more issues being found in one area than an other? If so should a deeper session be created for that area? Does that area become higher risk because more is being found there? Maybe, and that's where I might want to change my plan and strategy. I want to find the 'failure points' as quickly as possible. (Preferably the failure points that are important to the stakeholder(s) of the project)...What if you're finding no issues in a particular area but you intended to do lots more testing in that area? Do you still want to do that? You might want to change your focus away from that area...isn't that a change of plan and a potential change of strategy?
While writing this blog, I reverted to this document which I think aligns to what I am trying to say.
Also I can feel myself getting sucked into the checking and testing debate....eek....So should their be checking plans and testing plans? or is checking part a test strategy? i.e. sapient processes surrounding the check?, or is checking part of both..a checking plan and a testing strategy?...(It's now I'm probably confusing myself...)
Do your plans and strategies change during your testing? How do you document that? Do you need to? and how do you deal with the changes?
Michael Bolton alluded to the problem of measurement in his posts about testing here and here.
Michael will be able to explain things better than me no doubt in future posts ( like way way better) , but Jonathon Bach outlines an approach here.
I know this may not work for all situations but I have found that a customized low tech Testing Dashboard has helped me alot along with beginning of the day stand up meetings (What did you do yesterday, what are you doing today, what did you find, what is your feeling etc) I think these help in the project team understanding what you are doing, how you are doing it what issues you are incurring, understanding possible progress...