First of all i should clear-up what mean to me as test cases. For me, test cases are list of test ideas by which test ideas can be listed as bullet order, row wise or any other format convenient to me for particular project or client. It also required to add tips or guideline for those test ideas which i think other can face difficulties to understand my test idea such as (but all are optional):
Why required such test cases?
I know, many of the intelligent testers, test experts, consultants, will argue with me - why you required such test cases? Are you want to feed your testers? You should teach your testers - how to test, how to explore, how to use heuristics, how to use oracle, blah...blah...blah. Here i completely agree that i should teach my testers. And also i use those test cases to teach my testers, here might have another question....How?
Here i like to share my experience and context but others might have different context (i don't know). I have some well experienced intelligent testers, some are minimum experienced (less than 1~2 years) but promising, some are cub newbies and some are even fresher in my team. I never can expect i will get all the well experienced testers in my team (that will exceed my company budget!). So, what will happen when i assign to test a particular feature to an experienced/expert tester, suppose (s)he will generate 20 test ideas to test that feature. What if, i assign same feature to test to a cub newbies or even to a minimum experienced tester, will (s)he able to generate all the 20 test ideas which can generate by an expert tester? even-though, they are intelligent enough and i teach them how to test, how to explore.........often not possible because an experience tester learn/know more test ideas by his/her experience and those ideas may not be able to generate by an inexperienced testers, even though (s)he is promising and intelligent enough. So, finally what will happen if i assign to test a feature to cub newbies? (s)he can miss some important test to execute, by which some significant issues can be uncovered on that feature. Here test cases which contained listed test ideas will guide them for proper testing, help them to know several test ideas....and so on.
What more to do with test case?
Motivate testers to think beyond the test cases and to generate more test ideas while executing and list-out those test ideas by which it can be possible to know particular tester's thoughts, intelligence, exploring capabilities...and so on. Here another thing is related to my organization but may not in other organization, there are many clients want to see the test cases to ensure test coverages, which is valid and better than blindly trust to the test team and i think it somehow reduces some risk in some degree according to my context.
Readers, i like to know your point of view and experiences in this regard to clear-up my understanding and to remove misconceptions if i have any!
Enjoy testing! enjoy learning!
-----
(!) NOTE: There are few small updates made on the original post (mostly changed the choice of words for proper interpretation).
- Required test data to execute the test if my test have required any specific data.
- Environment related information if my test required to make any specific changes on the test environment.
- Steps to execute the test if there have any complicated step to execute the test.
- Also need to add tips for any special thing to follow while executing.
Why required such test cases?
I know, many of the intelligent testers, test experts, consultants, will argue with me - why you required such test cases? Are you want to feed your testers? You should teach your testers - how to test, how to explore, how to use heuristics, how to use oracle, blah...blah...blah. Here i completely agree that i should teach my testers. And also i use those test cases to teach my testers, here might have another question....How?
Here i like to share my experience and context but others might have different context (i don't know). I have some well experienced intelligent testers, some are minimum experienced (less than 1~2 years) but promising, some are cub newbies and some are even fresher in my team. I never can expect i will get all the well experienced testers in my team (that will exceed my company budget!). So, what will happen when i assign to test a particular feature to an experienced/expert tester, suppose (s)he will generate 20 test ideas to test that feature. What if, i assign same feature to test to a cub newbies or even to a minimum experienced tester, will (s)he able to generate all the 20 test ideas which can generate by an expert tester? even-though, they are intelligent enough and i teach them how to test, how to explore.........often not possible because an experience tester learn/know more test ideas by his/her experience and those ideas may not be able to generate by an inexperienced testers, even though (s)he is promising and intelligent enough. So, finally what will happen if i assign to test a feature to cub newbies? (s)he can miss some important test to execute, by which some significant issues can be uncovered on that feature. Here test cases which contained listed test ideas will guide them for proper testing, help them to know several test ideas....and so on.
What more to do with test case?
Motivate testers to think beyond the test cases and to generate more test ideas while executing and list-out those test ideas by which it can be possible to know particular tester's thoughts, intelligence, exploring capabilities...and so on. Here another thing is related to my organization but may not in other organization, there are many clients want to see the test cases to ensure test coverages, which is valid and better than blindly trust to the test team and i think it somehow reduces some risk in some degree according to my context.
Readers, i like to know your point of view and experiences in this regard to clear-up my understanding and to remove misconceptions if i have any!
Enjoy testing! enjoy learning!
-----
(!) NOTE: There are few small updates made on the original post (mostly changed the choice of words for proper interpretation).
10 comments:
Hi Selim,
A nice post. I am agree that test cases can be said as formatted test ideas but I have few questions:
1. As you said
"Here test cases which contained listed test ideas will guide them for proper testing, help them to know several test ideas....and so on."
- What is difference between a checklist and test cases in this scenario. I think you are using test cases here as checklist.
2. As you said
"Give open challenge to the testers to generate more test ideas while executing and list-out those test ideas by which it can be possible to know particular tester's thoughts, intelligence, exploring capabilities"
- I was wondering when you assign the test ideas from senior testers to newbies then how they (newbies) are being challenged and how they are learning. Aren't they just following the test ideas of other testers.
3. How you decide that a promising newbie tester is matured enough to create his own test ideas when you are making him to follow the test ideas of other testers?
@Mohit,
Thanks for your comments.
- What is difference between a checklist and test cases in this scenario. I think you are using test cases here as checklist.
it is not necessarily to have any difference between checklist and test case but there should have mentioned tips/guidelines to execute (if required).
- I was wondering when you assign the test ideas from senior testers to newbies then how they (newbies) are being challenged and how they are learning. Aren't they just following the test ideas of other testers.
Same problem can be happened whether you give them any test ideas or not. Who can not learn from your given test ideas or not being able to generate more test ideas, how s(he) will do it better without those?
3. How you decide that a promising newbie tester is matured enough to create his own test ideas when
Same like previous comment
Again thanks for your comments.
Hi Selim,
Great efforts in publishing the post and thank you for sharing your experience with the world of testers.
Documentation is required and I think to some extent you are right about some of the critical test ideas that could be executed by newbie testers [ But also one question in my mind is Why hire such testers who cannot think of cool test ideas - Let me not talk about that now ].
Hi Mohit,
I saw a question about Selim making his testers to execute test ideas but how about referring to those test ideas triggering new test ideas. Test Ideas can be still used as a team work because it is possible to miss some test ideas in different times or depends on thinking capability of a tester. All individuals think differently and have good thinking in different different areas.
Please do correct me if I am wrong.
Thanks,
Santhosh Shivanand Tuppad
My blog : http://tuppad.com/blog/
I am a little concerned about some of your assumptions in this post. But before that I would like to say that I am glad that you are blogging, and I hope you keep it up. I know I disagree with you a lot of the time, but only because I feel you are worth it.
It seems you worry a lot about test repeatability. There is no problem with that if you are aware of the side effects. Usually following scripts will make your testers stop thinking. It is very difficult to get over this cognitive bias. But not impossible. I am not that concerned about my new testers not executing the same tests. In fact I encourage them to always come up with new tests (which you probably do too). Every time I have hired new testers, they have discovered new problems that we had overlooked for a long time. I do have checklists for regression tests, but these are very high level lists, that promote thinking about how to test creatively. Also there is a common antiprocess of giving new testers scripts to execute to teach them the product and to teach testing. That does not work, ever! It only turns them into zombies.
Also, I don't agree with your comment:
"there are many clients want to see the test cases to ensure test coverages, which is valid and better than blindly trust to the test team and i think it somehow reduces some risk in some degree according to my context"
You will never be able to show them all the test cases you execute, because most are not scripted. So that doesn't ensure anything nor does it reduce risk. It only deceives your client and gives them an excuse to be satisfied with their delusions. If you really need to, then there are other ways to express to your clients what and how you are testing. Look up Session Based Test Management. Trust is important. They have to trust that you know more about testing than them. That has to be there. Without that trust, whatever you do will not appease them, or make them learn from you. Try to build that trust. Once you have trust your status reports will be more meaningful to them. It would even educate them about changing their expectations.
@Santhosh,
Thanks for your comments.
[ But also one question in my mind is Why hire such testers who cannot think of cool test ideas - Let me not talk about that now ]
would you expect that all of your testers have same level of skills and experiences, in my case i can't expect such.
I like your comments " to refer those test ideas triggering new test ideas. Test Ideas can be still used as a team work because it is possible to miss some test ideas in different times or depends on thinking capability of a tester. All individuals think differently and have good thinking in different different areas."
Again thanks for your valuable comments.
@Sajjad bhai, again thanks for your valuable comments. Below i have given my comments based on my experience, you may have different point of view.
But before that I would like to say that I am glad that you are blogging, and I hope you keep it up.
I have inspired by you, especially directly inspired by Pradeep Soundararajan, Parimala Shankaraiah, Santhosh Tuppad, Sharath Byregowda, Dhanasekar, Ravisuriya...i'm grateful to all of you.
I know I disagree with you a lot of the time, but only because I feel you are worth it.
As i mentioned earlier, i always enjoy transmitting conversation with you because sometimes it give me different perspective of my thoughts and help me learning as well.
It seems you worry a lot about test repeatability.
It was not only about test repeatability, it's not necessary to repeat all the test ideas/cases for all the test cycle.
Usually following scripts will make your testers stop thinking. It is very difficult to get over this cognitive bias. But not impossible.
I encourage testers to think about new test ideas and always testers come-up with new ideas whether i gave them any listed test ideas or not. Anyone may have different experience.
I do have checklists for regression tests, but these are very high level lists
If you think that all your testers are mature enough to execute from high level checklists, ofcourse it is a easier way to proceed.
Also there is a common antiprocess of giving new testers scripts to execute to teach them the product and to teach testing. That does not work, ever! It only turns them into zombies.
Might be still i have some lack, it's not understandable to me why your test ideas will not help testers to know/learn about important/interesting test and drive them to think about testing or even more. And how a new testers will generate all the test ideas you have learned/known from your long experiences, it may possible by some extra ordinary genius but can't expect from all new testers.
You will never be able to show them all the test cases you execute, because most are not scripted.
I agree, it's not possible to show them all the test which we executed.
So that doesn't ensure anything nor does it reduce risk.
At least we are able to show them how much we are think about the product and it's features, also client can help us to determine which we miss from client expectation.
If you really need to, then there are other ways to express to your clients what and how you are testing. Look up Session Based Test Management.
I have already tried SBTM but could fail to prove it's effectiveness and usefulness, ofcourse could not fail to try it again again.
Trust is important. They have to trust that you know more about testing than them. That has to be there. Without that trust, whatever you do will not appease them, or make them learn from you. Try to build that trust. Once you have trust your status reports will be more meaningful to them. It would even educate them about changing their expectations.
It is easy to establish such trust when you are working for a client, for a product, for the longtime. We have very few longtime project for a particular client. We are doing many short-duration/mid-duration projects for several clients. Also it is important to consider typical Japanese clients, it's not so easy to change their usual practice. We should do what client want from us to make client happy, still we are not in that position to ignore client.
Test cases in most places are the kind of Firist Information Report for any newbies or any one new to the product.A
If the question is it necessary or not? My answer would be it is not necessary for expert testers.They can learn quickly and can test with simple checklists.n expert tester may not need test case to learn\explore a new application,but we do not have experts everywhere.
But when it comes to good or bad. The perspective differs. From company perspective it is good because countries like India where job hopping is common, they need a very detail test cases,so that a tester is productive from day one of joining(mean starts executing the test cases blindly). But that is bad for a testers,because most of the testers are more than happy just executing them,they don’t add any value to company or to them self just be executing test cases.I would blame the testers for not coming out of it.
Even James Bach in his article http://www.satisfice.com/articles/what_is_et.shtml said "Scripting has its place. I can imagine testing situations where efficiency and repeatability are so important that we should script or automate them"
--Dhanasekar S
http://testingideas.wordpress.com
@Dhanasekar,
Thanks for your valuable comments.
i agree with your comments, test cases are not required for expert testers but i think it's required for newbies or who have very limited experience in testing. Here i like to mention, i'm not talking about detail test cases, rather i'm talking about list of test ideas with tips or guideline for those test ideas which i think other can face difficulties to understand to execute my test idea. At the same time it's important to motivate them to explore during executing those test cases to identify new test ideas.
Thanks,
- Selim
@Selim,
Nice post. I do agree with you to the point that sometimes you can not control client's requirements. So if client wants test cases you have to write cause testers are getting paid for it.
But if you continue the process of writing test cases and executing them you might be not utilizing testers creativeness effectively enough as they might be stuck with the ideas written in the test case format.
Actually our education system is like test cases written by others. To gather knowledge from that is expected but to blindly follow the education system will not improve your creativeness.
So Test Case write and execute scenario by newbie testers is acceptable to have a idea what might be testing is but to invent new ideas by executing the test cases should be the obvious expectation and should be encouraged to do that.
Regards
Ashik
@Ashik,
Thanks for your comments.
I always looking for creativeness of my testers, hence, i motivate testers to think beyond the test cases and to generate more test ideas during execution.
again thanks for your comment and for sharing your thoughts.
- Selim
Post a Comment