piątek, 9 stycznia 2015

Dla kogo jest ta książka ("Techniki testowania")?

Patrze także: spis treści projektowanej książki
Patrz także: literatura i inne źródła 
For Whom and Why Is This Book Needed?

What Is It Needed for?
Testers need to know how to design test cases.
Testers' main responsibility is to choose from infinite, or at least prohibitively huge number of possible combinations of paths and data values, those few - terrifyingly few - test cases that will really be exercised during test execution and thus be the final bulwark  of defence against risk of failure, loss and catastrophe.
If this was not for test design, testing would be just requirements engineering turned upside-down, plus some knowledge in how to manage bug reports, plus a handful of tools with funny names; a kind of last resort fire brigade, needed only when error prevention fails. But it is not so, because testing means test design. Test design (and analysis, which always precedes design) is the very essence of testing, is what makes it a separate, special set of skills. 
However, no fully comprehensive book on test analysis and design techniques is available. There are three main reasons for this.
One reason is, testers and their employers alike typically look for information that is needed here and now, and which provides immediate relief. Therefore, many books, web pages and other information sources on testing, are in this respect sadly similar to job advertisements: they value detailed technical or domain knowledge more than sound, generic test knowledge. Advertisements  for "test analyst with Selenium skill and insurance industry experience" are incomparably more common than those seeking specialists with general test knowledge, capable and willing to learn required domain and technical skills fast. The result of this is, many testers learn test design techniques on the fly and only within given technology and domain boundaries, and fail to see the whole picture.

Who cares? Well, everybody should, because this way the transfer of knowledge and skill is minimal. Next time, the tester who had already learned designing test cases based on UML activity diagrams for a banking system, will need to learn designing test cased based on UML state transition diagrams for an embedded system from scratch. This way, the waste of time and money is maximal!

I can understand, even if not completely forgive, such approach in hasty, we-need-it-tomorrow project situations, but the is definitely wrong approach to learning and teaching testing. If we want really good practitioners, we need effective textbooks for them. These textbooks must not mix up test design skills with everything else, including technology and domain knowledge. Even if test practitioners need to mix these topics in project realities, they must not mix them in their heads!

The second reason is that readers', authors' and publishers' short-sighted motives take prevalence over far-sighted approach, when they choose to put together test design, test organization, test management and even test automation knowledge in one volume or on one web site, or in one presentation. This sells better, probably because it gives an impression of no-nonsense pragmatism, while treating these subjects separately may give an impression of being theoretical and academic. Again, in order to become really high-quality professionals, capable of applying our skills in many different situations, we should learn - and teach - theses areas separately. Professionals need to be able to use their skills multi- dimensionally, not only as one or two or even ten specific sequences.
The third reason is, more than half of the book market for testing is monopolized by ISTQB-based textbooks. There is only a limited number of test book copies people can buy, and since books with "ISTQB" on the title page are safer bet for the publishers, they dominate. Good-bye to serious test design! 
And of course, the main main reason is, THICK VOLUMES SELL BAD!
By learning test analysis and design techniques separately, you'll be able to use them more effectively, and really efficiently, within different test strategies, processes and test organizations, equally well for various technologies, and in all conceivable domains and businesses.

What Is Available, and What is Not?
I searched the phrase "test design techniques" in amazon.com, and what I got highest up on the list was... "Exploratory Software Testing: Tips, Tricks, Tours, and Techniques", and "Agile Testing: A Practical Guide for Testers and Agile Teams", and some more such titles :(. Even keeping my general reservations about exploratory and agile approaches aside for a while, these books were not what I was looking for: not comprehensive textbooks on test design techniques. The ones I found mix test design with many other issues, and do not make any attempt to cover all available techniques. And of course, "tips, tricks and tours" are always easier to sell than boring lectures.
Let the search go on, back to good old names. Boris Beizer's "Software Testing Techniques" and "Black Box Testing" by the same author are very good, yet far from complete. Glenford Myers' "The Art of Software Testing" (first published as early as 35 years ago!) still is and always will be the ultimate in describing some techniques, but definitely not all that are worth knowing.
Then of course there is Lee Copeland's "A Practitioner's Guide to Software Test Design". Great book, in my opinion one of the best there is on this subject. However, it is still not fully what I believe testers need today; it does not offer a really comprehensive view. Saying this, I do not mean just omitting a few rather exotic and unimportant techniques; I mean omitting a lot. For example, UML contains fifteen types of models, which are extremely useful for designing test cases; Lee's book covers two of them use case diagrams, and state transition diagrams. That was the author's choice, and I respect it, but I think more is needed by many testers.

There are books based on the world's most popular test certification scheme: ISTQB Advanced Level syllabi for test analysts and technical test analysts. Even keeping my general reservations about the contents and structure of ISTQB syllabi aside for a while, the simple truth is, design techniques with ISTQB blessing present just a smallish part of the total amount of test design techniques. For example, they hardly cover techniques specific for non-functional testing, and they are mostly limited to test design for dynamic test execution, leaving out almost completely the rules of static testing.

Paul C. Jorgensen's "Software Testing: A Craftsman's Approach" is exceptional since it refers to mathematical concepts and logical reasoning. At last a book that dares use the words "discrete math" and "graph theory"! To tell the truth, I find it hard to understand why all other books choose to teach test design without mathematics. Perhaps because context-driven school pretends testing is more a social skill than anything mathematical? However, Paul's book is very much focused on functional testing, and formal test design: good, but I believe non-functional testing, or experience-based testing must not be left to intuitive magicians, but taken care of in the same volume. 
Last but not least, one must not forget Geoff Quentin's worthwhile attempt "The Tester's Handbook", whose chapter 7 "Specific Test techniques" contains the most comprehensive list of test techniques available now. However, I find it not complete, either.

This Book Fills a Market Gap
Please note: I do not criticize these books, nor do I think I am somehow wiser then their authors. On the contrary, I will use the knowledge they have already provided maximally. My wish is to fill a market and educational gap, not a knowledge gap.

That is it: my ambition is to provide the missing link. Not for the sake of theory, nor for the sake of academic ambitions - I will hardly present anything that is not already known and described somewhere. My goal is to provide a fully practical handbook, gathering in one place the as many as possible - not just chosen few - good techniques for designing test cases. This book will help testers of all kinds, exploratory and scripted, agile and traditional, waterfall and iterative, manual and automatic. It should be equally useful for regression-testing and for confirmation-testing, for unit-level and system-level, for testing embedded and database systems, written in object-oriented or in assembly language, static and dynamic, testing software, design, for models and for documents.

Let us disregard the misleading dichotomies of white-box versus black-box, formal versus informal, web testing versus database testing, etc. There are some issues specific for some domains, but first of all, there are test design techniques common for all of them. In project realities, domain and technology knowledge helps, but the lack of general test design knowledge kills. All testers all need to make the same kind of decisions: what to actually check, and what to leave alone. This book will help them to make the right decisions.

Brak komentarzy:

Prześlij komentarz