In my opinion ... they do not exist. This is probably because, that would make the world of testing rigid and narrow in most of the ways. This would disturb the "software ecology" balance - kind of having a perpetual motion machine.

or

they might exist in some one's world of testing not universally in irrefutable way ...

Shrini

Paul replies...

I've answered your other comments - they might help.

But I think this notion of rigidity misses the point. The axions, as I read them tell me - you have better think about X because there are some fearful consquences if you guess wrong or choose wrong or drift into some action blindly.

What do the axioms mandate? That you think and consider your options. That's all.

Paul:
As you’re now considerably aware, your requested debate over whether a set of irrefutable text axioms exists is getting up a head of steam over on James Bach’s blog, http://www.satisfice.com/blog/archives/122 . I’ve already posted an initial reply to James, and he has reply-replied engagingly. I intend to post further to that, one reason being to try to encourage James to reply directly to your blog! But I’m waiting for a few other people to state their position – it’s gone a little quiet.

The word “axioms” triggers vigorous debate – as you know, my own original name was “always-good-practices” and I did withdraw from that to “always-good-principles”. Even then, James I believe is more amenable if they are called heuristics, and I’ll concede that also, though I think my message is now that some heuristics are more fundamental than others. I’ve been suggesting (STARWest 2007) that heuristics in general are at the top (or fundament!) of a hierarchy, and heuristics may be refined into patterns, and patterns in time may become fossilised into techniques. The fossilised techniques are more impervious to the march of time: in the case of software testing (opposite of geology!) probably to their detriment. But leaving that aside – for now we’re talking just about the driving heuristics...

As regards “needing no proof, self-evident, irrefutable” – you put that up for debate, so I hope that debate continues. My “always-good-principles” (things to think about, and implement in an appropriate, or context-driven, way) currently number 33, but they start from only two that I would now claim to be axiomatic – testing must be effective, and should be efficient. The must/should distinction is deliberate. And this is one of my answers to “why attempt this, what is the mentality of these people?” – surely no-one in any context could claim they want to be ineffective or inefficient? So we all have our own ways to pursue those two axioms: all the other principles in my view flow from that and challenges to them are valid, and at some point may succeed. But it’s a framework – it seems to me more useful than saying “everything else depends, all heuristics are negotiable”. One of my early principles is “all testing must be based on risk”, whereas James states (slide 19 of the Rapid Testing course he gives with Michael Bolton) more cautiously “most testing is driven by questions about risk... so, it helps to relate test results back to risk”. James would probably say his wording is more thoughtful, and maybe it is, but for the utility of the rest of the world – is not some kind of Pareto principle useful here?

I don’t believe most of our clients expect to be invited to select between schools of testing – please see my response to http://uktmf.com/blog/paulgerrard/2008/02/schools_out.html .

And for the proposed content of the “axioms”, please see my separate reply to your corresponding post, http://uktmf.com/blog/paulgerrard/2008/02/the_12_irrefutable_testing_axi.html .

Regards,
Neil

Paul replies...

I've looked over at James' blog but not seen anything that I feel the need to comment on immediately or that I haven't already considered. His post is about a non-existent school. I will be doing more work on the axioms (as currently stated in their superficial way).

I'm trying, in my responses to the axioms posts, to answer with examples what the axioms mean. It's a slow business because a blog isn't the best media for that. It's been interesting to me to see the reaction to the axioms. I stand by them as axioms because NONE of them (except the coverage axiom) forces any kind of method, heuristic, process or practice on a tester.

I would like them to be read as directives. That is:
1. identify and engage with stakholders
2. identify and agree how you will identify things to test.
3. Identify your oracles
and so on.

The responses to these directives are either positive or negative. I can use the axioms as a checklist or as a way of selecting practices/heuristics etc. E.g. the prioritisation axiom says - you need a mechanism for prioritisation. It doesn't say what that mechanism is and depending on context, you'll make a different choice.

The axioms, as they appear, were written in not much more than one hour. they could be worded better, and need examples. I promise to do improve them and make them more concrete.

There isn't a risk-based testing axiom because I think that a risk could be used in so many ways. Your test basis (1), scope management (4), coverage (5), prioritisation (9), sequencing (10), design method (11) and repeat-test policy (12) all may be influenced by risk - it's your choice. Numbers are axioms.

There aren't efficiency/effectiveness axioms because these are relative concepts (efficient compared to what?) but I'm not sure they are testing-specific. One could say effective/efficient applies to all human activity. I could have said, 'create a work-breakdown structure' but that's more a project management activity than testing.

Does that make sense?