Monday, February 6, 2012

Automation/Requirements Document/Process/Certification cannot find bugs

Product being tested --- Breath analyser to detect alchohol!!!

Objective of product --- Analyze the air to identify if the person blowing air has had alchohol or not.

What the product did not do --- Analyze if the person being tested has actually blown his air or not.

And the test case --- Get Drunk. totally drunk. get analyzed by the breath analyzer, but don't blow air into the equipment.

And the test case result --- Failed. Since the breath analyzer does not detect if you actually blew air into the equipment or not.

And what's the bug? --- Expected behavior is that the system should detect if the person is blowing air into the equipment or not. Actual behavior is that it does not detect this.

And you won't find this test case in the requirements document; not in boundary value analysis or equivalence method or some such method; no testing certification can help you detect this flaw; no 6 sigma process or CMMi process can help you find this test; and no automation suite can help you prevent it.

In spite of all of the above, this bug has been around in breath analysing equipment for a long long long time. That proves the theory that there are more fake testers than me around :). Anyway, the point I was trying to make was that testing is best left to humans and not to automated suites, or processes, or methodologies. The best tester is still the man, and not the machine!!!


  1. I absolutely love your perception of de-meriting the automation vis-a-vis Manual testing, However I feel,its a one off thing and it does not mean that the Automation tests are any where lesser than Manual testing.

    Infact, when the things gets boring (same test cases for multiple cycles across months), the automation is best friend of a manual tester. Afterall, the ensuring of quality is a tester's responsibility. It tends to get boring and monotonous to check same thing again and again, and hence become a candidate for skipping an important change. Automation is the best help there.

    Hence, What you said is perfect, but only limited to the current example.

    1. what I was saying was that there's some amount of subjectivity in each product that we test; and we cannot leave it to an automated solution for the results.

  2. Well that could have been tested in the initial stage of product testing like smoke testing.. Its a tester's mistake ofcourse !

    1. well, nobody saw the "smoke" in smoke testing and passed the case; in this case, the smoke emanating from the machine :). BTW, what's a "smoke test"?

  3. Nice :)

    Off Track Thought: how much drunk would these manual-testers who would be actually testing the "Breath analyser to detect alchohol!!!".... officially getting drunk in the office premises during the office hours... lol...

    1. :)... well, I could have contacted you for testing, but you haven't left your contacts :)!!!

  4. Hello, Neat post. There's a problem along with your website in internet
    explorer, could test this? IE nonetheless is the marketplace chief
    and a big element of other people will miss your wonderful writing because of this problem.

    Also visit my web-site: cheap mlm leads