Wednesday, June 16, 2010

Software Testing and the "Hand of Clod"!!!

England Played USA on their first FIFA 2010 World Cup match and most of the world saw Rob Green's (England Goal Keeper) mistake that let the USA escape with a draw.

The next day, the entire world have started to blame Rob Green for his mistake, attributing the goal only to his mistake. People are contemplating firing him. We even have seen the "Hand of Clod" headline in many newspapers.

I am not trying to say that the England Goalkeeper is not at fault, but let's try to answer a few questions below...

1) Who let the possession of the ball to USA?
2) Where were the mid-fielders and why did they let the USA Offense inside?
3) Where were the defenders and why did they let the USA Offense so close?
4) Why could the defenders not intercept the shot to the goal?
5) Who was assigned to mark the scorer? Where was he when the scorer scored?
6) Who selected Rob Green to play in the match?
7) Who decided to buy the gloves that the Goalkeeper was wearing? Was it tested for circumstances wherein the ball meets the gloves when the gloves are wet? What were the test results and who tested it?

And so many many many more questions.....

If the answer is not "England Goalkeeper" for any of the above questions, then I don't think that it is right to blame him for the goal.

What is important is - a part of the blame is his; even more important is ONLY A PART OF THE BLAME IS HIS. The rest of the blame has to be owned by other members of England World Cup Soccer team.

Trying to draw a parallel between this and today's software tester, now think about a scenario wherein a tester would have leaked a very easy to detect showstopper defect.

Writer’s Irritating Personal Note:- My opinion is that all of us (the readers and writer of this post) have leaked defects in the past;but only a few honest testers would admit it. Whoever claims not, is a "fake tester"!!!!

Coming back to the story, the entire team blame the tester for leaking a very easy to detect showstopper defect.

Nobody questions the following which could have resulted in the defect...

1) Wrong Reqiurements sent by the Client --- We cannot question the client, can we?
2) Design problems which would have resulted in the defect
3) Lack of Unit Testing or Test Coverage
4) Waking hours of the tester before he detected the defect (What if he's been testing continuously for 12 hours and let the defect slip in the 13th hour of testing?)
5) Scope Creep in form of Change Requests
6) The boss who was breathing down the tester's neck when the tester was testing

and a million zillion other questions!!!

As part of "Most Convenient for all" Corrective action, the test team or tester gets fired and the world is happy!!!

Question yourself. Is that a corrective action or a "convenient" action?

My take on the incident
Now, I am not saying that Rob Green should not be blamed at all. He is definitely to be blamed. It was his job to stop a goal and he let in the goal.

But, what I am saying is that he is not the only person to be blamed. Likewise, the software tester is not an individual to be blamed when a defect is leaked.

Until and Unless the entire engineering team stands up and takes responsibility, such defect leaks will continue and recur sometime in the not so distant future!!! I think, that's what teamwork is all about... taking someone along with you when the going's not so good... If that does not happen, then i think we need another blog on "fake teams" ;) !!!

The Defects should start and stop with the entire team. If you are a part of the team, then the defect owner should be you, YOU and YOU!!!!! Until and unless you try to own a defect that you have come in contact with, defect leakage (and goal leakage) will continue!!!!!

Saturday, June 12, 2010

All the Fake Numbers

Someone once said "Keep It Simple, Stupid"!!!. I'd say, "Keep it Simple". If not, it means "You're Stupid!!!".

Let's try to see a scene that we’d all encounter during the end of the test cycle.

The Cast
Test Representative - Test Lead, Test Manager, Senior Tester, whoever. I call the person as a test representative since that's what he/she is
Boss - Master of the Daylight hours of the test representative
Big Boss - Master of the Daylight hours of the boss of the boss of the test representative,
Bigger Boss - Senior Management , Project Sponsors, etc. etc. etc...
Peer - Close friend/Best Buddy/Pal of the test representative, who is stuck in another project or wants to spend time browsing, or a workaholic, 1 who stays late @ office...

Project Stage ---> 2 Weeks before shipping

Time - 7 PM - Day 1

Scene - The test representative has been asked to send a daily status report on testing status by his boss.

Mental State of Test Representative ---> Panic Stricken!!! He has not sent such a report earlier in his life!!! His boss has re-directed him to the archives of the company to find a similar report. His peers have responded to his "call of panic" by flooding his mail boxes with format of a 10,000 status reports, some from his email archives and some from his google search.

What does the Test Representative do? - With a little time, he looks at whatever metrics is available with him, whatever he knows, and plugs it into a test status report and sends it to the whole world.

Fast Forward to next day.

Time - Day 2, 10 AM

Venue - Big boss's email

Activity - Big boss has received an email in the choicest language from his bigger boss's. They indicate a worthless status report.

Time - Day 2, 10:14 AM

Venue - Office of Big Boss

Scene - Big boss faulting the boss for a hopeless report, says the magic word “Appraisal”!!!

Language – The choicest words :) !!!!!

Time – 11:04 AM

Venue - Boss's office

Scene - Boss shouting at the Test representative for a very bad status report. Boss also mentions the word "Appraisal"!!!

Language – The very same choicest words :)!!!

What now? – Today, the boss wants to send the Test Report himself.

Time – 2:08 PM

Venue - Browser of the Test Representative

Activity - Test Representative having opened 7 Windows in IE and doing a search on test status reports and Metrics management. Finds a lot of data that he incorporates into the current day's test report.

Time frame– From 6:11 PM – 9:56 PM

Activity – 582 e-mails between the boss and the test representative on improving the report quality

Time - 10:02 PM in the evening

Venue - Boss's mail-box

Activity - Having reviewed the test report multiple times, and after so many updates between him and the test representative, the boss opens the current day's test status report and hits the "Send" button!!!

Boss’s Mental State - Relieved.

Time - 10:03 PM in the evening

Venue – Bigger boss’s mail-box

Activity - Opening the test status report sent by the boss and reads it.

Mental State – More Confusion!!! Is this what I said? Yes, he is confused. He is seeing so many details in the report, and he is confused on what to make of it.

Time - 10:46 PM

Venue - Hospital

Hospital? Yes, the bigger boss has been admitted for head injury. Witness seem to indicate that he has banged his head against a wall. His laptop has the latest status report open!!!

Time – Next day morning…

Venue – Bigger boss’s office

People - A bandaged bigger boss, the big boss and the test representative.

Activity – Well, I really don’t want to go into that… This blog is about fake software testing and I think we need to come back to our subject of software testing!!!

And the truth is--- Among all of the above people, there is definitely at least "1 fake software tester"!!!

What none of the above cast realized ---> The answer to Why, Why and Why. What questions does the status report, sent on an every day basis towards the end of the release cycle, try to indicate?

What these people should realize ---> The only 2 questions that people want to be answered from the test status report are ...--- How soon can we release? What is the health of the current build?

That's all, I believe, should the test status report try to indicate. It's very important for the test team to project these numbers so that all other stakeholders understand project health and ability to meet release dates.

Take a look at some of the metrics on test status reports that you can find defect reports - Time taken to test, no. of defects in all builds in the last year, Review count of test cases reworked on (from the previous generation), review time for test cases, Various delays that happenned in the project, Root Cause Analysis, Defect injected rate, Life-history of the entire project, Defect Removal Rate, Remaining defects rate, etc. etc. etc...

As in the story above, the boss and the bigger boss try to project a lot of information in these reports – The poor test team representative spends hours and hours of time in trying to get the above metrics into the report. Though each data point is valid at some stage, their usefulness is defined by the way it is useful to the project in the current stage. With very little time to ship the application, these numbers become useless.

Neither do these indicate the current health, nor do they indicate how soon can you ship the product.

Having said that, what do you think I say should be in the report?

1) Number of Open S1-P1/S2-P2 defects & time taken to resolve/re-test them - You will need to work very closely with the development team to predict these numbers.

2) No. of blocked test cases and the business priority of these blocked test cases - You will need to work very closely with Business/Dev team to say these numbers.

3) Ratio of no. of passed test cases to no. of total test cases - This is something that you should be able to say. Also indicate business priority of the passed test cases.

4) Sometimes people want to see a list of issues. Please feel free to include a list of open defects, but what will add value is an estimate on the time to fix/re-test these defects as well.

5) List of existing issues/blockers – Add to this list, details of who owns it and the ETA.

These not only tell you a status of the current health of the build, but also give you a view on how soon you can ship. That's all you need!!!

Of course, it would be very easy for all of you to tell me trillions of other metrics that you need...but then, if you subscribe to the thought of a trillion metrics in your daily test status reports, then you'd fall into the category of the fake software testers!!!

Happy Reporting!!!

Friday, June 4, 2010

Pressure --- It's all yours!!!!!

Pressure... --- It is a very familiar term by now. All of us have said that we are pressurized at the work place. All of us say that we feel pressurized due to deliverables, dates, schedules, still-open-must-fix defects, etc. etc. etc.

Expected Behavior Under Pressure --- That you remain unaffected!!!

Actual Behavior --- Unknown... each individual behaves differently under different situations!!!

Well, let's take a look at the following few scenes that you can come across in every day life.....

Scene 1
The Time ---> A lazy weekend when you have hours to spare
The Assignment ---> You are browsing a book website to buy the latest offering from Jeffery Archer.
The Scene ---> You encounter a JavaScript error message. You take a look at the error message, choose the relevant option and keep browsing. Out of your personal interest in testing, you sometimes take notes on what the error message was, but you keep browsing the website.

Scene 2 ---
The time ---> 11.30 AM... Day 3 of your first job in your new company.
The Assignment ---> Test the client's application. The application is a website that sells books.
The scene ---> you want to prove yourself to everyone in the new world. While you are testing the book website, you come across a JavaScript error. When you test the page, you work on an all-nerves-alert mode; make notes of the JavaScript error. Of course, with the new job, you feel a bit of the nerves, but you tell yourself that it's nothing that you cannot handle.

Scene 3
The time ---> 11.30 AM... 1 Day before shipping
The Assignment ---> Test the client's application that sells books.
The Scene ---> You have been asked to provide the sign-off by the day's evening. You start testing. Every 1 hour, you get calls from your bosses asking for updates on testing. You are still explaining test progress in a phone to your bigger bosses, when you encounter the same JavaScript error.
Next steps --> you report it to the world and while the defect is being fixed and deployed, you heave a sigh of relief and go out for a cup of coffee. (I don't smoke and so, I don't have a reference to smoking over here :)!!!

Scene 4
The time ---> 11.30 AM ... 8 hours before shipping
The assignment --> testing the client's web application that sells books.
Your Client ---> Wants to ship the product 24 hours before his competitor. Keeps calling the CEO telling him that the success of the project would mean more revenue. He wants a defect free product & the product has undergone a 1000 last-minute CRs.
Your company's CEO ---> feels that by releasing in time, the company would make a million dollars more in the financial.
Your manager ---> feels that he'd get a promotion if the defect-free product ships on time.
The Scene ---> With 8 hours to launch, a build arrives for the test team and you are asked to test the booking website. Your manager makes it very clear to you that a slip-up can cost you your job in the company. Your CEO keeps asking you for what's happening. You come across the JavaScript error. But, you feel the tension while you test since you feel a 100 pairs of eyes watching you test.

The common factors and the differences from the above listed...

The Common Factors

1) The product is the same
2) The defect is the same
3) The tested product is the same
4) The Stakeholders are the same

The Difference...
The Degree of Pressure...


Where does this Pressure stem from? My thoughts listed...

Fear.... Of failing... and of succeeding.......
1) Of Failing???
Most of our managers, talk about the consequences of failure. And this thought plants itself deep-rooted in our minds. Even CEOs let talk of the wide-spread belief (yes, may people believe this still!!!) that the tester is wholly responsible for finding a defect. In the name of "sending across the right message", managers have fired testers who have been in the project for a long time, due to minimal defect leakage... Plus, add the burden that a defect leakage would also mean that all those long hours and personal sacrifices for the current project becomes wasted..... With more and more eyes watching you, fear climbs to a much higher altitude and increases in magnitude... Over time, this fear breeds into nervousness.

2) Of Succeeding???
Can someone have a fear of succeeding??? Yes.... When a project is going to end, the mind goes on visit to a few places --- 1 place is called the future, where the mind starts to thank the people who congratulate you..... While the other part of the mind has a fear of stumbling at the last hurdle.
Result ---> The tester is unable to have a rational thought, unable to make a decision on a defect and tends to think that every defect found is severe. A case of where the individual falls down due to the fear of succeeding...

3) Pressure initiated by external agencies
I don't know about you, but I have had sleepless nights prior to a very critical release. When a product goes live, there may be a zillion items that can go wrong. The importance of the occasion of the release, and the so-called expectations can do a lot of damage to the untrained mind.

How do you respond to Pressure?
Every individual have different ways of responding to pressure. Some step up, while others break down. You will have to know how you work under pressure.
And, you will have to inform your bosses as to how you work under pressure. Good Managers (and I say good managers), get the work completed by exerting the "right amount of pressure"!!!

Having said that, how do you handle pressure?
If you google, then you would get a million answers (except this blog since this ain't listed in google). But, there's only 1 person who knows it... And that 1 person is YOU. Think about how you would work under pressure and be prepared for all possible items. Be prepared to handle pressure... so when it is delivered at your door-step, you'd know how to handle it.

Some Gyan on Handling Pressure?

What do I do?
Well, I don't know how you'd handle it, but what I do is... When I come across a pressure situation, I make a conscious decision to walk away from there. By completely moving away, it offers me enough time to ensure that the steam is let off.

Controlling the mind
Some people believe in controlling the mind. Mind-Control exercises to help them focus on the current situation at hand.

A team member's role in reducing pressure...
Confiding in team members and taking their help. Works fine for a few...Some people are able to handle pressure independently and some are able to handle it as a team.

Becoming a Detached soul...
This works fine for me. Getting mentally and emotionally detached saves you from getting drained out. Once you are completely detached from the situation, from the product hype, from the expectations, you can stay in the present without letting the pressure get to you.

Fake Testers gyan
And I like giving Gyan :)!!! And please do remember, even if you don't come across this situation in your current job, it's highly likely that you'd come across this situation sometime in the near future. So, in addition to increasing your testing skills, please spare a thought, spare a moment as to how you would be handling pressure.

And use every situation to practice how you would work under pressure... It will help you to handle pressure in the upcoming days.

That's just 1 stop in your journey towards better testing...Other stops to come in later posts!!!