Brakes and Defect Prediction...

Brake prediction and defect prediction are related. Defect Prediction is defined by me as the ability to predict the number of defects that would occur in the next development cycle. How is Defect prediction possible? Read on for my thoughts.

I did the following exercise. My workplace is an hour’s drive from my home. I challenged myself to predict the number of times I’d brake on a day, while driving from home to office.

What I did was as follows:-

1) From Day 1 - Day 5, I calculated the number of brakes on each day to create a data repository.

2) From Day 6 - Day 15 onwards, I used the information gathered from the previous days to try and predict the number of times I braked every day. I also updated the data repository at the end of every day.

Top 7 of my observations in italized below with their corrosponding learning in Software testing in BOLD inline…

1) Every day, I applied the brake at least once on my way to work.

All of us can predict at least 1 defect in the system. (I guess that's the closest we can get to with predicting defects).

2) Though the traffic conditions and road crowd was same on all 15 days, my predictions were wrong. They had a variance of + and – 50%.

Number of lines of code cannot be used as a factor for predicting defects

3) Regardless of how many times I braked, I always reached office in  60-70 mins. Average time to reach destination was not hampered by the number of brakes.

Your project schedule will not be changed by your number of defects. Rather they are determined by your ability to detect and fix them in quick time.

4) Sometimes, when I braked, I had to remain braked for long periods of time, due to different reasons. Traffic, crowd, signals, etc. etc. etc.

Severity of defects can never be predicted at all

5) Mostly, I thought I’d hit the brakes at the same spots on the days.Wrong. More than 50% of the time, I hit the brakes at different locations.

Defect Predictions becomes a highly mis-guiding factor. It also gives you the false notion that the total number of defects will not exceed a given number. It may be proved false after the application is launched.

6) On 2 days a week at least, I had to take a different path to work. My algorithm to predict defects went haywire on those days.

During the path, teams need to display a lot of agility. Many a time, you have to take a different direction to reach your goal. Your defect prevention algorithm, may not factor in for these change of direction

7) And False Alarms!!! Sometimes, I expected an obstruction and hit the brake, but it turned out to be a false alarm.

You cannot predict false alarms. Sometimes, you might spend a few hours on a trivial or "Difficult To reproduce" defect.

Well, I certainly was not accurate on predicting defects or my brakes, but at least learnt enough information I thought is worth a blog post :)!!!

And last, the fake tester's gyan

1) Dont bother about the future. Worry about the present.

2) Ask yourself the question --- If you had 5 mins at your disposal, would you try to test the product, or use the time to predict the total no. of defects the next development cycle would throw up?

3) And lastly, when we are saying that it's not possible to fully test a product, how can you predict the number of defects in the future?


If the number of defects varies by 50%, then how do you estimate work for dev teams during testing? Well, more on that another day!!!

Re-defined Definitions...

Someone said annualy 1 billion dollars is spent on Software Testing. I believe more than half of that goes into fake testing :)!!!

Following are some Software Testing Definitions. Please take a minute to read and enrich your knowledge.

Sandbox testing - Software Testing when we have received the build, but the Dev teams have not yet completed coding some of the functionality

Waterbox testing - Software Testing that happens when test team get the build, but for some features, the design is not yet completed

Gasbox testing - Software Testing when the test team receive a build, but the dev team have not yet started coding.

Casual testing - Software Testing on Fridays when the test team is dressed in casuals. The casual attitude to testing results in what's called as "Casual Defects".

Formal testing - Software Testing that happens on Mondays when the test team is fully dressed in formals. This testing type results in "Formal Defects".

Gamma testing - Software testing that happens after alpha testing and beta testing.

Lateral Testing - Software Testing done by a team member who does not belong to your team, but to a different team.

Serial Testing - Software Testing done by testers in serial. Teams test only 1 functionality at a time.

Parallel Testing - Software Testing done by testers in parallel. All functionalities are tested simultaneously.

And by now, if you have started thinking that the above definitions are true, then what's also true is that YOU ARE A FAKE TESTER LIKE ME!!!

Honestly, a year back, I really did think that stuff as above was true. What I did not realize was as follows.

Definitions... are man-made. They are created for the convenience of the author. As a reader, it would be good if you can spend time trying to understand the concept presented by the writer and the intent of the author. It would be much beneficial for you to understand the concept, rather than try to mug the definition and become a subscriber to definitions.

Fake Tester's Gyan

Stop being a believer of Definitions.

Start being a believer of Concepts.

It's about time, you stop believing definitions and start believing concepts!!!

Interviews – Stop thinking aloud. Start thinking, Channelize your thoughts and Reply…

In the past few years, I have done quite a number of testing interviews. Some of the questions that I've asked are:-

"How do you test a pen?”, “How do you test a mobile?”, “How do you test a remote controller?", "How do you test a random number generator?", "How do you test an application that generates the fibonacci series?" etc.

When I asked them "How do you plan your testing for a website selling mobile phones, interacting with 3 suppliers?", none of them paused to think. The answer came immediately like below.

“I'd plan for sanity testing. Will plan for testing the site against XYZ interfaces. L&P Testing needs to be a part of the test plan. I'll have daily stand-ups. I will talk about cost and variances to think about estimation. I'll have a risk plan for managing risks proactively. I will do a requirements-traceability-matrix..."... and he'd go on and on and on.

After talking to many candidates, it struck me that most of them, when answering the above questions, did not pause to think; or ask for time to think. Though it seemed that they were answering the question, they were only "thinking out their thoughts aloud".

Thinking a bit more, I guess the best way to answer such questions, in an interview would be in the following 4 steps:

STEP 1:- Think.
Ponder about the question for a minute and think about the answers and various possibilities for the next couple of mins and speak up when you are prepared to answer. If you want more time, please ask the interviewer for time.

STEP 2:- Channelize your thoughts.
Think about the solution and channelize your thoughts to ensure that your answer is structured correctly, or how you want it to be structured. A structured answer, will definitely earn you a lot of brownie points with your future employer. If you want, write down short points on paper before you start talking about the answer

STEP 3:- Prioritize the reply and speak it out accordingly.
Go ahead and speak up and start answering the question. If required, refer to the short points while you answer. If you need more time to think, please ask for more time.

STEP 4:- Invite him for discussion on your reply
Ask if the interviewer has any questions or invite him to discuss the finer points of your answer. Try to give logical reasons for your decisions for prioritizing.

FST Gyan section -
If you are being interviewed, then
Start ---> Asking for time, if you think you need time.
Stop ---> thinking out your thoughts aloud. Interviews are not a forum to think aloud. Secondly, interviewers will definitely be impressed if you ask them for time to think.

If you are the interviewer, then
Start ---> Asking the candidate to think and reply.
Stop ---> asking questions when the candidate immediately answers to your questions. Most probably, he's not answering the question, but "thinking out aloud"!!!

And yes, as always, have a happy interview!!!

“Non-Reproducible defect” – Non-occuring or “Unable to recreate?”…

Non-Reproducible defects, sometimes, are even more lethal than existing defects. If ignored, you can never predict when the non-reproducible defect would rear it's ugly face and kill the product. Trying to present a personal perspective of "top 5 myths" around non-repro defects.


Myth 1: That non-reproducible defects are harmless

Not really. When it's happened once, it's more likely that it can happen again. Non-occurance in a protected environment does not mean permanent non-occurance in production. Secondly, can the defect can be ignored since it cannot be re-produced? Definitely not. Test around areas of the defect to explore it further. You never know what you can unearth unless you dig further!!!

Myth 2: That the development team have fixed the non-reproducible defect

A personal favorite. I have heard many stories of dev fixing non-repro defects. How do you verify a bug-fix that's not reproducible in the first place? I don’t know. I have never figured it out. Maybe they know something that you don't. If you have good interpersonal relationships with the developers, you should be able to talk to them to understand the root cause to try and re-create the error conditions to reproduce the defect and confirm if it’s fixed.

Myth 3: Large number of non-reproducible defects? No, it doesn’t’ mean much

Maybe a large number of non-reproducible defects means the following
a) Test Conditions are varying due to test environment instability
b) Testers have not understood product design
c) Dev and testers don’t talk to each other
Whatever it is, it means chaos!!! And that's a ill-omen and bad-health for the product

Myth 4: That the non-reproducible defect is a scenario that's never happenned before

If it’s an application in production, (a maintenance application), maybe it’s happened before. Look around.
a) Talk to the support teams to check if it's happenned in the past.
b) Talk to the veteran project members to see if similar defects have been unearthed in the pasts.
c) Look into the bug database.
d) Check the support log files (if you have access to support calls).
e) Network with older project testers who you are in touch with (even though they have moved to other projects or companies).
f) Maybe it's happenned earlier in life and you can get some info from earlier people. Also talk to people who have tested the application earlier to see if it's happenned in the past.
Remember, History is a great teacher!!!

Myth 5: That the sole owner of a non-reproducible defect is the tester

Non-Repro defects are mentinoed as errors of the tester. I Disagree. It's more like a case of a person being punished for trying to reach objectives beyond boundaries. When a tester tries to invest time in trying to showcase a rarely occuring problem, we are actually punishing him by thrusting ownership on him. Metrics crazy companies have assigned metrics for each developer and tester, and I am pretty sure that this results in the tester being blamed if a defect’s non-reproducible.

My take --> I feel that this defect should be owned by the entire project team. No single person ownership for non-reproducible defects.

Fake Tester's Gyan

1) Dont' ignore them

Most of us tend to ignore the defects which cannot be reproduced again. It's akin to commiting suicide. Please ensure that you address all non-repro defects before going live. Ensure that the right people get to view them and discuss them accordingly, before deciding on next steps of these defects.

2) A non-repro defect session

Why is it non-reproducible? Have a session with the entire project team to brainstorm regularly on all non-reproducible defects in the project. Maybe we would understand patterns on why a defect is non-reproducible!!!.

3) Business Severity

And look at business severity, as always. If it's high priority, then please spend some more time looking into areas. Try defining test conditions. Look for cases outside the requirements. Maybe it’s the slow system, slow connection speed, memory leak, etc. that’s resulting in this behavior. Classify high priority non-reproducible defects into a bucket and spend 1 hr a week analyzing them. It's better spending time on bug research now instead of doing it after the system goes live.

That's all!!!

As always, happy testing!!!

Happy Programmer's day!!!

Hi Programmers,

Best wishes from the fake software tester for a wonderful and bug-free "Programmer's day" :) !!! Yes, for those who don't know what I am talking about, Sep 13 and Sep 12 (in a leap year) is celebrated as Programmer's day, at least in Russia!!! And I am not surprised.

The logic behind this is to celebrate the 256th day of the year as the Programmer's day. As you know, 256 was chosen since it can be represented with an 8-bit style. But, I have the following question behind celebrating the 256th day.

Don't you guys think that your counting should start from Day-0 and celebrate Day 256 (which'd fall on Sep 14 and Sep 13 on leap years) as a programmer's day? Your guess is as good as mine. Maybe we can log a defect against the above logic :) !!!

And What about a tester's day? Maybe it can be celebrated the following day, for the simple reason that testing follows development. Happy Tester's day too!!!

Look Closer…He's a Manager, not a mentor!!!

Today's world expects managers to play a mentoring role to all of their team members. But this expectation seems fundamentally flawed. It is actually very difficult for a manager to be an "effective" mentor to his team members. Why? Why? If you are really interested in my opinion, please read-on.

1) Difference in Priorities
A manager's priority would be the company's good health. A mentor's priority would be your good health. An example is -- your manager would never ever advise you to quit your job to reach your goal, your mentor would!!! Basic difference in Priorities is the 1st hurdle for a manager to be a mentor.

2) Interests Vs Loyalties & Following your Dreams
When your interests and corporate interests clash, a manager would advise you to do what's beneficial to the company. A mentor would nudge you towards what'd be beneficial to yourself.

For example, if you are a very good black box tester in a manual testing company, with an aspiration towards networks, a manager would show you a nice career path in the direction of black box testing, while your mentor would ask you to follow your dreams. A mentor would most probably say --- "Use this job for sustenance, do a course in networking and join Cisco"!!!

3) Favoritism towards top performers
Managers are bounded by loyalties towards old-timers, loyalists & top performers. The manager does not invest time in a weak performer. But Mentors don't let down a weak performer. You would want a mentor, who would not let you down, don't you?

4) Are you his Competitor?
When your mentor becomes your manager, the following question might creep into your mind --- "Is my career not progressing because he thinks I am his competition?” Somewhere down, your mentor, your manager, becomes your dreaded enemy and sadly, a rival!!!

5) And the Credit Crunch...
When your mentor becomes your manager, who do you think gets credit for a successful launch, and who do you think gets blamed for failures? Work becomes more difficult when this question takes root into your head. As we all know, a mentor would always take the blame for failure and credit you for success.

6) Confessing your faults...
It's very difficult to confess your weaknesses and faults to your manager. You would have the belief that it'd come back to haunt you during your "appraisal time". But, you would never have this fear confessing this to your mentor!!!

Those are some blockers that I could think of... as to why managers should not try their hand at mentoring... They are called Managers... and should strictly stick to Managing!!!

And no, I am not even hinting at asking you to get through life without a mentor... That borders on insanity!!! You always need a mentor to guide you in the right direction.... The results can potentially be hazardous when your manager becomes your mentor... or vice-versa!!! And below is my suggestion to solve this.

The Internet has shrunk the size of our world. That means it's possible for you to reach out to anyone you want who resides at another corner of the world. So, please reach out... scan the entire world to identify your mentor. If you are looking at your manager to do it, then...keep in mind, he's a Manager first...then a mentor!!!

Fake Software Tester’s Guide to attend meetings…

The title to this post tells what this post is all about. A common scenario as you grow up the perceived corporate ladder would be a need to attend a lot of meetings. Once you grow above the managerial level, plan to attend at least 4 hrs a day in non-productive meetings. And some of these meetings, you would have to attend even though you have no idea on what it is all about. And this post is about surviving such meetings.

Listed below are a few survival tips for such meetings – wherein you don’t have an idea on the topic being discussed.

1) The sandwich Blackberry method
A large sandwich and a blackberry can be your saviors. Walk into the meeting with a large sandwich and constantly, keep fiddling on your blackberry for the entire duration of the meeting. Walk out of the room every 10 minutes to attend to that very important phone call and keep walking back inside. You can be rest assured you won't be troubled by anyone for the duration of the meeting.

2) The copy-Paste method
Before the meeting, grab your team member who's attending it and ask him on what he's going to talk about. Ask him a few questions and ensure that you remember all his answers. Send him off on any errand to ensure that he doesn't attend the meeting. Now, re-state whatever he's told you and answer the same questions that you've asked him. Nobody would disturb you after you've made your point.

3) The Magic words
Remember these words - Bottom Line, Bird's eye view, Revisit, Thinking Hat, out of the box, Paradigm, Game Plan, win-win, Leverage, etc.
The trick is to pick 2 words from above and form a sentence, at any time during the meeting. Some examples I have are below:-

a) Guys, it's time to put on the thinking hats to think of a win-win proposal
b) Folks, let's come up with an out-of-the-box solution, which is pro-active and leverages our strengths.
c) The Bottom Line is that we need to revisit if we are doing things the right way. Take a step back to have a Bird's eye view of the solution.

You can try this exercise yourself and within a few practice tries, you'd become a master at it. You can create such statements from these words randomly and you'd sound like a veteran of senior management.

4) The Blame-it-on-others Method
Start off a conversation at the beginning of the meeting to talk about bad food in the canteen, rising fuel bills, lack of facilities in the company, bad candidates lined up by HR for interview, unrealistic client demands, etc...

5) Google will help you
If it's a meeting about a product evaluation, do a vigorous Google search 1 hr before the meeting. Scribble notes from all over the internet and present them as your viewpoint. Mostly, nobody would know and you can escape unscathed.

And my pick goes to the method described below... the best thing to do is....
6) Being honest
Yes. The last way is honesty. Stand up and tell everyone that you will be unable to provide any justice by your attendance, since you have no idea on what's being discussed. Tell everyone that you think you'd be better off doing something productive, than be a part of it and walk out. That's not something that happens very often, but I guess it's the most honest way.

If you can think of more such practices, I'd be happy to hear them and it would be great if you can post them in the comments section!!!

Well, what about those telephonic meetings that you need to dial in to attend??? Well, save that thought.... more on those another day!!!

The Defect I Leaked

Earlier in life, in the late 90’s, I was hired to test a software application's installation. After the usual late hours, Development fights, etc., I certified that the product works fine and Okayed the release.

What Happened then?
Within 1 day of going live, the support team was flooded with a zillion calls saying that the application does not get installed. All the installations aborted with the error message - "INVALID OS". The products had to be re-called, the defect fixed and re-shipped.

Root Cause on Investigation
The installation software had logic to determine the version of the Client machine's OS. Both Dev & Test assumed the location of this entry in a particular location, while most of the live systems had this info stored in a different location in the registry. As a member of the test team, I did not do sufficient research before testing and missed out this important detail, which at that time seemed insignificant to me.

Who's to be blamed for the Fiasco?
3 People who were responsible --- I, Me & Myself.

Defect Leakage --- A 1st for me
It was the 1st defect I leaked which had very high business impact. The fault was mine, mine & mine. Like so many other people in the project, I assumed that the operating system version is stored at a particular location only. I assumed that it cannot be stored elsewhere and so, I DID NOT TEST FOR IT. BY ASSUMING, I LEAKED A DEFECT!!!

Till today, the bug haunts me every time I take up a testing assignment. It is the fact that I can still leak a defect makes me more determined to ensure that I don't leak defects at least now.

A Fake Software tester...
--- Never admits his fault to a defect that he's leaked
--- Always thinks someone else is to blame for a defect that he's leaked
--- forgets the leaked defect too soon in life and invariably, leaks many more
--- Tries to pass the root cause for defect leakage due to a different reason --- bad environment, bad specification, bad coding, time pressure, no requirements, etc. etc. etc....

In every interview of mine, I ask the test engineers who are applying for the job about any defect that they'd have leaked earlier. And to date, most of them have said that they have never leaked a defect in their entire life!!! This is coming from testers who have an experience of 6-10 years in the Software Testing Industry.

Do I see an entry here for the files of Ripley's believe it or not? (Personally speaking a tester who's never leaked a defect can be equated with a developer who has never had a single bug filed against his name).Unbelievable!!! Truly, Unbelievable!!!

The truth is that even experienced testers leak defects. And only we know about the defect that we leaked. And we have to ensure that we have to remember a leaked defect all the time to remind us that we are also vulnerable, so that we will at least try to avoid leaking defects in the future!!!

Fake Tester's Gyan (of course, the gyan giving is very important too)

1) Admit Blame --- Yes. Admit blame at least to yourself, if not to the entire world!!!
2) Shameful, it is not --- Though you might feel some shame in admitting a defect, trust me, it’s not. What’s actually shameful is NOT TO ADMIT IT TO EVEN YOU!!!
3) If I leaked defects, am I a failing tester? --- A common misconception is that you'd seem a failed tester to your peers if you admit it. You are not failing. You are the only one speaking the truth and you are sub-consciously scaling the success peak by your honest admissions.
4) It takes Guts. Do you have them? --- It takes guts to admit a defect you leaked. Am very sure that you do have a leaked defect. But do you have the guts to admit it?

If you are mentally strong enough to admit to a defect that you leaked, please feel free to share it on the Comment section below. If you are not bold enough to share it with the world, spend 10 mins in retrospection, admitting it to yourself!!! To own up to yourself about blaming self for a leaked defect makes you a better person; and a much better tester in the long run!!!

Happy Admissions, if you dare to admit!!!

To Trust, or Not to Trust - Part 1 of 7...

"To be or not to be --- That remains the question..." - started Hamlet in his soliloquy. But, what he did not know was that he was being watched by a few other people when he was delivering it.....

Similarly, a software tester also has to, over time, build credibility with other team members – Be it peers, or bosses, or bigger bosses, or sub-ordinates. Having said that, a tester should also know a list of people he can trust.

Trees have been felled in the go-green world on writing millions of zillions of articles on why the test teams need to build credibility with other team members. But, the question that’s posed here is -- Have the other teams built their credibility with the testers?

Need for the Question
Is there a need for the question posed above? Why does the test team need to know which other team have built their credibility with the test teams?

Well, to cut a long story short, let’s take a quick look and ask the tester the following questions, before he starts testing.

1) What’s your confidence level on the fact that there will not be scope creep?

2) What’s your confidence level on coverage of non-functional requirements?

3) How sure are you that the defect is fixed, when the developer checks-in code at 11 PM?

4) Do you believe that the Dev teams have completed the code review? Or are they under pressure from senior management to start testing at the earliest and have turned a blind eye to the code being reviewed?

5) A problem has occurred in production and you have been asked to call the support engineer. How confident are you that the production support engineer has given you all logs? How confident are you that he’d support you?

6) You have goofed up big time. How confident are you that, when you confide this in your boss, your boss won’t blow it out of proportion, but would correct you?

7) You have just conveyed the biggest risks to your project managers, whose objective is on-time delivery. How confident are you that they would have conveyed the risks to the clients?

And the answer is…
The answer to all would be a confidence %. Or confidence quotient. And that directly will be more, or less, depending on how much credibility that the concerned person has built with you over time.

For the software tester, life is tough, since people start noting him only at the fag end of the test chain. He is the last line of defense and has to handle all the pressure.

During this time, the tester will need to make decisions on who can and cannot be trusted. How is this decision made? Only on the basis of your relationship with the person and how they have reacted to a similar situation in the past. And that’s why you need to know who have built their credibility with you.

I will try and pen my thoughts on future posts, on what can potentially happen when there’s lack of trust between the tester and the peer. Will try to post them in another 7 or 8 short boring posts…..

Till then … to Trust, or not to trust... remains your question to be answered!!!

Software Testing and the "Hand of Clod"!!!

England Played USA on their first FIFA 2010 World Cup match and most of the world saw Rob Green's (England Goal Keeper) mistake that let the USA escape with a draw.

The next day, the entire world have started to blame Rob Green for his mistake, attributing the goal only to his mistake. People are contemplating firing him. We even have seen the "Hand of Clod" headline in many newspapers.

I am not trying to say that the England Goalkeeper is not at fault, but let's try to answer a few questions below...

1) Who let the possession of the ball to USA?
2) Where were the mid-fielders and why did they let the USA Offense inside?
3) Where were the defenders and why did they let the USA Offense so close?
4) Why could the defenders not intercept the shot to the goal?
5) Who was assigned to mark the scorer? Where was he when the scorer scored?
6) Who selected Rob Green to play in the match?
7) Who decided to buy the gloves that the Goalkeeper was wearing? Was it tested for circumstances wherein the ball meets the gloves when the gloves are wet? What were the test results and who tested it?

And so many many many more questions.....

If the answer is not "England Goalkeeper" for any of the above questions, then I don't think that it is right to blame him for the goal.

What is important is - a part of the blame is his; even more important is ONLY A PART OF THE BLAME IS HIS. The rest of the blame has to be owned by other members of England World Cup Soccer team.

Trying to draw a parallel between this and today's software tester, now think about a scenario wherein a tester would have leaked a very easy to detect showstopper defect.

Writer’s Irritating Personal Note:- My opinion is that all of us (the readers and writer of this post) have leaked defects in the past;but only a few honest testers would admit it. Whoever claims not, is a "fake tester"!!!!

Coming back to the story, the entire team blame the tester for leaking a very easy to detect showstopper defect.

Nobody questions the following which could have resulted in the defect...

1) Wrong Reqiurements sent by the Client --- We cannot question the client, can we?
2) Design problems which would have resulted in the defect
3) Lack of Unit Testing or Test Coverage
4) Waking hours of the tester before he detected the defect (What if he's been testing continuously for 12 hours and let the defect slip in the 13th hour of testing?)
5) Scope Creep in form of Change Requests
6) The boss who was breathing down the tester's neck when the tester was testing

and a million zillion other questions!!!

As part of "Most Convenient for all" Corrective action, the test team or tester gets fired and the world is happy!!!

Question yourself. Is that a corrective action or a "convenient" action?

My take on the incident
Now, I am not saying that Rob Green should not be blamed at all. He is definitely to be blamed. It was his job to stop a goal and he let in the goal.

But, what I am saying is that he is not the only person to be blamed. Likewise, the software tester is not an individual to be blamed when a defect is leaked.

Until and Unless the entire engineering team stands up and takes responsibility, such defect leaks will continue and recur sometime in the not so distant future!!! I think, that's what teamwork is all about... taking someone along with you when the going's not so good... If that does not happen, then i think we need another blog on "fake teams" ;) !!!

The Defects should start and stop with the entire team. If you are a part of the team, then the defect owner should be you, YOU and YOU!!!!! Until and unless you try to own a defect that you have come in contact with, defect leakage (and goal leakage) will continue!!!!!

All the Fake Numbers

Someone once said "Keep It Simple, Stupid"!!!. I'd say, "Keep it Simple". If not, it means "You're Stupid!!!".

Let's try to see a scene that we’d all encounter during the end of the test cycle.

The Cast
Test Representative - Test Lead, Test Manager, Senior Tester, whoever. I call the person as a test representative since that's what he/she is
Boss - Master of the Daylight hours of the test representative
Big Boss - Master of the Daylight hours of the boss of the boss of the test representative,
Bigger Boss - Senior Management , Project Sponsors, etc. etc. etc...
Peer - Close friend/Best Buddy/Pal of the test representative, who is stuck in another project or wants to spend time browsing, or a workaholic, 1 who stays late @ office...

Project Stage ---> 2 Weeks before shipping

Time - 7 PM - Day 1

Scene - The test representative has been asked to send a daily status report on testing status by his boss.

Mental State of Test Representative ---> Panic Stricken!!! He has not sent such a report earlier in his life!!! His boss has re-directed him to the archives of the company to find a similar report. His peers have responded to his "call of panic" by flooding his mail boxes with format of a 10,000 status reports, some from his email archives and some from his google search.

What does the Test Representative do? - With a little time, he looks at whatever metrics is available with him, whatever he knows, and plugs it into a test status report and sends it to the whole world.

Fast Forward to next day.

Time - Day 2, 10 AM

Venue - Big boss's email

Activity - Big boss has received an email in the choicest language from his bigger boss's. They indicate a worthless status report.

Time - Day 2, 10:14 AM

Venue - Office of Big Boss

Scene - Big boss faulting the boss for a hopeless report, says the magic word “Appraisal”!!!

Language – The choicest words :) !!!!!

Time – 11:04 AM

Venue - Boss's office

Scene - Boss shouting at the Test representative for a very bad status report. Boss also mentions the word "Appraisal"!!!

Language – The very same choicest words :)!!!

What now? – Today, the boss wants to send the Test Report himself.

Time – 2:08 PM

Venue - Browser of the Test Representative

Activity - Test Representative having opened 7 Windows in IE and doing a search on test status reports and Metrics management. Finds a lot of data that he incorporates into the current day's test report.

Time frame– From 6:11 PM – 9:56 PM

Activity – 582 e-mails between the boss and the test representative on improving the report quality

Time - 10:02 PM in the evening

Venue - Boss's mail-box

Activity - Having reviewed the test report multiple times, and after so many updates between him and the test representative, the boss opens the current day's test status report and hits the "Send" button!!!

Boss’s Mental State - Relieved.

Time - 10:03 PM in the evening

Venue – Bigger boss’s mail-box

Activity - Opening the test status report sent by the boss and reads it.

Mental State – More Confusion!!! Is this what I said? Yes, he is confused. He is seeing so many details in the report, and he is confused on what to make of it.

Time - 10:46 PM

Venue - Hospital

Hospital? Yes, the bigger boss has been admitted for head injury. Witness seem to indicate that he has banged his head against a wall. His laptop has the latest status report open!!!

Time – Next day morning…

Venue – Bigger boss’s office

People - A bandaged bigger boss, the big boss and the test representative.

Activity – Well, I really don’t want to go into that… This blog is about fake software testing and I think we need to come back to our subject of software testing!!!

And the truth is--- Among all of the above people, there is definitely at least "1 fake software tester"!!!

What none of the above cast realized ---> The answer to Why, Why and Why. What questions does the status report, sent on an every day basis towards the end of the release cycle, try to indicate?

What these people should realize ---> The only 2 questions that people want to be answered from the test status report are ...--- How soon can we release? What is the health of the current build?

That's all, I believe, should the test status report try to indicate. It's very important for the test team to project these numbers so that all other stakeholders understand project health and ability to meet release dates.

Take a look at some of the metrics on test status reports that you can find defect reports - Time taken to test, no. of defects in all builds in the last year, Review count of test cases reworked on (from the previous generation), review time for test cases, Various delays that happenned in the project, Root Cause Analysis, Defect injected rate, Life-history of the entire project, Defect Removal Rate, Remaining defects rate, etc. etc. etc...

As in the story above, the boss and the bigger boss try to project a lot of information in these reports – The poor test team representative spends hours and hours of time in trying to get the above metrics into the report. Though each data point is valid at some stage, their usefulness is defined by the way it is useful to the project in the current stage. With very little time to ship the application, these numbers become useless.

Neither do these indicate the current health, nor do they indicate how soon can you ship the product.

Having said that, what do you think I say should be in the report?

1) Number of Open S1-P1/S2-P2 defects & time taken to resolve/re-test them - You will need to work very closely with the development team to predict these numbers.

2) No. of blocked test cases and the business priority of these blocked test cases - You will need to work very closely with Business/Dev team to say these numbers.

3) Ratio of no. of passed test cases to no. of total test cases - This is something that you should be able to say. Also indicate business priority of the passed test cases.

4) Sometimes people want to see a list of issues. Please feel free to include a list of open defects, but what will add value is an estimate on the time to fix/re-test these defects as well.

5) List of existing issues/blockers – Add to this list, details of who owns it and the ETA.

These not only tell you a status of the current health of the build, but also give you a view on how soon you can ship. That's all you need!!!

Of course, it would be very easy for all of you to tell me trillions of other metrics that you need...but then, if you subscribe to the thought of a trillion metrics in your daily test status reports, then you'd fall into the category of the fake software testers!!!

Happy Reporting!!!

Pressure --- It's all yours!!!!!

Pressure... --- It is a very familiar term by now. All of us have said that we are pressurized at the work place. All of us say that we feel pressurized due to deliverables, dates, schedules, still-open-must-fix defects, etc. etc. etc.

Expected Behavior Under Pressure --- That you remain unaffected!!!

Actual Behavior --- Unknown... each individual behaves differently under different situations!!!

Well, let's take a look at the following few scenes that you can come across in every day life.....

Scene 1
The Time ---> A lazy weekend when you have hours to spare
The Assignment ---> You are browsing a book website to buy the latest offering from Jeffery Archer.
The Scene ---> You encounter a JavaScript error message. You take a look at the error message, choose the relevant option and keep browsing. Out of your personal interest in testing, you sometimes take notes on what the error message was, but you keep browsing the website.

Scene 2 ---
The time ---> 11.30 AM... Day 3 of your first job in your new company.
The Assignment ---> Test the client's application. The application is a website that sells books.
The scene ---> you want to prove yourself to everyone in the new world. While you are testing the book website, you come across a JavaScript error. When you test the page, you work on an all-nerves-alert mode; make notes of the JavaScript error. Of course, with the new job, you feel a bit of the nerves, but you tell yourself that it's nothing that you cannot handle.

Scene 3
The time ---> 11.30 AM... 1 Day before shipping
The Assignment ---> Test the client's application that sells books.
The Scene ---> You have been asked to provide the sign-off by the day's evening. You start testing. Every 1 hour, you get calls from your bosses asking for updates on testing. You are still explaining test progress in a phone to your bigger bosses, when you encounter the same JavaScript error.
Next steps --> you report it to the world and while the defect is being fixed and deployed, you heave a sigh of relief and go out for a cup of coffee. (I don't smoke and so, I don't have a reference to smoking over here :)!!!

Scene 4
The time ---> 11.30 AM ... 8 hours before shipping
The assignment --> testing the client's web application that sells books.
Your Client ---> Wants to ship the product 24 hours before his competitor. Keeps calling the CEO telling him that the success of the project would mean more revenue. He wants a defect free product & the product has undergone a 1000 last-minute CRs.
Your company's CEO ---> feels that by releasing in time, the company would make a million dollars more in the financial.
Your manager ---> feels that he'd get a promotion if the defect-free product ships on time.
The Scene ---> With 8 hours to launch, a build arrives for the test team and you are asked to test the booking website. Your manager makes it very clear to you that a slip-up can cost you your job in the company. Your CEO keeps asking you for what's happening. You come across the JavaScript error. But, you feel the tension while you test since you feel a 100 pairs of eyes watching you test.

The common factors and the differences from the above listed...

The Common Factors

1) The product is the same
2) The defect is the same
3) The tested product is the same
4) The Stakeholders are the same

The Difference...
The Degree of Pressure...


Where does this Pressure stem from? My thoughts listed...

Fear.... Of failing... and of succeeding.......
1) Of Failing???
Most of our managers, talk about the consequences of failure. And this thought plants itself deep-rooted in our minds. Even CEOs let talk of the wide-spread belief (yes, may people believe this still!!!) that the tester is wholly responsible for finding a defect. In the name of "sending across the right message", managers have fired testers who have been in the project for a long time, due to minimal defect leakage... Plus, add the burden that a defect leakage would also mean that all those long hours and personal sacrifices for the current project becomes wasted..... With more and more eyes watching you, fear climbs to a much higher altitude and increases in magnitude... Over time, this fear breeds into nervousness.

2) Of Succeeding???
Can someone have a fear of succeeding??? Yes.... When a project is going to end, the mind goes on visit to a few places --- 1 place is called the future, where the mind starts to thank the people who congratulate you..... While the other part of the mind has a fear of stumbling at the last hurdle.
Result ---> The tester is unable to have a rational thought, unable to make a decision on a defect and tends to think that every defect found is severe. A case of where the individual falls down due to the fear of succeeding...

3) Pressure initiated by external agencies
I don't know about you, but I have had sleepless nights prior to a very critical release. When a product goes live, there may be a zillion items that can go wrong. The importance of the occasion of the release, and the so-called expectations can do a lot of damage to the untrained mind.

How do you respond to Pressure?
Every individual have different ways of responding to pressure. Some step up, while others break down. You will have to know how you work under pressure.
And, you will have to inform your bosses as to how you work under pressure. Good Managers (and I say good managers), get the work completed by exerting the "right amount of pressure"!!!

Having said that, how do you handle pressure?
If you google, then you would get a million answers (except this blog since this ain't listed in google). But, there's only 1 person who knows it... And that 1 person is YOU. Think about how you would work under pressure and be prepared for all possible items. Be prepared to handle pressure... so when it is delivered at your door-step, you'd know how to handle it.

Some Gyan on Handling Pressure?

What do I do?
Well, I don't know how you'd handle it, but what I do is... When I come across a pressure situation, I make a conscious decision to walk away from there. By completely moving away, it offers me enough time to ensure that the steam is let off.

Controlling the mind
Some people believe in controlling the mind. Mind-Control exercises to help them focus on the current situation at hand.

A team member's role in reducing pressure...
Confiding in team members and taking their help. Works fine for a few...Some people are able to handle pressure independently and some are able to handle it as a team.

Becoming a Detached soul...
This works fine for me. Getting mentally and emotionally detached saves you from getting drained out. Once you are completely detached from the situation, from the product hype, from the expectations, you can stay in the present without letting the pressure get to you.

Fake Testers gyan
And I like giving Gyan :)!!! And please do remember, even if you don't come across this situation in your current job, it's highly likely that you'd come across this situation sometime in the near future. So, in addition to increasing your testing skills, please spare a thought, spare a moment as to how you would be handling pressure.

And use every situation to practice how you would work under pressure... It will help you to handle pressure in the upcoming days.

That's just 1 stop in your journey towards better testing...Other stops to come in later posts!!!

Re-writing Status Reports...

"A Test Execution Status Report a day keeps all the Stakeholders at bay" :)!!!!! - Very true. When your project enters the test execution phase, please keep sending every day status reports on progress, issues faced, blockers, etc. and all stakeholders would be pleased... at least, most of the time!!!

Now, some time or the other, all of us have written status reports... “Management Teams are known for asking Delivery teams periodic updates on status, status on progress and a progress on the update :) “!!! But, have we ever given a thought to RE-Writing Status Reports? What happens when you re-write a status report? The first report that we write would be a direct outpour of data in its current form. It’s only when you try to re-write it and tweak it, that you’d be able to create more visibility to what needs to be seen and ensure that the right information is present at the right place.

Few Important factors, which people keep forgetting, while writing a status report, oops… while “Re-writing” a status report ... -- Quantitative, Concise, Accuracy, Font Usage, Priorities, Risks with business impact!!!!! There might be a Zillion more, but I feel that these are 6 main “Forgotten Factors” while "re-writing" a status report.

Here is a list of common problems that I have seen in Status Reports:-

1) Forgotten Big Picture -- Forgetting the "Big Picture". Sometimes, we tend to forget the big picture and concentrate on talking about trivial issues, which may not form a part of the bigger picture.

2) It becomes an Essay instead of a Report -- Some Status Reports might run for a few pages. How many of us even try to scroll down a daily status report?

3) Hidden Critical Information -- Some reports may have the most important piece of information hidden somewhere in the middle, or in a corner. Nobody would end up seeing this, whereas, this needs to be the first information that’s seen.

4) 2 Hour Status Report -- I have seen some people spend at least 1-2 hrs every day trying to consolidate a status report. If you do this, then something somewhere’s potentially wrong.

5) Copy Pasting -- A common habit of the fake tester is to copy-paste the information from some text and circulate it. A true tester comprehends what's being sent before sending the report. A fake tester copies and pastes information without understanding what the information’s all about.

Having given the "gyan" of what's important while writing, oops.... re-writing, these daily status reports, let's try to see as to why they are important.

Why Quantitative?
Always say it with numbers. Compare the statements -- "Worked on Logout Test Cases", "Execution of 50% Logout Related Test cases. Unable to execute 20% due to show-stoppers. Pending 30%". Having a number in every statement in a report gives a very quantitative understanding of what's been accomplished.

Why Concise?
Most people do not spend more than 30 seconds on status reports. Most of them don't even scroll down. So, please use the top most part of the report to talk about Main issues, blockers, concerns, risks, etc. – Items that need immediate resolution.

Why Accurate?
Your status reports have to be accurate enough to reflect current status. People would not be interested in the status as of the day's morning or afternoon.
In the long run, you might end-up losing your credibility if your reports are not accurate.

Font Usage --- What for?
Fonts are available to be used. Use RED for Blockers, Green for Concerns, etc. With a different color, issues get highlighted more easily and are more readable.

Do-next Priorities ---- In a status report?
Yes. Assuming you have identified 3 blockers and 5 defects, ensure that you put the name of the person who needs to look into it against each one of them. It would ensure everybody would know who is looking into a potential issue and ask him for status, accordingly. If it is a group, please find out who would be replying to you on this, and include their name in the report.

Risks in a daily status report?
Yes. If there are some defects, which can result in a business risk, please highlight it accordingly. It always pays to highlight the business risk so that the right person would look into it at the right time.

Project Risks and a Business Risk are different things. Please understand the difference while highlighting them.

After you write and re-write your status reports, do a quick check as to whether the reports are Quantitative, Concise, Accurate, Usage of Fonts, Do-next Priorities mentioned and Risks explained clearly.

Other than the above, there are other factors such as System Health Check, Severe defects, Priority fixes required, Work completed in the current day and plan for the next day, etc. etc. etc., but, these would be a part of many status reports. I tried to talk about the above since those are very crucial factors, which can be easily missed…. Project Pressure does so many funny things… so does the late night cab that you have booked which keeps honking while you try to send this report in a hurry and catch the cab!!!

Try this if you want to… Write a Status report… And use the above checklist to re-write it --- See the difference… Keep doing it till they become a habit with you… Happy Re-writing :) !!!

Continuing the saga of UI Testing – The Sequel (Part 2)…..

Ever wondered what would happen to all the keyboards of the world if a new alphabet is added to the English language? Well, I definitely don’t know what’d happen…... and will not attempt to answer this question with this post….... but, this would have been identified in the previous century if there had been in existence, a UI tester who could have tested the keyboard UI design with an eye on scalability... This would be 1 truly defined hallmark of a person with the mind of a "True Tester"!!!!! But we never had any testers in the profession in the Middle Ages, did we?

When I read the above paragraph, I think it's highly likely that I sound more like an idiot.... why on earth would someone want to add a new alphabet to the English language? But again, to answer that..... “Why not???”

This is my 2nd post on the never ending saga of UI Testing - And personally speaking, I'd classify the 12 pillars of UI Testing as follows - Accessibility, Alignment, Appearance, Compatibility, Error Handling, Hot Keys, Menus, Navigation, Security, Style sheets, Toolbars and Client-side UI Validations.

What? Did I read it right? Client-side UI Validations? Yes, of course. I believe that these UI validations also should be an integral part of UI testing…. Mainly because an integral pre-requisite to UI testing is to understand very specific customer requirements, and these might take the form of client-side validations, which is why I am including this section over here.

Now, please note that the perfect UI classification would be found only in the world of the "Fake Software Tester"... In the true tester’s world, nothing and nobody is perfect.... Neither is this post... If you disagree, please feel free to email those rotten tomatoes along with your set of curses, along with the areas of disagreement, so that we can debate it and arrive at a logical conclusion…

Below, am listing a few UI practices that I have been allowed to share with you… I am listing only the ones that I have seen and heard in my life…. And beware, BE AWARE… Practices the below at your own peril, for these are certified fake practices and can kill the customer…

1) Not Understanding the Difference between Functional and UI Testing
I asked a friend, whom I perceive as a “Fake Software Tester” to do some UI testing on Facebook….and below is what she did…
a) Created an account for herself and logged into facebook
b) Went ahead and added a few friends, added a few applications
c) Checked out the Privacy settings of Facebook......
d) Started to send messages and check if the messages were reaching people…
e) Started testing the wall feature…
f) And went on and on and on……

Sadly, this friend of mine did not understand the difference between functionality and UI. Understanding this difference is very important so that you can focus all your testing skills on the right areas.

2) Ignoring the client bandwidth while doing UI Testing for Web-based applications
There was this customer, for which a friend's team was involved in UI testing. After the usual project life-cycle, the project went live. But, customers started complaining of slowness, which was routed to the performance teams. The project team said that the application stability was just fine. The problem --- Most of the clients were accessing the application from 14 KBPS low-band width networks.

Sadly, when the test team did the UI testing, they did not understand the network band-width of the customer. There were so many unwanted GIF and JPEGs; so much of text was getting downloaded. It was a typical example where the test team could have suggested the usage of a "HTML Shrinker" type utility, but this suggestion was never made. Some of you might disagree stating that this would be a design problem, but this is an example to re-iterate the fact that we need to understand customer networks while doing our testing.

3) Understanding the true needs of the customer - A "timely" check
There is this story of this testing team testing the clock feature. The entire team tested these using local clock settings. The entire feature was tested with a DD-MM-YYYY format. But, the client was based out of a time zone that preferred an YYYY-MM-DD format. This issue was detected very late only in the UAT phase, when the UI testing bit should have caught this.

4) Understanding any specific Client Requirements - An Appearance Incident
For an online-form that is to be used by an eye hospital, the missing fields were highlighted in RED. As per the specified requirements, the test team tested if the fields are highlighted in RED. But, when the application went live, the customers were unable to understand the missing fields, since they were color blind and could not recognize the letters.

5) A cold story of Hot-Keys
An ex-boss of mine narrated an interesting incident. This team did a lot of research and hard-work to develop a text editor component along with their application. When they created this text editor, they made a decision to have their own hot keys for this text editor. What they had forgotten was that the users are mostly used to "Ctrl+C" for copy and "Ctrl+v" for pasting, and not having this combination would mean that the users thought that this feature of copy-Paste was unavailable in the text editor. Sadly, for all their hard-work, the text editor remained unused.

The creators of the application insisted that the hot-keys of the application were listed in the Help Manual. Sad but true, in today’s world, most of our users do not read the application manual. Whatever appears in the help manual remains mostly unread. And the application dev team forgot to understand that most of us are most comfortable with the windows specific hotkey combinations.

6) When Compatibility testing takes a vacation…
Once, Compatibility went on holiday. There was a very smart Project Manager (smart since he considered himself very smart), who decided that he can reduce testing time and project cost by making an assumption that if 1 Page works on most browser-screen resolution combinations, all other pages would. The biggest problem with this assumption was that the "Search" and "Registration" pages were not tested on the other screen resolution combinations. When the app went live, the product owners realized that the most commonly used fields of the application were hidden and the user had to do a lot of scrolling.

Compatibility testing means that you test all features of the application on the browser-screen resolution combinations. Such incidents commonly occur even today, when you have a very inexperienced team of project managers handling the delivery of the application.

7) Error-handling
Do the following error messages make any sense?
i. Error 2003- Please contact the admin.
ii. Error -03841769-Error Occurred.
iii. Error 80184436-Duplicate is existing.

Above mentioned error messages are meaningless. The tester should ensure that he tries out all possible error messages and the fact that they have meaningful information that's passed on to the reader. This is where the aspect of a manual tester scores over automation, since an automated application cannot detect when error messages are stupid.

Moving on.... let me tell you my bit about the aspect of Error Handling, Compatibility and Hot Keys in the world of UI Testing...

Compatibility - What do you most commonly look for while doing Compatibility Testing?
What is compatibility testing? My answer's very simple.... Ensure adherence to all screen resolutions of the client. When you do compatibility testing, ensure the following:-
1) Make a list of all browsers that would be used to access your application
2) Make a list of all Operating systems, or other dependant frameworks or software
3) Make a list of all screen resolutions
And make a list of the above combination and test with all the possible combinations. Do not be a Fake tester and limit yourself to this combination alone… Each application is unique and demands that you make up your combination, which is specific to the application.

Error Handling - What do you most commonly look for while doing "Error Handling" Testing?
1) Consistency - Error Handling has to be consistent. The same error condition should throw the same error messages across the application.
2) Meaningful - The error message should be meaningful and the customer should be able to understand it.
3) Educative - Error messages should assume that the user does not know a lot about the application and should guide the user of what the mistake was and what are the rectifying steps.
4) Returning Tab Focus - Also check if the tab focus returns to the erroroneous fields, when there is an error.
5) Working closely with Business - Work with the business user and make a list of all error messages that can occur. Ensure that you test for all the error message combinations and for consistent error messages.


Hot Keys - What do you most commonly look for while doing "Hot Keys" testing?
What are hot keys? We all know about it... but the fake tester does not. Let's simply say hot-keys are keys that are provided by windows to execute OS specific commands. It is not right for us to replace those keywords to execute my program specs. My program should never over-write the operating system commend. What do you look for while doing “Hot Key” testing?
1) Over-riding - Hot Keys provided by the Operating system cannot be over-ridden (all standard hot keys such as (ctrl+c, ctrl+v etc.) are working as expected)
2) Cancel & Escape - Functionality of the cancel buttons and the escape key need to be similar
3) Duplication - No duplication of hot keys
4) F1 - F1 always takes me to help from anywhere in eh application
5) Alt-F4 - Alt+F4 triggers the window.close event and.

Cut to the Chase
During my limited time so far in the world of testing, I admint to having been a “Fake Tester” for a very long time. And in some aspects, I still am one. I have interviewed/questioned many fake testers and their understanding of UI testing. Some of the stories listed above come from what I have listened to them in these interviews.

It is very important for the tester to understand the 12 pillars of UI testing and plan the tests around these. I have talked about the A's earlier and about the other 3 over here.

The list of items under these classifications is huge... and I believe the classification list is endless. But, what I have put in here are only my thoughts on this and feel free to comment/write to me and let me know whatever I have missed out over here.

And saying adios with the promise to write about the last 6 pillars -- Menus, Navigation, Security, Style sheets, Toolbars and Client-side UI Validations -- in a subsequent post..... Have a great week ahead!!!

Beginner's Navigation Guide to the Galaxy of "Report Testing" in Software...

Every application has Reports... But are they mostly over-looked by testers? Do we really devote enough thought to plan on testing reports?

Well, I could not find enough stuff on the 1st few pages returned by Google for the search "Tips and Tricks for testing software reports", which prompted me to post this... Actually, there's another reason too... Reports is 1 part of the system, which cannot be neglected since their importance by the project teams are undermined, but they form a very crucial part of the system... Why so? Mainly because, it is on the information presented in reports that business make a lot of their decisions. This is something which will not be understood by the "Fake Testers" that are in business today.

Trying to broadly classify the reports that are a part of every software solution, I have come across 2 kinds of Reports - "Operational" and "Historical Reports". (If you can think of a broader classification, which I am sure you would, then please write it to me so that I can be proved to be a member of the Fake Software Testing Community).

Operational Reports - Reports in the application that is used to drive day-to-day applications.

Historical Reports - Reports on the basis of data retrieved from the database that forms the basis for key management decisions.

Am sure that you would have encountered both types... (and you can classify even more to it...)

Top 6 aspects that are mostly overlooked by the fake software tester, when planning on testing software reports are, in my humble opinion are as follows:-

1) Data Preparation for Testing Reports
a. Importance of Test Data - It's impossible to prepare test data for testing software reports in a single day!!! You will need to understand the business context and create all your test data accordingly, as close to real-life data as possible. The fake software his this irritating habit to create a "data
dump" with data that comes to his head, without understanding if this would be close to like-live data. This would Rank @ No. 1 in the "Most Horrible Practices of Software Testing" list. The Data that is being prepared should be close to like-live data and it's 1 of the best practices to have the business users take a quick look at this data that'd be tested.

b. Life Span of Application and Frequency of Report Generation - Mostly, you would need to create/generate your own data and so, please understand the life span of the application being tested, before coming up with like live data creation for testing. Also understand the frequency with which the reports would be
generated so that your data preparation can be like that.

c. Sourcing for Reports - If the data can come from an external data feed, please work with the corresponding systems to get the data in place for your testing. If the data is going to change almost on a daily basis, there would be situations wherein you will have to change dates or other attributes in the system. See if you can have an automated program setup to generate this data for you.

Please plan for data preparation for testing Reports as part of your "Data Preparation" Activity.

It is very difficult for me to imagine a fake tester dedicate so much time and plan for the above mentioned activities as part of testing. To quote an example, I remember seeing data prepared by a fake tester contain "Sundays", when the report is on a stock movement over a 5 week period. The fake software tester does not realize that the stock market is closed on Sundays, mostly!!!

2) NFRs for Reports
a. Response Times - Do reports need NFRs? This is 1 question that is asked by the fake software tester. Another set would state that the business never gave them any NFRs. Sample this, if the business user expects a report to be generated in 4 seconds, then that needs to be something which you need to test. A
true tester would question NFRs for Reports and would ensure that testing for this would be incorporated in the report.

b. Report Data Volumes - Another thing that the fake tester does not ask for is for how many records get processed for a particular report, at least a range. There is no point trying to test with 200 trillion records, if the business expectation of data is only around, say 40381 records :)!!!

Response Time and Record Volumes need to be understood before starting to test reports.

3) SQL Related Testing of Reports
This is 1 area that I believe that the fake tester is completely blind to.

a. SQL Profiler - After executing the report, it is a good habit to make use of a database profiler tool (SQL Profiler for SQL and not sure for other
databases), to run a trace and check for performance improvement. Giving the trace to the developer on the test systems would fully tell the developer on what tweaking he needs to do and would execute the report in a much quicker time.

b. Dynamic SQL Query - A lot of freshers miss this. I was once asked to do a check as to why reports take a lot of time to respond. When I looked into the Sproc, I realized that there was a lot of static SQL, when the need was for it to be dynamic. There was no point in me raising a defect stating that the
query takes a long time. Though you are the developer's enemy, please remember it is such small actions, which would go a long way in bridging the gap between you and the development teams.

c. Overnight reports - Now, in case the data is not based on daily transactions, and the report is required during the day, it makes a lot of sense to have the data archived into a separate table and a separate Sproc to get data from this table for the reports. I am really not sure if all the SQL Gurus of the world
would point their guns at me if this is incorrect, but again, please remember I have a lot of traits of the Fake Software Tester, and am currently in the
transition phase :) !!!

4) Print Functionality of Reports
Every Report would print into a paper. Though all the "Go-Green" people of the world would hate me for this, a true tester has to be "Non-Go-Green" and would, I believe waste a lot of paper in testing this functionality.

A true tester, before designing his tests for reports, would go the extra mile to understand the following
a. What is the printer used when the application is like-live
b. What is the printer configuration data when the application is like-live
c. What kind of paper is used for printing? (A4, A3, A2, etc... Any other specific configuration)

Importance of type of Paper - There’s reason for understanding the kind of paper. I remember 1 client of my peer, who used "Ruled Paper" for his testing, so that the test report had data in the corresponding columns. For some reason (I remember that it was incompatibility with other applications), that they got all reports printed on a paper. Now, this type of "Special Paper" was never used for testing by the test team and this resulted in a lot of surprises when the application went live.

Importance of Printer Configuration - Also, if the customer is using landscape printing on A2 for a type of report, then what value does the test team add in testing those reports on A4 sheet using Portrait configurations?

Importance of trying to print many times before Go-LIVE - And, please do a lot of print-outs to see how the print-outs would look in actual, how much ever times you can!!! Only if you take print-outs can you fix problems in areas where the printed copy goes out of the paper borders, totals are printed in a
separate page, proper page alignment for report header and footer, proper alignment for the summary, font related issues, etc.

This area, sadly, is also mostly neglected by the fake tester.

5) Import and Export Functionalities/Search Criteria/Localization
Every report has import and export possibilities. Export, definitely yes, but import???

Of course... I have seen cases where the user wants to import the query from a text file. So, the possibility of "Import" always exists, though it is very minimally used.

Now, we need to understand the various applications and versions of the applications to which the export should support and test with all of those... I do know of cases where the user was using very old versions of Excel and the application refused to export into these versions of the Office software. Sadly, there were a lot of escalations questioning delivery capability, by the time this escalation happened!!!

The search criteria are tested, but needs to be tested to ensure that the user does not search with what is not possible. Search can throw up a lot of defects. Would I want to save a search?

6) And understanding the business context and business logic behind the reports...
And as I had stated at the beginning, please understand the business logic and the business context behind the report that you are testing. Ask the following questions
"Why do you need the report for business?"
"What data is shown and what decisions can be made by this report"
"Why do you feel some columns are not required while some are required?"
"Why do you think that this Report makes sense to business and if not, why do you think that this report is not required"

And to re-iterate, there are a lot of other aspects of software reports that need to be tested. Not just the 6. Listed above is what I feel are the mostly overlooked aspects, which need to be re-looked at. And since Nobody's Perfect, neither am I.... If you really feel that there is more to add, please keep those comments coming in...

Very unfortunately, the fake software tester does not realize the importance of testing reports, which can, indirectly lead to a very wrong business decision, which can potentially bring down the entire company or cause huge losses to the company... It is my wish that there come a day every tester understands the business context of the reports that he tests and tests it using the business perspective...

And until this day arrives, the ranting of this fake software tester would definitely continue.....!!!

Standard Disclaimer - And an Apology...

Ever since that I started this blog a couple months back, I have been asked by a lot of peers and ex co-workers if I'd categorize them as members of the group of "Fake Software Testers".

My answer to all --- Yes and No.

To clarify, this blog does not have any direct reference to my current and past co-workers, nor do I even try to implicate that they are fake testers. To be honest, I have had the privilege of working with a lot of the brightest minds in the past couple of decades. And though some of my current colleagues do not realize it, their approach to the field of software testing makes me pale in comparision!!!

But, having said that, I have been seeing a lot of fake testing in the past few years happen, which is what I am trying to set right with my blog. It saddens the heart to see a lot of these fake testers, being hailed as very good testers. I don't have a problem with them being rewarded... but, when they are given bigger responsibilities, we are setting them up for failure.... I am not too worried about the projects failing, but.... if these testers are pushed into testing software which saves lives, or flight software, then that is very dangerous.... Their failure to detect a defect might kill a few lives!!! --- that's what I am worried about.

Even I --- for a considerable amount of time, have been (and sometimes, continue being) a fake software tester ----- and I realized I'd remain so until someone pointed it out!!!

That is why, I am trying to create a blog which would also talk about some of the fake/wrong practices of testing, which is so much prevelant these days. I am also trying to share a few of my experiences with this.....Not all of it is right, and not all of it is wrong. It's only when I try to don the hate of a fake tester that I am able to see what is right and what's wrong... from purely my perspective!!!

And 1 more thing... I have not declared my identity on this blog. That's for only 1 reason ----- it gives me immense satisfaction, to being referred to as "The Fake Software Tester"!!!!!

English - "A thinking language"...!!!

Was just reading Pradeep's blog on "Coaching testers on Bug Reports, Advocacy and Credibility" @ http://testertested.blogspot.com/, when the following thought took shape in my head.

In all our interviews, we ask for a list of known languages of the candidates. But, we never ask for the "Primary Thinking Language" of the applicant, neither do we make an effort to find out what that is.

Going through some of the awful bug reports that Pradeep has reported, and having sampled much more of the same in my life, I am of the opinion that we need to look at ways to identify the tester's "Primary Thinking Language", and all our training programs on communication skills should also look at how English becomes our primary thinking language - since English is the universally accepted language for defect reporting.

We ask them for languages that they speak, they write and they can read. But have we ever, for a moment, thought of the importance of identifying the "Primary Thinking Language" at all? Having posed this question, what do I mean when I say "Primary Thinking Language"???

The way I'd define it is... Your Primary Thinking Language is the language of the word that comes immediately to mind when you think/look at an object or think of an emotion. We did the following experiment with a group of people who were predominantly tamil speaking and another group who are management grads from IIM.

We showed them a door, An Angry Amitabh Bachan scene from a movie, a key to the door, a picture of a child sleeping and the photo of a tomato.... and asked them to answer whatever comes to mind in the 1st second.... The answers that they gave...

Majoring of answers from the tamil speaking group --- Kadavu/maram, padam/cinema/kovam, saavi/pootu, kuzhandhai/azhagu/thookam, thakkali... (Words that first came to their mind were in their mother tongue... Conclusion ---> the primary thinking language --- Tamil)

The English speaking Management Graduates rattled off the following words --- Door, anger/flick/movie, key, child/sleeping/bliss, veggie/sandwich/rotten tomato..... That helped us conclude what's their "Primary Thinking Language" was --- English, Obviously.

You can do this experiment with someone you know to identify your primary thinking language and you would know what I am trying to say here...

Training Deparments --- Please take note.... Please help employees identify their "Primary Thinking Language" and work with them long-term to make English as the "Primary Thinking Language", so that it would be beneficial for both the employee and the company in the long run....

And I don't think that this is a problem specific to India... I am sure this problem exists in many countries where English is not the primary speaking language.....

And yes.. this article has nothing to do with fake testing... Just some thoughts that came to mind on reading Pradeep's blog... The fake tester's off on a long weekend... See you in March!!!

The 1st 10 mins...

What is the most important time of the day? For me, it's always the first 10 mins of today and the last 10 mins of the previous working day. Most often than not, these 10 mins have always decided on the future course of actions for the entire day. They have determined if the entire day is going to have a smooth flow, or if there are going to be rough patches.

As soon as you enter work, your investment on the first 10 mins would reap the results for the rest of the day. Please put all stops in place and finalize your day's plan during the 1st 10 mins. Please tell your boss that you can't be disturbed during this time. Please inform your sub-ordinates that you cannot be disturbed during this time. Having put all these stops in place, what do you do in the 1st 10 mins?

Well, I don't know what you do (how can I :), but here's what I do during these 10 mins.

1) A lot changes during the previous day....
Most probably, most of our project team members live across the world in a different timezone. Please check your emails and make a quick check of items that you need to accomplish during the day.

2) Plan for the top 3 things that you want to do for the day.
On Paper, draw a simple list of top 3 things that you need to complete for the day. It can be completing a review, completing some documentation, doing some testing, doing some test data preparation, or whatever... But, please have a good clarity on the top 3 tasks that you plan to achieve for that particular day at work.

3) Planning for interruptions...
All of us get interrupts.... how do we handle interrupts? We don't have to look far... we only need to take a good look into the mind of a very effective operating system to understand how the system handles interrupts and if you study it, you would know that the system handles interrupts in the best possible manner. You need to mimic the same... it's too simple, really!!! The moment you get an interrupt, all you need to do is to see if you have to place that task among the top 3 activities for the day. If not, it can wait!!!

4) Plan for what to discuss during the meetings on this day...
Make a quick list of meetings that you need to participate... And plan for the top 2 things that you want to talk about in those meetings. Nobody likes someone who keeps talking and talking... By talking less, you would be able to create the focus on the most wanted items. Make a list of the things that you want to talk in meetings for the day.

5) If you feel the need to decline or postpone meetings, do it at this time.
You cannot be in all the places at all the time. If you feel that you need not attend meetings, please decline them 1st thing in the day. There is no harm in declining meetings. You have to do prioritizations effectively.

6) Finalize the effective working time during the day. The time when you will not be disturbed by anyone...
This is something that I do religiously... For at least 4 hours during the day, please ensure that there is no disturbance. Do not take coffee breaks.. do not answer phones... do not take gossip breaks... email breaks... browsing breaks.... break from breaks, etc.etc.etc... For 2 hours in the AM and 2 hours in the PM, please focus only on deliverables. I call this EFFECTIVE WORK HOURS. Makes a world of difference to your deliverables...

7) Playing hooky from work is always important.... -
Finalize the time of the day when you play hooky and read fun-emails and internet browsing, Office Gossips, Purchasing stocks on the stock market, etc. None of us refrain from this and in a way, this is always important.

And very important... though it is very important for you to be at the beck and call of your boss, please don't do that in the first 10 mins of your day...

Please keep your boss, your boss's boss and your boss's boss's boss far far away during this time :) !!!

The fake tester always reaches for a cup of coffee and checks all fun forwarded emails before starting off. His entire day gets screwed up, and he cannot prioritize. He spends most of his working time in something which is not important at all. For example, If you have 90% notion that a feature will not be delivered, why would you need to spend time on testing it? Please remember... investing in the 1st 10 mins of the day would be very effective for the entire day, and you would reap results for the rest of your long career!!!

What the fake tester does not realize is that...... Dedicated planning during the first 10 mins of the day, would ensure that you own the entire day!!!!!