Saturday, March 20, 2010

Continuing the saga of UI Testing – The Sequel (Part 2)…..

Ever wondered what would happen to all the keyboards of the world if a new alphabet is added to the English language? Well, I definitely don’t know what’d happen…... and will not attempt to answer this question with this post….... but, this would have been identified in the previous century if there had been in existence, a UI tester who could have tested the keyboard UI design with an eye on scalability... This would be 1 truly defined hallmark of a person with the mind of a "True Tester"!!!!! But we never had any testers in the profession in the Middle Ages, did we?

When I read the above paragraph, I think it's highly likely that I sound more like an idiot.... why on earth would someone want to add a new alphabet to the English language? But again, to answer that..... “Why not???”

This is my 2nd post on the never ending saga of UI Testing - And personally speaking, I'd classify the 12 pillars of UI Testing as follows - Accessibility, Alignment, Appearance, Compatibility, Error Handling, Hot Keys, Menus, Navigation, Security, Style sheets, Toolbars and Client-side UI Validations.

What? Did I read it right? Client-side UI Validations? Yes, of course. I believe that these UI validations also should be an integral part of UI testing…. Mainly because an integral pre-requisite to UI testing is to understand very specific customer requirements, and these might take the form of client-side validations, which is why I am including this section over here.

Now, please note that the perfect UI classification would be found only in the world of the "Fake Software Tester"... In the true tester’s world, nothing and nobody is perfect.... Neither is this post... If you disagree, please feel free to email those rotten tomatoes along with your set of curses, along with the areas of disagreement, so that we can debate it and arrive at a logical conclusion…

Below, am listing a few UI practices that I have been allowed to share with you… I am listing only the ones that I have seen and heard in my life…. And beware, BE AWARE… Practices the below at your own peril, for these are certified fake practices and can kill the customer…

1) Not Understanding the Difference between Functional and UI Testing
I asked a friend, whom I perceive as a “Fake Software Tester” to do some UI testing on Facebook….and below is what she did…
a) Created an account for herself and logged into facebook
b) Went ahead and added a few friends, added a few applications
c) Checked out the Privacy settings of Facebook......
d) Started to send messages and check if the messages were reaching people…
e) Started testing the wall feature…
f) And went on and on and on……

Sadly, this friend of mine did not understand the difference between functionality and UI. Understanding this difference is very important so that you can focus all your testing skills on the right areas.

2) Ignoring the client bandwidth while doing UI Testing for Web-based applications
There was this customer, for which a friend's team was involved in UI testing. After the usual project life-cycle, the project went live. But, customers started complaining of slowness, which was routed to the performance teams. The project team said that the application stability was just fine. The problem --- Most of the clients were accessing the application from 14 KBPS low-band width networks.

Sadly, when the test team did the UI testing, they did not understand the network band-width of the customer. There were so many unwanted GIF and JPEGs; so much of text was getting downloaded. It was a typical example where the test team could have suggested the usage of a "HTML Shrinker" type utility, but this suggestion was never made. Some of you might disagree stating that this would be a design problem, but this is an example to re-iterate the fact that we need to understand customer networks while doing our testing.

3) Understanding the true needs of the customer - A "timely" check
There is this story of this testing team testing the clock feature. The entire team tested these using local clock settings. The entire feature was tested with a DD-MM-YYYY format. But, the client was based out of a time zone that preferred an YYYY-MM-DD format. This issue was detected very late only in the UAT phase, when the UI testing bit should have caught this.

4) Understanding any specific Client Requirements - An Appearance Incident
For an online-form that is to be used by an eye hospital, the missing fields were highlighted in RED. As per the specified requirements, the test team tested if the fields are highlighted in RED. But, when the application went live, the customers were unable to understand the missing fields, since they were color blind and could not recognize the letters.

5) A cold story of Hot-Keys
An ex-boss of mine narrated an interesting incident. This team did a lot of research and hard-work to develop a text editor component along with their application. When they created this text editor, they made a decision to have their own hot keys for this text editor. What they had forgotten was that the users are mostly used to "Ctrl+C" for copy and "Ctrl+v" for pasting, and not having this combination would mean that the users thought that this feature of copy-Paste was unavailable in the text editor. Sadly, for all their hard-work, the text editor remained unused.

The creators of the application insisted that the hot-keys of the application were listed in the Help Manual. Sad but true, in today’s world, most of our users do not read the application manual. Whatever appears in the help manual remains mostly unread. And the application dev team forgot to understand that most of us are most comfortable with the windows specific hotkey combinations.

6) When Compatibility testing takes a vacation…
Once, Compatibility went on holiday. There was a very smart Project Manager (smart since he considered himself very smart), who decided that he can reduce testing time and project cost by making an assumption that if 1 Page works on most browser-screen resolution combinations, all other pages would. The biggest problem with this assumption was that the "Search" and "Registration" pages were not tested on the other screen resolution combinations. When the app went live, the product owners realized that the most commonly used fields of the application were hidden and the user had to do a lot of scrolling.

Compatibility testing means that you test all features of the application on the browser-screen resolution combinations. Such incidents commonly occur even today, when you have a very inexperienced team of project managers handling the delivery of the application.

7) Error-handling
Do the following error messages make any sense?
i. Error 2003- Please contact the admin.
ii. Error -03841769-Error Occurred.
iii. Error 80184436-Duplicate is existing.

Above mentioned error messages are meaningless. The tester should ensure that he tries out all possible error messages and the fact that they have meaningful information that's passed on to the reader. This is where the aspect of a manual tester scores over automation, since an automated application cannot detect when error messages are stupid.

Moving on.... let me tell you my bit about the aspect of Error Handling, Compatibility and Hot Keys in the world of UI Testing...

Compatibility - What do you most commonly look for while doing Compatibility Testing?
What is compatibility testing? My answer's very simple.... Ensure adherence to all screen resolutions of the client. When you do compatibility testing, ensure the following:-
1) Make a list of all browsers that would be used to access your application
2) Make a list of all Operating systems, or other dependant frameworks or software
3) Make a list of all screen resolutions
And make a list of the above combination and test with all the possible combinations. Do not be a Fake tester and limit yourself to this combination alone… Each application is unique and demands that you make up your combination, which is specific to the application.

Error Handling - What do you most commonly look for while doing "Error Handling" Testing?
1) Consistency - Error Handling has to be consistent. The same error condition should throw the same error messages across the application.
2) Meaningful - The error message should be meaningful and the customer should be able to understand it.
3) Educative - Error messages should assume that the user does not know a lot about the application and should guide the user of what the mistake was and what are the rectifying steps.
4) Returning Tab Focus - Also check if the tab focus returns to the erroroneous fields, when there is an error.
5) Working closely with Business - Work with the business user and make a list of all error messages that can occur. Ensure that you test for all the error message combinations and for consistent error messages.


Hot Keys - What do you most commonly look for while doing "Hot Keys" testing?
What are hot keys? We all know about it... but the fake tester does not. Let's simply say hot-keys are keys that are provided by windows to execute OS specific commands. It is not right for us to replace those keywords to execute my program specs. My program should never over-write the operating system commend. What do you look for while doing “Hot Key” testing?
1) Over-riding - Hot Keys provided by the Operating system cannot be over-ridden (all standard hot keys such as (ctrl+c, ctrl+v etc.) are working as expected)
2) Cancel & Escape - Functionality of the cancel buttons and the escape key need to be similar
3) Duplication - No duplication of hot keys
4) F1 - F1 always takes me to help from anywhere in eh application
5) Alt-F4 - Alt+F4 triggers the window.close event and.

Cut to the Chase
During my limited time so far in the world of testing, I admint to having been a “Fake Tester” for a very long time. And in some aspects, I still am one. I have interviewed/questioned many fake testers and their understanding of UI testing. Some of the stories listed above come from what I have listened to them in these interviews.

It is very important for the tester to understand the 12 pillars of UI testing and plan the tests around these. I have talked about the A's earlier and about the other 3 over here.

The list of items under these classifications is huge... and I believe the classification list is endless. But, what I have put in here are only my thoughts on this and feel free to comment/write to me and let me know whatever I have missed out over here.

And saying adios with the promise to write about the last 6 pillars -- Menus, Navigation, Security, Style sheets, Toolbars and Client-side UI Validations -- in a subsequent post..... Have a great week ahead!!!

Sunday, March 7, 2010

Beginner's Navigation Guide to the Galaxy of "Report Testing" in Software...

Every application has Reports... But are they mostly over-looked by testers? Do we really devote enough thought to plan on testing reports?

Well, I could not find enough stuff on the 1st few pages returned by Google for the search "Tips and Tricks for testing software reports", which prompted me to post this... Actually, there's another reason too... Reports is 1 part of the system, which cannot be neglected since their importance by the project teams are undermined, but they form a very crucial part of the system... Why so? Mainly because, it is on the information presented in reports that business make a lot of their decisions. This is something which will not be understood by the "Fake Testers" that are in business today.

Trying to broadly classify the reports that are a part of every software solution, I have come across 2 kinds of Reports - "Operational" and "Historical Reports". (If you can think of a broader classification, which I am sure you would, then please write it to me so that I can be proved to be a member of the Fake Software Testing Community).

Operational Reports - Reports in the application that is used to drive day-to-day applications.

Historical Reports - Reports on the basis of data retrieved from the database that forms the basis for key management decisions.

Am sure that you would have encountered both types... (and you can classify even more to it...)

Top 6 aspects that are mostly overlooked by the fake software tester, when planning on testing software reports are, in my humble opinion are as follows:-

1) Data Preparation for Testing Reports
a. Importance of Test Data - It's impossible to prepare test data for testing software reports in a single day!!! You will need to understand the business context and create all your test data accordingly, as close to real-life data as possible. The fake software his this irritating habit to create a "data
dump" with data that comes to his head, without understanding if this would be close to like-live data. This would Rank @ No. 1 in the "Most Horrible Practices of Software Testing" list. The Data that is being prepared should be close to like-live data and it's 1 of the best practices to have the business users take a quick look at this data that'd be tested.

b. Life Span of Application and Frequency of Report Generation - Mostly, you would need to create/generate your own data and so, please understand the life span of the application being tested, before coming up with like live data creation for testing. Also understand the frequency with which the reports would be
generated so that your data preparation can be like that.

c. Sourcing for Reports - If the data can come from an external data feed, please work with the corresponding systems to get the data in place for your testing. If the data is going to change almost on a daily basis, there would be situations wherein you will have to change dates or other attributes in the system. See if you can have an automated program setup to generate this data for you.

Please plan for data preparation for testing Reports as part of your "Data Preparation" Activity.

It is very difficult for me to imagine a fake tester dedicate so much time and plan for the above mentioned activities as part of testing. To quote an example, I remember seeing data prepared by a fake tester contain "Sundays", when the report is on a stock movement over a 5 week period. The fake software tester does not realize that the stock market is closed on Sundays, mostly!!!

2) NFRs for Reports
a. Response Times - Do reports need NFRs? This is 1 question that is asked by the fake software tester. Another set would state that the business never gave them any NFRs. Sample this, if the business user expects a report to be generated in 4 seconds, then that needs to be something which you need to test. A
true tester would question NFRs for Reports and would ensure that testing for this would be incorporated in the report.

b. Report Data Volumes - Another thing that the fake tester does not ask for is for how many records get processed for a particular report, at least a range. There is no point trying to test with 200 trillion records, if the business expectation of data is only around, say 40381 records :)!!!

Response Time and Record Volumes need to be understood before starting to test reports.

3) SQL Related Testing of Reports
This is 1 area that I believe that the fake tester is completely blind to.

a. SQL Profiler - After executing the report, it is a good habit to make use of a database profiler tool (SQL Profiler for SQL and not sure for other
databases), to run a trace and check for performance improvement. Giving the trace to the developer on the test systems would fully tell the developer on what tweaking he needs to do and would execute the report in a much quicker time.

b. Dynamic SQL Query - A lot of freshers miss this. I was once asked to do a check as to why reports take a lot of time to respond. When I looked into the Sproc, I realized that there was a lot of static SQL, when the need was for it to be dynamic. There was no point in me raising a defect stating that the
query takes a long time. Though you are the developer's enemy, please remember it is such small actions, which would go a long way in bridging the gap between you and the development teams.

c. Overnight reports - Now, in case the data is not based on daily transactions, and the report is required during the day, it makes a lot of sense to have the data archived into a separate table and a separate Sproc to get data from this table for the reports. I am really not sure if all the SQL Gurus of the world
would point their guns at me if this is incorrect, but again, please remember I have a lot of traits of the Fake Software Tester, and am currently in the
transition phase :) !!!

4) Print Functionality of Reports
Every Report would print into a paper. Though all the "Go-Green" people of the world would hate me for this, a true tester has to be "Non-Go-Green" and would, I believe waste a lot of paper in testing this functionality.

A true tester, before designing his tests for reports, would go the extra mile to understand the following
a. What is the printer used when the application is like-live
b. What is the printer configuration data when the application is like-live
c. What kind of paper is used for printing? (A4, A3, A2, etc... Any other specific configuration)

Importance of type of Paper - There’s reason for understanding the kind of paper. I remember 1 client of my peer, who used "Ruled Paper" for his testing, so that the test report had data in the corresponding columns. For some reason (I remember that it was incompatibility with other applications), that they got all reports printed on a paper. Now, this type of "Special Paper" was never used for testing by the test team and this resulted in a lot of surprises when the application went live.

Importance of Printer Configuration - Also, if the customer is using landscape printing on A2 for a type of report, then what value does the test team add in testing those reports on A4 sheet using Portrait configurations?

Importance of trying to print many times before Go-LIVE - And, please do a lot of print-outs to see how the print-outs would look in actual, how much ever times you can!!! Only if you take print-outs can you fix problems in areas where the printed copy goes out of the paper borders, totals are printed in a
separate page, proper page alignment for report header and footer, proper alignment for the summary, font related issues, etc.

This area, sadly, is also mostly neglected by the fake tester.

5) Import and Export Functionalities/Search Criteria/Localization
Every report has import and export possibilities. Export, definitely yes, but import???

Of course... I have seen cases where the user wants to import the query from a text file. So, the possibility of "Import" always exists, though it is very minimally used.

Now, we need to understand the various applications and versions of the applications to which the export should support and test with all of those... I do know of cases where the user was using very old versions of Excel and the application refused to export into these versions of the Office software. Sadly, there were a lot of escalations questioning delivery capability, by the time this escalation happened!!!

The search criteria are tested, but needs to be tested to ensure that the user does not search with what is not possible. Search can throw up a lot of defects. Would I want to save a search?

6) And understanding the business context and business logic behind the reports...
And as I had stated at the beginning, please understand the business logic and the business context behind the report that you are testing. Ask the following questions
"Why do you need the report for business?"
"What data is shown and what decisions can be made by this report"
"Why do you feel some columns are not required while some are required?"
"Why do you think that this Report makes sense to business and if not, why do you think that this report is not required"

And to re-iterate, there are a lot of other aspects of software reports that need to be tested. Not just the 6. Listed above is what I feel are the mostly overlooked aspects, which need to be re-looked at. And since Nobody's Perfect, neither am I.... If you really feel that there is more to add, please keep those comments coming in...

Very unfortunately, the fake software tester does not realize the importance of testing reports, which can, indirectly lead to a very wrong business decision, which can potentially bring down the entire company or cause huge losses to the company... It is my wish that there come a day every tester understands the business context of the reports that he tests and tests it using the business perspective...

And until this day arrives, the ranting of this fake software tester would definitely continue.....!!!

Sunday, February 28, 2010

Standard Disclaimer - And an Apology...

Ever since that I started this blog a couple months back, I have been asked by a lot of peers and ex co-workers if I'd categorize them as members of the group of "Fake Software Testers".

My answer to all --- Yes and No.

To clarify, this blog does not have any direct reference to my current and past co-workers, nor do I even try to implicate that they are fake testers. To be honest, I have had the privilege of working with a lot of the brightest minds in the past couple of decades. And though some of my current colleagues do not realize it, their approach to the field of software testing makes me pale in comparision!!!

But, having said that, I have been seeing a lot of fake testing in the past few years happen, which is what I am trying to set right with my blog. It saddens the heart to see a lot of these fake testers, being hailed as very good testers. I don't have a problem with them being rewarded... but, when they are given bigger responsibilities, we are setting them up for failure.... I am not too worried about the projects failing, but.... if these testers are pushed into testing software which saves lives, or flight software, then that is very dangerous.... Their failure to detect a defect might kill a few lives!!! --- that's what I am worried about.

Even I --- for a considerable amount of time, have been (and sometimes, continue being) a fake software tester ----- and I realized I'd remain so until someone pointed it out!!!

That is why, I am trying to create a blog which would also talk about some of the fake/wrong practices of testing, which is so much prevelant these days. I am also trying to share a few of my experiences with this.....Not all of it is right, and not all of it is wrong. It's only when I try to don the hate of a fake tester that I am able to see what is right and what's wrong... from purely my perspective!!!

And 1 more thing... I have not declared my identity on this blog. That's for only 1 reason ----- it gives me immense satisfaction, to being referred to as "The Fake Software Tester"!!!!!

Saturday, February 27, 2010

English - "A thinking language"...!!!

Was just reading Pradeep's blog on "Coaching testers on Bug Reports, Advocacy and Credibility" @ http://testertested.blogspot.com/, when the following thought took shape in my head.

In all our interviews, we ask for a list of known languages of the candidates. But, we never ask for the "Primary Thinking Language" of the applicant, neither do we make an effort to find out what that is.

Going through some of the awful bug reports that Pradeep has reported, and having sampled much more of the same in my life, I am of the opinion that we need to look at ways to identify the tester's "Primary Thinking Language", and all our training programs on communication skills should also look at how English becomes our primary thinking language - since English is the universally accepted language for defect reporting.

We ask them for languages that they speak, they write and they can read. But have we ever, for a moment, thought of the importance of identifying the "Primary Thinking Language" at all? Having posed this question, what do I mean when I say "Primary Thinking Language"???

The way I'd define it is... Your Primary Thinking Language is the language of the word that comes immediately to mind when you think/look at an object or think of an emotion. We did the following experiment with a group of people who were predominantly tamil speaking and another group who are management grads from IIM.

We showed them a door, An Angry Amitabh Bachan scene from a movie, a key to the door, a picture of a child sleeping and the photo of a tomato.... and asked them to answer whatever comes to mind in the 1st second.... The answers that they gave...

Majoring of answers from the tamil speaking group --- Kadavu/maram, padam/cinema/kovam, saavi/pootu, kuzhandhai/azhagu/thookam, thakkali... (Words that first came to their mind were in their mother tongue... Conclusion ---> the primary thinking language --- Tamil)

The English speaking Management Graduates rattled off the following words --- Door, anger/flick/movie, key, child/sleeping/bliss, veggie/sandwich/rotten tomato..... That helped us conclude what's their "Primary Thinking Language" was --- English, Obviously.

You can do this experiment with someone you know to identify your primary thinking language and you would know what I am trying to say here...

Training Deparments --- Please take note.... Please help employees identify their "Primary Thinking Language" and work with them long-term to make English as the "Primary Thinking Language", so that it would be beneficial for both the employee and the company in the long run....

And I don't think that this is a problem specific to India... I am sure this problem exists in many countries where English is not the primary speaking language.....

And yes.. this article has nothing to do with fake testing... Just some thoughts that came to mind on reading Pradeep's blog... The fake tester's off on a long weekend... See you in March!!!

Thursday, February 18, 2010

The 1st 10 mins...

What is the most important time of the day? For me, it's always the first 10 mins of today and the last 10 mins of the previous working day. Most often than not, these 10 mins have always decided on the future course of actions for the entire day. They have determined if the entire day is going to have a smooth flow, or if there are going to be rough patches.

As soon as you enter work, your investment on the first 10 mins would reap the results for the rest of the day. Please put all stops in place and finalize your day's plan during the 1st 10 mins. Please tell your boss that you can't be disturbed during this time. Please inform your sub-ordinates that you cannot be disturbed during this time. Having put all these stops in place, what do you do in the 1st 10 mins?

Well, I don't know what you do (how can I :), but here's what I do during these 10 mins.

1) A lot changes during the previous day....
Most probably, most of our project team members live across the world in a different timezone. Please check your emails and make a quick check of items that you need to accomplish during the day.

2) Plan for the top 3 things that you want to do for the day.
On Paper, draw a simple list of top 3 things that you need to complete for the day. It can be completing a review, completing some documentation, doing some testing, doing some test data preparation, or whatever... But, please have a good clarity on the top 3 tasks that you plan to achieve for that particular day at work.

3) Planning for interruptions...
All of us get interrupts.... how do we handle interrupts? We don't have to look far... we only need to take a good look into the mind of a very effective operating system to understand how the system handles interrupts and if you study it, you would know that the system handles interrupts in the best possible manner. You need to mimic the same... it's too simple, really!!! The moment you get an interrupt, all you need to do is to see if you have to place that task among the top 3 activities for the day. If not, it can wait!!!

4) Plan for what to discuss during the meetings on this day...
Make a quick list of meetings that you need to participate... And plan for the top 2 things that you want to talk about in those meetings. Nobody likes someone who keeps talking and talking... By talking less, you would be able to create the focus on the most wanted items. Make a list of the things that you want to talk in meetings for the day.

5) If you feel the need to decline or postpone meetings, do it at this time.
You cannot be in all the places at all the time. If you feel that you need not attend meetings, please decline them 1st thing in the day. There is no harm in declining meetings. You have to do prioritizations effectively.

6) Finalize the effective working time during the day. The time when you will not be disturbed by anyone...
This is something that I do religiously... For at least 4 hours during the day, please ensure that there is no disturbance. Do not take coffee breaks.. do not answer phones... do not take gossip breaks... email breaks... browsing breaks.... break from breaks, etc.etc.etc... For 2 hours in the AM and 2 hours in the PM, please focus only on deliverables. I call this EFFECTIVE WORK HOURS. Makes a world of difference to your deliverables...

7) Playing hooky from work is always important.... -
Finalize the time of the day when you play hooky and read fun-emails and internet browsing, Office Gossips, Purchasing stocks on the stock market, etc. None of us refrain from this and in a way, this is always important.

And very important... though it is very important for you to be at the beck and call of your boss, please don't do that in the first 10 mins of your day...

Please keep your boss, your boss's boss and your boss's boss's boss far far away during this time :) !!!

The fake tester always reaches for a cup of coffee and checks all fun forwarded emails before starting off. His entire day gets screwed up, and he cannot prioritize. He spends most of his working time in something which is not important at all. For example, If you have 90% notion that a feature will not be delivered, why would you need to spend time on testing it? Please remember... investing in the 1st 10 mins of the day would be very effective for the entire day, and you would reap results for the rest of your long career!!!

What the fake tester does not realize is that...... Dedicated planning during the first 10 mins of the day, would ensure that you own the entire day!!!!!

Tuesday, February 9, 2010

The Forever-Incomplete Test Plan.....

Ha Ha... Is this the billionth blog on a test plan?(Can a billionth blog exist.... food for thought :))!! Ever wondered what most of these nerds above the managerial hierarchy do on an almost every day basis? If the tester slogs the whole day, what does the test manager contribute to the project at all? Let's face facts!!! All Managers are over-heads!!! But, are they??? In a way, Ain't them all infamous for planning :) ?

Well... like it or not, most of their daylight hours... and moonlight hours... goes into planning... and for the test manager, the test plan should be a holy-book of sorts for the entire project, or program!!!

What does the test plan mean to the stakeholders, project management group and testers? Why do you need a test plan and what does it try to achieve?

Ask the fake tester, who, without a blink, would reply stating that the test plan is created for the sake of the Quality team who does regular audits. I don't blame him. That's his true belief!!!

And what would the true practitioner say? He would not give an answer in the first place!!! Test Plans mean different things to different people and the true manager would understand it. For the Stakeholder, it would mean about knowing release cycles, turn-around time.... for the project management group, it would list out the hardware/software requirements, etc.... for the tester, it'd say who'd be testing what module, etc.... for the customer, it'd talk about acceptance criteria, etc. etc. etc.

My top 4 myths about the test plan... - And what a fake tester would call - "4 Top requirements for a test plan :)"

1) It is always an MPP/word-document
I have seen this so many times in my life. A test plan is always seen to be a MPP or a test document. Why? A plan written for a 100 year project written very simply on a single sheet of paper, would also constitute a test plan. Why can't a test plan take the form of a simple PPT addressing whatever it needs to address? What a fake tester does not realize is this - Planning is all in the mind!!!

2) After customer sign-off, please do not update the document at all.
A very common occurance. The customer is always very eager to see the test plan... and the project management usually blackmails him to sign-off the test plan. Now, the test manager, having seen the customer sign-off, checks in the test plan document into the VSS/project repository archives. He tries to dig it out only during the audits... Completely WRONG!!! Please understand that the test plan is a living document and needs to be constantly updated... Else, it
will not server the purpose in the first place. Please update whatever needs to be updated into the test plan on a very regular basis.

3) Look out for the company template and fill in your test plan as per that template...
Whenever someone asks the test manager for a test plan, he does a deep-dive into the company document archives, digs out any document that has a test plan in the name, removes content from all sections, fills content in all the sections and submits it to you for review. It's not uncommon for you to see a L&P sections in a test plan for a 3-4 page website, which would eventually get decommissioned in the next month. The true practitioner, would use his common
sense to include only relevant information in the plan, which would be very short, but.. very important... serve the purpose!!!

4) It has to be a 100 pager document :)
And the mother of all myths... A test plan has to be a 100 pager. So many times, you review test plans to find it filled with useless information... Information soon going to be extinct, or information that you already know... If it's known information, why do we need to put it into a test plan? Sample this... A windows application, having just 3 screens... used to check a job interface.... ( a job that runs once a quarter, yea.. the same job that's gonna be decommissioned in the next 2 quarters)... has a 243 page test plan. A 243 test plan for a 3 screen application? Sometimes, in our eagerness in pursuit of perfection, we tend to overdo things.. and this is an example... Visualize this!!! The project would have an estimate of 20 days, but the estimation to
create the test plan would be 40 days!!!

Having said all that, from my personal experience, what I feel are some of the top 6 things to keep in mind while creating the test plan:-

1) Why do you need the test plan? Duration of the project? Internal/External?
Quick questions... Why do you need the test plan at all? Is it to satisfy the client or to satisfy project management requirements? How long is the project going to go on? Is this an internal or an external customer facing application? Please answer these questions before starting off on the test plan. Answering these questions would provide you the mostly correct direction that you need to take off in... Mostly, internal applications planned for 1 months seldom require a test plan.

2) Highlight all risks , issues , Assumptions and Constraints-
Having decided to create a test plan for your project, please list all risks, issues, assumptions and constraints in separate sections with appropriate contingency and mitigation plans. What is very important over here? That you, as the writer of the test plan, understand the differences between a risk, issue, assumption and constraint. You would also need to be able to recommend suitable contingency and mitigation plan. These are project management terminologies. The fake tester would easily be able to call out the risks, but it's only the TTP (True Testing Practitioner) who would be able to give a lot of thought to this section and sometimes, ask for help from other people to fill out this section. Sometime in the future, I'll write a separate topic on this.
Even when reviewing a test plan, it's very easy to look at this section and this would clearly differentiate between the true and fake testers!!!

3) Automation plans...
Automation test plan. This is something for which, as part of test plan, you would also need to know the development plans. Not only dates, but how development is planned and delivered to test so that the atuomation plans can cover this. Also use this section to explain to the readers, if automation would be an overkill. Also talk about criteria to choose the best automation tool, so that it provides cost requirements also to the project amnagement teams. It also gives them sufficient time to procure licenses, or re-use existing unused licenses.

4) Plans for testing Non-functional requirements...
Sadly speaking, half of the world start thinking of the non-functional requirements when they start to fill in this section in the test plan. The other half? They dont even think about it while filling this section. It is very difficult to find testers in this world, who understand the concept of testing for non-functional requirements, and who are able to foresee the potential business impact if these are not met. Trust me, plans for NFRs cannot be created overnight. A lot of thought needs to go into filling, or rather, thinking about this and how you would proceed to testing these items.

5) Plans for test data, masked like-live data...
Never try reinventing the wheel. why? For the simple reason that it's already been invented!!! ... Likewise, if you have an input stream coming into the live servers, try to have the same feed coming into the test systems so that you can test with like-live data. Do not compromise on test data, since they would be the driving parameters to predict behavior in production after go-live!!! If data received in the test systems is masked, please work with your bosses to
ensure availability of unmasked data. Always remember - The more you have to unmasked like-live data, the more defects you'd unmask!!!

6) And most definitely, to state what is out of scope...
My personal favorite section, that I have saved for the last!!! As much as what you are going to be testing is important, equally important is the areas that you are not planning on testing. Business, Stakeholders need to understand this scenario before approving your test plan, so that they would know what is not tested.

Having listed the above, here's what you need to keep in mind while you write the test plan- if you are still awake, or haven't browsed to another site by now ----- A Test Plan, however hard you try, is never ever a single day effort. All those, who disagree with me, is either a "Pure" Genius, or "just Another Fake Tester"!!!!!

Adios.....

Monday, January 25, 2010

Should I report to work on the Indian Republic Day? or any Indian National Holidays?

Furious thought... resulting furious rantings and ramblings... I would never want to write about a non-testing topic, but try as I might, I am unable to put these thoughts out of my mind.

As I write these, a nation of a billion is celebrating their republic day. All schools and institutions have been declared a holiday. I am reading news articles about how people are rebelling against organizations that have asked them to come in to work on a republic day to meet client commitments and project deadlines. "Should I go to work on an Indian Republic day? Is it moral for the boss to call me to work on this day??? -" At the outset, it seems like a million dollar question??? Is it one??? Well, I would think definintely not!!! mostly because, it's not worth even a few pennies, let alone a million dollar...

To answer the question, "Should I work on an Indian Republic day? Definitely not!!! For god's sake, it's a NATIONAL HOLIDAY" seems to be every body's answer".. Well, the let's spend some time pondering over this thought...

Agreed that police, military, medicine and transporation are essential service? A quick checklist of questions for the same person who does not want to go into work on a Republic day.

1) Do you ask your maid to come into work on a republic day?
2) Do you get pissed off when you go to shopping on a republic day and find there are not enough shop assistants to assist? Do you complain to the shop manager about the lack of resources on a holiday?
3) Do you get angry when you find that your apartment security person has gone off to attend
celebrate republic day and is not into work on that day?
4) Do you get angry if you are watching a movie on cable on republic day and the movie conks
off? Would you want the cable guy to fix the problem immedieately or are you absolutely all
right if he fixes it after a day?
5) If there's a plumbing problem at home on the republic day, would you want to call the plumber immedieately to fix it or you want him to fix it later?
6) Do you plan to get the odd carpentry jobs around home to be completed on a republic day?
7) If you lose your ATM card, or if you have a problem with your mobile bills, do you expect customer service to be available on a republic day??? Or are you all right if they shut shop for the day and resume services the next day?

Sad but true -----> Sometimes, we as clients, or when we represent a form of the clients and show the inclination to have unjust demands. Similar to how we expect our clients to understand human sentiments, we would also need to understand human sentiments and have our expectations accordingly.

We show a lot of hypocratic tendancies wherein our expectations of people that we pay to serve us is completely different on what we want the people that we serve to expect from us. If the answer for most of those questions in the checklist is YES, then it means that we belong to the category of clients with unjust expectations. Thank god that you are not a client :)!!!

Fake Testers Gyan of the day ---> Please keep human sentiments in mind on National holidays!!!