I Don’t Report Bug
Dear Tester,
You read it right.
I don’t report the bug. As soon as it is found. I do other activates, which help me to write good bug. Rather make the bug reporting great.
All this process I have learned in my 20 years experience.
I have learned by mistakes. And learnt from other mistakes too.
I do some analysis before reporting bug.
They are:
- Make sure it is bug
- Not reported already
- Collect and document details.
By this, I report only valid bugs and which are not already reported.
The aim is to have the quick fix by developers and deliver quality product.
What all I do when I find bug ?
In Summary, before I report the bug, I do :
- Review Requirements
- Confirm if it is bug
- Identify the root cause
- Check if already reported
- Get all information
- Narrow down scenario
- Capture evidence
- Discuss with team
- Test on other environments
- Check consistency of bug
Note that, these are NOT the sequential steps.
Let’s dive in.
Review requirements
First things first.
Whenever I see an issue, I first go to requirements. Requirements can be documents or a stories in tools like Jira.
When I feel something is wrong in the product, I make sure that it is bug.
And how do I do that?
By referring to these documents or story. Mainly I verify the acceptance criteria of these. The issues can be regulatory violation also, which are not documented in story.
So, I refer the clauses in the regulation documents as well.
Some times, no… most of the times, user expectations are not documented.
Specifically usability. The bug related to these are difficult to verify in requirements.
Confirm if it is bug
Bug.
The moment we find bug, we get excited. And start to report it in bug management tool. Before that, I check if it is really bug. I check if it is any setting/configuration issue in the product.
Also, I will check if it is test data issue.
I also keep visiting requirements/stories.
There can be some footnotes or exceptional flows.
There can be some issues related to network connectivity also.
If I report bugs for these issues, they they will be rejected. Most likely !
Also read : Why Software Has Bugs ?
Try to identify the root cause
Bug need not be straightforward.
It can be due to many reasons. A defect in one feature can have impact on other features. Or sub features too. So it is crucial to identify the root cause of the bug. For example: Order value in ecommerce product.
I see wrong calculation of order value in checkout page. Then I don’t report it as Check out page issue.
On analysis, it can reveal that actual issue with taxing feature. There the tax calculation logic would be wrong.
If root cause is not identified, there can be multiple issues.
Or there are chances that bugs escape to production.
Check if it is already reported
If I conclude that issue is actually bug, then I would report that.
But wait, I don’t report directly. I would get in to bug management tool. Search if bug is reported. I also search if similar bug is already reported.
I don’t stop here. If the bug or similar bug is already reported, I check it’s status. Bug can have any status.
It can be Open, or closed. If open then I do not log it. If it is closed, then I check why it is closed.
Closure can be because of fix, duplicate, rejection, deferred etc.
If bug is closed as fixed. Then it is a problem. Really a problem. There can be multiple reasons for this. Example: wrong code check-in, regression issue etc.
Checking the existence of bug reduces chances of rejection, duplication etc.
Get all related information about bug
How I arrive at the bug.
This the stage where I try to get details related to bug. The steps to reproduce, actual result, expected result.
Also, operating system, browser where bug is found.
This analysis is very important from fixing point of view.
From this analysis, I can conclude that the bug is found in Firefox browser. The functionality might be working as expected in Chrome and Safari etc. Or the issue can be observed only in Windows and not in Mac operating systems.
I also get the logs or error messages.
I provide all these details in bug when I report.
This helps the developers to fix easily. I would save their time in this analysis.
Otherwise they may fix partly and may miss to fix other part.
Also read : Reasons Behind Common Software Testing Failure
Narrow down the scenario
Scenario for bug.
Many scenarios can lead to bug. Or bug is triggered only when certain steps are performed. I try to narrow down.
I try to identify circumstances that trigger the bug.
I analyse if bug is related to specific user actions, data inputs, or certain conditions?
For example, the bug may be triggered for only certain user types.
It can not be triggered for other user types like ‘Subscriber’ user types, but only if the user type is ‘Anonymous’.
This analysis and information can provide valuable information to developers.
Capture evidence
Bug needs to have proof.
So, I would capture the screenshot. Even better, wherever possible I record the steps to reproduce in video.
I prefer to capture build number, date of testing too.
Also, I collect the logs.
Once this information is collected, it helps me to understand if this is genuine bug.
Also, once reported, developers can quickly fix.
Discuss with team member
Some times I might not be confident about bug.
The requirements might be unclear. The acceptance criteria mentioned in the story might some times be not sufficient.
Bug can be usability issue also.
In these and other cases, I might not conclude if it is bug.
So, I prefer to discuss with my team. Everybody has his/her own thought process. Also, I get the views from fresh eyes.
So, I ask team members to have a look at this possible bug.
I provide them steps to reproduce. Screenshot etc. This helps them to see what is going on. My steps might be wrong or invalid. I might have missed some requirements. It might be working as expected.
I found it useful when I discuss with team members before I report.
Verify on other environment
As I discuss with team, I also prefer to test on other environments.
Example, I test on Windows, Mac (different versions of OS), different browsers like Chrome, Edge, Firefox, Safari etc. If I have access to previous release versions of the product, I test on previous version also.
This helps me to conclude if the bug is pre-existing or new to this release.
Some bugs can be browser specific or OS specific.
This analysis helps me documenting a clear bug report.
Consistency of bug
Ideally, bug should be consistently reproducible.
Every time I follow the steps to reproduce, bug should be triggered. Then it will be easy to do further analysis — if its actually a bug or some settings issue.
I have some cases, where the bugs are inconsistent.
Believe me, these are nightmares. Both for testers and developers.
If it is consentient, then it will be easy debug and fix.
If bug is randomly triggered, then it would be challenging to reproduce, fix and test.
Conclusion
I have tried to put together the process or steps I follow when I find a bug. Rather a possible bug.
These steps, I learnt over period of time. Learnt from mistakes- my own mistakes and others mistakes.
By properly analysing the bugs and then reporting it in the tool helps in quick fixes and delivering quality product to customers.
You can use these steps and create a clear, valid and complete bug.
That’s all for now !
P.S : Please share this, so that your friends can learn. And grow !
P.S.S : Subscribe to by newsletter to learn from my experience.
