How Do I Do Regression Testing

You know about regression testing.

Right?

You should be.

If not, here is a quick shine of light on this.

In my two decades of experience, I’ve come to realize that regression testing is not merely a routine task. It is a critical process in delivering high-quality software.

And delighting users !

It is testing to make sure that any newly added code or fixing a bug is not impacting existing code.

Regression testing is to make sure that no new bug is introduced in the existing functionality.

In other words:

Regression testing is the process of testing software product that has recently undergone changes. This testing is to ensure that the changes have not introduced new defects or broken existing functionality.

So I test the application parts or features, which are impacted by new code.

Based on the size of the application to be tested, it can be time consuming effort. I test them again, though it was tested (more than) once in the earlier test cycles or releases. So can be boring.

But it is crucial activity and I don’t take risk of skipping regression testing.

But, Why Regression Testing Is Important ?

Regression testing can be a complex and time-consuming process.

You saw why is to so just now.

But it is essential for ensuring the quality of software application.

According to my experience, here is why regression testing is important.

• To ensure that the software is still working as expected after code changes.
• To uncover defects that may have been introduced due to changes.
• To prevent fatal regression issues-where a change to code causes a previously working feature to break.
• To improve the overall quality of the software application.

Let’s dive in to my way of performing regression testing.

Steps of My Regression Testing

My testing is context driven.

That means, testing is based on the context and need of the release.

However, I follow these steps generally.

1. Impact Analysis & Scoping
2. Test environment setup
3. Test data preparation
4. Test execution
5. Defect reporting
6. Defect re-testing
7. Reporting results
8. Sign-off

Though each step is equally important, I will explain more on the Impact analysis here.

I will try to explain other steps in future. In detail.

My Method of Regression Testing

You know it by now.

Regression testing is testing of already developed and tested features. And how to test these ? Popular method is to execute regression test cases. Easy right?

But should I execute all the test cases?

No. I don’t do that.

I don’t do exhaustive testing. I do risk based testing.

Now you might be thinking what is risk based testing.

Well, risk based testing is testing the functionality or a feature, which we think can pose risk if not tested. It can be any risk associated with software application.

How to analyse the risk?

One of the ways is , I do impact analysis.

In impact analysis, I try to identify which all features or functionality of the product is impacted by the change in code.

It helps determine which test cases should be included in the regression suite based on the impact of changes or updates to the software application.

Basically it helps me to scope the testing for regression.

Here is my approach for impact analysis for test scoping.

1. Understand The Changes:First, I know the changes made in code.I begin by understanding the nature of the changes or updates in the application.This can include changes to code, configuration, features, bug fixes, and enhancements.This understanding helps me to identify the impacted areas. If this crucial step is done properly, then half the battle is won !Remaining process will be hassle free.
2. Identify Affected Areas:In this step, I determine which areas of the software are directly affected by the changes. This could include specific modules, functions, or features.Mind you, understanding the changes is the pre-requisite for this.
3. Prioritize Affected Areas:Next step is prioritization.I prioritize the affected areas based on their criticality. And also importance of the feature to the overall functionality of the software.I focus more on critical areas.
4. Review Existing Test Cases:Then I get in to test management tool.I search existing regression test suite/test cases. Identify the test cases that cover the areas affected by the changes.Of course, I search test cases based on the priority set above.
5. Select Relevant Test Cases:I select the test cases that are directly related to the changes or updates.These test cases might have executed already. May be during last release.I re-execute them in current release.These are the test cases that are most likely to uncover issues resulting from the changes.They uncover regression bugs. Most likely !
6. Consider Indirect Impact:It’s time now to check for indirect impact.I assess whether the changes have any indirect impact on other parts of the application.Even if a change doesn’t directly affect a module, it could introduce new bugs.Example, if a variable is changed in the code, it can have adverse impact on other functionality.So I search and identify test cases for these features.
7. Add New Test Cases:If any test scenario doesn’t have test cases, I do develop new ones.Of course, for newly added feature, test case addition is a must.I make sure that all requirements have test cases. And all impacted test cases are executed as part of regression testing.
8. Evaluate Historical Data:Now some data analysis !I log in to defect tracking tool. Review historical data. I try to identify if there are any patterns. If certain areas have been prone to defects in the past, I prioritize them.If there are more defects reported by customer in a certain area, I pick them first.And if there are no or minimum production bugs in certain functionality, I don’t usually scope for testing. (Context driven !)
9. Balance Coverage:Now, some balancing act.My aim is to balance between thorough test coverage and efficiency.I select enough test cases to provide confidence in the software’s quality without overburdening the testing process.I also do consider time availability for testing.If test cycle time is less (usually when release date is fixed), then I have to descope the low priority testing.
10. Automation Selection:Search for automated scripts.If I have already automated scripts for the impacted feature, I search for them.If I have automated test cases, I prioritize executing them first.But I never ever start automating during regression cycle.Automation is a project in itself.It demands separate resources, investment and time.
11. Documentation:When scope is finalized, I document it.In this document, I mention the rationale behind my test case selection decisions. This documentation helps justify my choices. And also provides transparency to the testing process.This document helps me to communicate testing scope to stake holders-Management, Product team, Dev team, etc.
12. Regular Updates:I revisit the regression suite.I do this regularly.I update my regression test suite continuously as the software application evolves. Same regression suite might not be applicable for all the releases.Patch releases might need subset of this suite.

Read all my newsletters !

To summarize,

Since impact analysis and scoping of regression testing is a major task according to me, I have explained that in more detail.

If you think I should explain more about other steps, do let me know. I am happy to do that on priority !

If you gained something out of it, please do share to your friends. Let them also grow !

Subscribe to my weekly newsletter. You will get learnings from my experience.

Please let me know if you need posts on any specific topics on linkedin

Happy testing.