Testing is Contextual — No Best Practices, Please

Jayateerth Katti
8 min readOct 12, 2024

--

Are you following testing best practices?

Be cautious.

Best practices aren’t always the “best”.They are not the golden rules you might think they are.

They’re highly dependent on context.

What works wonderfully in a situation might backfire in another.

I will explain this concept unpacking why context is important in testingPut.And how you can approach to face the challenges.

Why There’s No Universal “Best Practice”

“Best practice” implies that it is a silver bullet.

It also suggests that there’s a universally optimal way to approach testing. And that can be applied to any situation.

But it is not !

Software development and testing are complex activities. They are influenced by various factors. Those factors can be project requirements, team dynamics, stakeholder expectations and technology stacks.

The same practice can yield different results depending on these factors.

Consider the example of agile methodologies.

Agile practices, such as continuous integration and continuous testing, are widely regarded as best practices.

Yet, these practices may not always be suitable for you.

In a small startup environment with limited resources, implementing a full-fledged CI/CD pipeline might not be feasible due to cost or time constraints.

This can be different in large enterprises.

They might have multiple teams working on the same product, CI/CD is almost a necessity to ensure smooth collaboration and rapid delivery.

An Example from My Experience

Here’s an experience from my career that illustrates it.

I was working on a project.

In this project, my team was responsible for testing a specific story. The intended functionality of the story was working as expected.

So I closed it.

Bugs I found during testing were reported and linked to the story, which seemed like a logical and effective approach.

This process worked well in that project.

And I concluded it as a “best practice.”

Obviously. Right?

Now I started working on a new project.

In that different project, I applied the same approach and quickly ran into trouble. The project manager escalated the issue.

Reason was alarming to me.

He argued that the story should not be closed if there were any bugs related to it.Even if the primary functionality was working fine.

He was correct.

The reason was that closing a story could signal to developers that the feature was complete.And is ready for production.

This could have potentially led to bugs being overlooked or deprioritized.

This experience taught me a lesson.

What I considered a best practice in one context was not applicable in another.

The project’s specific needs, stakeholder expectations and the way success was measured all contributed to my failed approach.

Here we go…

The Importance of Understanding Context in Testing

Each project can be different.
Your testing and process vary based on following factors:

  1. Project requirements
  2. Stakeholder expectations
  3. Technology used
  4. Team dynamics
  5. Risk tolerance

Let me explain each of them for you.

1. Project Requirements Vary

Every project is unique.

It has unique requirements, objectives and constraints. A practice that is efficient in one scenario may not be feasible in another.

Take this for example:In a project with a tight deadline, time of completion might be prioritized over thoroughness.

This necessitates a different testing strategy compared to a project where quality is the top priority.And time is not a constraint.

In one of my projects, we were working on a critical financial application.

The stakeholders had zero tolerance for bugs.

We had a process set. To develop test cases, review them. These were done before the product was ready for testing.
When the product was ready for testing, we executed them.

Any bugs found were reported.

In the next subsequent builds,we tested bug fixes. And on the ‘release candidate build’ we ran those test cases as part of regression testing.

Now, another project.
I was working on a social media app. Much lower risk product. The client was more concerned with getting the product to market quickly.

So I adopted a different approach.

I instructed my testing team to adopt exploratory testing. No prior test case development. We explored the product and wrote high level tests.

Regression was again exploratory combining the tests identified in the exploration earlier.

2. Different Stakeholder Expectations

Your project/product will have different stakeholders.

They can be project managers, product managers, developers and clients.

They have varying expectations.

Some may prioritize quality delivery over speed, while others might focus on meeting tight deadlines.

You need to align your testing approach based on your priority for stakeholders.

Consider this example.

I was involved in a project where the product manager was highly detail-oriented.
She insisted that every bug, no matter how minor, be addressed before the product was shipped.

This approach of the PM demanded us to have a different approach for testing.

We had to dedicate significant time and resources to testing. This led to delayed release.But ensured a high-quality product.

Now see this example:

In another project, I worked with a different client.

That client was more interested in rapid deployment. The client was willing to accept a few minor bugs as long as the product was delivered quickly.

We adjusted our testing strategy accordingly.

That was risk based testing. We focused on high-priority areas. I, along with the PM, made a conscious decision to release the product with known bugs !

3. Technology Stack and Tools

Now the technology aspect.

The technology stack, tools and environment also impact what practices are suitable.

A practice that works well with one set of tools might not integrate smoothly with another.

I remember another project.

We were using a particular test automation tool that was excellent for functional testing of an application.

But it was not suitable for another product testing.
It was not able to uniquely identify all the objects in the product.
Initially, we tried to use the same tool for both the applications.

Because we were considering it a best practice to standardize on a single toolset.

However, the results were not so good.

We had to switch to a different tool that was better suited for automating other product’s testing.

This experience reinforced the lesson that the right tools must be chosen based on the specific needs of the project.

4. Team Dynamics

Now comes the huma part.

Your team may have multiple members.

The skill set, experience level and dynamics of your team play a significant role.
A junior tester might need a different approach compared to a seasoned expert.

So, you need to customize your practices to the strengths and weaknesses of your team.

I will give an example.

In one of my projects, I was leading a team of junior testers,

I realized that the traditional approach of having testers write their own test scripts was not giving the desired results.

Those testers were struggling.

This led to errors, multiple reviews.

To address this, I introduced a function library with reusable components. This approach helped the testers to focus on testing rather than scripting.

This adjustment improved the quality and speed of testing.
More than that, it eased the life of testers. And mine too !

5. Risk Tolerance

Risk management is crucial.
Some projects can tolerate more risks than others. In high-stakes environments, rigorous testing with no shortcuts is crucial.

But, a low-risk project might allow for faster, less thorough testing practices.

I worked on a healthcare application.

The risk tolerance was extremely low due to the sensitive nature of the data and the potential impact of any bugs.

We implemented a very detailed testing process. That included multiple levels of review and approval before any changes were made to the codebase.

On the other hand, in a low-risk internal tool project, we adopted a more flexible approach.

We had faster releases and less comprehensive testing.

The stakeholders understood and accepted the risks associated with this approach.

The reason was the tool was not customer-facing and could be updated quickly if issues were discovered.

By these examples, now you might have got a feel of how testing is contextual. You can follow the process everywhere.

Practical Tips for You

Based on my experience, here are some tips for you.
These would help you to adjust yourself with dynamics of testing.

  1. Assess the context first
  2. Be flexible
  3. Communicate with your team
  4. Learn from mistakes
  5. Seek feedback
  6. Document your process

Let me explain each of them a bit.

1. Assess the Context First

Context is king.
Before applying any testing practice, take a step back. Assess the context.

Understand:

  • What are the project’s goals?
  • What’s the technology stack?
  • Who are the stakeholders?

For instance, if you’re working on a project with a tight deadline, you might prioritize test automation to speed up the testing process.

However, in a project with a longer timeline and higher quality expectations, you might focus more on testing to ensure thorough coverage.

2. Be Flexible

Stay adaptable.
Be ready to tweak your approach based on the context.

Flexibility is a crucial skill in testing.

In one project, I had to switch from a manual testing approach to an automated one.
That was midway through the project due to changing requirements.

This flexibility allowed us to meet the project’s goals.

3. Communicate with Your Team

Communication is crucial.

Ensure that everyone in your team is on the same page. Discuss and agree on the practices that suit the current project.

Remember, what worked in a previous project might need adjustments this time.

In a project where I was leading a globally distributed team, clear communication was essential. That is to ensure that everyone understood the testing approach and their role in it.

We held regular meetings to discuss progress, challenges and any necessary adjustments to our testing strategy.

4. Learn From Mistakes

Learn, learn, learn.
Every project offers new lessons. Use these experiences to refine your understanding of what practices work best in different contexts.

Keep a ‘Lessons Learned Document’.For every project.

For example, after encountering issues with a testing tool in one project, I made it a point to thoroughly evaluate tools before selecting them for future projects.

This continuous learning process has helped me avoid similar mistakes again.

5. Seek Feedback

Get others’ opinions.
Regularly ask for feedback from your team and stakeholders.
They can provide valuable insights. Those insights help you adapt your practices to the project’s needs.

In one project, I received feedback from developers that our testing approach was too time-consuming. That was slowing down the overall development process.

We adjusted our approach by implementing exploratory testing by writing high level tests.
This process improved efficiency without slippage.

6. Document Your Process

Make notes.
Keep a record of what works and what doesn’t in each project.

Over time, this will build a knowledge base. This knowledge base can guide you in future projects.

I started maintaining a testing playbook. That document included approaches we used, the challenges we faced and how we overcame them.

This playbook has been an invaluable resource in subsequent projects. That helped me and my team avoid mistakes we had made and build on our successes.

Conclusion

Testing isn’t a one-size-fits-all activity.
Avoid rigidly adhering to so-called best practices. Instead, focus on understanding the unique context of each project. By doing so, you will avoid potential pitfalls.Also you position yourself as a thoughtful and adaptable tester who adds real value to any team.

Accept the fact that testing is contextual.You will be better equipped to face the challenges of each unique project.

If you need guidance on your growth? Set up call with me

Follow me on Linkedin

--

--

Jayateerth Katti

20 Years of experience in testing. I write about testing and growth.