In this chapter we will study about smoke testing and sanity testing. Both the tests have their own objectives and priorities. These two types of testing play key role in success of a project.Most of the times we get confused between the meaning of Sanity Testing and Smoke Testing. First of all, these two testings are way “different” and are performed during different stages of a testing cycle.
What is Smoke Testing in Software Testing?
Smoke Testing is not thorough testing but it is a group of tests that are executed to verify if the basic functionalities of that particular build are working fine as expected or not. This is and should always be the first test to be done on any ‘new’ build.
It helps not to waste the testing time to simply testing the whole application when the key features don’t work or the key bugs have not been fixed yet.Our focus will be on primary and core application work flow.
Who Should Perform the Smoke Test?
Not the whole team is involved in this type of testing to avoid the wastage of time of all the QA’s.Smoke Testing is ideally performed by the QA lead who decides based on the result as for whether to pass the build to the team for further testing or reject it. Or in the absence of the lead, the QA’s themselves can also perform this testing.
At times, when the project is a large scale one, a group of QA can also perform this testing to check for any showstoppers. But this is not so in the case of SCRUM because SCRUM is a flat structure with no Leads or Managers and each tester has their own responsibilities towards their stories.Hence individual QA’s perform this testing for the narrative that they own.
How to Conduct Smoke Testing?To conduct smoke testing, we don’t write test cases. We just select the necessary test cases from already written test cases.
Do we really write test cases for all testing types? Here in this chapter, we have given clear idea on choosing testing types to write test cases.
As mentioned earlier, here in Smoke Testing, our main focus will be on core application work flow. So we pick the test cases from test suite which cover major functionality of the application. In general, we will pick minimal number of test cases that won’t take more than half an hour to execute.
This testing can be treated as the entry point for complete Functional Testing of functionality or system (as a whole). But before that, the QA team should be very clear about what tests are to be done as smoke tests. This testing can minimize the efforts, save time and improve the quality of the system. It holds a very important place in sprints as the time in sprints is less.
This testing can be done both manually and also with the help of automation tools. But the best and preferred way is to use automation tools to save time.
Real Time Example: Assume, you are working for an eCommerce site. When a new build is released for testing, as a Software QA you have to make sure whether the core functionalities are working or not. So you try to access the eCommerce site and add an item in to your cart to place an order. That’s a major flow in most of the eCommerce sites. If this flow works, you can say this build is passed. You can move on to functional testing on the same build.
What is Sanity Testing in Software Testing?
Sanity Testing is done when as a QA we do not have sufficient time to run all the test cases, be it Functional Testing, UI, OS or Browser Testing.
“Sanity Testing as a test execution which is done to touch each implementation and its impact but not thoroughly or in-depth, it may include functional, UI, version, etc. testing depending on the implementation and its impact.”Sanity Testing is done during the release phase to check for the main functionalities of the application without going deeper. It is also called as a subset of Regression testing. It is done at the “release level”. Sanity test should be done only when you are running short of time, never use this for your regular releases. Most of the times, we don’t get enough time to complete the whole testing. Especially in Agile Methodology, we will get pressure from the Product owners to complete testing in few hours or end of the day. In this scenarios we choose Sanity Testing.
How to Conduct Sanity Testing?
Same like Smoke testing, we don’t write separate test cases for Sanity testing. We just pick the necessary test cases from already written test cases.
As we mentioned early, it is a subset of regression testing. When it comes to Sanity testing, the main focus is to make sure whether the planned functionality is working as expected.
Real Time Example: Let’s take the same example as above. Assume you are working on an eCommerce site. A new feature is released which is related to Search functionality. Here your main focus should be on the Search functionality. Once you make sure that the Search functionality is working fine then you move on to other major functionality such as payment flow.
Smoke Testing Vs Sanity Testing
Example to show the difference between Smoke and Sanity Testing:
For example: In a project for the first release, Development team releases the build for testing and the test team tests the build. Testing the build for first time is to accept or reject the build. we call it as Smoke Testing. If the test team accepts the build then that particular build goes for further testing. Imagine the build has 3 modules namely Login, Admin, Employee. The test team tests the main functionalities of the application without going deeper. we call it as Sanity Testing.
Some more differences between Smoke Testing and Sanity Testing:
| SMOKE TESTING | SANITY TESTING |
| Smoke Test is done to make sure if the build we received from the development team is testable or not | Sanity Test is done during the release phase to check for the main functionalities of the application without going deeper |
| Smoke Testing is performed by both Developers and Testers | Sanity Testing is performed by Testers alone |
| Smoke Testing exercises the entire application from end to end | Sanity Testing exercises only the particular component of the entire application |
| Smoke Testing, build may be either stable or unstable | Sanity Testing, build is relatively stable |
Do we automate smoke tests?
Yes, we can automate Smoke test cases. It saves a lot of testing time. Assume you have 50-100 smoke test cases. To execute these 50-100 test cases, it may take time around 4-6 hours approximately. If you have automation scripts for these test cases, you can execute them once the build is released and confirm whether the build is passed or not in less than the time you spend on manually executing smoke test cases. So most of the teams automate smoke test cases.