Everybody heard about automation testing. Without doubt it makes manual testing easier and reduce the time spent on whole testing process. But why is it needed to you and your application? We have simple answers:
- first of all it increases quality of your software;
- it really can reduce cost of manual testing that is repeated on each development life cycle
- test coverage of application will be heightened
- autotests can be run frequently and fast
- most of automation frameworks generate clear statistics which can help to point out the weak sides of application
- when first phase of automatization is completed you can see the report with sanity test run results at any time you need to make sure that application is stable
- higher coverage of cross browserness testing
When automation testing is necessary
On the one hand, if your project would be developed for more than 4 months, it is the first time you should think about automation testing. In first two months enough functionality to start automation process for manual test cases would be developed. In three month manual QA engineer will already have a regression checklist that should be run after every sprint ending and after every serious defect fixes. During this period the additional costs are already needed for manual testing. But in the end of the fourth month regression testing for the implemented functionality would take more than a day. Automation testing can help to speed up the process and reduce costs. Manual QA engineers will have enough time for proper testing of new functionality and features. The impact of implemented features on old functionality can be verified via two mouse clicks to run automation suit.
On the other hand, if software under development is only MVP and would take about two months you should also think about automation testing. That is needed for the future development of extended version of a product for the quick sanity and smoke testing after. As the complexity of the application grows, errors control becomes much harder.
For example, if registration flow must be tested, the QA engineer should run a number of test cases that:
- verify mandatory fields filling;
- check correctness of entered data (phone or email)
- email verification
- validation message displaying
Note that if there are more than one user type in the application all test cases should be run for each of them. So now you can realise how much time testing only of the registration and sign in flow takes. And now can you imagine how much time would core functionality manual testing take, which should be provided with code base growth? Regression is always necessary, as well as time consuming.
Also you should consider human factor. I mean that the most conscientious and accurate regular tester can miss defects during monotonous manual testing.
The world's best QA practice is to test manually developed functionality, properly verify as many test scenarios as tester can and then create automate script for future regressions.
How we do it in SteelKiwi
Our QA engineers can cover main functionality by automated use cases using page object model (POM) design pattern. The main advantage of POM is easiness of maintenance, non-brittle test code and reducing of duplicate test code. In SteelKiwi we chose Agile methodology for development process. That is why we are always ready for your changes. Obviously, changes in code base will be reflected on automated use cases. Basically, not created using POM autotests should be rewritten, that is really time and cost consuming. Obviously, if automatisation QA engineer has used POM maintain of autotest takes much less time.
We use next technologies for automation:
- the first one is using Java + Selenium + Serenity + JUnit framework. This behavior driven development which helps create maintainable and clean code for automated regression and acceptance tests. General advantage of Serenity framework is detailed reports with step by step description of application functionality with screenshots on each user step. You can see a sample of the generated report on the screenshot provided.
- the second one is using Python + PyTest + Allure + Selenium. Allure framework serves to get reports of test runs. This reports are understandable for manual testers and project managers. Also, you can add screenshots, logs or other files to these reports, and even add tests to tickets in the bug tracking systems. Allure allows to separate difficult tests by steps, includes information about tests parameters and environment.
In conclusion please assume that cost of automation testing is higher than manual only at first stages of product development. After a while code base will grow and the situation always changes vise versa. And in that moment automation testing can strongly reduce cost, human resources and time of product release.
