Sometimes, as quality assurance professionals, we often have to decide which would be the best strategy to follow in order to assess the quality of a feature. To do that, we also need to select the correct test technology (framework, tool, language etc).
We need to do this as early as possible in the process, as the quality of the implementation will depend on these decisions. Failing to do that might lead to increased effort and poor outcomes.
In this article we draw from some of the most representative Quality Assurance strategies we’ve followed at Parser, which are also commonly used right now to test and ensure the quality of a backend implementation.
Backend Automation (BE) Strategies
SpecFlow
We applied this strategy with a client that already used Azure Pipelines and Continuous Integration. SpecFlow is a .NET open source framework for Behavior Driven Development, Acceptance Test Driven Development and Specification by Example. Although examples can be written in a native language with the easy-to-understand Gherkin Syntax (Given-When-Then), sometimes SpecFlow requires a dictionary to translate the methods used to implement test examples or test case scenarios. Scenarios are written inside what is called FeatureFiles and methods called dictionaries are implemented in C# for .NET.
Evaluation
The best part of using SpecFlow to test the Backend implementation is that scenarios are written in a readable language, so anyone, even business people, can easily understand what each test is meant to do. In a sense, these scenarios can be considered a way of data documentation. SpecFlow is commonly used for Unit Testing which allows developers to easily cover the first layer and find low-level bugs at the early stages of an implementation.
SpecFlow is best used for long-term projects because it requires some previous work to be done (for example when integrating with Azure Pipelines, the pipelines need to be setup first before using SpecFlow). It can also be time-consuming when implementing methods for a dictionary. Finally you’d need to know the code of the application that is being developed and how the functionalities are implemented. Despite all that, SpecFlow is the best Backend Quality Assurance strategy when the project is based on .NET.
Looking at the pros and cons of selecting SpecFlow as a BE automation strategy, the best way to use the framework is working with BE Developers and QA Automation Engineers. BE Developers understand the specific code and how it’s implemented and are also the ones who will implement the dictionaries. QA Automation Engineers write the specific scenarios in Gherkin language (inside FeatureFiles). Together, developers and QA engineers agree on the way to implement the scenarios and the methods to execute them. This approach ensures a reliable implementation and good quality coverage.
Cucumber
Cucumber is another of our examples: a testing framework that supports Behavior Driven Development (BDD), allowing users to define application operations in plain text. Cucumber uses Gherkin, letting developers and testers write complex tests while keeping it comprehensible to even non-technical users. We commonly see Gherkin used alongside Ruby, but it also supports Java, Javascript and Scala.
Cucumber is used primarily in test automation and it works best in cases where there are some real-world actors interacting and achieving some sort of outcome. It’s especially useful when it can be written from the point of view of the user.
| 1 2 3 | Given User is a premium club member When Sarah logs into the homepage Then she sees the premium club member call to action |
Though this example is about the screen, it’s also talking in the language of the users and product. This is a comfortable specification.
The whole structure must be written into a file with the .feature extension, as in the case of SpecFlow.
Both Cucumber and SpecFlow support BDD (Behavior Driven Development), and they use Gherkin for the test case scenarios. A key difference is that Cucumber files can be used with JUnit (JUnit is a unit testing framework for the Java programming language) to implement Unit Testing, or with JSON files to test REST API endpoints. In the case of SpecFlow, it’s being used to execute tests for .NET projects.
Evaluation
Cucumber uses simple syntax, is really easy-to-use and it is a good option for REST API testing, since the test scenarios are readable. It also provides an easy way to define the JSON configuration files to be used in REST API tests, and can support different languages to implement the test scenarios. But it does require some collaboration between developers and testers to write correct test scenarios in the same syntax that the code is implemented.
To summarise, Cucumber is ideal when we are working with REST API testing, or with projects that are based on Ruby or similar languages.
REST Assured
Testing and validating REST services in Java is harder than in dynamic languages. REST Assured is one of the most popular open source tools, introducing the simplicity of using languages like Ruby into the Java domain. It is a Java library that provides a domain-specific language (DSL) for writing powerful, maintainable tests for RESTful APIs. From a JSON response, users can use REST Assured to validate different fields helping automate the testing of APIs.
REST Assured can be used easily in combination with existing unit testing frameworks, such as JUnit and TestNG. The fluent API used by REST Assured, supports the familiar Given/When/Then syntax from behavior-driven development (BDD), resulting in a test that is easy to read and takes care of everything (setup, execution, and verification) with just a single line of code.
Evaluation
With REST Assured, you can not only verify response body contents, but also check the correctness of technical response data, such as the HTTP response status code, the response content type, and other response headers.
Often, users will want to repeat the same test with various sets of (input and output) parameters, a concept known as data-driven testing. Instead of writing a new test for each test data record, users can create a parameterized test and feed it with as many test data records as your desired test coverage requires.
Compared with Cucumber, which allows you to define the high-level scenarios, REST Assured helps to simplify the actual definition of each part of the test and the interactions with endpoints. So, REST Assured can be best used for testing Restful Web services and JSON based web services.
Which strategy to use
These are just two of the Backend automation strategies that we should be considering. Which will be the best for the assessment at hand, depends on the architecture and language used for the code implementation. It is therefore our responsibility as QA professionals to investigate which are the most appropriate frameworks and languages to use when starting an automation project.
If you want to find out how to identify, select and implement the best Quality Assurance strategy for Backend implementations, have a look at our article How to create an effective QA automation strategy.
