Who we are?
As you can easily guess after the title of the entry, we were apprentices in Syneo and our tasks included writing automatic tests for the web application. Every day we study at the University of Technology and Life Sciences in Bydgoszcz, in the field of applied IT.
What did we use to create tests?
- Language: C#,
- IDE: Visual Studio 2015,
- Testing framework: NUnit 3.4.1,
- Tool for browser support: Selenium Web Driver,
- Browser: Mozilla Firefox (before Google Chrome),
- Version control system: Git + Visual Studio Team Services
What are we going to write about?
We would like to present some of our insights about automatic testing from the point of view of a novice programmer and describe the advantages and disadvantages of the tools we used to write them. We will also try to determine whether automated tests bring a great value to the project for which they are written. We invite you to read.
A few tips for beginners
The automatic tests were a complete novelty for us, we were learning on a regular basis while writing. Of course, during this process we encountered several problems, the solution of which was often not so obvious. That is why we would like to present some tips for people who have never encountered this topic before.
- 1.Familiarize yourself with the application
Perhaps this is a trivial statement and for many it will seem obvious, but before you start writing tests it is worth spending a few hours to familiarize yourself with the application. Test it manually. Knowing the mechanisms by which the program under test is guided, we will significantly accelerate and facilitate the process of writing automated tests.
2. Clean up after yourself
Imagine an example test scenario:
- logging into the administrator,
- setting up a new user,
- testing its edition.
What’s missing here?
Now let’s do this test 150 times.
Yes, testers have to (!) Clean up after themselves. The user’s example is quite innocent but often in the application we have the ability to add pictures or videos. We personally reached the critical point in which the server had almost a few dozen GB effects of our tests.
3. Create tests that “think”
Assuming in advance that certain elements in the application will exist when the tests are run is a good solution. But only at the beginning, when we are more interested in testing a given functionality than on the intelligence of our scripts. However, should we leave it in this form?
Let’s assume that the test user’s account is required. In the event that it runs out as a result, we will return the failure of the tests in which we log into this account. The only thing we can do then is ask if it’s the fault of not having an account or maybe some error occurred in the application. The correct answer will appear only after registering the user and switching on the tests again. Where’s the problem?
We have to handle the application ourselves and manually create a user account and run the test scripts again. After all, these are automatic tests, they should do this for us. Actually, they are created for that. It is in our interest to build tests that will be able to react to such situations.
In our application, two criteria had to be fulfilled for the user to be logged in correctly: an appropriate budget had to be set up and an account had to exist for which we want to log in. An example of an auxiliary method that itself has responded to the lack of an appropriate budget and the lack of a test user can be seen in the screenshot below.
4. Move the cursor
While working with Selenium, we encountered a problem that took a long time to resolve. Namely, at one of the tests, this tool did not always click in the place that we pointed to. The code itself was checked a good dozen times about reworking it, not to mention it.
It turned out that the mouse cursor is the culprit. When it was inside the browser window, it disrupted testing. Selenium behaves like us before it acts on an item. It finds an element, just like a human ‘invades’ it with a mouse, which can be observed by changing the style of a button, for example. Therefore, when our cursor was in the browser window, Selenium could not click on this element.
To solve this problem, we had to use the Cursor class from the System.Windows.Forms space, and just with each launch of the browser the mouse cursor was moved to the top of the screen where no one could fault it.
You can see an example of the code that moves the mouse pointer in the screenshot below.
5. You do not always have to download files
In many applications it is possible to download files to disk. They can be pictures, movies, music and the like. However, if you write tests that check the possibility of downloading a given file, you have to download it to disk every time?
It depends on whether we want to perform actions on it once it is on our computer. If we are going to do it then, we must necessarily download it. In case when we only want to test the possibility of downloading a file, we can do it without physically downloading the file to the disk.
To do this, search the site for the link to the file you want to download, then send an HTTP query to it and read the code that we will get in response. If it is 200 (OK), it means that the file can be downloaded. Unfortunately, if you need an active user session to download the file, you need to draw a “cookie” or “token” session and perform an HTTP query with it.
Sample code checking if files can be downloaded on the listing below.
Introducing … Page Object Pattern
Tests are in themselves an application. A code that is seemingly independent of the program being tested. Seemingly because any change in the tested application must be included in the tests as well. How does it look like? Let’s look at a piece of code.
But where are any tests in this code? After all, you can see at a glance that this class is only used to find items on the page and provides a method to log in to the administrator’s account.
This is what is mentioned in the title of the Page Object Pattern. It is a pattern that allows you to separate the code responsible for searching for HTML elements on the page and interacting with them, from the code whose task is to test the application. Let’s now look at the test code that uses the AdminLoginPage class.
You can not disagree with the fact that this pattern allows you to create truly readable tests. What are its advantages?
At the beginning of the chapter, we mentioned that every change in the application must be reflected in the tests. Imagine now that we do not use the Page Object Pattern.
- In every test that uses a particular element, we must search for it on the page.
- Elements can be up to a dozen on each side.
- Let’s say that the developer developing the application has changed the class or ID of several elements.
In how many places do we have to make changes in the tests? Probably in many. However, when implementing the pattern we present, we search for each element only once, so each such change is also applied only in one place. We do not need to say what convenience is such a convenient code update.
In more complex projects, such as the one at which we were able to work, we can not imagine working without a described pattern. It made it very easy for us to introduce fixes to tests that stopped working due to changes in the application. The code created with it has become much clearer. However, with smaller projects, you have to consider whether it is worth investing additional time devoted to its implementation.
Firefox vs Chrome
At the beginning of the post, we mentioned that we used both browsers listed in the title. In the end we had to use Firefox, but if the decision belonged to us, we would stay with Chrome.
Mozilla is slower. When we launched our tests using the Google browser, the time in which they were performed was about 11-13 minutes. After changing the browser, about 20-23 minutes. So it doubled.
At the time of testing, you can turn a blind eye if the browser would be more friendly to the tester. Unfortunately it is not. Chrome each time waited for the page to load completely and only then did actions on it. Unfortunately, this can not be said about Mozilla. This resulted in the fact that we had to add WebDriverWait to the test scenarios after each page reload. So, simply force the browser to wait for the page to load completely. Without this, the actions included in the test were performed on non-existent elements, which resulted in the failure of half of the tests that passed on Chromie flawlessly.
The only thing that appealed in favor of the Mozilla browser is the built-in WebDriver, which has been updated with the browser. This allows you to avoid problems (which may never happen) related to the fact that we will use the older WebDriver on a newer browser.
It should only be remembered that browsers and their support for Selenium are changing quite dynamically and at the time of reading this text, it may turn out that Selenium is best supported under a completely different browser. However, at this time, this is Chrome.
Is it worth to write automatic tests?
This is a very difficult question, which unfortunately does not have a simple and unambiguous answer. Commonly saying: “it depends.”
From our perspective, as apprentices, it is worth it. Writing tests is an interesting experience that definitely develops programming skills and allows you to work on a real project. At the same time, we work with other people responsible for applications, but on the other hand our shortages in knowledge and skills do not reflect on the project as much as when working with the source code of the proper program.
However, automated tests have their downsides. First of all, to create them you have to direct people who could at that time devote to developing or creating a product for the client (unless we can afford people who would only do tests). However, this is not the end. Once written tests, unfortunately, you can not leave yourself after creating them. You must continue to support them as the app develops. Therefore, we have like two products developed simultaneously.
We already know that tests need to be constantly improved along with the development of the application, which can sometimes consume additional time and money. It would be unjust, however, to end it.
The value of the test creation process itself should be taken into account. During our practices, with the programmer responsible for the development of the proper application, we sat “back in the back”, so that at the time of designing the test scenario we were able to notify him about any errors on the site.
In total, all unpaid tests we wrote about 90. Each checking a different functionality of the site. How long did the proper testing take?
It took a total of about 21 minutes. At the speed of the computer that does not have to look for the element on the page, or move the cursor to click on it. He also does not have to be careful not to enter incorrect data, because we as test developers have anticipated such an eventuality.
Please imagine how much it would take one or even two people. It should also be added that the tests were completed by us in about 2.5 weeks. It is therefore safe to assume that it is a quick and accurate way to check the correctness of the product.
What would we change by writing tests for the second time?
Looking at it from the perspective of time (post we write over a month after creating tests), we would change really little. We believe that the code we write is legible and there should be no problems with its maintenance. Of course, while the tests are updated on a regular basis, along with the novelties introduced in the application. Otherwise, even the best-written code will be difficult to grasp.
The changes that we would introduce at the moment are rather directed to a larger ordering of the project and include extension methods that allow more natural testing of test results.
In order to achieve the first of the above-mentioned assumptions, we would introduce a breakdown of basic tests that test only the overall reaction of the site. For example, whether the link redirects to the right place or whether sending a form results in a message about (no) success. And for those that are testing the logical aspect, or what results should be triggered by certain actions, for example if removing a user causes a change in the number of users displayed in the appropriate tab and the like.
The extending methods we mentioned are called Fluent Assertions. We have talked about the fact that they enable more natural testing of test results. Perhaps they are more useful for unit / integration tests where the results can be more complicated than in the automatic tests, but nevertheless we recommend that you familiarize yourself with them yourself. They are much more intuitive and clear than those built into NUnit. Below is a simple example that checks the result of a unit test with their use.
We hope that our post has helped a little people who, like us, have never had any contact with automated tests. If we found this type of post at the very beginning, it would surely save us a lot of time. At the same time, we apologize to all those who are more familiar with this topic for all the errors that unfortunately we do not see at our level.
We tried to pass our (modest) experiences in the best way. However, this is our first published text and we are aware that it may not be perfect. Therefore, we invite you to check our entry (both in terms of content and our skills in using the native language) and to share the results in the comments. Maybe it is you who will find a mistake that we will have to correct? ?
The authors of the entry are:
Students of the University of Technology and Life Sciences in Bydgoszcz, major in applied IT.