. Don’t be that developer who is woken up in the middle of the night because of some problem with the web application. After all, you need your beauty sleep – some of us more than others. The best way to avoid problems with your application is to test thoroughly. Of course, there is a cost to testing, and it is easy to jump in too deeply such that every piece of code is outrageously well tested.
Finding the right balance of just what to test and which ASP.NET Core testing tools to use is something that comes with experience. Testing simple code that has a low impact too thoroughly is not as bad as failing to test complex code with a high impact, but it will still have a negative impact on the success of your project. Complicating the problem is that we have a number of ways to test code. You may be familiar with the idea of the testing triangle or, for those who think a bit higher dimensionally, the testing pyramid. The Testing Triangle The purpose of the triangle is to demonstrate the number of tests you should have of each type. The bulk of your tests should be unit tests: these are tests which test a single public method on a class. As you move up the testing triangle the number of tests at that level decrease; conversely the scope of the tests increase.
The cost of an individual test increases towards the top of the triangle. In this article, we’ll look at testing tools which can help us out on each level of the pyramid. Some of these tools will be specific to.NET, but others will be portable to almost any development platform, and we’ll call out which is which as we progress. You will notice that as we increase the level of abstraction by moving up the triangle, the testing technologies become broader and applicable to more web development technologies. The testing triangle is a good guide for your ASP.NET core testing strategies. Unit Tests Unit tests are the smallest sort of tests that you’ll write. Ideally, they exercise a single method and should be trivial to write.
In fact, many people suggest that if you find unit tests difficult to write then it is an indication that the code being tested is doing too much and should be split up. I use unit tests as an indicator that methods should be split up into multiple methods or even split into separate classes. These are the sorts of tests you should create during test-driven development. Unit testing tools are not new in.NET or in any modern language. The migration to.NET Core has brought with it most of the first-class unit testing tools.
Unit testing tools are divided into a number of categories: test frameworks, test runners and assertion libraries. The frameworks are a set of attributes that allow you to decorate your code such that they can be found and run. Typically the testing frameworks also include a runner which will run the tests without starting up your entire application. Finally, there are assertion libraries that are used to check the conditions inside a test – most people stick with the assertions provided by their testing frameworks. All of these tools can run against any.NET project, not just ASP.NET.
Test frameworks My personal favourite testing framework is, which is now part of the open source.NET Foundation. This ensures that it will have a long life and is well recognized in the community. XUnit is a fairly opinionated framework that spends a lot of effort on being fast. Its terminology is slightly different from what you might have seen in the past, substituting terms like Fact and Theory in place of Test and ParameterizedTest. During the incubation of.NET Core, xUnit was the only unit testing framework that kept up with the constant betas.
Recently Microsoft open sourced their as v2. MSTest has been around for years and has been sorely in need of updating for much of that time. However, the new version is quite good. Even two years ago I would not have even considered MSTest over xUnit, but it is now quite competitive. Erik has an on this very blog doing a deeper comparison of several.NET unit testing frameworks. Test runners Tests Runners execute the tests in your suite and present the results in an understandable format.
If you’re using Visual Studio then there is no need to look any further than the built-in unit test runner. It works very well for ASP.NET core testing.
This part of the tooling has received a lot of love from the team over the last few releases. The speed of running tests and of test discovery is remarkably better. In addition to the standard runner, there are now live unit tests. This tooling runs your tests continuously as you write your code to tighten up the feedback loop between writing code and getting feedback. Command-line tooling for running tests is also excellent on.NET Core.
Tests can be run as easily as running dotnet test. The official documentation has a on running command-line tests, and this approach is suitable for running on a build server. XUnit will actually detect that is running in a continuous integration (CI) environment and alter its output format to one that the CI server can parse. Assertion libraries Almost everybody makes use of the built-in assertion libraries that come with the testing frameworks. However if, like me, you’re particular about the way your assertions are constructed there are some really nice alternatives.
And provide a more BDD style of assertion. What does that mean? Instead of writing Assert.That(location.X, Is.EqualTo(1337)); we can write the much more legible location.X.ShouldBe(1337); The error messages produced by these libraries are also much easier to read, saving you time tracking down testing failures.
Unit tests for other languages Because these tools run on the.NET framework, they can be used to test F# code as well, just as F# code can be used to test C# code. There are some very nice testing tools in the F# space which, if you’re feeling adventurous, are worth looking at in more detail. F# for Fun and Profit have a wonderful series of entries on. Integration Tests Integration tests are larger than unit tests and typically cross over the boundaries between modules. What is a module you might ask: that’s a great question. A module could be as small as a single class, so an integration test could be as small as just exercising the interactions between two classes.
Conversely, a module may cross process boundaries, and be an integration between a piece of code and a database server. While the catchword for unit tests is speed (so you can run them rapidly on your development box without interrupting your flow), the catchword for integration tests is parallelization. Each test is inherently going to take a long time, so to compensate we find something for all those dozens of cores you can get on your machine these days to do. Request-based testing tools More often than not, the integration tests on a web project will involve submitting requests to the web server and seeing what comes back.
Previously integration tests of this sort have been quite tricky to write. For ASP.NET Core testing, the situation has been improved with the introduction of the. This server allows you to submit requests to an in-memory HTTP server. This is much faster and provides a more realistic representation of what a fully-fledged server would return.
You can read a much more in-depth article on integration testing at ASP.NET Core in the. One handy tool for integration tests which examine the HTML returned from an endpoint is.
AngleSharp provides an API for parsing and exploring the DOM which makes checking attributes of the returned HTML a snap. Database integration testing In the past, I’ve written some pretty impressive pieces of code that stand up and tear down databases against which integration tests can be run. Although it was a pretty snazzy piece of code at the time, the speed at which the tests could run was quite limited. Fortunately, we’ve moved a little bit out of the dark ages with Entity Framework Core. Just like the TestServer, EF Core provides an in-memory implementation of a database.
This database can be used to with great rapidity. The one shortcoming of this approach is that queries written in SQL for performance or clarity reasons cannot be tested with the in-memory implementation. In those cases, I recommend reading Dave Paquette’s article on integration testing with. Acceptance Tests Acceptance tests cross-module boundaries like integration tests, but they are written from a user’s point of view.
These tests are usually written in a way that describes the behaviour of the system rather than the function. By this, I mean that your tests simulate a user’s experience with the application. In.NET are two major flavours of tools that facilitate these tests:. The primary difference between these tools is their approach to describing the tests. SpecFlow uses the Gherkin language from the Cucumber tool language to describe tests. NSpec uses its own dialect to describe similar tests. Either of these tools will do a good job building tests, but I find SpecFlow to be a more usable tool; your mileage may vary.
Software Application Testing Tools
UI Tests The highest level of automated tests are UI tests. These tests actually drive the web browser in an automated fashion performing mouse clicks, typing text, and clicking links. In the.NET space, I actually like more than any other. In my experience, UI tests tend to be incredibly fragile and inconsistent. I’d like to believe that it is just my own incompetence that leads them to be such, but in discussions with other people I’ve yet to uncover any counterexamples. Much of the issue is that web browsers are simply not designed for automation and are missing hooks to program against.
Typically testing ASP.NET web apps becomes an exercise in adding sleeps in the hopes that the UI has updated itself. Recently I’ve been hearing good things about a new tool called, but I’ve yet to use it. If you have I’d love to hear about it in the comments. UI test tools are portable to any web application and not limited to ASP.NET.
Manual Tests Manual testing is often seen as terrible drudgery. A lot of companies try to avoid manual testing in favor of cheaper automated tests. Automated tests cannot tell you what the user experience of your website is like. It is important to get a real variety of typical users to look at your site. The problems that manual testers are able to find are different from the ones you will find using unit, integration, or any other test. Bonus: Performance Testing ASP.NET Core ASP.NET Core is really, really fast.
The benchmarks for it place it in the top two or three web frameworks. This is a very impressive accomplishment considering that the team started their performance optimizations near the bottom of the list. However, as performant as the framework is, its performance can easily be ruined by running poor code on top of it. There are two classes of tests which can be run in this space: low-level and load tests.
Low-level tests Low-level performance tests answer questions like “what is the quickest way to deserialize this data stream?”, or “will heavy load in this section of the code cause too many memory allocations?”. These questions are rarely asked and for most applications that simply retrieve data from the database and display it, they don’t matter a great deal. But as soon as your code starts doing anything computationally expensive or memory intensive, they become important. The most popular tool in this space, and the one which you’ll frequently see used by the.NET Framework team itself is. This ASP.NET Core testing tool provides attributes and a test runner, not unlike xUnit, but focused on testing. The output looks something like As you can see in this example, the code is tested on a number of platforms with varying sample sizes.
These tests can be used to find issues and then kept to ensure that there aren’t any regressions in performance as your application matures. Load testing The best thing that could happen to your web application is that it becomes very popular and people flock to it; it can also be the worst thing. Thiruttuvcd malayalam. Knowing how your application performs under load will help keep your nights and weekends uninterrupted. The best tool in this space and one that I’ve seen recommended even by those who won’t otherwise touch the Microsoft stack is Visual Studio Team Services’. Using the power of the millions of machines in Azure VSTS is able to spin up very high-scale load tests that can stress your website far beyond what any desktop tool could. It is priced very reasonably and I highly recommend it.
VSTS load testing uses Azure for testing, consequently it scales to the moon. Request Tracing Both load testing and low-level tests are great tools for gauging the performance of your application before launching it. However, nobody can predict what actual user load on your website might look like.
It is really useful to have a tool like which is able to drill into the performance of your live site highlighting slow queries, code bottlenecks and hidden errors. During development, the sister tool to Retrace is invaluable at showing a high level overview of application performance. Load testing will only tell you that your site is slow while Prefix will give you the why. The price for Prefix (free!) is certainly attractive too.
Double Bonus: Accessibility Testing Accessibility testing unlocks your application to more people. Your users may be blind, have limited mobility, or be color blind, and your site should be usable by all of them. There are as many tools in this space as there are different impairments. This article is too short to do a full analysis, but others have. A Zillion Choices is a real problem in the modern world and the variety of testing tools available to you will not make the problem any better. I hope that this article will at least point you in a direction that resembles success. If you’re testing ASP.NET Core at all, then that is a huge win, since any tests are better than none.
Go forth and testify! About Simon TimmsSimon is a polyglot developer who has worked on everything from serial port drivers on an Android tablet, to NServiceBus, to processing tens of thousands of messages a second using stream analytics, to building Angular web applications. All that in the last year.
He is the author of a number of books on JavaScript and ASP.NET and blogs far less regularly than he should on both the Western Dev's site (and on his blog (He is also the least visible member of the ASP.NET monsters (interests change as frequently as he changes his underwear. At the moment he is interested in F# and how AI is going to change how we develop software. Diablo 2 patch 1.13c download. He is a big picture specialist who can maintain a cogent view of an entire system and pick the right technology to solve a problem.
Website Testing Tools
The performance of your web application affects your business more than you might think. Top engineering organizations consider performance not as nice-to-have, but as a crucial feature of their products. Unfortunately, most engineering teams do not regularly and scalability of their infrastructure, and most lack the tools to properly do so.
Although you can find plenty of commercial tool options out there, for the right organization, free and open source tools may be a good alternative—or the perfect complement to your commercial tool set. The value of performance Before talking tools, let’s consider load times and the value of performance. When you understand that performance is key to a great user experience, you need tools that will do one thing very well, and that’s to measure the user’s perceived load time. There are many out there, but ultimately, that's the only performance metric that matters. How fast is fast enough for a web application?
Here’s a quick overview of:. Under 100 milliseconds is perceived as instantaneous. A 100 ms to 300 ms delay is perceptible. Users expect a site to load in 2 seconds. After 3 seconds, 40% of visitors will abandon your site.
Engineering teams should. The goal of performance testing is to understand how your applications behave under heavy load conditions. To get started, you need to understand the baseline performance of your application and that the performance of each transaction is unique. For example, in an e-commerce application, a home page transaction is likely highly cached and very fast, whereas a checkout transaction is more complicated and must talk to a payment service, shipping service, etc. To ensure that users have a great experience, you must test the most common flows for your users and understand performance both in the browser and on the server. To get the job done, you’ll need server-side, client-side, and performance tools, and you can find free and open source options that fall into each of these categories.
Webinar: Tools for understanding server-side performance and are great for quick load tests from a single endpoint. If you just need to get a sense of the requests per second for an endpoint, these are great solutions. A more advanced approach—and my personal preference—is, an open source load-testing framework that enables complex transactions and can generate high levels of concurrency with ease. I’ve included it in the list below, along with a few other tools to consider. Locust.io. – This is a great tool for understanding the performance on the server side. – The authors describe it as “a utility for arming (creating) many bees (micro EC2 instances) to attack (load test) targets (web applications).”.
– This is an open source framework for performance and load testing that runs concurrent Python scripts to generate load (synthetic transactions) against a remote site or service. It’s commonly used for web performance and scalability testing, but you can also use Multi-Mechanize to generate a workload against any remote API accessible from Python. – This http load-testing and benchmarking utility was designed to let web developers measure code under duress, to see how it will stand up to load on the Internet. Siege supports basic authentication, cookies, and HTTP and HTTPS protocols, and lets the user hit a web server with a configurable number of simulated web browsers.
Those browsers place the server “under siege.”. – Use this tool for benchmarking your Apache HTTP server, to get an idea of how Apache performs. – This tool measures web server performance and provides a flexible facility for generating varied HTTP workloads and measuring server performance. The focus is not on implementing a particular benchmark but on providing a robust, high-performance tool that facilitates the construction of both micro- and macro-level benchmarks.
The three distinguishing characteristics of httperf are its robustness, which includes the ability to generate and sustain server overload; support for the HTTP/1.1 and SSL protocols; and its extensibility to new workload generators and performance measurements. – Use Apache JMeter to test performance both on static and dynamic resources (files, servlets, Perl scripts, Java objects, databases and queries, FTP servers, and more). You can also use it to simulate a heavy load on a server, network, or object to test its strength or analyze overall performance under different load types.
Finally, consider using it to make a graphical analysis of performance or to test your server/script/object behavior under a heavy concurrent load. Tools for understanding client-side performance Modern applications spend more time in the browser than on the server side.
The best tool to use to understand client-side performance is., a service that analyzes the content of a web page and generates suggestions to make your pages load faster. Reducing page load times reduces bounce rates and increases conversion rates. Google PageSpeed Insights Tools for understanding real-world performance is my favorite tool for evaluating client-side performance from real browsers. Open source tool analyzes your website’s speed and performance based on performance best practices and timing metrics. You can analyze one site, analyze and compare multiple sites, or let your continuous integration server break your build when you have exceeded your performance budget.
Sitespeed.io It is not always possible for teams to modify the applications to optimize client-side performance. Fortunately, Google invested in making and as web server extensions to automate performance improvements without the need for code changes. Here’s a description of each, as well as one other useful tool:. speeds up your site and reduces page load time. This open-source server module automatically applies web performance best practices to pages and associated assets (CSS, JavaScript, images) without requiring you to modify your existing content or workflow. speeds up your site and reduces page load time. This open-source Apache HTTP server module automatically applies web performance best practices to pages and associated assets (CSS, JavaScript, images) without requiring that you modify your existing content or workflow.
provides deep insights into the performance of the client side in a variety of real browsers. This utility will test a web page in any browser, from any location, over any network condition—and it’s free. WebPagetest.org Choices and tradeoffs While you have a wide choice of open source tools from which to choose for performance management, it’s not always wise to build and manage your own performance testing tools and infrastructure. You may determine that it’s not worth the engineering resources required to load test when you can pay for a proven commercial product or service.
Such tools let you more easily build, execute, and analyze performance tests, but you have to be willing to pay for it. Should you go with open source, or will a commercial tool provide more value? Only you can decide if it makes sense to use open source—and if your organization has the engineering resources required. If you do, many of the tools I’ve discussed above should go right into your toolbox.
Taken together, these free and open source tools offer a path to get started on capacity planning and load testing on the server side, optimizing and performance testing the client side, and monitoring performance from end to end to derive meaningful insights from performance tests. What are your favorite tools? Add your comment below.
Image credit.
Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |