Six months ago I was trying out different BDD frameworks for .NET / C#. SpecFlow was one of the tools I already had experience with. I have been using it now and then for a few years. Recently I have been exploring a technique to reuse SpecFlow features to test a system from different angles. The same scenarios can be used to test a system both via API and browser.

Why?

Why do I want to reuse SpecFlow feature files?

I can get started with one test approach, and later change to another without rewriting the feature files.

I am forced to think hard about writing good feature files.

In general, tests via API are:

  • easier to write
  • easier to maintain
  • faster to run

In general, tests via browser are:

  • truly end-to-end

The subject

I will be using the same subject as in the previous blog post:

The stack, a last-in-first-out (LIFO) collection:

And I will test the same things as before:

  • Empty vs nonempty stack
  • Peek() and Pop() methods
  • InvalidOperationException

I created a single page application as a UI for the stack. It will provide the opportunity to test the stack both via a REST API and via the browser.

The SPA is a React + Redux app on ASP.NET Core generated with yo aspnetcore-spa:

yo aspnetcore-spa

The code for the SPA is available at: https://github.com/hlaueriksson/ConductOfCode/tree/master/SpecFlow/ConductOfCode

To run the site:

  • Install .NET Core 1.1
  • set ASPNETCORE_ENVIRONMENT=Development
  • dotnet run
  • Browse http://localhost:5000

SpecFlow

I will be using the same feature file as before:

The step definitions for testing the Stack<T> class directly also looks the same as before:

Both these files are located in a VS project called ConductOfCode.Specs.

The code is available at: https://github.com/hlaueriksson/ConductOfCode/tree/master/SpecFlow/ConductOfCode.Specs

How can we reuse the SpecFlow feature file?

  1. Create a new VS project
  2. Copy the feature file to the new project
  3. Implement the step definitions in the new project

Step number 2 can be done by:

  • Copy and paste the file
  • Use Add As Link
  • Include the file via a git submodule

I will demonstrate the Add As Link way. With this approach it is important that the new VS project has the same default namespace as the original project. Change the default namespace via the project properties:

Default namespace

Then use the right-click menu to Add / Existing Item..., select the feature file, and Add As Link:

Add As Link

The csproj will look like this:

csproj

With the Add As Link approach it matters what unit test framework you use together with SpecFlow:

  • NUnit works fine with the ReSharper test runner
  • xUnit has some issues with the ReSharper test runner

Api

I created a StackController in the SPA project. The class provides a REST interface for an in-memory stack. To make the testing of the API easier, I added Swagger to the project with NSwag. The response types for each action is annotated with attributes based on HTTP status code.

Use the Swagger UI to get started with some manual testing:

Swagger UI

The ConductOfCode.Specs.Api VS project contains the tests that hits the API.

With NSwag.MSBuild I generated a client for the API from the Swagger definition:

.\packages\NSwag.MSBuild.8.5.0\build\NSwag.exe swagger2csclient /input:http://localhost:5000/swagger/v1/swagger.json /classname:StackClient /namespace:ConductOfCode.Specs.Clients /output:./ConductOfCode.Specs.Api/Clients/StackClient.cs

With the generated client in place, I created a synchronous facade:

The facade will be used as the subject in the step definitions. When the API changes and the client is regenerated, it is hopefully only the facade that we need to update.

The step definitions for testing the API:

Web

The React + Redux app uses the REST API to access the in-memory stack. It is not exactly a wonder of ux and design:

SPA

The ConductOfCode.Specs.Web VS project contains the tests that hits the browser.

The site is tested with Coypu and Chrome.

Access to the site is handled with the Page Object Pattern.

The page object will be used as the subject in the step definitions. When the site and markup changes, it is hopefully only the page object that we need to update.

The step definitions for testing the web:

Together with a Should library, I also like to create my own should extensions:

Reports

SpecResults and SpecResults.WebApp can be used to generate reports after SpecFlow runs.

SpecResults.WebApp

Create a base class and let your step classes inherit from it:

It matters what unit test framework you use together with SpecFlow:

  • xUnit works fine with SpecResults
  • NUnit does not work with SpecResults

Conclusion

It is possible to reuse SpecFlow features to test a system from different angles.

Unit Test Sessions

When writing features:

  • try to be agnostic on how the system is tested
  • avoid being too detailed in the steps
  • identify the lowest common denominators between test approaches
  • Copy and paste the file

This approach is good if you don’t want 100% identical feature files between projects.

The projects can have different default namespaces.

  • Use Add As Link

This approach is good if you want identical feature files between projects.

The projects must have the same default namespace.

Unfortunately you will get this annoying dialog when trying to open the feature file from different projects:

This document is opened by another project.

  • Include the file via a git submodule

I have not tried this approach and it certainly has a bit of overhead.