Building and testing on multiple .NET versions with GitHub Actions

We recently started migrating our CI/CD pipeline from AppVeyor to GitHub Actions. While doing so we ran into several issues not supported by the default template. In this post, I'll go through how we have set up building and testing .NET (Core) code on multiple versions of .NET Core and .NET 5.

Before digging into GitHub Actions I want to put a few words on the move from AppVeyor. AppVeyor has been groundbreaking in a lot of ways. I have used everything from CruiseControl.NET to TeamCity in the past. AppVeyor introduced a lot of awesome features like free build for open source projects, a visual editor better than most other systems at the time, and YAML-based config that you could include in your Git repository. AppVeyor has served us very well for sure. So why switch? There's a couple of reasons like minimizing the number of dependencies and moving everything to our GitHub organization.

With that out of the way, let's take a look at GitHub Actions. For those of you who don't know Actions, it's a great addition to both private and public repositories on GitHub. Actions will allow you to automatically build, deploy, publish, etc. changes in Git repositories, much like you know it from similar CI/CD systems like Azure DevOps, AppVeyor, and TeamCity.

For the rest of the post, I'll use a simple class library with a StringExtensions class as well as a test project. The overall structure looks like this:

Adding build and test with GitHub Actions is only a few clicks away. After adding the projects above to a repository on GitHub, click the Actions tab on GitHub:

GitHub Actions already identified this as .NET Core project. Click the Set up this workflow button. The .NET Core workflow comes with a set of default steps:

name: .NET Core

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.301
    - name: Install dependencies
      run: dotnet restore
    - name: Build
      run: dotnet build --configuration Release --no-restore
    - name: Test
      run: dotnet test --no-restore --verbosity normal

As of writing this post, .NET Core 3.1.301 us set up and the code is build and tested. The .NET version will probably change to .NET 5 but the overall structure should look similar.

Click the Start commit button, input a commit message, and commit the new build file. GitHub Actions will pick up the commit and start building:

If your class library is targeting .NET Core 3.1 or lower, everything should build and test successfully.

The first problem we ran into is caused by our assemblies targeting multiple versions of .NET Core (and now .NET 5). As long as you make sure to install the highest version needed, dotnet will make sure to build for all of the specified targets. Let's expand the csproj file for the GitHubActions project with multiple versions of .NET (Core):

<PropertyGroup>
  <TargetFrameworks>netcoreapp2.1;netcoreapp3.1;net5.0</TargetFrameworks>
</PropertyGroup>

The project is now build for both .NET Core 2.1, .NET Core 3.1, and .NET 5. To run all of our unit tests on all supported versions of .NET (Core), make sure to include the same list of target frameworks in the unit test project as well. When trying to build this with GitHub Actions we get the following build error:

/home/runner/.dotnet/sdk/3.1.404/Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v5.0 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application.

This makes sense since we cannot build .NET 5 with the .NET Core 3.1 SDK. As already mentioned, we can build old versions with a new version of the SDK, so let's replace .NET Core 3.1 with .NET 5 in the build script:

- name: Setup .NET Core 5.0
  uses: actions/setup-dotnet@v1
  with:
    dotnet-version: 5.0.x

Rather than hardcoding the minor version like we did previously, I am simply setting up 5.0.x which will install the newest version of .NET 5.0.

When adding the change to Git, GitHub Actions will rebuild the project. The build step now succeeds but we are stuck with the next problem in the Test step:

Testhost process exited with error: It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '3.1.0' was not found.

• The following frameworks were found:
  5.0.0 at [/home/runner/.dotnet/shared/Microsoft.NETCore.App]
  You can resolve the problem by installing the specified framework and/or SDK.
  The specified framework can be found at:
• https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=3.1.0&arch=x64&rid=ubuntu.18.04-x64

Please check the diagnostic logs for more information.

The error is logged when trying to execute your unit tests on a target framework not available for GitHub Actions. While a new version of .NET (Core) can target older versions during compile, you will need the old version of the SDK when running the code. Since the Test step is running your code, multiple versions of .NET (Core) needs to be installed.

To install multiple versions of .NET (Core), expand the build script like this:

- name: Setup .NET Core 2.1	
  uses: actions/setup-dotnet@v1	
  with:	
    dotnet-version: 2.1.x	
- name: Setup .NET Core 3.1	
  uses: actions/setup-dotnet@v1	
  with:	
    dotnet-version: 3.1.x
- name: Setup .NET Core 5.0
  uses: actions/setup-dotnet@v1
  with:
    dotnet-version: 5.0.x

Now both .NET Core 2.1, .NET Core 3.1, and .NET 5 are installed during the build phase. GitHub Actions automatically pick up the newest version of dotnet during the Build phase, why the order you set up the versions isn't important. When reaching the Test phase, all of the targeted .NET (Core) SDKs are now available and the build succeeds.

See how we can help you monitor your website for crashes Monitor your website