itch.io

Thanks for the feedback! We'll try to take a look at that to see if we can figure out what might have happened.

Thanks! We appreciate it! Hope you look forward to the final release later this year.

I'm sorry, but the game is designed around there being 2-4 players as there are some puzzles that require at least two players and communicating with each other. The online service provider we use also doesn't allow the same computer to connect twice.  Theoretically if you have two computers you could host on one and join with the other and switch between the two when needed, but I can't promise the best experience with that.

Introduction

Early on in the development of our project, War for Athelay, my co-developer and I decided that it might be wise to try to come up with ways to prevent ourselves from introducing basic bugs by creating unit tests. Originally we ran these manually before each pull request, but we quickly realized that it was easy to forget to do as we didn’t have any way to truly force ourselves to be accountable for running them. That’s where Azure Pipelines comes in. What I’d like to talk about is the basics of how to use Azure Pipelines for a Unity project to automatically run unit tests when creating pull requests.

If preferred, you can access a Google Docs version of this post here: https://docs.google.com/document/d/1UdnzXNSqurBJWy7SONwxBSHAhGO8-XCh-ATdekkH_io/...

What Are Unit Tests?

To start off, I should introduce what unit tests are for those who aren’t familiar. In a normal business environment, whereas things like QA testing and regression testing are spearheaded by some sort of QA team, unit tests are developer created tests meant to be run after making code changes and before those code changes get merged into the main development branch. They’re typically developed in the code itself and can be run automatically, which is what makes them so useful. Once created, they can be run easily and frequently.

It also just so happens that Unity has pretty good unit testing support built in (see https://docs.unity3d.com/560/Documentation/Manual/testing-editortestsrunner.html). I’m not going to go into details on creating unit tests, but they can vary from testing the inputs and outputs of specific functions, verifying that prefabs are setup correctly to prevent human error, or even running parts of the game and mock player input. They are pretty versatile once you get used to them, and I’d highly recommend that you use them in your future projects, even if you aren’t following the rest of this post!

What Are Azure Pipelines?

Azure Pipelines are part of Azure DevOps, which is a site that bundles code repositories, work items for managing the project, a wiki, and other items similar to GitHub, GitLab, etc. The pipelines themselves are basically processes that you can setup to achieve certain tasks. These can vary from compiling code, uploading files, running applications, etc. Using this, we can essentially run Unity, tell it to run the unit tests, and verify the results of the unit tests.

These Pipelines can then be attached to the branch policies so that they are automatically run when someone creates a pull request into those branches. By doing that, you can force people to pass the unit tests before their changes are merged in.

You may be wondering why we’re using Azure DevOps instead of something like GitHub. Simply put, when we were first starting, GitHub had only recently started allowing private repositories, and I don’t think we were aware of it at the time. That being said, I don’t see why this couldn’t be adapted to run through the GitHub Actions, though I haven’t used them myself. Azure DevOps can also use external repos like GitHub, so you could simply use Azure DevOps just for its pipelines.

Prerequisites

There’s going to be a few things I’m not going to cover here as they’re already documented elsewhere. So, before moving on, make sure you have the following:

1. A spare Windows computer with Unity installed (it won’t be able to run Unity for unit tests if you have it already running for development)
2. An Azure DevOps organization and project with a repo setup
3. The DevOps self-hosted agent setup on the spare computer (see: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=a...)

Pipeline Setup

Step 1: Create the new pipeline

When you’re in a project in DevOps, the Pipelines can be found under the section with the blue rocket ship icon (1 in the below picture). Once you’re on that page, you can just click on the “New pipeline” button to create a new pipeline (2 in the below picture).

Normally it would walk you through a step-by-step process, but I’d recommend clicking on “Use the classic editor” as in my experience the classic editor is easier to use and gives you more options.

Step 2: Select the code repository

Here it’ll ask where your code is located. This is where you can link it to a GitHub project if you don’t have your code repository within DevOps. For this example, though, we’re using a Repository in one of our projects. Note that the branch doesn’t matter too much, so you can leave it as the master branch. If you run the pipeline manually, you can specify the branch at the time of running it.

Once you have everything selected, click on Continue.

Step 3: Setup basic pipeline information

Again it’ll try to offer to do some basic options, but just go for the link at the top for starting with an Empty job as the option we want isn’t in the default list.

It’ll now start asking for some basic information. Fill in the name of the pipeline (something like “Unity Unit Tests” will do). For the Agent pool, select the pool for the self-hosted agent. It should be under the “Private” section.

Step 4: Creating the unit test task

Now it’s time to create the task for running the unit tests. Click on the blue plus sign to the right of Agent job 1.

Search the list of results for Powershell and add it (don’t use the PowerShell on target machines option).

Select on the PowerShell Script task now, and there will be some options. Instead of File Path, select Inline. You could add this as a ps1 file underneath your project, but for simplicity’s sake I’m going over the inline option as it’s easier. Paste the following script in the inline box:

https://drive.google.com/file/d/1-jfOydgKH5YDIlt1cisCZZXanotV4exD/view?usp=shari...

Edit the $UnityPath = “” to have the path to the folder with Unity Editor exe file (e.g. C:\Unity\UnityHub\Editors\2019.1.9f1\Editor) and update the $ProjectName = “” to match the product name in your build settings. If you have the Unity project in a subfolder in the repo, you’ll need to put the subfolder name as $Subfolder.

Your task settings should now look like the following:

Step 5: Saving and running the unit test

With that, you should hopefully be done! Simply click on “Save & Queue” at the top (1 in the below screenshot), select “Save & Queue” again. You can then save it and run it on any branch by hitting the "Save and run" button (2 in the below screenshot).

Viewing Results

You can go to the pipelines to see a list of all of your created pipelines. It’ll show the status of the latest run there (or if it’s currently running).

If you click on your pipeline, you can see a list of all of your previous runs and if they succeeded or failed. You can also rerun the unit tests at any time manually by selecting the Run pipeline option in the top right.

If you select a run, you can see a page that goes over its status, how long it ran, and what’s related to it. If there’s an error, you can click on the error to go directly to it (denoted by the 1 icon in the below picture), otherwise you can click on the job at the bottom to go to the general logs (denoted by the 2 icon in the picture)

The logs should hopefully explain any issues. Here’s an example where it failed a unit test. You can see that the unit test that failed (“Tests.MainMenuTests.CheckVersionNumber” in the example below), the failure message (“Version date needs to be greater than…”), and a small stack trace for the location of the failure in the code.

Setting Up Branch Policies

So, this is nice and all, but it still doesn’t force it. That’s what branch policies are for! With branch policies, you can enforce certain restrictions on branches, such as requiring pull requests, code reviews, or firing off a pipeline and ensuring that it’s successful. This is not only a good idea because of the pipeline but can also be good for group projects as you can make sure someone else takes a look at code before it’s merged in.

To add policies, you’ll first want to go to the branch list underneath your repo (the orange branch icon on the left).

If you have multiple repos, DevOps may default to the wrong one. To change the repo you’re under, you can select it from the dropdown at the top.

If you don’t see the branch you’re looking for, you may need to select on All to see it. In the list of branches, you can see which branches already have policies by seeing the little ribbon icon to the right of its name (such as master and FebruaryRelease in the below screenshot). To add or edit the policies for the branch, select the … when hovering over the branch on the right and select “Branch Policies”.

Here you can see all your options for branch policies. All of these can be useful depending on how many people are in the project and how it’s managed. But the option we’re looking for is the Build validation. Click on Add build policy.

Select the unit tests pipeline you created, and then look over the other options. Once you’ve got them figured out, hit save, and then you should be good!

With this done, whenever someone submits a pull request it will automatically fire off the build, which you can see in the pull request page.

Conclusion

That about wraps everything up for this! I hope this will be helpful to at least some of you out there as I think it’s been a very big help for our project. I’d love to hear any feedback about this tutorial if anyone has any! In the coming weeks I’m hoping to post more tutorials like these over how to use Azure DevOps to create Unity builds, manage multiple releases, and other items that I think may be helpful to other people. My co-developer and I have also been considering creating a write up of our experiences in grooming features and how we go about working together in the same project and codebase.

If anyone is interested, our project that we’ve been using this on is War for Athelay, a turn-based strategy roguelike game that is out in early access and has a free demo. You can view the project or follow us at any of the links below!

itch.io: https://noblephantomgames.itch.io/war-for-athelay
Twitter: https://twitter.com/Noble_Phantom
Discord: https://t.co/fjCXgAcnyX