Managing ephemeral self-hosted GitHub Actions runners with Tartelet

In the beginning of 2023 we set out to move from our current CI solution to GitHub Actions. There were three aspects that motivated us to revise our setup.

• Consolidate our services. At Shape we’re happy users of GitHub and by moving our CI setup from one provider to GitHub Actions, we can reduce the number of services developers need to use daily.
• Decreasing build times. The faster the build times are on our CI, the faster we can iterate on pull requests and distribute builds for testing.
• Lowering the monthly cost. While not a dealbreaker, our CI setup was getting quite expensive, and it would be a welcome side-effect if we can reduce the monthly cost of our CI setup.

During our initial testing of GitHub Actions we found that jobs building an iOS app run 20-25% slower on GitHub’s macOS runners compared to the CI provider we were moving from. This made GitHub’s runners a total no-go as we don’t accept an increase in build times 🙅‍♂️

Shifting our focus to self-hosted runners

We began investigating GitHub’s self-hosted runners that allow developers to use any machine to act as a runner and ultimately have builds run faster and in environments that the developers are in control of. We had the following requirements for our self-hosted runners.

• Feature parity with GitHub’s macOS runners. We want full control over the software installed on our self-hosted runners, so we can ensure that it has the same software installed on GitHub’s runners making it easy to use GitHub’s runners, in case our self-hosted runners are down.
• Provide isolated and ephemeral environments for jobs. It’s important that the jobs run in isolated environments that are recreated between each job. This ensures that no data is leaked between workflows, and a workflow always starts with a clean slate.
• Physical access to hardware is not assumed. We are not interested in maintaining the hardware ourselves. Instead we want to rent Mac hardware. As such, the solution should not assume that we have physical access to the hardware.

As we researched our options, we were surprised to discover that the market for self-hosted GitHub Actions runners is not as mature as we had assumed. There were few to none off-the-shelf solutions that fulfilled all of our needs. So being developers, we started building it ourselves. As Bob the Builder says, “Can we fix it? Yes, we can!” 💪

Introducing Tartelet

Screen sharing to a physical machine rented from MacStadium. The machine maintains two virtual machines, each operating a GitHub Actions runner, using Tartelet.

Today we are open-sourcing Tartelet, our take on managing self-hosted GitHub Actions runners. Tartelet has a few benefits that make it stand out from alternatives.

• Jobs run in ephemeral virtual machines. Tartelet will run jobs initiated by GitHub Actions in ephemeral virtual machines, meaning that once a job’s completed the virtual machine will be destroyed and a new virtual machine will be created for the next job.
• Standing on the shoulders of giants. Not only is Tartelet open-source but it’s also built on top of Tart, an open-source command line tool for managing macOS virtual machines.
• It’s fast. Because Tartelet is built on Tart, which in turn uses Apple’s recently introduced Virtualization framework, our virtual machines run at near-native performance.
• Virtual machines run in parallel. Each physical machine can run up to two virtual machines at once. This increases the throughput while keeping the cost of renting machines down.

While we’re still evaluating the solution, we are running Tartelet on two physical machines, allowing us to have four virtual machines running in parallel. The machines are two Mac mini M1 from 2020 with 16 GB of memory rented from MacStadium and Hetzner.

We are seeing that our iOS apps build 3 - 4 times faster on our self-hosted runners compared to GitHub’s macOS runners and we have reduced our monthly cost by more than 20%. All of this while also improving the developers’ experience. Not only are we as developers happy that builds are lightning fast but since we’re consolidating services on GitHub, we also have one less service to use.

If this has piqued your interest and you want to learn more about Tartelet, then we recommend starting with the README in the Git repository and then heading over to the wiki on GitHub.