1. Home >
2. Developer Blogs >
3. Migrating Continuous Integration (CI) from TeamCity to GitHub Actions

---

Like any organization that wants to ensure changes to a code base are actively being tested, OneLogin leverages Continuous Integration (CI) to run tests suites for each project in the OneLogin product. In this article, I’ll talk about OneLogin’s journey to migrate 50+ repositories from TeamCity to GitHub Actions.

The Era of TeamCity

TeamCity served our needs in the early days of CI. We leveraged Packer to build AMIs for TeamCity agents so that our projects had the necessary OS dependencies for testing as well as Puppet to automate deploying TeamCity and the agents. As more projects were onboarded to TeamCity, we made use of build configuration templates to help simplify managing the multitude of projects.

At a high level, the design of the TeamCity pipeline architecture is illustrated in the following diagram:

As you can see, a fairly straightforward architecture for a CI pipeline. As the requirements of the CI pipeline evolved (i.e. using Docker Buildkit, switching registries, etc.), we were able to propagate these changes through the build configuration template to 50+ repositories effortlessly.

We were able to get by with TeamCity for quite awhile as it met many of our needs. However, as time went on, our CI infrastructure was becoming fragile. TeamCity agents would need redeploying because they got in a weird state that would cause test executions to fail. Or, builds would be queued for an intolerable amount of time because our monolith projects exhausted all available TeamCity agents. These pain points with the CI infrastructure forced us to re-evaluate what we wanted from a CI platform.

Reassessing CI Requirements

It became clear that TeamCity was no longer going to meet our needs. For the Platform team, the #1 need that we had was CI infrastructure would not be managed by us. We no longer had the capacity to continue managing the infrastructure that would support the CI platform. Furthermore, we need CI infrastructure to be “infinitely” scalable, which was not feasible with TeamCity due to licensing costs. We could no longer tolerate long build queues because it was starting to impact new releases of the OneLogin product.

Not only did we have new infrastructure requirements, but we also had new usability requirements. Much like our infrastructure, which leverages Infrastructure-as-Code, we wanted CI-as-Code (CIaC) to describe the CI pipeline. We were also looking for a solution that allowed changes to the CI pipeline to be localized to the branch modifying the CIaC file. The lack of this feature in TeamCity made it difficult to roll out changes to CI without impacting other projects.

Once the requirements were gathered, it was time to examine the options available to us. We had looked at different open source solutions like Concourse CI, but these are self-hosted solutions and therefore didn’t meet the infrastructure requirements. We considered vendors like CodeFresh, AWS CodePipeline, GitLab CI, and GitHub Actions. Since AWS and GitHub were already cleared by the security team as acceptable vendors for our infrastructure and VCS, respectively, we narrowed the choices down to AWS CodePipeline or GitHub Actions. AWS CodePipeline and GitHub Actions checked off many of the same boxes, however, GitHub Actions provided branch-level CI pipelines. Given this, we felt that GitHub Actions was the better solution for our needs.

The Dawn of GitHub Actions at OneLogin

After deciding to pursue GitHub Actions, it was time to evangelize it to the rest of the engineering team. With the help of many adventurous colleagues, several projects showcased GitHub Actions as a CI solution. We also demonstrated how to handle secrets in CI to get the support of our security team. The portfolio of success stories provided the evidence needed to get organizational buy-in to make the transition from TeamCity to GitHub Actions. It was time to write up a migration strategy and design a new CI pipeline, or as it will now be called, a Workflow, using GitHub Actions.

Designing the GitHub Actions Workflow

Leveraging the tools used in the TeamCity build configuration was important to carry over to the GitHub Actions workflow. Our goal was to keep as much of the existing CI pipeline architecture as possible. It would help reduce the amount of work needed to port our CI pipeline to a CI workflow.

To help us understand how we might port our TeamCity build configuration to a GitHub Actions workflow, we met with the GitHub support team and engineers that worked on GitHub Actions. They gave helpful guidance on how to leverage GitHub Actions as well as an opportunity to talk about needs that were not supported by the existing features. Most notably was that workflow composability was not as capable as other CI solutions, such as GitLab CI.

Following these meetings, we set out on our mission to design a new workflow in August 2020. It was at this time that composite actions were added to GitHub Actions. This had a significant impact on how we structured the new workflow. Instead of having many small steps or few large steps, we could bundle steps into a composite action and just use it from the workflow. Providing abstractions in CI would later be instrumental in how CI is maintained, as well as simplifying adoption.

Something to note about GitHub Actions, if you are referencing a third-party action in your workflow and there is a typo in the reference, the workflow will never run because GitHub Actions was unable to prefetch the action. However, while toying with different approaches to using composite actions locally, a less than obvious behavior was discovered within GitHub Actions. Consider the following example:

./path/to/dir

The path to the directory that contains the action in your workflow’s repository.

jobs:
     my_first_job:
          steps:
               - name: My first step
                 uses: ./.github/actions/my-action

This looks like it should work, however, as noted in a community support thread, this fails at runtime with an error message stating that the action could not be found with a follow up question, “Did you forget to run actions/checkout before running your local action?” If this isn’t guarded before runtime like third-party actions are, then this means that as long as the action becomes available at the specified path before the workflow arrives at that step, you can dynamically load actions from any source, even other repositories. This newly discovered capability just opened the door for making CI maintainable at scale on GitHub Actions.

We designed our new workflow to checkout the source code of the repository using the workflow as well as the repository containing the composite actions. The workflow references the composite actions to perform building and testing on our projects. The new workflow’s architecture now looks like this:

As you can see, it is mostly the same as the TeamCity CI pipeline architecture. We have added the steps to checkout the repositories as the very first step. Furthermore, cleanup after testing is no longer necessary since GitHub Actions disposes of runners once workflow job execution finishes. Here is an example of one of our service repos using the new workflow:

'''
name: Docker IKE Test, Build, and Push

on: [push]

env:
  SERVICE: mappings-api-service
  CPU_LIMIT: 2

jobs:
  build_push:
    if: ${{ !startsWith(github.ref, 'refs/tags') }}
    runs-on: ubuntu-latest
    steps:
      - name: Dump GitHub context
        env:
          GITHUB_CONTEXT: ${{ toJson(github) }}
        run: echo "${GITHUB_CONTEXT}"
      - name: Checkout Service
        uses: actions/checkout@v2
        with:
          path: $
    - name: Checkout OneLogin Actions
        uses: actions/checkout@v2
        with:
          repository: onelogin/.github-private
          token: $
          path: onelogin-actions

    - uses: actions/setup-node@v1

    - uses: ./onelogin-actions/general/actions/secrets-to-env/v1
        with:
          SECRET_CONTEXT: ${{ toJson(secrets) }}
    - uses: ./onelogin-actions/platform/actions/install-ike/v1/beta1
    - uses: ./onelogin-actions/platform/actions/ike-aws-configure/v1/beta1
    - uses: ./onelogin-actions/platform/actions/docker-registry-login/v1/beta1
    - uses: ./onelogin-actions/platform/actions/prepare-registry/v1/beta2
    - uses: ./onelogin-actions/platform/actions/ike-image-build/v1/beta2
    - uses: ./onelogin-actions/platform/actions/static-code-analysis/v1/beta2
    - uses: ./onelogin-actions/platform/actions/ike-test/v1/beta2
    - uses: ./onelogin-actions/platform/actions/ike-image-push/v1/beta2
'''

With a new workflow ready, the next major hurdle is to migrate projects from TeamCity to GitHub Actions.

The Great Migration to GitHub Actions

Migrating 50+ repositories from TeamCity to GitHub Actions was not going to be easy. Automating the roll out would have been the easiest strategy to make the switch, but as the owner of this initiative, I felt it was important to live up to OneLogin’s core value, collaboration. We worked with the managers and directors of the engineering teams to carry out the migration.

As engineers were delegated the responsibility of migrating their services from TeamCity to GitHub Actions, we held office hours to give engineers a chance to see the new workflow in action as well as ask questions about how the workflow works. When engineers saw how concise the new workflow was thanks to the abstractions provided by composite actions in a centralized repository, the task of migrating a project from TeamCity to GitHub Actions was less daunting. It took all of Q4 (Oct-Dec for OneLogin) to start and finish the migration.

Epilogue

The approach to doing CI will not always look the same for every organization. Some companies have dedicated teams to build CI solutions. At OneLogin, we needed a solution that would allow engineering teams to feel empowered to handle CI themselves so that the Platform team could focus on its main mission of improving the reliability of the OneLogin product.

Thank you so much for taking the time to read my blog!

---

---