Ready, Set, Github Action!
Late last year, Github beta-released something they have called Github Actions. It responds to Github Events and starts off workflows that uses docker containers to do the work. It resembles Bitbucket pipelines, but using the myriad of Github Events gives Github Actions extra granularity than just reacting to git push events. Apart from that, the workflow UI is a real treat and unlike anything I have seen before. At Unacast we found Github Actions particularly interesting, and I’ll tell you why and how to use them.
How Unacast does CI
At Unacast we have a sound focus on transparency, for example in how we collect data, GDPR and a high transparency within the company when it comes to sharing the inner workings. In engineering we add to transparency by using chatops. Deploys and error handling with Pagerduty are all done within Slack. This gives everyone an easy insight in how we do operational tasks. Our VP in engineering Andreas Heim did a presentation on Kubecon in 2016, which nicely summarises how we do it. In essence our deployment follows this this diagram:
+-----------+ +--------+ +----------+ +--------+
|Slack/Hubot| | GitHub | | Heaven | | Server |
+-----------+ +--------+ +----------+ +--------+
| | | |
| Create | | |
|-------------->| | |
| | | |
| Created | | |
|<--------------| | |
| | | |
| | Deployment Event | |
| |----------------->| |
| | | Deploy |
| | |--------------->|
| | | |
| | Deployment Status| |
| |<-----------------| |
| | | |
| | | Completed |
| | |<---------------|
| | | |
| | Deployment Status| |
| |<-----------------| |There is two pain points with this, and that is the Hubot deploy plugin and Heaven. Both repositories are no longer maintained. In addition Heaven is giving us some hell. We have a lot of projects, well over 120 repositories, and a well of different programming languages and servers to deploy to. This demands that Heaven needs to accommodate all of these deployment-scripts, as well as being able to authenticate to all servers. In a fast paced development, this hinders us a bit. It’s hard to update Heaven, and only a select few have access to deploy Heaven. Also the extensive authorization Heaven has, is not something we are comfortable with.
Started deploying with Github Actions
So when we got our hands on the Github Actions beta, we set out to see if it could replace Heaven. In short Github Actions is for us the new Heaven. One should try to look at Github Actions as a third party solution, and not an integral part of Github, although it kinda is. The real good part is that within the workflow, you are already authenticated with Github. That means you can easily fire new Github Events inside your workflow, or docker containers to be more precise.
Our event of interest is the Create Deployment Event, which triggers a deployment of a particular branch or git-sha. Github does the git checkout for you, so by the start of the workflow everything is good to go. Next it is up to us to do the deploy. A picture says more than thousand words, a diagram even more:
Github has created some really nice pre-built actions for everyone to use or build upon. One being gcloud auth which we have used in the Authenticate action in this example. It uses a token for a service account and does a log in, which later steps(containers) are able to use. This gives us the freedom to create or pick and choose which docker container we would like to use in the actual deploy. In our case we use the official gcloud docker image from Google.
All in all, this works great when everything is successful. But what happens if one of the actions fail? Well, first of all every parallel actions are killed immediately. Secondly, there is no update to the deployments api, except for what you do yourself. That means there isn’t any notifications sent to Slack that says this went sideways, and the deploy still is listed as Deploying and after some time it gets status Abandoned in Github. Meaning we have to update the api, within the containers, ourselves. Which is a bit tricky, because the containers themselves differ a lot and could potentially contain only a bare minimum of tools. So, to create a tool that can be reused across actions, it needs to have as little container-dependencies as possible. For this, we created a repository of actions, the first being deployment.
Unacast public actions repo
To keep the dependencies to as small as possible, we set out to write all our tools in shell-scripts. This way they are runnable from all Github Actions containers. To add them to the workflow, and make them available to later actions, we copy all of the scripts to ${HOME}/bin. ${HOME} is a mapped folder, so all subsequent actions has that same folder available. Now it is just a matter of using the scripts in our workflow:
action "Add deploy scripts" {
uses = "unacast/actions/github-deploy@master"
}
.....action "Deploy Composer" {
uses = "docker://google/cloud-sdk:228.0.0-alpine"
needs = ["Import Airflow variables"]
runs = "/github/home/bin/deployment-exec-try"
secrets = ["GITHUB_TOKEN"]
args = "./scripts/deploy-composer.sh"
}
action "Notify success" {
uses = "docker://byrnedo/alpine-curl"
needs = ["Deploy Composer"]
runs = "/github/home/bin/deployment-create-status"
secrets = ["GITHUB_TOKEN"]
args = "success"
}
Now, if something fails in “Deploy Composer”, the function deployment-exec-try ensures that the deployment api is notified of the failure, and pass on the exit code which result in the workflow being stopped. We have put all of these functions in an open repository, free of use for all of you out there. So please use it, comment, create issues or better yet create pull requests at unacast/actions/github-deploy.
Should I abandon my CI for Github Actions?
Keep your pants on, just a little longer I would say. First and foremost, it’s just in beta, so you might have to endure some rapid changes. And who knows, Github might even drop Actions totally. Although I think that is highly unlikely, given their competitor Bitbucket already has fully working pipelines and Github seems to invest heavily in it.
So far we haven’t gotten any hints on pricing. A good guess could be to compare to other services as Circle CI, Travis and Bitbucket. For Github Actions to be competitive I think they have to put themselves in that range.
There is also some hiccups, that I am certain will be smoothed out before final release, such as slow startups, streaming logs, integration with the Checks API and general error handling.
Here at Unacast we are testing Github Actions on a small set of repositories and are enjoying it immensely. We are still using Hubot creating deploy events, but I want to give a big shout out to Slashdeploy, who not only replaces the need for Hubot deploy and Github Slack Integration, but also moves configuration from the Hubot repository to the repository that is actually deployed. A lot more comprehensible!
So that is us from Unacast, I would love to hear if any of you have started using Github Actions!
This story is also posted on Unacast company blog https://www.unacast.com/post/ready-set-github-action, where you as well can read that we were voted best place to work in NYC 2019 https://www.unacast.com/post/unacast-is-awarded-best-places-to-work-in-nyc-2019 (hint, hint apply for work)
