Sharing Travis-CI generated files

Intro

You probably already know about GitHub, and there is also a chance, that you've heard a word or two about Travis-CI, maybe you have even used it. Now Travis is a continuous integration server that seamlessly integrates with Github and can be used to build and test all your public projects for free. Of course there is always some limit in free resources. Travis provides you only with a resources for running your jobs some time, but they won't give you any storage space. This means that everything that was generated during build process will be deleted together with your repository as soon as your job finishes. To be honest it's not a big deal, as you are getting continuous testing absolutely free, and so you can always tell is there were problems with a projects you've pushed to GitHub. But after developing for a while you start to notice that there are some things that are generated by your project and it will be nice to make available to the broad audience with each successful update. For example code coverage statistics, pdf of your LaTeX publication, maybe some documentation for your library, or even binary of the application itself. Today I spent whole day trying to figure out how to do it, an lets hope that this blogpost will help you to get started in 15 minutes.

Example

I will demonstrate everything on my super-simple rails app that had some tests, and code coverage report. So the main Idea is that after running tests a coverage/ directory appears in project's root directory. It contains index.html and some other stuff. Our goal is to put the contents of the coverage/ directory somewhere on the web, so everyone can se our coverage report :) So eventually you can check out Uko/Rubidium-WHOIS as an example.

GitHub Pages

As our hosting solution we will use GitHub Pages. They are free, and when using Travis, you have a GitHub project anyway. So our first goal is to create a GitHub page for the project, to do this just follow official instructions. As a firs commit I've used my current coverage data.

OAuth Token

As GitHub pages use git mechanics (commits, pushes) to update date. As we do not want to use our GitHub login/pass in some publicly available setups, lets generate and OAuth GitHub token that can be revoked any time you want. To do so run:

$ curl -X POST -u <your_github_username> -H "Content-Type: application/json" -d "{\"scopes\":[\"public_repo\"],\"note\":\"token for pushing from travis\"}" https://api.github.com/authorizations

As a result you should get a JSON response like this one:

{
  "scopes": [
    "repo"
  ],
  "app": {
    "url": "http://developer.github.com/v3/oauth/#oauth-authorizations-api",
    "name": "token for pushing from travis (API)"
  },
  "created_at": "2012-11-29T15:55:19Z",
  "url": "https://api.github.com/authorizations/123456",
  "token": "abcdef1234567890abcdef1234567890abcdef1234567",
  "updated_at": "2012-11-29T15:55:19Z",
  "note_url": null,
  "note": "token for pushing from travis",
  "id": 123456
}

a string next to token: key is the one we are looking for. From now on I will be referring to it as token.

Travis Config

If you are not familiar with travis config, please check original doc.
In general we have 2 thing to configure. First of all it's getting Travis to know your token. Config file can be seen by everyone, but Travis provides a way to encrypt values of environment variables. This can be achieved by executing two lines in a shell:

gem install travis
travis encrypt -r <user>/<repository> GH_TOKEN=<token> --add env.global

second one will encode your token, assign it to GH_TOKEN variable and add in to .travis.yml config file.
Note: you can encode login/pass the same way. OAuth token is just more secure solution for this particular problem.

Second thing is calling the "upload script" (that we will write in next section) when that build succeeds. Just call a shell script from .travis.yml:

after_success: ./update-gh-pages.sh

Upload Script

This is how the contents of update-gh-pages.sh should look like:

if [ "$TRAVIS_PULL_REQUEST" == "false" ]; then
  echo -e "Starting to update gh-pages\n"

  #copy data we're interested in to other place
  cp -R coverage $HOME/coverage

  #go to home and setup git
  cd $HOME
  git config --global user.email "travis@travis-ci.org"
  git config --global user.name "Travis"

  #using token clone gh-pages branch
  git clone --quiet --branch=gh-pages https://${GH_TOKEN}@github.com/Uko/Rubidium-WHOIS.git  gh-pages > /dev/null

  #go into diractory and copy data we're interested in to that directory
  cd gh-pages
  cp -Rf $HOME/coverage/* .

  #add, commit and push files
  git add -f .
  git commit -m "Travis build $TRAVIS_BUILD_NUMBER pushed to gh-pages"
  git push -fq origin gh-pages > /dev/null

  echo -e "Done magic with coverage\n"
fi

As you can see, we are checking if the build is not from pull request, to save only data from our commits. In the same way we can check the branch from which commit origins, and depending on it do different stuff. Also please note that some output is redirected to /dev/null to avoid leaking of decrypted token.

Feel free to ask questions in comments and let the Defunktocat be with you!