Amazon S3 testing in Travis CI

I recently started integrating Travis CI into some of my projects. Travis is a continuous integration service that will execute your test cases in multiple different environments. I've been putting off integrating Transit into Travis because of Amazon S3 testing, but I found the time to dig into it and implement it successfully.

Preparing S3

To integrate S3 into Travis, the AWS environment must be prepared. This section can get quite lengthy and complicated, but I will keep it brief and simply. The first step is to create a new bucket and apply a lifecycle rule of 1 day to the entire bucket. This rule will delete all files in the bucket every 24 hours, which in turn requires no maintenance, cleanup and low costs on our end.

The next step is to create a new IAM group and assign the "Amazon S3 Full Access" policy permission. Once created, we need to modify the policy and change the permissions. I used the following policy for testing, which allows full S3 access to the ci-testing bucket. 

{
	"Statement": [
		{
			"Effect": "Allow",
			"Action": "s3:*",
			"Resource": "arn:aws:s3:::ci-testing/*"
		}
	]
}

After the group is setup, create a new user and assign it to the group. Be sure to save/store the credentials (access and secret key) of the user as we will need them later on. It's also a good idea to set/generate a password for this user.

Integrating Travis

To make use of S3 in Travis, the appropriate variables must be set in the environment. This can be accomplished using the env:global: setting in the .travis.yml file. The region and bucket can be set using plain text.

env:
  global:
    - AWS_S3_REGION="us-east-1"
    - AWS_S3_BUCKET="ci-testing"

Like any smart developer, the access and secret keys should not be set using plain text, as this would give S3 access to anyone viewing the Travis logs. However, we can still set the private keys using encryption keys. This process requires Ruby and RubyGems to be installed (either with Cygwin, Homebrew or another package manager).

Installing RubyGems on Cygwin is rather easy. Simply download the zip and extract it to the C:/cygwin/home/ directory. Once extracted, run the following command from within the extracted folder (this requires ruby to be installed via the Cygwin setup).

ruby setup.rb

Once installed, install Travis.

gem install travis

Then use the encrypt command coupled with the -r flag (repository) to generate a secure environment key. In the example, I will be using my Transit repository.

travis encrypt -r milesj/Transit AWS_S3_KEY="<access_key>"

Add the encrypted string to the .travis.yml file. Do the process again for the secret key.

travis encrypt -r milesj/Transit AWS_S3_SECRET="<secret_key>"

The file should now look like the following:

env:
  global:
    - AWS_S3_REGION="us-east-1"
    - AWS_S3_BUCKET="ci-testing"
    - secure: "<encrypted_access_key>"
    - secure: "<encrypted_secret_key>"

These environment variables can now be accessed from the $_SERVER global within the test cases. For a full implemented example, check out my Transit library.