In this post, I’d like to share how you can secure your deployment pipeline using multiple accounts, profiles, and multi-factor authentication (or MFA). I’ll specifically be talking about Amazon Web Services (AWS) in combination with aws-vault. Let me start off by giving you some context.
For a product I’ve recently worked on, we have different stages of deployment. ‘Development’ for doing our development in (you guessed it) and ‘production’ for the actual production side of things. We don’t really mind the dev environment to break, but we’d strongly prefer to keep prod (as we call it) up and running all the time.
both profiles, we’ve set up completely different accounts in AWS. For
maximum protection, we opted for MFA to secure our accounts. It’s a
fairly sane industry practice. Amazon allows “virtual MFA devices” to be
used for its MFA, which means you could use Google Authenticator (which
is probably installed by default on your Android phone). In addition,
we’ve started using
which can securely store the Access Key and Secret Access Key you’ll
get from Amazon. No need to store your keys in plain text format any
Let’s first install
aws-vault. You can either simply download the relevant package for your system from their releases page on GitHub (https://github.com/99designs/aws-vault/releases), or install it using Go:
If you download and install it yourself, copy the executable to a ‘bin’ folder that’s in your
PATH and ensure it’s executable by running
chmod +x aws-vault inside the directory where you installed it. You may need to rename it when copying. Test your installation by running
usage: aws-vault [<flags>] <command> [<args> ...]
A vault for securely storing and accessing AWS credentials in development
--help Show context-sensitive help (also try --help-long and
--version Show application version.
--debug Show debugging output
--backend=file Secret backend to use [file kwallet secret-service]
--prompt=terminal Prompt driver to use [terminal]
add [<flags>] <profile>
Adds credentials, prompts if none provided
rotate [<flags>] <profile>
exec [<flags>] <profile> [<cmd>] [<args>...]
Executes a command with AWS credentials in the environment
remove [<flags>] <profile>
Removes credentials, including sessions
login [<flags>] <profile>
Generate a login link for the AWS Console
Run an ec2 instance role server locally
(The latest version of
aws-vault at the time of writing is 3.7.1, which is what I am using.)
Next, you need to set up a user in IAM who has the permissions to perform all sorts of operations through CloudFormation. For the sake of clarity, I’ll settle for a user with Administrator Access. In AWS, click on Services and select IAM. (You can sort AWS services alphabetically, which can be a tremendous help for finding them.) Go to Users and click on Add User:
In the next screen, enter a username and select “Programmatic Access”, then click ‘Next’.
You can now select the permissions for the new user. As indicated earlier, we will give this user Administrator Access. Click on “Attach existing policies directly” and from the list select AdministratorAccess:
After clicking next, you can review what you’ve done so far. If you’re happy, simple click ‘Create User’.
reached the last step of the process, and this is the only opportunity
you have to download the new user’s credentials. (If you ever lose those
credentials, you cannot retrieve them. However, you can always create
new credentials.) We will use these credentials to create a new profile
We will create a profile called ‘sls-administrator’, which is also the user’s name in IAM.
$ aws-vault add sls-administrator
Enter Access Key ID: AKIAIBCXXXXXX253JEJA
Enter Secret Access Key: U3kE/XXXXxxxxXXXXy/aeLEIJKktDd6ZR+Ce9
Enter passphrase to unlock /home/sander/.awsvault/keys:
Added credentials to profile "sls-administrator" in vault
(The keys have been changed to protect the innocent.)
We have now securely stored the user’s Access Key and Secret Access Key in the vault. All that’s left is to set up MFA. In IAM, click Close if you haven’t yet, and you’ll be sent back to the User section. Click on the newly created ‘sls-administrator’ user, then select Security Credentials.
As you can see, no MFA device has been assigned yet. Click on the pencil icon next to “Assigned MFA device … No”. A popup window will be shown:
For this tutorial, we’ll assume you’ll be using your smartphone, so select “A virtual MFA device” and click ‘Next Step’. You will now be shown a QR code that you must scan with you authenticator app (like Google Authenticator). After scanning the QR code, you should enter two consecutive authentication codes. Click ‘Authenticate’ and you’re good to go.
As a final step, we need to tell
that we’ll be using MFA. From the security credentials view, copy the
ARN that is now shown next to “Assigned MFA Device”. Using your
favourite editor, open the file
Enter a new profile, indicating the profile expects MFA for
authentication purposes. The name should be same as the profile name you
mfa_serial = arn:aws:iam::xxx0417xxx80:mfa/sls-administrator
Don’t forget to save the file. (Sorry, couldn’t help myself.)
Now, if you want to deploy, for instance, a serverless service (I mean the Serverless Framework here), you’ll run this command:
$ aws-vault exec -- sls-administrator sls deploy -v -s prod
This command executes (
exec) the command
sls deploy -v -s prod using the profile
sls-administrator. The flag
-v means the deployment will be verbose, and the
option defines the stage. It will ask you for the vault’s passphrase
and, because you’ve set up MFA, for an authentication code. Retrieve
this code from your phone and enter it.
And that’s it. You have now successfully set up MFA for your deployment pipeline (or at least for one stage). Simply repeat the above steps for different stage you may have, e.g., development, staging, production.