Renew AWS credential for a long run AWS CLI process

Cloud, Scripting, Security 1 Minute

We use aws s3 sync to synchronise a big mount of files (800,000+ objects) from on-premise to AWS S3 bucket. Due to security restrictions, the Maximum CLI/API session duration is configured for 1 hour. So it is most likely the credential will expire before the sync job is completed.

There are generally two places to configure the AWS credential: Environment variables or AWS credential config file.

  • Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY & AWS_SESSION_TOKEN) is not a solution here, as it is considered to be permanent and not looked upon again until end of the command.
  • Credential config file (e.g ~/.aws/credential) can be updated anytime, but the running AWS CLI won’t automatically re-read this file if the credential file is configured in the following format:
[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token=jkjw3rjlEAMPLETOKEN

Here is a solution we implemented and it works like a charm!

  1. Create a codebuild project or lambda function in AWS, which is scheduled to run every 50 minutes to assume a IAM role (that is only allowed to put objects into the target S3 bucket), then saves the temp credential on the remote on-premise (e.g ~/.aws/temp.json) server in following format. 
{
  "Version": 1,
  "AccessKeyId": "an AWS access key",
  "SecretAccessKey": "your AWS secret access key",
  "SessionToken": "the AWS session token for temporary credentials", 
  "Expiration": "ISO8601 timestamp when the credentials expire"
}

2. Update the AWS credential file (e.g ~/.aws/credential) on the on-premise server to

[default]
credential_process = cat ~/.aws/temp.json

In this way, AWS CLI will be smart enough to automatically renew the AWS credential that it is currently using by rerunning the credential_process command before they expire which timestamp is recorded int the Expiration key

References:
Sourcing credentials with an external process

Unknown's avatar

Published by Jackie Chen

Published

3 thoughts on “Renew AWS credential for a long run AWS CLI process”

  1. Pingback: `aws s3 sync` lessons learned – Jackie Chen's IT Workshop
  2. Pingback: Setup “Instance Profile” for on-prem servers – Jackie Chen's IT Workshop

  3. Thanks for documenting this, it helped me greatly. We have a local file repository of about 12TB that needed to sync to a customer’s S3 bucket. I mostly followed this, however instead of having an AWS process renew the token, I configured it to have the token renew automatically itself, when it needed a new token.

    My config, credentials, and scripts are at https://gist.github.com/withanhdammit/2b0fe015e6d76c611f877be2f7a750ac

    I ran ‘sync.sh’ once via cron to do the initial file push, once that was done, I schedule sync.sh to run once a day via cron.

    Worked like a champ!

    Reply

Leave a comment