Abstract: With the infrastructure platform growing, the number of secrets in the AWS Secrets Manager increases. This article provides the solution to effectively backing up and restoring them.

Introduction

This article continues the work regarding providing the possibility for bulk backing up and restoring SSM parameters and secrets. It has the same motivation and design considerations as in the case of parameters. Here we will mention this information in brief, and in case if you would like to read about it more - you can find more information here.

Copying secrets from one environment to another using the AWS console is a time-consuming, highly repetitive and inconvenient operation in case you have to transfer hundreds of them. The amount of effort to make changes to secrets for the new environment can be reduced with the use of directory structure and local editing in your favourite IDE. In particular, the secrets values are placed inside individual files in ./secrets directory with the path equal to the secret name. This makes it easy to delete, add or edit secrets in bulk.

This article is devoted to describing the two shell scripts: one for bulk backup of secrets and the other - for their restore. They can be used to prepare for the DR situation or to spin up the new environment in AWS.

Script to backup secrets

First, we check if secrets directory exists and stop execution in this case, as we don’t want to overwrite edits to secrets performed by the developer. After this script stores each secret value as a file with the path matching the secret name:

aws secretsmanager list-secrets --filter Key="name",Values="$secrets_base_path" --query "SecretList[*].Name" --max-items=2000 --output json --profile $aws_profile_name > $secrets_parent_dir/secrets.txt

{
  read

  while IFS= read -r secret_path
  do 
    if [ "$secret_path" = "]" ]; then
      break;
    fi

    secret_path=`echo $secret_path | tr -d '", '`
    echo "reading: $secret_path"

    file_path="./$secrets_parent_dir/$secret_path"

    value=`aws secretsmanager get-secret-value --secret-id $secret_path --query "SecretString" --no-paginate --no-cli-pager --output json --profile $aws_profile_name`
    value=${value:1:-1}
    value=`echo $value | tr -s '\\\"' '\"'`

    secret_dir="./$secrets_parent_dir/$(dirname $secret_path)"
    mkdir -p $secret_dir

    printf '%b' "$value" > "$file_path"
  done
} < $secrets_parent_dir/secrets.txt

Script to restore secrets

The following script is used to restore secrets to another environment:

cd 'secrets'

for file_name in $(find . -type f -print)
do
  if [ $file_name = './secrets.txt' ]; then
    continue
  fi

  secret_value=`cat $file_name`
  if [ -z "$secret_value" ]; then
    echo "Error: secret value cannot be empty - add value to $file_name"
    continue
  fi
  
  secret_name=${file_name:2}
  echo "writing $secret_name"

  aws secretsmanager put-secret-value --secret-id $secret_name --secret-string $secret_value --no-cli-pager --profile $aws_profile_name > /dev/null
done

How to use the solution

• Copy secrets from the source environment: ./get_secrets.sh aws_profile_name secrets_base_path. For example, for customer-management in the prod environment you can run: ./get_secrets.sh customer-prod prod/customer-management. Each secret value will be saved to a file in the ./secrets directory, which matches its path.

• Edit secrets using your favourite text editor. Don’t forget to rename the environment directory if needed, for example ./secrets/prod/... → ./secrets/dr/...

Note: If you don’t want to update values for some secrets in the next step - simply delete them or even delete entire directories.

• Overwrite secrets in the destination environment: ./put_secrets.sh aws_profile_name. For example: ./put_secrets.sh dr.

Warning: It will overwrite the values of all secrets from ./secrets directory in the destination environment.

Warning: The script does not create new secrets - it overwrites values only for existing secrets. If a secret does not exist in the destination environment - you will get an error while running the script. For the DR environment, when TF repository, which contains the service, is deployed - you should already have all secrets for the application created.

Conclusion

This article complements and concludes the task of creating tools for convenient backing up and restoring parameters and secrets from one environment to another. You can use scripts to build automated pipelines to prepare for the DR situation, conduct regular DR tests, or use them in an interactive mode if needed.

Using the provided code, you can create a customized solution for your own needs for real-life usage. I hope you enjoyed this article and that you will find it useful.

Happy coding!

Disclaimer: Code and article content are provided ‘as-is’ without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of code or article content.

You can find the full source for building this solution here.