Abstract: With the growth of the infrastructure platform number of SSM parameters increases. This article provides the solution to effectively backing up and restoring them.

Introduction

After the creation of infrastructure in the DR scenario or during regular DR tests, developers spend a sizeable amount of time copy-pasting SSM parameters from existing environments. For example, some applications can have 50 SSM parameters to fill (!). Partial or full automation of this step allows developers to save time and effort, opening the possibility to work on other DR-related activities or to simply finish DR tests earlier. In the case of a real DR scenario, it will potentially allow the platform to be ready to serve customers faster.

AWS console is convenient when changing a few SSM parameters but not useful when you would like to back up or restore hundreds or even thousands of them. Moreover, frequently the contents of SSM parameters are environment-specific and need to be changed when restoring them to the new environment. Thus, the ability to easily edit their content would be nice to have. In addition to this, search and replace capability over all SSM parameter values can greatly simplify the process of creating and populating SSM parameters with values. This article will describe shell scripts for making this task convenient. They can be handy when spinning up the new AWS environment, as well as in the DR situation.

Design considerations

It is common to organize SSM parameter names in a directory-like structure, for example:

/dev/myapp1/app-config

/dev/myapp2/app-config
/dev/myapp2/features

Based on the requirements listed in the introduction and fact above, it was decided to write a shell script to backup SSM parameters in the directory structure based on their name:

./parameters/dev/myapp1/app-config

./parameters/dev/myapp2/app-config
./parameters/dev/myapp2/features

This way, after backing them up, we can open ./parameters in our favourite IDE for bulk parameters editing and searching or replacing string values. Moreover, deleting parameters in bulk or adding new ones if needed is easy. Indeed, let’s say we are not going to run myapp2 in the new environment. In this case, we can just delete the directory ./parameters/dev/myapp2 with all files inside. When we run the second script to create SSM parameters in the new environment, it will use the file’s directory path as a parameter name and its contents as a parameter value.

Using a directory structure and separate files to represent parameters is much more convenient than using a single file to store the names and values of all the parameters. Indeed, imagine that you would like to avoid restoring parameters for myapp3, which has 50 SSM parameters with tremendous JSON values. Searching for them in single file and deleting significant portions of text is inconvenient, error-prone and time-consuming, while just deleting directory ./parameters/dev/myapp3 is instant and user-friendly.

Script to backup SSM parameters

As we don’t want to overwrite edits to parameters values if a user will accidentally reruns the script, we stop its execution if the parameters directory exists, otherwise we create it:

params_parent_dir='parameters'

if [ -d $params_parent_dir ] 
then
  echo "Error: './$params_parent_dir' directory exists. Please delete/rename it." 
  exit 1
fi

mkdir $params_parent_dir

After it, we get all parameters, and for each of them, put its value to the file in the directory, which matches the parameter path:

aws ssm get-parameters-by-path --path $params_base_path --recursive --query="Parameters[*].[Name, Value]" --with-decryption --max-items=2000 --output json --profile $aws_profile_name > $params_parent_dir/params.txt

{
  read

  while IFS= read line
  do 
    if [ "$line" = "]" ]; then
      break;
    fi

    read -r param_path
    param_path=`echo $param_path | tr -d '",'`

    read -r value
    value=${value:1:-1}
    value=`echo $value | tr -s '\\\"' '\"'`

    read

    param_dir="./$params_parent_dir$(dirname $param_path)"
    mkdir -p $param_dir

    file_path="./$params_parent_dir$param_path"
    echo "reading: $file_path"

    printf '%b' "$value" > "$file_path"
  done
} < $params_parent_dir/params.txt

Script to restore SSM parameters

After editing parameters values in the IDE, we can restore SSM parameters with a separate script, in which we use file path as the parameter name and file contents as its value:

cd 'parameters'

for file_name in $(find . -type f -print)
do
  if [ $file_name = './params.txt' ]; then
    continue
  fi

  parameter_value=`cat $file_name`
  if [ -z "$parameter_value" ]; then
    echo "Error: parameter value cannot be empty - add value to $file_name"
    continue
  fi
  
  parameter_name=${file_name:1}
  echo "writing $parameter_name"

  aws ssm put-parameter --name $parameter_name --value $parameter_value --overwrite --no-cli-pager --profile $aws_profile_name > /dev/null
done

Here we are skipping processing ./params.txt, as it comes from parameters read script and is left for debugging purposes if needed.

How to use the solution

• Copy parameters from the source environment: ./get_params.sh aws_profile_name parameters_base_path. For example, for customer-management in the prod environment, you can run: ./get_params.sh customer-prod /prod/customer-management. Each parameter value will be saved to a file in the ./parameters directory, which matches the parameter path.

• Edit parameters using your favourite text editor. Don’t forget to rename the environment directory if needed, for example ./parameters/prod/... → ./parameters/dr/....

Note: If you don’t want to update values for some parameters in the next step - simply delete them or even delete entire directories.

• Overwrite parameters in the destination environment: ./put_params.sh aws_profile_name. For example: ./put_params.sh dr.

Warning: It will overwrite the values of all parameters from ./parameters directory in the destination environment.

Warning: The script does not create new parameters - it overwrites values only for existing parameters. If the parameter does not exist in the destination environment - you will get an error while running the script for this parameter. For the DR environment, when TF repository, which contains the service, is deployed - you should already have all SSM parameters for the application created.

Conclusion

In this article, we discussed the solution for convenient backing up and restoring SSM parameters from one environment to another. It can be used to prepare SSM parameter values in advance for utilizing them in case of the DR situation, as well as to build an automated pipeline to create periodic backups of parameters from environments. Even if the parameters values are not prepared for use in the DR account, in the actual DR situation, knowing their expected structure and sample values can save a lot of time and allow us to avoid browsing applications code to figure it out. In addition, the solution can be applied in an interactive mode if needed. Using the provided code, you can create a customized solution for your own needs for real-life usage.

I hope you enjoyed this article and that you will find it useful.

Happy coding!

Disclaimer: Code and article content are provided ‘as-is’ without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of code or article content.

You can find the full source for building this solution here.