Substitute environment variables in manifests

[iamhopaul123]

Background

Currently Copilot manifest does not support any form of pseudo-variables. On one hand, users cannot refer to the shell environment variables. On the other hand, they have to specify very similar values for different environments repetitively using environment override. For example, in our secret init doc, users during next step after the secret creation, they have to specify different DB_PASSWORD for test and dev environment.

Given the above-mentioned reasons, it is worth for us to introduce appropriate pseudo-variables to our manifest to make it more flexible.

Related Issues

• Docker build args sourced from Shell environment variables #1599
• Support for utilizing environment variables in manifest.yml #1992
• construct DB URLs from DB Secrets #2483
• Expose AWS account ID in test stage #2445
• Proposal: Input $env into the secrets #2590
• Support for passing env to docker build as args #2769

Proposal

Shell environment variables

name: my-service
type: Load Balanced Web Service
image:
  location: id.dkr.ecr.zone.amazonaws.com/project-name:${TAG}
  port: 3333

Pre-defined variables

secrets:
   DB_PASSWORD: /copilot/my-app/${COPILOT_ENVIRONMENT_NAME}/secrets/db_password
environments:
    prod:
      secrets: 
          <specific environment variable for prod>
    dev:
      secrets:
          <specific environment variable for dev>

Available pre-defined variables:

• COPILOT_APPLICATION_NAME
• COPILOT_ENVIRONMENT_NAME

It’s worth noting that we won’t allow users to override any of our predefined variables. Once we detect they have shell variables with the same name, we’ll log warning messages saying we won’t override these variables for them. Otherwise, it is possible they have ${COPILOT_ENVIRONMENT_NAME} defined in the shell session where they run Copilot without realizing its existence.

Also Copilot should error out if users are trying to refer to an undefined env var in their manifest.

[afgallo]

Oh, it's great that I've found this issue. I came across this limitation when I was trying to let copilot build my docker image but needed the ability to pass in an env variable to the build process. I guess the workaround now is to build the image manually and change my manifest to point to the image and not the dockerfile, @iamhopaul123 and @efekarakus ?