Read secure environment variable from python code

[Davi Rodrigues]

Hi all,

I have this piece of code on my bitbucket-pipelines.yml

image: python:3.7.2

pipelines: 
  branches: 
    dev: 
      - step: 
        name: Deploy to Dev 
        deployment: Dev 
        cache: 
          - pip 
        script: 
          - pip install --no-cache-dir boto3 botocore 
          - python ./scripts/update-dms-endpoints.py

and on my python code I am trying to use secure env variables to get the aws access key and secret, and also some other db passwords, such as below:

# The pipeline itself should already work without this, but this is safer to do
AWS_ACCESS_KEY_ID = os.getenv('CF_DATAPLATFORM_AWS_ACCESS_KEY_ID', '')
AWS_SECRET_ACCESS_KEY = os.getenv('CF_DATAPLATFORM_AWS_SECRET_ACCESS_KEY', '')

If I use this way, for some reason it can't find the values on the variables. The way I know that is that it tells me that the key/secret are not being used correctly, as shown below:

The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

 But, if I send the env variables under the environment block in the bitbucket-pipelines, it works, such as below:

image: python:3.7.2

pipelines: 
  branches: 
    dev: 
      - step: 
        name: Deploy to Dev 
        deployment: Dev 
        cache: 
          - pip 
        script: 
          - pip install --no-cache-dir boto3 botocore 
          - python ./scripts/update-dms-endpoints.py
        environment:  
          AWS_ACCESS_KEY_ID: $CF_DATAPLATFORM_AWS_ACCESS_KEY_ID 
          AWS_SECRET_ACCESS_KEY: $CF_DATAPLATFORM_AWS_SECRET_ACCESS_KEY

My question is, what is the correct way to consume secret env variables inside your python code?

Thanks!

[mkleint · Accepted solution]

your CF_DATAPLATFORM_AWS_ACCESS_KEY_ID env variable should be freely available in the build once defined in the repository/account/deployment. We don't perform any encoding or encrypting of the secret variables, we just make sure the value is not printed in the output.

We don't define 'environment' section under 'step' and any unknown content will be silently ignored. Your python code looks ok as well. So I'm not sure how adding the 'environment' section could fix the problem, probably some additional changes were performed at the same time.