atlassian/aws-elasticbeanstalk-deploy:0.5.5 upload artifact to S3 successful, download from S3 fails

[Justin]

Version of pipe: 

atlassian/aws-elasticbeanstalk-deploy:0.5.5

Relevant logs and error messages:

The upload succeeds

✔ Artifact uploaded successfully to s3://-redacted-

-----

And my pipeline,

custom: # Pipelines that can only be triggered manually
  deploy-to-staging:
    - step:
        caches:
          - node
        script:
          # This file uses the NPM_TOKEN environment variable
          - cp ./bitbucket-pipelines/.npmrc-config ./.npmrc
          - npm install
          - npm run rebuild
        artifacts:
          - dist/**
    - step:
      name: Deploy to Staging
      trigger: manual
      deployment: staging
      script:
        - apt-get update && apt-get install -y zip
        # Creates application.zip, using the latest commit of the current branch
        - git archive --format=zip HEAD -o application.zip
        # Adds dist to application.zip, since dist is never committed
        - zip -ur application.zip dist
        - pipe: atlassian/aws-elasticbeanstalk-deploy:0.5.5
          variables:
            AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
            AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
            AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
            APPLICATION_NAME: $STAGING_APPLICATION_NAME
            COMMAND: 'all'
            ENVIRONMENT_NAME: $STAGING_ENVIRONMENT_NAME
            ZIP_FILE: 'application.zip'
            S3_BUCKET: $S3_BUCKET
            WAIT: 'true'
            DEBUG: 'true'

-----

Is there anything that I could be doing incorrectly?

-----

It seems like this website isn't allowing me to exceed 20k characters... So, I had to snip some of the log.

Here's the error,

The download fails:

INFO: Creating application version in Elastic Beanstalk...
-snip-

2020-01-15 22:05:58,096 - MainThread - botocore.hooks - DEBUG - Changing event name from operation-args-parsed.elasticbeanstalk.create-application-version to operation-args-parsed.elastic-beanstalk.create-application-version
2020-01-15 22:05:58,097 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.application-name to load-cli-arg.elastic-beanstalk.create-application-version.application-name
2020-01-15 22:05:58,097 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.application-name: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,098 - MainThread - botocore.hooks - DEBUG - Changing event name from process-cli-arg.elasticbeanstalk.create-application-version to process-cli-arg.elastic-beanstalk.create-application-version
2020-01-15 22:05:58,098 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.elastic-beanstalk.create-application-version: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7ff413a66290>
2020-01-15 22:05:58,099 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'redacted-staging' for parameter "application_name": u'redacted-staging'
2020-01-15 22:05:58,099 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.version-label to load-cli-arg.elastic-beanstalk.create-application-version.version-label
2020-01-15 22:05:58,099 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.version-label: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,100 - MainThread - botocore.hooks - DEBUG - Changing event name from process-cli-arg.elasticbeanstalk.create-application-version to process-cli-arg.elastic-beanstalk.create-application-version
2020-01-15 22:05:58,100 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.elastic-beanstalk.create-application-version: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7ff413a66290>
2020-01-15 22:05:58,101 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'redacted-staging-50-b3d8248e' for parameter "version_label": u'redacted-staging-50-b3d8248e'
2020-01-15 22:05:58,101 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.description to load-cli-arg.elastic-beanstalk.create-application-version.description
2020-01-15 22:05:58,101 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.description: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,102 - MainThread - botocore.hooks - DEBUG - Changing event name from process-cli-arg.elasticbeanstalk.create-application-version to process-cli-arg.elastic-beanstalk.create-application-version
2020-01-15 22:05:58,102 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.elastic-beanstalk.create-application-version: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7ff413a66290>
2020-01-15 22:05:58,102 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'Application version created from https://bitbucket.org/redacted/redacted-prototype-03/addon/pipelines/home#!/results/50' for parameter "description": u'Application version created from https://bitbucket.org/redacted/redacted-prototype-03/addon/pipelines/home#!/results/50'
2020-01-15 22:05:58,102 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.source-build-information to load-cli-arg.elastic-beanstalk.create-application-version.source-build-information
2020-01-15 22:05:58,103 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.source-build-information: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,103 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.source-bundle to load-cli-arg.elastic-beanstalk.create-application-version.source-bundle
2020-01-15 22:05:58,103 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.source-bundle: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,103 - MainThread - botocore.hooks - DEBUG - Changing event name from process-cli-arg.elasticbeanstalk.create-application-version to process-cli-arg.elastic-beanstalk.create-application-version
2020-01-15 22:05:58,104 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.elastic-beanstalk.create-application-version: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7ff413a66290>
2020-01-15 22:05:58,104 - MainThread - awscli.argprocess - DEBUG - Parsing param --source-bundle as shorthand
2020-01-15 22:05:58,105 - MainThread - awscli.arguments - DEBUG - Unpacked value of u'S3Bucket=redacteddev,S3Key=redacted-staging/redacted-staging-50-b3d8248e.zip' for parameter "source_bundle": {u'S3Bucket': u'redacteddev', u'S3Key': u'redacted-staging/redacted-staging-50-b3d8248e.zip'}
2020-01-15 22:05:58,105 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.build-configuration to load-cli-arg.elastic-beanstalk.create-application-version.build-configuration
2020-01-15 22:05:58,105 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.build-configuration: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,106 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.auto-create-application to load-cli-arg.elastic-beanstalk.create-application-version.auto-create-application
2020-01-15 22:05:58,106 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.auto-create-application: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,106 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.process to load-cli-arg.elastic-beanstalk.create-application-version.process
2020-01-15 22:05:58,106 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.process: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,106 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.tags to load-cli-arg.elastic-beanstalk.create-application-version.tags
2020-01-15 22:05:58,107 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.tags: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,107 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.cli-input-json to load-cli-arg.elastic-beanstalk.create-application-version.cli-input-json
2020-01-15 22:05:58,107 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.cli-input-json: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,107 - MainThread - botocore.hooks - DEBUG - Changing event name from load-cli-arg.elasticbeanstalk.create-application-version.generate-cli-skeleton to load-cli-arg.elastic-beanstalk.create-application-version.generate-cli-skeleton
2020-01-15 22:05:58,107 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.elastic-beanstalk.create-application-version.generate-cli-skeleton: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7ff4131c5ad0>
2020-01-15 22:05:58,108 - MainThread - botocore.hooks - DEBUG - Changing event name from calling-command.elasticbeanstalk.create-application-version to calling-command.elastic-beanstalk.create-application-version
2020-01-15 22:05:58,108 - MainThread - botocore.hooks - DEBUG - Event calling-command.elastic-beanstalk.create-application-version: calling handler <bound method CliInputJSONArgument.add_to_call_parameters of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x7ff413110890>>
2020-01-15 22:05:58,108 - MainThread - botocore.hooks - DEBUG - Event calling-command.elastic-beanstalk.create-application-version: calling handler <bound method GenerateCliSkeletonArgument.generate_json_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x7ff413110a90>>
2020-01-15 22:05:58,108 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2020-01-15 22:05:58,108 - MainThread - botocore.credentials - INFO - Found credentials in environment variables.
2020-01-15 22:05:58,109 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python2.7/site-packages/botocore/data/endpoints.json
2020-01-15 22:05:58,195 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x7ff414c6a5f0>
2020-01-15 22:05:58,199 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.elastic-beanstalk: calling handler <function add_generate_presigned_url at 0x7ff414c8fb90>
2020-01-15 22:05:58,199 - MainThread - botocore.args - DEBUG - The s3 config key is not a dictionary type, ignoring its value of: None
2020-01-15 22:05:58,204 - MainThread - botocore.endpoint - DEBUG - Setting elasticbeanstalk timeout as (60, 60)
2020-01-15 22:05:58,206 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python2.7/site-packages/botocore/data/_retry.json
2020-01-15 22:05:58,211 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: elasticbeanstalk
2020-01-15 22:05:58,213 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.elastic-beanstalk.CreateApplicationVersion: calling handler <function generate_idempotent_uuid at 0x7ff414c6a848>
2020-01-15 22:05:58,214 - MainThread - botocore.hooks - DEBUG - Event before-call.elastic-beanstalk.CreateApplicationVersion: calling handler <function inject_api_version_header_if_needed at 0x7ff414c6ce60>
2020-01-15 22:05:58,214 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=CreateApplicationVersion) with params: {'body': {u'ApplicationName': u'redacted-staging', u'VersionLabel': u'redacted-staging-50-b3d8248e', u'Description': u'Application version created from https://bitbucket.org/redacted/redacted-prototype-03/addon/pipelines/home#!/results/50', u'SourceBundle.S3Bucket': u'redacteddev', u'SourceBundle.S3Key': u'redacted-staging/redacted-staging-50-b3d8248e.zip', 'Version': u'2010-12-01', 'Action': u'CreateApplicationVersion'}, 'url': u'https://elasticbeanstalk.redacted.amazonaws.com/', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'aws-cli/1.16.185 Python/2.7.15 Linux/4.19.78-coreos botocore/1.12.175'}, 'context': {'auth_type': None, 'client_region': 'redacted', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7ff412b8f3d0>}, 'query_string': '', 'url_path': '/', 'method': u'POST'}
2020-01-15 22:05:58,215 - MainThread - botocore.hooks - DEBUG - Event request-created.elastic-beanstalk.CreateApplicationVersion: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7ff412b8f390>>
2020-01-15 22:05:58,215 - MainThread - botocore.hooks - DEBUG - Event choose-signer.elastic-beanstalk.CreateApplicationVersion: calling handler <function set_operation_specific_signer at 0x7ff414c6a758>
2020-01-15 22:05:58,216 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2020-01-15 22:05:58,216 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/
content-type:application/x-www-form-urlencoded; charset=utf-8
host:elasticbeanstalk.redacted.amazonaws.com
x-amz-date:20200115T220558Z
content-type;host;x-amz-date
5529abb5335ce91f4a8a3f8daa5ff8bb626d9298fa2c7fea5d2a0fad8775c86d
2020-01-15 22:05:58,216 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20200115T220558Z
20200115/redacted/elasticbeanstalk/aws4_request
80d41d5ace837e6c2bd1db8c7609eee7225bde582bfa00352d7d782da003560d
2020-01-15 22:05:58,216 - MainThread - botocore.auth - DEBUG - Signature:
ccf589dbb8bb94ae4dd50a5eccf80ba804955bc37e29cb4bd4faac9ae3b9dc5e
2020-01-15 22:05:58,217 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://elasticbeanstalk.redacted.amazonaws.com/, headers={'Content-Length': '409', 'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'Authorization': 'AWS4-HMAC-SHA256 Credential=redacted/20200115/redacted/elasticbeanstalk/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=ccf589dbb8bb94ae4dd50a5eccf80ba804955bc37e29cb4bd4faac9ae3b9dc5e', 'X-Amz-Date': '20200115T220558Z', 'User-Agent': 'aws-cli/1.16.185 Python/2.7.15 Linux/4.19.78-coreos botocore/1.12.175'}>
2020-01-15 22:05:58,218 - MainThread - urllib3.util.retry - DEBUG - Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
2020-01-15 22:05:58,218 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): elasticbeanstalk.redacted.amazonaws.com:443
2020-01-15 22:05:59,231 - MainThread - urllib3.connectionpool - DEBUG - https://elasticbeanstalk.redacted.amazonaws.com:443 "POST / HTTP/1.1" 400 418
2020-01-15 22:05:59,232 - MainThread - botocore.parsers - DEBUG - Response headers: {'Date': 'Wed, 15 Jan 2020 22:05:59 GMT', 'x-amzn-RequestId': 'a84c8b46-4278-4b3a-b193-4faf981de0ff', 'Content-Length': '418', 'Content-Type': 'text/xml', 'Connection': 'keep-alive'}
2020-01-15 22:05:59,236 - MainThread - botocore.parsers - DEBUG - Response body:
<ErrorResponse xmlns="http://elasticbeanstalk.amazonaws.com/docs/2010-12-01/">
<Error>
<Type>Sender</Type>
<Code>InvalidParameterCombination</Code>
<Message>Unable to download from S3 location (Bucket: redacteddev Key: redacted-staging/redacted-staging-50-b3d8248e.zip). Reason: Bad Request</Message>
</Error>
<RequestId>a84c8b46-4278-4b3a-b193-4faf981de0ff</RequestId>
</ErrorResponse>
2020-01-15 22:05:59,237 - MainThread - botocore.hooks - DEBUG - Event needs-retry.elastic-beanstalk.CreateApplicationVersion: calling handler <botocore.retryhandler.RetryHandler object at 0x7ff412b51450>
2020-01-15 22:05:59,237 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2020-01-15 22:05:59,251 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 207, in main
return command_table[parsed_args.command](remaining, parsed_args)
File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 348, in __call__
return command_table[parsed_args.operation](remaining, parsed_globals)
File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 520, in __call__
call_parameters, parsed_globals)
File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 640, in invoke
client, operation_name, parameters, parsed_globals)
File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 652, in _make_client_call
**parameters)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call
raise error_class(parsed_response, operation_name)
ClientError: An error occurred (InvalidParameterCombination) when calling the CreateApplicationVersion operation: Unable to download from S3 location (Bucket: redacteddev Key: redacted-staging/redacted-staging-50-b3d8248e.zip). Reason: Bad Request
2020-01-15 22:05:59,253 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255
An error occurred (InvalidParameterCombination) when calling the CreateApplicationVersion operation: Unable to download from S3 location (Bucket: redacteddev Key: redacted-staging/redacted-staging-50-b3d8248e.zip). Reason: Bad Request

I've checked the S3 bucket and key, and the file exists there. So, the upload definitely succeeded. I downloaded the archive onto my computer and verified that the files are fine.

[Alexander Zhukov]

Hi @Justin . Does your IAM user have permissions to read from the $S3_BUCKET?

[Justin]

It had the permission. But it seems like the S3 bucket and the eb environment had to have the same region.

So, using a different bucket with the same region as the environment fixed it.