How to deal with spaces using AWS S3 integration in Bitbucket Pipelines?

[Oscar Navarrerete Estévez]

I am running the aws s3 integration pipeline:

pipe: atlassian/aws-s3-deploy:1.1.0
  variables:
    ...
    EXTRA_ARGS: '--metadata={"style-src":"self inline"}'

It seems the space is not being process properly because I get:

awscli.argprocess.ParamError: Error parsing parameter '--metadata': Invalid JSON: Unterminated string starting at: line 1 column 75 (char 74)
JSON received: {"style-src":"self

I have tried different quotes arrangements and also tried the shorthand syntax:

EXTRA_ARGS: '--metadata=style-src=self inline'

The error being:

awscli.customizations.exceptions.ParamValidationError: Unknown options: inline

I did simplify the pipeline for readability, but it is longer and all previous arguments in EXTRA_ARGS were accepted, so I do not believe there are other problems besides the space.

Any suggestion is appreciated, thanks in advance.

[Oscar Navarrerete Estévez · Accepted solution]

To whoever may be relevant.

The solution we ended up using in the end is to provide the metadata from an external file and decode it in the pipeline:

- export CSP_POLICY_B64=$(grep -v "^\s*#" default.csp | tr '\n' ' ' | tr -d '\r' | base64 --wrap=0)
- pipe: atlassian/aws-s3-deploy:1.1.0
  variables:
    ...
    EXTRA_ARGS: '--metadata cf-csp-b64=$CSP_POLICY_B64'

 In the above example, a default.csp file has been provided on the root of the project. The contents looks like so:

# This file is parsed by bitbucket-pipelines so it can contain
# commented lines starting with a # and newlines are formatted
# to a space when deploying.
style-src 'self' 'unsafe-inline';

[o_kyrdan]

Hi @Oscar Navarrerete Estévez 

Thank you for your question!

Could you try this:

pipe: atlassian/aws-s3-deploy:1.1.0
  variables:
    ...
    EXTRA_ARGS: "--metadata '{\"x-amz-meta-cms-id\":\"34533452\"}'"

Best regards,
Oleksandr Kyrdan

[Oscar Navarrerete Estévez]

Hello @o_kyrdan ,

Thank you for your answer, but we solved the problem in the answer I provided.

For the record I did try your suggestion, and I got this error:

2022-05-30 08:00:43,481 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
File "awscli/argprocess.py", line 335, in _parse_as_shorthand
File "awscli/shorthand.py", line 166, in parse
File "awscli/shorthand.py", line 171, in _parameter
File "awscli/shorthand.py", line 190, in _keyval
File "awscli/shorthand.py", line 349, in _expect
awscli.shorthand.ShorthandParseSyntaxError: Expected: '=', received: ''' for input:
'{"x-amz-meta-cms-id":"34533452"}'
^

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "awscli/clidriver.py", line 459, in main
File "awscli/customizations/commands.py", line 197, in __call__
File "awscli/customizations/commands.py", line 164, in __call__
File "botocore/session.py", line 672, in emit_first_non_none_response
File "botocore/hooks.py", line 227, in emit
File "botocore/hooks.py", line 210, in _emit
File "awscli/argprocess.py", line 313, in __call__
File "awscli/argprocess.py", line 338, in _parse_as_shorthand
awscli.argprocess.ParamError: Error parsing parameter '--metadata': Expected: '=', received: ''' for input:
'{"x-amz-meta-cms-id":"34533452"}'
^

Error parsing parameter '--metadata': Expected: '=', received: ''' for input:
'{"x-amz-meta-cms-id":"34533452"}'
^