Sed in pipelines doesnt like the word SECRET

[Oli Girling]

Had an odd issue with sed and wanted to share it.

image: node:14.15.4

pipelines:
  branches:
    dev:
      - step:
          script:
            - npm -g config set user root
            - npm i
            - npm i -g @cloudflare/wrangler
            - echo $CLOUDFLARE_API_TOKEN | wrangler config
            - mv wrangler-sample.toml wrangler.toml
            - sed -i "s/account_id = \"\"/account_id = \"${CLOUDFLARE_ACCOUNT_ID}\"/" wrangler.toml
            - sed -i "s/zone_id = \"\"/zone_id = \"${CLOUDFLARE_ZONE_ID}\"/" wrangler.toml
            - sed -i "s/RAPID_API_SECRET = \"\"/RAPID_API_SECRET = \"${RAPID_API_SECRET}\"/" wrangler.toml
            - sed -i "s/API_KEYS\", id = \"\"/API_KEYS\", id = \"${API_KEYS_NAMESPACE_ID}\"/" wrangler.toml
            - cat wrangler.toml
            - wrangler publish

 This line:

sed -i "s/RAPID_API_SECRET = \"\"/RAPID_API_SECRET = \"${RAPID_API_SECRET}\"/" wrangler.toml

Should replace RAPID_API_SECRET = "" with RAPID_API_SECRET = "${RAPID_API_SECRET}"

But instead it does this:

sed -i "s/$$$$RAPID_API_KEY = \"\"/$$$$RAPID_API_SECRET = \"${$$$$RAPID_API_SECRET}\"/" wrangler.toml 

Notice the $$$$ it adds

If i change it to RAPID_API_KEY it works no problem.

Ive tried a few things and its not the full word RAPID_API_SECRET, looks like its the word SECRET.

[ktomk]

Hey @Oli Girling ,

very interesting question. I'd consider that in the transition

RAPID_API_SECRET -> $$$$RAPID_API_KEY

there are four secrets in effect.

The first secret you have revealed your own is (likely) "RAPID_API_SECRET". And as secrets turn into their variable name prefixed with a dollar-sign, that variable name again is another secret, so the second dollar sign and again the variable name of the second secret is another secret and so on until the fourth secret and this time that was of the $RAPID_API_KEY.

Perhaps you have secured many variables, even those which do not pose any secret. These then catch a lot in the output filter. You perhaps triggered that, but I can only guess as I don't know your vars.

And you wanted to know how to open a remote-shell to Bitbucket Pipelines, right: Works with tunshell, likely tmate as well as in your reference to Travis-CI.

When you're in the remote shell, you can simulate pipeline output by writing to the standard-output of the parent PID of the tunshell command, here the agent temporary shell-script, PID 33:

$ sh <(wget https://lets.tunshell.com/init.sh -O - 2> /dev/null) L XXX XXX eu.relay.tunshell.com
Client already installed...
Connecting to relay server...
Waiting for peer to join...
52.204.xxx.xxx joined the session
Negotiating connection...
Attempting direct connection to 52.204.xxx.xxx
Direct connection to peer established
~ # ps af
PID USER TIME COMMAND
1 root 0:00 /bin/sh -c exit $( (/usr/bin/mkfifo /opt/atlassian/pipelines/agent/tmp/build_result && /bin/cat /opt/atlassian/pipelines/agent/tmp/build_result) || /bin/echo 1)
8 root 0:00 /bin/sh -c exit $( (/usr/bin/mkfifo /opt/atlassian/pipelines/agent/tmp/build_result && /bin/cat /opt/atlassian/pipelines/agent/tmp/build_result) || /bin/echo 1)
9 root 0:00 /bin/cat /opt/atlassian/pipelines/agent/tmp/build_result
11 root 0:00 /bin/sh /opt/atlassian/pipelines/agent/tmp/wrapperScript5987169154391062016.sh
32 root 0:00 /bin/sh /opt/atlassian/pipelines/agent/tmp/buildScript3732042037729391829.sh
>> 33 << root 0:00 /bin/sh /opt/atlassian/pipelines/agent/tmp/shellScript14411652748113454298.sh
47 root 0:00 sh -s -- T XXX XXX eu.relay.tunshell.com
48 root 0:00 [wget]
...
~ # printenv | grep SECRET
SUPER_SECRET=very, very secret
VERY_SUPER_SECRET=very, very secret (and secured) # this one is a secure variable
~ # echo "very, very secret (and secured)" >> /proc/33/fd/1 # pipeline log: $VERY_SUPER_SECRET
~ # exit
[2021-02-12T01:10:12Z ERROR client] Error occurred: Broken pipe (os error 32)

Pipeline image used in this example: ktomk/pipelines:busybox

Output in log

To snapshot and dump stuff out of the build-system, tar it into the project root, and have the pipeline create a build artifact out of it. Artifacts need to be relative to the project root.

For local debug: this is likely an output only issue for the log, your command is good. However if you would like to do local pipeline running there are also runners like mserranom/bbrun or ktomk/pipelines - none of these two has secrets hiding in the output.

Let me know if you still miss some options.

[Theodora Boudale]

Hi Oli,

I have attempted to reproduce this issue, but I'm afraid I haven't been able to.

What I did was create a test repo with a wrangler.toml file that included the line RAPID_API_SECRET = "".

I then created a bitbucket-pipelines.yml file using the same image you do (node:14.15.4), but I only included the cat and sed commands, like this:

image: node:14.15.4

pipelines:
  default:
    - step:
        script:
          - cat wrangler.toml
          - sed -i "s/RAPID_API_SECRET = \"\"/RAPID_API_SECRET = \"${RAPID_API_SECRET}\"/" wrangler.toml
          - cat wrangler.toml

This seems to succeed for me, and when the cat wrangler.toml command is executed after the sed command, I can see in the contents of the file that the line RAPID_API_SECRET = "" has been replaced with RAPID_API_SECRET = "$RAPID_API_SECRET".

I would like to ask a few questions so we can investigate this:

1. Do you see the following command you mentioned with the dollar signs in the Pipelines log file?

sed -i "s/$$$$RAPID_API_KEY = \"\"/$$$$RAPID_API_SECRET = \"${$$$$RAPID_API_SECRET}\"/" wrangler.toml

2. Can you open this Pipelines build, then in the Pipelines log select this sed command and expand it, take a screenshot just of that command and the content below it after you expand it, so we can see what you are seeing?

3. Is the line RAPID_API_SECRET = "" replaced in the wrangler.toml file, and if so with that? Can you take a screenshot from the output of the cat command, that shows only what this line has become after you executed the sed command?

4. If you have some time, one suggestion I can make is to debug this build locally with docker as per the instructions here 

Then check the output of the commands locally and and see if you get the same output and behaviour as in Pipelines. This is one way to narrow down whether the issue is specific to the build running in Pipelines or not.

Kind regards,
Theodora

[Oli Girling]

Hey @Theodora Boudale 

Thanks very much for the reply. 

Glad you cannot replicate it as must be something wrong my end.

1. Yep can see this in the pipeline log when expanding it but not when the command is run. 

2. Yep, attached screenshot
3. Yep, attached screenshot
4. Can do this when I have more time, but this is great to know you can do this. Will def be using this in the future.

If you need a link to the build let me know.

[Oli Girling]

Lastly, wondering if bitbucket allowed you to go into the server and dump stuff out? Like travis debug mode where you can ssh into the container and run commands?

[Oli Girling]

Notice also that the other sed commands look fine and seem to work perfectly. Just doesn't like when using the word SECRET my end.

[Theodora Boudale]

Hi Oli,

Thank you for the info.

When did you run the build/builds that had this issue?

Looking at the image node:14.15.4 in Dockerhub, I can see that it was updated a day ago:

• https://hub.docker.com/_/node?tab=tags&page=1&ordering=last_updated&name=14.15.4

There is a possibility that this issue is related to the image. If you ran this build earlier than a day ago (so, before the latest update of this image), I would suggest running it one more time now to see if the issue still occurs (perhaps the issue was caused by the earlier version of the Docker image).

If it still occurs, or if you have already run a build after the Docker image was updated and the issue is still there, my suggestion would be to open a support ticket with Bitbucket Cloud support team, give them a link to this Community post so you don't have to explain everything all over again, and also a link to the Pipelines build with this issue.

We can only access a Pipelines build if there is a support ticket open, and the engineer that will be working on your ticket can then investigate further.

You can open a ticket via https://support.atlassian.com/contact/, in "What can we help you with?" select "Technical issues and bugs" and then Bitbucket Cloud as product.

If you have any questions, please feel free to let me know.

Kind regards,
Theodora