Overriding variables via BITBUCKET_PIPELINES_VARIABLES_PATH

[Maurizio Pillitu]

Dear Community,

I'm trying to setup a pipeline to perform a checks against a deployed environment, and I'm making use of $BITBUCKET_PIPELINES_VARIABLES_PATH to set variables to use in my steps (see `broken-logic.yaml` in https://gist.github.com/maoo/4c62c3ec6fbc8f6e42b0bb3d17152d5e )

This code fails with the most weird error:

The ENV variable is not defined in the output variables. Define this output variable,

referring to echo ENV=qa > $BITBUCKET_PIPELINES_VARIABLES_PATH (4th step mentioned above), even if the code is exactly the same as the first step, and the first 3 (sequential) steps succeed.

Here comes the fun part; if I "simplify" the pipeline as follows, everything works as expected (see `working-logic.yaml` inhttps://gist.github.com/maoo/4c62c3ec6fbc8f6e42b0bb3d17152d5e )

Is there anything obvious that I'm missing? Also, based on what I'm trying to achieve, are there better architectures to achieve it? I probably haven't fully grasped BitBucket Pipelines architecture to be honest.

Thanks in advance.

PS - sorry for the links to gist, but apparently this editor takes out code formatting.

[Ben · Accepted solution]

Hi @Maurizio Pillitu 

A few things here:

• YAML B always uses single arrow (>) instead of (>>):
  This ensures that the file at $BITBUCKET_PIPELINES_VARIABLES_PATH is overwritten with a single value (ENV=dev, ENV=qa, or ENV=preprod) each time. There is no risk of appending multiple values, which could cause errors.

• YAML A uses shared configurations like <<: *check-java-images and <<: *check-completed. While this is valid YAML syntax, ensure that the anchors (*check-java-images, *check-completed, etc.) are properly defined elsewhere in the pipeline.

• The output-variables section is used to define variables that can be passed to subsequent steps. However, if a step that relies on ENV runs before the Set ENV step completes, the variable will be undefined. Ensure that steps depending on ENV are executed after the step that sets it.

Fixed example (sorry about the formatting):

definitions:
  steps:
    - step: &check-java-images
        script:
          - echo "Checking Java images for $ENV environment"
    - step: &check-ui-image
        script:
          - echo "Checking UI images for $ENV environment"
    - step: &check-completed
        script:
    - echo "Check completed for $ENV environment"

pipelines:
  branches:
    main:
      # Check image's CommitIDs against Dev
      - step:
          name: Set Dev environment
          script:
            - echo ENV=dev > $BITBUCKET_PIPELINES_VARIABLES_PATH
          output-variables:
            - ENV
      - step:
          <<: *check-java-images
          name: Check Java images commitID against dev env
      - step:
          <<: *check-ui-image
          name: Check UI image commitID against dev env
      - step:
          <<: *check-completed
          name: Check completed on dev
          deployment: dev

      # Check image's CommitIDs against QA
      - step:
          name: Set QA environment
          script:
            - echo ENV=qa > $BITBUCKET_PIPELINES_VARIABLES_PATH
          output-variables:
            - ENV
      - step:
          <<: *check-java-images
          name: Check Java images commitID against QA env
      - step:
          <<: *check-ui-image
          name: Check UI image commitID against QA env
      - step:
          <<: *check-completed
          name: Check completed on QA
          deployment: qa
      # Check image's CommitIDs against Preprod
      - step:
          name: Set Preprod environment
          script:
            - echo ENV=preprod > $BITBUCKET_PIPELINES_VARIABLES_PATH
          output-variables:
            - ENV
      - step:
          <<: *check-java-images
          name: Check Java images commitID against preprod env
      - step:
          <<: *check-ui-image
          name: Check UI image commitID against preprod env
      - step:
          <<: *check-completed
          name: Check completed on preprod
          deployment: preprod

 

If this still fails, please raise a ticket with support - as we need to look at your build environment/YAML config:

• https://support.atlassian.com/contact/

Cheers!

- Ben (Bitbucket Cloud Support)

[Maurizio Pillitu]

Thanks Ben, thanks for your help!

Answering in order to your points....

- Not sure I understand this first point.... is `>` the right way to go in my case? I think so, as `>>` (if I remember correctly) fails on the second invocation

- Yes, good point, I didn't add the syntax of the shared step definitions, but yes, they're there (if not, the pipeline would have probably failed differently). Thanks for checking!

- Yes, that is already the case: ENV is used after being defined in the step with `output-variables`; I've compared the "working-logic" with the "broken-logic" for a while already, but maybe I'm missing something obvious.

I'll do another couple of experiments, but after that I'll probably reach out to support.

Thanks for your help!

[Ben]

Hi @Maurizio Pillitu 

- Correct, > (single arrow) is the right way to go - I should have made this clearer :) 

If you are unable to reach out to support for whatever reason - please let me know your timezone and I'll raise a ticket on your behalf.

Cheers!

- Ben (Bitbucket Cloud Support)

[Maurizio Pillitu]

@Ben - here's the ticket - https://support.atlassian.com/requests/BBS-232008/