How to limit maximal concurrent steps in parallel?

[Niko Dojan]

Hi. 

I am working with multiple self-hosted runners on Windows and multiple concurrent pipelines. Two pipelines are serial, one is supposed to run parallel steps.

As far as I can see, each parallel step is reserving one separate runner. This could potentially block the serial pipelines, if the parallel steps keep taking free runners.

Is it possible to limit the number of concurrent steps in a parallel execution on self-hosted Windows runners?

Thanks in advance.

[Ben · Accepted solution]

Hi Niko,

It is not possible to limit the maximum number of concurrent steps that are able to execute in parallel.

If you wish to avoid blocking the serial pipelines - you can try using separate runner pools (with different labels) for parallel and serial pipelines, so they do not compete for the same runners.

For example: if you gave a group of runners the "parallel" label when creating them, and a group of runners the "serial" label when creating them, then defined your parallel steps to use the "parallel" label and the serial steps the "serial" label - they will pull from their respective pools, and ensure that the serial steps are not blocked.

- step:
      name: "Serial Step 1"
      runs-on:
          - self.hosted
          - windows
          - serial

- parallel:
       - step:
            name: "Parallel Step 1"
            runs-on:
                - self.hosted
                - windows
                - serial

I hope this helps. Please let me know if you have any further questions :)

Cheers!

- Ben (Bitbucket Cloud Support)

[Niko Dojan]

Thanks for the reply.

I was also thinking on adding more runners and pooling them.