Bitbucket Pipelines - using an external Docker daemon in build containers.

[Konrad Stolarczyk]

Hi Team,

My current setup consists of self hosted runner that is deployed as docker container, next to my small application environment managed by docker compose.

I need to be able to control my app deployments through dedicated pipeline steps that would run something as simple as "docker compose up" that would start the workload on the host.

Just a simple use of docker service in the pipeline step, like:

services:

- docker

doesn't do the trick, as the build container is not connected to the host's docker engine, thus falling into "Docker in Docker" situation. Inspection of build container validated, that docker.sock is not being shared (mounted) when build container is started by the runner.

https://support.atlassian.com/bitbucket-cloud/docs/run-docker-commands-in-bitbucket-pipelines/#Using-an-external-Docker-daemon mentions that it's possible to use docker deamon outside of build container, but fails to explain how to achieve that.

My question would be, if achieving this setup is even possible with self hosted runner in docker, and if so, how to implement that.

[Mark C]

Hi @Konrad Stolarczyk

Welcome to the community.

This is actually possible by mounting your environment's Docker daemon via docker.sock. You can check this link with an example: https://support.atlassian.com/bitbucket-cloud/docs/use-your-docker-images-in-self-hosted-runners/

Regards,
Mark C

[Konrad Stolarczyk]

Hi @Mark C 

Thank you for a quick reply!

I'm aware that I can bind the docker.sock to a self hosted runner by adding "-v /var/run/docker.sock:/var/run/docker.sock" when starting the runner.

The issue is, the runner as docker container doesn't actually execute the pipeline steps itself, it starts another "build" container to do it.
In another words, when pipeline is being executed, I have two distinct containers running (actually 4, as there's also _system_auth-proxy and _pause, but they're irrelevant to the problem):

1. runner_RUNNER_UUID based on image docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner, that I've started myself with docker.sock mount

2. RUNNER_UUID_<random_hash>_build based on default image atlassian/default-image:latest that was started by the runner to execute the pipeline step, that doesn't have the docker.sock mounted.

So going back to my original question, is it possible to "tell" the runner to start the build container with mounted docker.sock?
Maybe there's a way to enforce the runner not to schedule another build container, but rather execute the pipeline step by itself?

And if not, are there any other options to attach host's Docker daemon to environment/container in which pipeline step is being executed?

Big thanks for your help!
Regrads,
Konrad S