How to deploy docker container on host machine using bitbucket self hosted runner?

[Hans]

Hi community,

I asked this question before but somehow my post got flagged as 'In moderation' and was not visible anywhere. Sorry if I duplicate the question by posting this again. Anyway, my question is:

I am trying to set up a bitbucket runner to run on our on-premise machine. We would like to use the runner to deploy a container on the host machine. 

I deploy the runner in docker using the provided command:

docker container run -it -d \
-v /tmp:/tmp \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
-e ACCOUNT_UUID={<snip>} \
-e RUNNER_UUID={<snip>} \
-e RUNTIME_PREREQUISITES_ENABLED=true \
-e OAUTH_CLIENT_ID=<snip> \
-e OAUTH_CLIENT_SECRET=<snip> \
-e WORKING_DIRECTORY=/tmp \
--name bitbucket-runner docker-public.packages.atlassian.com/sox/atlassian/bitbucket-pipelines-runner:1 

Since the docker.sock is mounted I assume I can use the socket to communicate to docker on the host machine?

However, the following pipeline does not work:

definitions:
  steps:
    - step: &deploy-container
        name: Setup workdir
        runs-on:
          - 'runner-test'
          - 'self.hosted'
     
        script:
          - docker build -t test-image -f ./deploy/Dockerfile .
          - docker stop test-image || true
          - docker rm test-image || true
          - docker run test-image

pipelines:
  custom:
    DEPLOY:
      - step: *deploy-container

The above pipeline results in an error when executing the pipeline: bash: docker: command not found

Docker is not installed inside of the runner container. Adding service: docker to the pipeline:

definitions:
  steps:
    - step: &deploy-container
        name: Setup workdir
        runs-on:
          - 'runner-test'
          - 'self.hosted'
        services:
          - docker
        script:
          - docker build -t test-image -f ./deploy/Dockerfile .
          - docker stop test-image || true
          - docker rm test-image || true
          - docker run test-image

pipelines:
  custom:
    DEPLOY:
      - step: *deploy-container

does work. I believe it spins up a second container which does have docker available, and all the docker commands are executed just fine. However, the container is run inside of the runners container, and not on the host machine.

Now here is where I get stuck: how can I adjust the above pipeline to run the container on the host machine? I have tried adding:

script:
  - export DOCKER_HOST='unix:///var/run/docker.sock'
  - docker build .. 

to the script portion of the pipeline, but this only results in: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

The docker daemon is running on the host machine, so what am I doing wrong here? What is the correct way of allowing the bitbucket runner to run a container using the host docker? Or is this not possible at all with the dockerised runner, and should I be using the Linux shell runner instead?

Thanks in advance for any help you can provide!

[Theodora Boudale · Accepted solution]

Hi @Hans and welcome to the community!

When you use a Linux Docker Runner, the pipelines builds that you trigger do not run on the runner's container. For every step that runs on a runner, another Docker container (the build container) starts based on the image you specify in your bitbucket-pipelines.yml file:

• https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/

If you don't specify an image, then the DockerHub image atlassian/default-image:latest will be used. When the commands of the step's script finish successfully or if one of them fails, then the build container gets destroyed.

If you want to use Docker commands during a step, you will need to use a Docker service in your bitbucket-pipelines.yml file for that step. It is not possible to use the host's Docker.

You could use a Linux Shell Runner instead. In this case, the build will not run in a Docker container, but directly on the host machine. In order to use the host machine's Docker with a Linux Shell Runner, you will need to add the following command in your bitbucket-pipelines.yml file, at the beginning of the script that uses Docker:

- export DOCKER_HOST=""

This is because we are setting the variable DOCKER_HOST to tcp://localhost:2375, which can lead to issues.

If you use a Linux Shell Runner, please keep in mind the following (I copied from our documentation):

Linux Shell Runners use Bash to run pipeline steps on your Linux machine (host device). This allows the runner to execute applications on the host, but does not provide a clean build environment for every step. Any side effects generated by the step (such as, installing any applications, starting a database service, or editing a file outside of the build directory) would potentially affect the next step to be run (including new pipeline runs). To compensate for this, the runner try to empty the build directory empty after each step. It is your responsibility to make sure the scripts you run in each step won’t have a major impact on other steps.

Please feel free to reach out if you have any questions!

Kind regards,
Theodora

[Hans]

Hi Theodora, 

Thank you for the answer! I kind of expected this seeing as how difficult it looked to implement what I wanted to do. For my specific use-case I will try running the bitbucket shell runner instead.

Kind regards,

Hans

[Theodora Boudale]

Hi Hans,

You are very welcome! I just realize I didn't include the documentation for Linux Shell Runners in my previous reply, this is the link if you want to take a look:

• https://support.atlassian.com/bitbucket-cloud/docs/set-up-runners-for-linux-shell/

If you have any other questions, please feel free to reach out!

Kind regards,
Theodora

[Alex Harper]

Hi @Theodora Boudale 

I stumbled on this post when troubleshooting my own runner setup. Can you explain this part?

>If you want to use Docker commands during a step, you will need to use a Docker service in your bitbucket-pipelines.yml file for that step. It is not possible to use the host's Docker.

[Theodora Boudale]

Hi Alex,

If you use a Linux Docker runner, the docker service can be specified as follows:

pipelines:
  default:
    - step:
        runs-on:
          - self.hosted
        services:
          - docker
        script:
          - docker version

The relevant lines are

services:
  - docker

It is also possible to use a custom dind image with a Linux Docker runner, see the documentation below:

• https://support.atlassian.com/bitbucket-cloud/docs/configure-your-runner-in-bitbucket-pipelines-yml/#Custom-docker-in-docker-image

If you need further assistance troubleshooting a problem you have with your runner, please create a new question via https://community.atlassian.com/t5/forums/postpage/board-id/bitbucket-questions with more details about your setup and issue.

Kind regards,
Theodora