docker build in pipeline differs from local build

[Ivars Tiltiņš]

I am building a php docker container that uses a Dockerfile.dockerignore file to only allow php files inside the container. The dockerignore file contains only two lines:

project/**/*
!*/**/*.php

Inside the Dockerfile itself, it installs some php extensions, and copies the php.ini as well as ./project to . . In the pipeline everything is built from the root directory, using docker build . -f "path/to/Dockerfile".

On the local machine, the image builds fine, and includes all expected files. But when build on the Bitbucket Pipeline, it only includes the php files in the root directory of the project.

On locally built:

$ docker run -it org/project bash 
root@69afc4123684:/var/www/html# ls 
PHPMailer cameraAssign cron favicon.php forgot home inc.php index.php login logout recover signup

In Bitbucket pipeline built:

$ docker run -it org/project bash
root@e1557e7def2b:/var/www/html# ls
favicon.php  inc.php  index.php

As you can see, the container built in the Bitbucket pipeline is missing all directories that should be in the base directory.

I have attempted different methods of running the docker build command, but have not found what could cause this discrepancy. Should I attempt to use docker buildx? Or is there some other way to get this to build correctly?

[Theodora Boudale · Accepted solution]

Hi Ivars and welcome to the community!

I would like to ask some questions so I can try to reproduce this and understand what is happening:

1. Could you please provide the COPY commands you have in your Dockerfile?

2. You provided the output of ls when the image is built locally. You mentioned that some directories listed there are missing when building with Pipelines (I assume PHPMailer, cameraAssign, cron, etc are directories). Do these directories exist at the root of your repo's source code? Or in another directory of your repo's source code?

3. Are you using Bitbucket Pipelines on Atlassian's infrastructure, or with one of our self-hosted runners?

4. In case you use a self-hosted runner, is it a Linux Docker, Linux Shell, MacOS or Windows runner?

• If it is a Linux Docker runner, are you using a custom dind image, and if so, which one?
• If it is a Linux Shell, MacOS or Windows runner, what is the version of Docker client and engine on the host machine of the runner?

You mentioned something about buildx, so I wanted to add that buildx is not currently supported in Pipelines builds running on Atlassian's infrastructure. You can find more info on restricted commands here:

• https://support.atlassian.com/bitbucket-cloud/docs/run-docker-commands-in-bitbucket-pipelines/

Kind regards,
Theodora

[Ivars Tiltiņš]

Hi Theodora,

Thank you for the reply, here's the answers to your questions:

1. WORKDIR /var/www/html
   
   COPY ./containers/project-php/php.ini /usr/local/etc/php/php.ini
   COPY ./project .

2. The additional entries in the output of ls are indeed directories, and they  exist in the project's directory.
3. I am using the Bitbucket Pipelines on Atlassian's infrastructure.

The image in use in the pipeline is atlassian/default-image:4

Here's also the output of ls from the project's root directory. Directories such as assets/ should be excluded from the image, as they do not contain any .php files. This is to reduce the total size of the image, as a lot of the non-php files are large images.

$ ls -p portal
assets/ fa/ home/ invite/ manifest.json test.htm
browserconfig.xml favicon.ico images/ jquery/ PHPMailer/ test.json
cameraAssign/ favicon.php inc.php login/ recover/
cron/ forgot/ index.php logout/ signup/

[Theodora Boudale]

Hi Ivars,

Thank you for the info. I'm a bit confused with the ls -p portal command.

Does the directory structure in your repo look like this?

Or is there a directory named portal under the project directory, where assets, home, etc exist?

I have been able to reproduce this with a repo that has directory structure like in the screenshot. Please let me know if I got something wrong in the directory structure.

I suspect (but I am not 100% sure) that the ignore patterns may behave differently with different versions of Docker.

Could you please let me know:

• Are you reproducing locally on Linux, Windows, or MacOS?
• What version of Docker Engine and Docker client do you have on this machine? You can find that out if you execute the command docker version on a terminal application.

Kind regards,
Theodora

[Ivars Tiltiņš]

Hi Theodora,

Sorry about the confusion about the ls -p portal command, I have multiple projects using basically the same dockerfiles but with different paths, and was simplifying to just 'project'. I guess I forgot to change that there, was supposed to be ls -p project 😅

The screenshot provided looks like the repository structure I have!

I am building locally on a Linux machine, with Docker engine and client both at version 27.3.1.

I hope there's a simple workaround or fix to the ignore pattern discrepancy on the version of Docker that's in the Atlassian pipelienes...

[Theodora Boudale]

Hi Ivars,

Thank you for the info!

I have been able to reproduce the issue on a Linux machine (outside Pipelines) with Docker client version 20.10.24 and Docker engine version 25.0.2, which is what Pipelines currently supports. When I used version 27.3.1 then the directories with the php files were included.

I'm afraid that we don't support Docker clients greater than v20.10.24 or different versions of the Docker Engine for builds running on Atlassian's infrastructure.

You could use one of our self-hosted runners on a server of yours:

• https://support.atlassian.com/bitbucket-cloud/docs/runners/

If you use a Linux Shell runner, then the builds are going to run directly on your host machine and they will use the Docker Engine and client on that machine. You will just need to add the following command in the yml file of a step that uses a Linux Shell runner in order to be able to use Docker:

- DOCKER_HOST=""

Kind regards,
Theodora

[Ivars Tiltiņš]

Hi Theodora,

Thank you for the help!

If I understand correctly, I will have to use a self-hosted Linux Shell runner to be able to build the images without the issue. Is that right?

Also, do you know if there are any plans in the near future for Atlassian Pipelines to support Docker version 27.3.1?

[Theodora Boudale]

Hi Ivars,

Yes, since newer versions of the Docker client are not currently supported, a Linux Shell runner is the best option considering the patterns in your .dockerignore. A Windows or MacOS self-hosted runner may also work, if you have a Docker installation that matches the patterns the same way as your Linux machine.

With the Linux Docker runner you can have a newer version of the Docker Engine if you specify a custom dind image, but the Docker client is the same (v20.10.24). I tried manually installing a newer version of the Docker client during the build, but this lead to other issues (my .dockerignore file was not being read).

I don't know when we are going to release a newer version of Docker. We post updates on Pipelines infrastructure changes here:

• https://confluence.atlassian.com/bbkb/infrastructure-changes-in-bitbucket-pipelines-1142431176.html

This includes updates to the Docker Engine and also the Docker client. So, when there is such a release, it is going to be posted there.

Kind regards,
Theodora