Docker pull takes a long time

[Stevedan Ogochukwu Omodolor Omodia]

I notice that docker pull takes a long time when running in pipelines. Is there a way to cache the latest docker image and use it as cache when docker pulling?

[Theodora Boudale · Accepted solution]

Hi Stevedan and welcome to the community!

Can you please clarify which of the following two you are referring to?

(1) The build container's Docker image that is being pulled at the beginning of every Pipelines step
(2) Pulling an image during a Pipelines build with a docker pull image command

If you're talking about (1), we cache public Docker Hub images. If you use a public Docker Hub image as a build container, it's going to be pulled from our cache. We do not cache private Docker Hub images or images from other registries at the moment.

If you're talking about (2), you can cache the image you pull during your build. Please keep in mind though that there is a bug with the Docker cache at the moment and it is not being consistently used. This means that the cache may sometimes be used but other times not.

If you'd like to try it, you can adjust your step as follows:

- step:
    script:
      - docker image pull <your image here>
    services:
      - docker
    caches:
      - docker

If you try this, please do check if the docker cache is being used in your builds and if there's a benefit; otherwise, you may want to remove the docker cache until the bug is fixed, so that your builds won't spend time uploading and downloading a cache that isn't used.

Please feel free to reach out if you have any questions!

Kind regards,
Theodora

[Stevedan Ogochukwu Omodolor Omodia]

isnt there a limit in caching size?

[Theodora Boudale]

Hi Stevedan,

If we're talking about a Docker image that is cached during the build, like in the example I shared above in point (2), then yes, there is a limit. Only caches under 1 GB once compressed are saved. We compress the Docker image before uploading it to cache, and the size of the compressed image needs to be under 1 GB; otherwise, it won't be uploaded.

Kind regards,
Theodora

[Stevedan Ogochukwu Omodolor Omodia]

My docker image is larger, so this might not work for me. Is there a chance that we can get higher cache size?

[Theodora Boudale]

Hi Stevedan,

I'm afraid that it is not possible to increase the cache size limit.

Please keep in mind that 1 GB is for the compressed image. A Docker image with a size < 2 GB should be able to compress to less than 1 GB. If it doesn't, you will see a message in the Pipelines build log, in the last section titled Build teardown.

If the compressed image is larger than 1 GB and doesn't get uploaded to the cache, another option would be to use Runtime v3 for that Pipelines step and use Buildx caching with an external cache. This is the documentation for Runtime v3 and buildx caching:

• https://support.atlassian.com/bitbucket-cloud/docs/enable-and-use-runtime-v3/
• https://docs.docker.com/build/cache/optimize/

Please keep in mind the following:

• If you use Runtime v3 for a Pipelines step, the Docker CLI is no longer automatically mounted in the build container. You will need to either use a Docker image that has the Docker CLI  as a build container for this Pipelines step or install the Docker CLI during the Pipelines step.
• The Pipelines Docker cache is deprecated in Runtime v3, which is why we recommend Buildx caching.

Please feel free to let me know if you have any questions.

Kind regards,
Theodora

[Stevedan Ogochukwu Omodolor Omodia]

OK thanks

[Theodora Boudale]

You are very welcome. Please feel free to reach out if you ever need anything else!