Docker pipeline cache goes stale

[moritz_matchory_com]

Recently, we started noticing changes not being present in our Docker images pushed to Docker Hub from a BitBucket pipeline. After clearing the pipeline cache, the new changes were picked up and properly pushed. I promptly disabled the cache.  

If I can't rely on the cache being updated correctly and our production deployments thus using stale code, the cache is actively harmful and I'm not going to enable it again! What is happening here?

[Theodora Boudale]

Hi @moritz_matchory_com and welcome to the community.

I'm afraid that, at the moment, the cache in pipelines doesn't get updated when the dependencies are updated.

A cache gets saved on a successful build (if the cache is empty), it gets stored for a week (unless manually cleared), and after it gets cleared a new cache is created on the next successful build. The feature to have it automatically updated while it is saved is not available yet.

We have a feature request for this in our issue tracker:

• https://jira.atlassian.com/browse/BCLOUD-16314

I would suggest that you add your vote there (by selecting the Vote for this issue link) as the number of votes helps the development team and product managers better understand the demand for new features. You are more than welcome to leave any feedback, and you can also add yourself as a watcher (by selecting the Start watching this issue link) if you'd like to get notified via email on updates.

Implementation of new features is done as per our policy here and any updates will be posted in the feature request.

Please feel free to let me know if you have any questions.

Kind regards,
Theodora

[Jason Harrison]

@Theodora BoudaleTo confirm,

• the cache is not cleared when the build fails. 
• the only time there is no cache is when
  • the pipeline has never been run before, or
  • if the cache has been manually cleared, or
  • the pipeline runs https://bitbucket.org/atlassian/bitbucket-clear-cache -- but there will only be no cache AFTER running that pipeline

[Caroline R]

Hi, @moritz_matchory_com!

You are right in your assessment here. The caches are not cleared if a build fails, that is true. You can find how a cache is saved, restored, and cleared here:

• https://support.atlassian.com/bitbucket-cloud/docs/cache-dependencies/

Also, you are right about the scenarios in which there is no cache as well. You can leverage the pipe you shared to clear the cache, as you mentioned, and it will only be cleared at the end of the pipe run. 

I hope this helps, but please let me know if you have any questions. 

Kind regards,
Caroline