Forums

Articles
Create
cancel
Showing results for 
Search instead for 
Did you mean: 

Support incremental caching

Ben Scholzen March 26, 2025

We are currently using quite a few hacks to make caching for Rust projects work, but there are certain limitations.

Mtime
Rust relies on the `mtime` within the target folder. We currently work around this by creating a tar of the target folder and cache that, and on restore unpack it again. Bitbucket should really preserve the original mtime from caches.

Dependency updates
This is the real issue. We are basing our cache on the `Cargo.lock` file. This works generally okay, build dependencies are only recompiled when we change any dependency. The problem here is that when only one dependency is updated, Rust has to recompile all dependencies.

Ideally we'd have an option to update caches incrementally. What the Github rust-ache action (https://github.com/Swatinem/rust-cache) does is to try to restore from the last Cargo.lock version, uses potential old build artifacts, and then updates the cache.

Additionally, the Github action does not only take the Cargo.lock file into account, but also the rustc version, other toolchains and a few other things. This could mostly be covered by taking the Bitbucket pipeline image into account, so switching to a different image would invalidate the cache completely.

Incremental caches would also allow to cache artifacts from local crates, so if multiple binaries are compiled, only changed ones would have to be processed.

1 answer

0 votes
Mark C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 28, 2025

Hi @Ben Scholzen,

Welcome to the community.

For the Mtime issue, we do have an existing enhance request here: https://jira.atlassian.com/browse/BCLOUD-19760
Although it is closed at the moment, I'd encourage you to add your comments there, as enhancement requests, even if they're closed, are regularly reviewed by our internal team.

For the caching dependency, would you be able to confirm if you're currently Caching with file-based cache keys

Regards,
Mark C

Ben Scholzen March 28, 2025

Yes, I am caching with file based cache keys, specifically Cargo.lock.

Mark C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 30, 2025

Hi @Ben Scholzen

Thanks for that.

In this case, would like me to create a feature request for Pipelines to support incremental caching?

Regards,
Mark C

Ben Scholzen March 31, 2025

That'd be great, yes! :)

Mark C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
April 7, 2025

Hi @Ben Scholzen

Apologies for the late response.

I created a feature request for supporting incremental cache in Bitbucket Cloud Pipelines. You can find the feature request here: https://jira.atlassian.com/browse/BCLOUD-23656
In the meantime, please feel free to upvote and watch the existing feature request to let you know of any updates. Our policy on implementing new features is here.

Regards,
Mark C

 

Suggest an answer

Log in or Sign up to answer
DEPLOYMENT TYPE
CLOUD
TAGS
AUG Leaders

Atlassian Community Events