how permanently wipe big files from master

[wesko]

I used git for years but I hardly need nor understand the complexity. I push and diff a few times per day and for me it is a backup and rollback mechanism. I work alone.

I got a warning that the repo was over 1GB. Fine, paid the $10 and I am safe. But I wondered how to shrink the online repo and found a lot of binary junk that can be removed forever (.class/.o etc.). Tried everything, scripts, jars, but whatever I do, the online repo more than doubled.

1. I am totally not interested in the history forever. So a wipe from a certain date would be nice.

2. I am not interested in big files. They can be wiped for good.

3. Anything else to shrink back to below 1GB?

How can I achieve 1/2/3?

[Ben · Accepted solution]

Hi @wesko 

After cleanup operations - it is expected that the repository size will increase temporarily, as cleanup operations create garbage (dangling commits) which consume space and must be cleared from the server-side. This is performed automatically once a certain garbage threshhold is met, or a support engineer can perform this manually.

I have attempted to reduce the size of your largest repositories by executing a garbage collection, but your workspace is still > 1GB.

• 1.2GB > 151.4MB

• 726.3MB > 220.3MB

• 711.4MB > 282.2MB

• 580.5MB > 570.0MB

• 339.7MB > 335.6MB

• 288.2MB > 225.5MB

I would suggest the following, given that you're on a Standard plan currently:

1. Identify the largest files in your repositories by executing the following command:
git rev-list --objects --all \
| git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \| awk '/^blob/ {print substr($0,6)}' \
| sort -r --numeric-sort --key=2 \
| numfmt --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

2. Perform cleanup operations locally to reduce the size

3. Push the cleaned repository back to the workspace, and let us know once you're done - so we can execute a server-side gc from our end to finalise the size reduction.

If we cannot resolve this in community - I will raise a ticket with our support team on your behalf as per your support entitlement. Please let me know your timezone so I can ensure to raise it with the team operating in your region.

Please let me know how this goes.

Cheers!
- Ben (Bitbucket Cloud Support)

[wesko]

Ben, thanks a lot for this customer-oriented answer, great. 

There's a small thing: I cannot seem to run the script at bullet 1. It claims "usage: git cat-file" is wrong.

[wesko]

I believe \| awk should be | awk \

[wesko]

The script worked, now I get a warning that the repo is 3.5GB, please run a git gc!

[Ben]

Hi @wesko 

I've just executed gc's against your repositories and these have been reduced significantly:

3.5GB > 228.8MB
1.5GB > 40.8MB
1.1GB > 783KB
399MB > 282.2MB
232MB > 770.9KB
222MB > 4.7MB

Your workspace is now under the 1GB limit at 596.7 MB.

Cheers!

- Ben (Bitbucket Cloud Support)