How do I clean up workspace size? Which repos are too big?

[makeitTim]

One of my older workspaces is 1.7 GB. I'm not actively using much, but I don't want it locked. There are a couple dozen repos going back over a decade, some are C++ and Java, probably have things committed that shouldn't be. I'm sure it could be cleaned. How? All Bitbucket tells me is the overall workspace size.

How do I find the big files?

How do I clean them out?

Thanks for any help.

[Ben · Accepted solution]

Hi @makeitTim 

There is currently no feature in Bitbucket Cloud to show you all repositories by size:

• We have a feature request raised to make this possible - please Watch it for future updates and Vote for it to improve its visibility with regard to customer demand:
  https://jira.atlassian.com/browse/BCLOUD-22153
• There is a workaround detailed in the above feature request using a Forge app - instructions are here (note: we do not provide support for Forge):
  https://community.atlassian.com/forums/Forge-for-Bitbucket-Cloud/Instantly-Access-and-Share-Repository-Size-Data-Across-All/ba-p/2877076
• Alternatively, there is an unofficial python script you may use (this is not supported by Atlassian) to list all repositories by size:
  https://bitbucket.org/bbcloud-support-scripts/get_repository_details/src/master/

In terms of figuring out what the large file sizes are, you may use this command against the large repository - it will sort the largest files to the top:

• git rev-list --objects --all \
  | git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
  | awk '/^blob/ {print substr($0,6)}' \
  | sort -r --numeric-sort --key=2 \
  | numfmt --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

Once you have a list of those large files, you may reduce the repository size by removing those large files or converting them to GIT LFS:

• https://support.atlassian.com/bitbucket-cloud/kb/how-to-remove-files-from-the-commit-history-in-bitbucket-cloud/
• https://support.atlassian.com/bitbucket-cloud/docs/use-bfg-to-migrate-a-repo-to-git-lfs/

To specifically remove files above 1mb, you can make use of the same BFG tool described in the links above to do so - instructions and a sample command for this are below:

• https://rtyley.github.io/bfg-repo-cleaner/
• bfg --strip-blobs-bigger-than 1M my-repo.git

Please note that any cleanup operations create garbage/dangling commits, which will inflate the size. We need to run a server-side gc from our side to complete the process, so please let me know when you're ready to do so.

Cheers!
- Ben (Bitbucket Cloud Support)

[makeitTim]

Is there a way to easily just backup a repo to a Google Cloud bucket, Cloud Build already does stuff like that?

...

This page under a Workspace DOES seem to give the repo sizes. The feature already exists?

/{workspacename}/workspace/projects/PROJ

Google Cloud Source Repository allows you to easily mirror a repo from Bitbucket (or github). But it's not meant to be the "source of truth," so deleting the underlying repo afterwards doesn't look like maintainable even though I think the files are mirrored on google. I'll backup stuff somewhere else as well.

[Ben]

Hi @makeitTim 

You could always just go to Repository Settings > General for each of your 16 repositories to check each individual size, albeit this would be a bit time-consuming. 

To maintain confidentiality/privacy - I can reveal your largest repositories (but asterix them out to not reveal their full names in public):

• Hyp***** (533.6MB)

• Triv-***** (355.7MB)

• Proj**** (327.1MB)

• Core******** (249.5MB)

• Alf********** (141.6MB)

• Tr** (134.5MB)

There is not a function to backup in this manner, if you want to get a copy of your repositories - I'd suggest cloning them locally and uploading these to a storage service.

Cheers!

- Ben (Bitbucket Cloud Support)

[makeitTim]

Yeah, that's them. The large files are image and music assets that I do not want to delete. I'll backup elsewhere in Google and somewhere else, and migrate off Bitbucket.

[makeitTim]

As far as I can tell, the only big files are various image assets an original .PSD image files stored in the repo. They are all over old C++, Android, and iOS projects which had mipmap raster approaches. Finding and cleaning them out doesn't look manageable?

Is there a way to move all the image assets over 1 MB to LFS? What's the approach here?