What does the announcement about Datacenter going away mean for Bamboo?

[Martin Winter]

As Atlassian now announced that the Datacenter products go away, I'm curious what that means for Bamboo. Does this mean an end for Bamboo ? Or is there a new Cloud version coming for Bamboo?

I doubt that we are ever able to move to a cloud version of something like a bamboo. Our Bamboo instance grew massive - too massive to ever make sense for a cloud and all the agents would have to be on our network anyway as they run on very specific OS and hardware.

Unfortunatly, no mention of Bamboo on the recent announcement... Would love some clarification

[Shashank Kumar · Accepted solution]

Hello Martin,

Welcome to Atlassian community.

Unlike other Atlassian products, Bamboo does not have a cloud equivalent. For customers seeking cloud-based CI/CD, we recommend evaluating Bitbucket Pipelines for code build and deployment needs. We encourage you to analyze the feasibility of migrating your workflows to Bitbucket Pipelines to ensure a smooth transition.

Regards,

Shashank Kumar

[Martin Winter]

Can you clarify your response?

Does this mean that Bamboo is going away and Bitbucket Pipelines is the only Atlassian product for CI/CD in the future or does that mean that Bamboo is (for now) not impacted by this announcement?

(To clarify, we currently use Bamboo Datacenter in a setup with > 1000 remote agents across all common Linux and *BSD enviroments with agents running on x86_64, i386, arm64, ppc64el, ppc32 and riscv. Also archiving around 100TB of artifacts as part of the Bamboo builds. I don't think anything close to this is possible with Bitbucket Pipelines)

[Shashank Kumar]

Hi Martin,

Yes you are correct, with this announcement Bamboo is also affected and in future Bitbucket pipelines is the only solution available for CI/CD from Atlassian.

However there are few exceptions to it and to know further I would kindly request you to raise request at https://support.atlassian.com/contact/ where you can speak to our support to look at your available options. Kindly use the Bamboo SEN while raising the request. 

Regards,

Shashank Kumar

[Raul Gomis]

Hi Martin,

Thanks for reaching out and sharing your perspective. I’m the Head of Engineering for Bitbucket Pipelines and would love to learn more about your use cases and help.

We strongly recommend migrating to our Cloud version to eliminate the burden of managing, scaling and patching servers, a change that has brought significant benefits to our customers (including Atlassian's internal CI/CD processes) when migrating from Bamboo to Bitbucket Pipelines. Having all said, if you need more control and need to run behind the firewall, we also offer a self-hosted option with runners.

Bitbucket Pipelines powers all of Atlassian’s CI/CD automation for our most mission-critical workloads and products, even our large frontend monolith which is operated by over 1500 engineers on a daily basis. In the recent few years, we have made substantial investments in improving our product and expanding our infrastructure. As a result, for example, today Atlassian runs more than 1 billion build minutes per month in Bitbucket Pipelines, for over 9,000 internal monthly users.

Please reach out to our support team with your details. We would be happy to chat with you to understand more the details of your use cases.

Helpful reads:

• https://www.atlassian.com/blog/atlassian-engineering/how-we-reinvented-atlassian-cicd
• https://www.atlassian.com/blog/atlassian-engineering/how-we-migrated-complex-ci-cd-workflows-to-bitbucket-pipelines

[Martin Winter]

Raul,

happy to talk to you or the support team in detail, but here is a quick overview for you and everyone else interested:

We run the CI system for an open source community (FRRouting) on Github (and a few other related projects). The code we develop is mostly in C and we use Bamboo to build it for various platforms and test on them. As we work on a Linux/*BSD portable project which is used from small (home systems) to large cloud and hardware platforms, we build and test mostly in full virtual machines and LXC containers on various hardware platforms. Mostly Intel 64bit and ARM 64 bit at this time, but also (to make sure we find any issues) on 32 bit Intel and ARM, PowerPC (pp64el on IBM Power8/9/10 Architecture and ppc32 as it's a different endian system) and RiscV (experimental at this time).

We also build on all the major Linux and BSDs and their respective current main supported versions (multiple Long Term Versions).

Same goes for testing, where we simulate large network topologies. Testing is all done in VMs - LXC, Docker etc won't work as the shared kernel will neither scale (Certain network operations are bound to specific CPUs on Linux) or we need a specific kernel versions. Some of the test use licensed software where the site license restricts us on the physical location on where it can be used (and it requires specific hardware/software to run)

As the project is all Open Source, anyone can push a PR to Github which will trigger the Bamboo CI. 

All this ends up in (currently) about 600 Bamboo Agents, controlling around 1000 VMs, MicroVMs and LXC containers. They are all triggered by Github Webhooks and a build/test runs for around 2..3 hrs for a single PullReq push. We could trivially speed it up by adding more agents, but we try to keep to find a useful balance between infrastructure needed ($) and runtime.

We also need to archive and keep most of the history online - currently we have around 100TB of artifacts online for the public to look at. At the same time - while keeping this online - we need to filter to keep all the webcrawlers and AI crawlers out of it (for bandwith and CPU reasons as artifact downloads take some toll) (AI companies seem to love our data...)

Our main Bamboo CI is currently a server with 768GB RAM, 64 cores and 100TB of NVMe storage (as compressed ZFS to save space for all the artifacts)

(Nearly) all the agents and servers are running on our own servers across multiple datacenters - with the high load we see, this is by far the most cost effective way and only way for a lot of the specialized architecture. It also allows us to control access to our infrastructure by allowing normal public access, but using things like Anubis to block AI crawlers and blocking some providers which are not responding to abuse complains.

As a Open Source Company and a US based non-profit, we currently enjoy the free license from Atlassian and we highly appreciate it. I truly believe that the Atlassian Bamboo is a fantastic system and we are using it for about 10 years now. 

Maintaining all the infrastructure is actually not a bit problem for us - a lot of it is automated (Ansible), but there is still work by constantly adding new platforms (setup is usually the simple part, but getting all the bugs found and fixed is the hard part - and a cloud won't help there)

In case you want to see the main CI in action: It's at https://ci1.netdef.org - as I mentioned before - most is public visible

I have to admit that I'm not too familiar with Bitbucket Pipelines, but I think the key challenges for us are:

• We use Github and not Bitbucket as webserver
• Running on our own Infrastructure is way cheaper than paying for the resources on Amazon or another cloud (a single agent usually has between 4 and 32 GB RAM and 4-12 CPU cores and we have on average around 50-400 of them running at any time). Same goes for main CI
• We have limited control on protecting the Website against crawlers on a "service" in difference to our own hardware
• Most agents would probably need to be continued to run in our own infrastructure as they can't be hosted for technical reasons (like no possibilty to do specific OS Distributions, Kernels, no MicroVM possible etc). This would also increase our bandwidth requirements (now all the artifacts may need to be sent remote - which is approx 4GB for each run)
• Be able to keep artifacts online over years back may not be possible or too expensive on a hosted platform compares to the our self-hosted version.

Anyway, as I said before - we are a very happy Bamboo User for many years and happy to discuss in detail with anyone at Atlassian.

[Brian Clair]

This decision to discontinue Bamboo Data Center is pretty catastrophic. We've setup most of our infrastructure on it and now will need to move back to Jenkins.  Apologies, but Pipelines is super difficult to use and debug - it's simply not a viable option.

[Martin Winter]

Same for us. Bitbucket Pipelines is no option and doesn't have the features we need. If this is the choice by Atlassian (and I'll reach out to them to discuss) then I have to move to something else.

But I have to say that I was pretty happy with Bamboo. Using it for a very long time and in general it just works as expected. Sad to see it go.