Approaches for upgrade automation with clustered, containerized Confluence?

[Connor]

Hello all, we are working on setting up a Confluence cluster in AWS using ECS containers. We have the environment set up, but now we're looking at upgrade path approaches for both minor and major version upgrades. It seems like all of Atlassian's documentation on clustering is written under the assumption that one is deploying on regular servers or VMs and will be upgrading in place.

I'm wondering if anyone in the community has experience with upgrade automation for a cluster of containers, both for zero-downtime upgrades and for major upgrades, who would be able to walk me through the process they are following.

Thank you!

[Charlie Misonne · Accepted solution]

Hi Connor,

I have no experience with ECS but I used to manage a clustered instance of Jira and Confluence on AKS.
I hope this can help you or put you on the right trck

One mistake I made back then is the usage of customer docker images and not the helm charts provided by Atlassian. They might not support you for any support request you raise. In any case, they will not support the technicalities of a containerized setup. So it's a risk you might take.

They have documentation about EKS on Available examples - Atlassian DC Helm Charts
I also looked at the official helm charts as inspiration to see which parameters and values they use, mainly for the health probes. atlassian/data-center-helm-charts at 3fd60fac57e97d36c23befce73591b5ea088bb7c
For ECS I can not find anything official.

For the flow:

1. Read upgrade notes. There might be breaking changes or a version where zero-downtime upgrades are not supported.
2. Backup everything (database, storage, ...)
3. Put Confluence in upgrade mode via the UI
4. Adapt the image tag to the new version
   In case of K8S it is enough to do this in the stateful set. Each pod will be upgraded 1 by 1 using the rolling update strategy.
   It is important to correctly configure the readiness probes for this! Your cluster should check the /status page of Confluence and only proceed with the next node once this returns an OK status.
   No idea how this works in ECS but there is probably something similar
5. Wait for the rolling update to finish
6. In the UI: finish the upgrade. You'll have a button to proceed to complete the upgrade once each node is on the same version. This will basically perform any necessary DB changes.

[Connor]

Thanks for your reply, Charlie! We have talked about EKS a couple of times during this project, but we opted not to go for it as we don't have much institutional experience with Kubernetes yet. The supportability question is definitely important, though, so I will definitely take that back for discussion.

I appreciate your help, this will definitely point us in the right direction. We'll start looking into the readiness probes and see how we can accomplish this with ECS. Thanks again!

[Charlie Misonne]

You're welcome!

Good luck with the setup. It's definitely an exciting project.