Build on bare metal remote agents vs builds in Docker vs remote agent and builds in Docker

[Jason Templeman]

Want to have a conversation regarding the best way to proceed for a growing on-prem solution.

Our group has 30+ Linux based remote agents that we use for building our source code.  We are having issues keep all the systems in sync as requirements change and the fact that these are not dedicated machines so developers and testers are using them as well.

We are in process of creating a Docker image so all systems will have the same environment but having issues getting Docker Runner (Isolated build running in Docker container) to work (even simply with a CentOS container - could use help here).  We have a very complex build and test system where we run QEMU kvm so that is controlled inside the container and controlled using a bamboo user (not root).

Now I'm being asked to have the remote agent also be in the Docker container so that the bare metal machine only needs Docker installed.  The Bamboo plan spins up a container making it available for the next job (guess we would need one remote agent on-line to begin with and a container for each our our remote agents as our agents are dedicated to our project on the server).  Not sure this is even possible and seems overly complected if it is even possible.  I don't like this solution and looking for a better one.

We are running on-prem with no support for using AWS elastic agents.  Our IT is looking at a possible on-prem kubernetes solution but that won't be till next year and getting pressured to move towards a unified environment solution.

Any guidance in getting Docker Runner working, managing the environment, etc. would be greatly appreciated as I find real world examples on the Internet lacking.

[Jeyanthan I]

Hey @Jason Templeman ,

I understand you are using Bamboo to run KVM images as a part of your build process and you are planning to dockerise this activity. Did I get it right?

If so, I just wanted to clarify that running a virtualised environment within a container might not work well and there can be challenges while setting up. I think it is best to run your KVM images is on a dedicated remote agent.

Cheers,
Jey