How do you deploy from Pipelines to a remote Docker Swarm cluster ?

[Sandeep Srinivasa]

hi,

I'm trying to get a Bitbucket Pipelines CD setup with a Docker Swarm cluster.

I messaged Geoff on Twitter and he suggested I post here - https://twitter.com/realfakegeoff/status/943729735703576577

You guys have deployment documentation for Kubernetes - https://confluence.atlassian.com/bitbucket/deploy-to-kubernetes-892623297.html

But there is no equivalent documentation for Docker Swarm. Please add documentation for deploys to docker swarm.

[Pavel G]

Here are more details how to use Portainer webhooks for updating Docker swarm service from Bitbucket pipeline:

1.  get docker service webhook url: https://appsaloon.be/blog/service-webhooks-in-portainer/
 url will be in https://portainer.url.com/api/webhooks/TOKEN format

2. store TOKEN to a PORTAINER_WEBHOOK_TOKEN secret variable: https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/

3. 
- step:
  name: Publish app to Docker Hub
  trigger: automatic
  script:
  - docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PWD
  - docker build -t XXX:$BUILD_VERSION_APP .
  - docker push XXX:$BUILD_VERSION_APP
- step:
  name: Deploy to staging
  deployment: staging
  trigger: manual
  script:
  - wget --server-response --method POST "https://portainer.url.com/api/webhooks/$PORTAINER_WEBHOOK_TOKEN?tag=$BUILD_VERSION_APP"

[Phagedo]

Hi! We are using webhooks as well. As last step in our bitbucket pipeline I trigger a webhook provided from  swarm management tool portainer which is deployed in our swarm cluster with a simple ‚curl -C WEBHOOKURL‘. Portainer fetch the images from a configured registry and make a rolling restart of the images within swarm. Works very smooth. Cheers, phillip

[Sandeep Srinivasa]

@wiesson that is superb! Thanks for taking the effort. Your recipe is something on the lines that we want to follow. 

However SSH is not an option - we want this to run on every commit/merge. 

Looking forward to see this working on Swarm!

[Philip Hodder]

Hi @Sandeep Srinivasa,

Why is SSH not an option? Is it acceptable if SSH is run on every commit/merge as part of the pipeline script?

We have some documentation on setting up SSH commands inside of you pipeline here: https://confluence.atlassian.com/bitbucket/use-ssh-keys-in-bitbucket-pipelines-847452940.html

Does that help at all?

Thanks,

Phil

[Sandeep Srinivasa]

Thanks for replying.

For the same reason why kubectl tries not to use it..and every platform provides a SSL/TLS API.

Being able to physically log in to a server is a different security envelope with different considerations. For example, once you are on the server - you can attempt Spectre/Meltdown. You cannot do that with remotely issued docker commands over TLS.

In general, we would strongly request for a transport level security over being able to SSH.

IMHO both k8s and swarm have it - it's just a question of how to set it up with your infrastructure.

[Trent Murray]

Did this ever get anywhere, @Sandeep Srinivasa - did you manage to find a solution?

[Sandeep Srinivasa]

Nope.

If you do - please do post here.

[peksipatongeis]

Any updates on this? Would be really nice to have a proper way of doing Docker deployments to remote hosts. 

[tomasras]

+1

[peksipatongeis]

I was hoping I could execute the `docker stack deploy` remotely from Pipelines but I guess it's not going to happen due to how the remote host connection is set up with Docker. 

Ended up setting up a pipeline where docker-compose.yml is copied to remote host and running commands on the remote host as well. 

[tomasras]

We could try something like this : 

https://medium.com/@iaincollins/docker-swarm-automated-deployment-cb477767dfcf

[Philip Hodder]

Hi Sandeep,

I'm not fully sure what Geoff had in mind as a workaround here. He's on leave until next week so I can't ask him until then. As far as I know this isn't possible as Swarm commands aren't available at the moment. But, I'll check with Geoff when he's back.

Thanks,

Phil

[Sandeep Srinivasa]

> Swarm commands aren't available at the moment.

well I guess that was the point for this ! Swarm is the second biggest deployment infra after kubernetes and morever is bundled inside Docker itself.

We would appreciate Bitbucket Pipelines team to tell us how to do this effectively. Swarm already comes with TLS security - https://docs.docker.com/engine/security/https/ 

but unfair to only support kubernetes  :(

[Philip Hodder]

Hi Sandeep,

You can follow this guide to deploy to your Swarm cluster: https://docs.docker.com/swarm/swarm_at_scale/deploy-app/

You will also need to update the DOCKER_CERT_PATH to point at the API keys of your Swarm cluster, mentioned here: https://docs.docker.com/engine/reference/commandline/cli/#description

I don't have an ETA on when we may have a similar guide on Swarm in our own documentation. However, those links should suffice for you by translating the appropriate commands into commands in your deploy step script.

Thanks,

Phil

[Sandeep Srinivasa]

couldnt figure out how to do it with pipelines. maybe im making a stupid mistake.

I pray that you guys do get around to it /hopeful

[wiesson]

stage: deploy
 variables:
   DOCKER_HOST: tcp://XXXX:2376
   DOCKER_TLS_VERIFY: 1
   SERVICE_NAME: barfbarf
 image: docker:latest
 script:
   - mkdir -p ~/.docker
   - echo "$TLSCACERT" > ~/.docker/ca.pem
   - echo "$TLSCERT" > ~/.docker/cert.pem
   - echo "$TLSKEY" > ~/.docker/key.pem
   - docker login -u gitlab-ci-token -p "$CI_BUILD_TOKEN" "$CI_REGISTRY"
   - docker service update --detach=false --with-registry-auth --image registry.gitlab.com/barfbarf/$CI_PROJECT_NAME:$CI_BUILD_REF barfbarf
 environment:
   name: production
   url: https://barfbarf.com
 only:
   - master

Basically it works like this code snipped that I posted above (gitlab pipeline). I'm currently trying to "translate" it to bitbucket.

[wiesson]

What you can do - SSH to one of your managers and run those commands

docker service update ...