401 Unauthorized in Docker Build which does GO MOD DOWNLOAD from Private Workspace Repository

[Banani Karma]

I have a bitbucket (cloud) Workspace, Repository ( private ). Have some Golang code under such a repository.

I am trying to Dockerise a simple Golang app which needs to download code (dependent) from the private bitbucket repo. 

I have added SSH keys, generate App passwords etc. Tried couple of ways, but have been unlucky for "go mod download" step to work in my dockerfile.

docker build error:

go mod download: reading https://api.bitbucket.org/2.0/repositories/MyWorkspace/MyRepo?fields=scm: 401 Unauthorized

Anyone faced a similar situation by any chance? Any help, pointers will be appreciated. Thanks. 

[Patrik S]

Hello @Banani Karma

Welcome to Atlassian Community!

I'm not that familiar with go lang, but from the logs you've shared with us, it looks that behind the scenes your "go mod download" is calling the Bitbucket API to get some information about your repository. However, from the error message "401 Unauthorized" it seems that this API call is not using valid credentials when authenticating.

To authenticate to Bitbucket API you can use your bitbucket username and an AppPassword. Following is an example API call using username and password :

curl -X GET -u bb_username:app_password 'https://api.bitbucket.org/2.0/repositories/<workspace>/<repo_slug>'

That said, would you please check on your script which credentials are being used by go mod command ?

If after fixing the credentials with the above suggestion it still doesn't work, we kindly ask you to share what commands are being executed in the background by the "go mod download" command.

Let us know if you have any questions.

Thanks, @Banani Karma.

Patrik S

[Banani Karma]

Hello Patrik,

Thanks much for your reply / support / help on this. I did another trial with a slightly different and simpler approach than past trials.

----------------------

Basically,

- copying the PRIVATE KEY into the image ( multi stage build )

- starting the ssh-agent 

- adding the key to the agent

before GO MOD DOWNLOAD which tries to pull code from BB private Repo. 

----------------------------

This time, error message is little different:

go mod download: reading https://api.bitbucket.org/2.0/repositories/MyWorkspace/MyRepo?fields=scm: 403 Forbidden  server response: Access denied. You must have write or admin access.

------------------

"You must have write or admin access."

Dockerfile steps:

------------------------------

FROM golang:1.17-buster as builder

WORKDIR /app

COPY go.* ./

// copy private ssh key

COPY id_rsa .

RUN eval $(ssh-agent) && \
ssh-add id_rsa && \
ssh-keyscan -H bitbucket.org >> /etc/ssh/ssh_known_hosts

RUN go mod download

...

* I am able to pull / download code from direct command prompt ( which uses the pub / private keys related to ssh from my "/home/.ssh" directory or ssh-add on command prompt ). But i wanted to DOCKERISE my app/code.

Hope you get more clues / knowledge of what i am doing.

 Thank you for your help on this again!!

-Banani

[Patrik S]

Hello @Banani Karma ,

After doing some research for similar cases, I found the fix for the error you are facing is usually the following :

1. Edit git.config

git config --global url."git@bitbucket.org:".insteadOf "https://bitbucket.org/"

This will force SSH to be used, instead of HTTPS.

2. Set the GOPRIVATE environment variable

ENV GOPRIVATE="bitbucket.org/<your workspace>"

 Sources :

• https://community.atlassian.com/t5/Bitbucket-questions/How-do-I-run-go-get-on-a-private-repo/qaq-p/1385040
• https://stackoverflow.com/questions/63791976/go-get-private-bitbucket-repo-giving-403-forbidden

Hope that helps. Do let us know in case you have any questions.

Thanks, @Banani Karma .

Patrik S

[Banani Karma]

Thank you for the update Patrik.

As mentioned, i do not have issues if i am doing from the HOST Machine.

The issue is specifically wrt Dockerising / when i want to build an image/container of the Go App.

This means somehow being able to do the steps like you mentioned in the context of Docker Build. Thats where things are giving issues and doing Trial & Error.

** If you get something specific to doing GO GET or GO MOD DOWNLOAD from a Dockerfile / Docker Build, kindly share. Still unlucky DOCKERISING from BB and even GitHub.  

Thanks much Patrik.

[Patrik S]

Hello @Banani Karma ,

Have you tried adding those commands to your Dockerfile, before the GO MOD download command ?

The Docker file would look like the following :

ROM golang:1.17-buster as builder

WORKDIR /app

COPY go.* ./

// copy private ssh key

COPY id_rsa .

RUN eval $(ssh-agent) && \
ssh-add id_rsa && \
ssh-keyscan -H bitbucket.org >> /etc/ssh/ssh_known_hosts

RUN git config --global url."git@bitbucket.org:".insteadOf "https://bitbucket.org/"

ENV GOPRIVATE="bitbucket.org/<your workspace>"

RUN go mod download

Please make sure to replace the value of the variable GOPRIVATE above with your correct workspace name.

Thanks @Banani Karma .

Patrik S

[Banani Karma]

Patrik,

Appreciate your continued follow-up and help on this.

Well the issue was with making SSH thing happen rightly / as desired while in the  Docker Build. Did some mix/match, trial & error of 2 approaches i referred / got from online search. And even GO GET each repo vs 1 GO MOD DOWNLOAD.

And yes, it worked in a different machine where only 1 git account and SSH key is set up.

Still need to have this work in the first machine where multiple git accounts and git providers (GH, BB) are used.

Will update the ticket finally, so that others may get help. This kind of things are sometimes not very logical or can be generalised - specific cases behave different (based on the dynamics of the env/context/machine etc.)

Thanks much for your suggestions and analysis. Will close the ticket after my next update.

Good day!

[Banani Karma]

Multiple things i had to Trial & Error for GO MOD DOWNLOAD to succeed with bitbucket.org  private repository:

Top 1

------

It seemed to me / in my trials, when i use a custom name to ssh key file (ssh-keygen) other than the default: id_rsa, it gives the access / permission errors!

** I got success when key name is "id_rsa" **  ( not with a custom name: strange )

Then, 2. SSH related rituals/steps in Dockerfile

----------------------------------------------

ADD id_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa

RUN eval $(ssh-agent) && \
ssh-add /root/.ssh/id_rsa && \
ssh-keyscan -H bitbucket.org >> /etc/ssh/ssh_known_hosts

RUN echo "Host bitbucket.org\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config

Lastly

------

RUN git config --global url.ssh://git@bitbucket.org/.insteadOf https://bitbucket.org/

RUN git config --global url."git@bitbucket.org:".insteadOf "https://bitbucket.org/"

ENV GOPRIVATE=bitbucket.org/MyWorkspace

before 

RUN go mod download in Dockerfile

----------

Bitbucket Team, can have a look at their end, if it works with ssh keys named other than default id_rsa / id_rsa_pub. In my trials, it didn't work with a different name. Not sure if it's some other issue which got interpreted this way (key name issue).

Thanks much Patrik! I am good now.