Dockerfile - Host key verification failed.

[William Bailey]

Hi all -

We are trying to build a CI Pipeline starting our application with webpack-dev-server through docker & testing with nightwatch.js after. 

Here is the current dockerfile & pipeline.yml.

----------------------------------------------------------------------------------

Dockerfile

FROM node:latest

MAINTAINER

#Set working directory in the container
WORKDIR /Users/xxxxxx/Repositories/xxxxxxxx

# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./

#Install local packages listed in package.json
RUN npm install

# Bundle app source
COPY . .

#Export port 8080
EXPOSE 8080

#Set command to run when the container is started
CMD [ "npm", "run", "start", "test" ]

--------------------------------------------------------------------------------------

Pipelines.yml

image: atlassian/default-image:2
pipelines:
default:
- step:
name: NewHomePage
script:
- docker build --tag=build5 .
- docker run -p build5
services:
- docker
caches:
- docker # adds docker layer caching

-------------------------------------------------------------------------------------

We are getting an error related to ssh private keys - 

npm ERR! Error while executing:npm ERR! /usr/bin/git ls-remote -h -t ssh://git@bitbucket.org/xxxxxxxxxxx/xxxxxx.gitnpm ERR! npm ERR! Host key verification failed.npm ERR! fatal: Could not read from remote repository.npm ERR! npm ERR! Please make sure you have the correct access rightsnpm ERR! and the repository exists.npm ERR! npm ERR! exited with error code: 128

Does anyone have any suggestions on how to solve this?

Any help would be appreciated!

[Leonardo M]

Hey William,

Looking at your configuration and the error you have received, I believe that you are using SSH to attempt to fetch a dependency on a repository that's hosted on Bitbucket, but are failing due to missing public SSH keys.

I believe that following the steps below should do the trick for you:

1. Add the following to your yml file, as the first command of your build step:

git config --global url.“git@bitbucket.org:“.insteadOf “https://bitbucket.org/”

^This will set up your remote to use SSH instead of HTTPS.

2. Set up a Pipelines SSH key by navigating to the repository > settings > Pipelines > SSH keys > Create an SSH pair.
Once the SSH is created, copy the public SSH key to your clipboard.

3. Once the key is on your clipboard, as an admin, navigate to the account that you are currently trying to fetch your dependencies from (the one that is showing on the "npm ERR! Error while executing:npm ERR! /usr/bin/git ls-remote -h -t ssh://git@bitbucket.org/xxxxxxxxxxx/xxxxxx.git" error you mentioned) > Settings > Security > SSH keys.
Click on add key, and paste the public SSH key that is on your clipboard to the SSH key field, then save.

Once that's done, you should be able to fetch the repository using SSH, as you were trying before.

Cheers!

[dtsn]

Thanks for the reply!

We need to inject the SSH key into the docker container at run time from the pipelines access keys.

We are currently running a webpack dev server and a set of Nightwatch tests in docker. This works perfectly locally (using local SSH keys), however, we need to give docker access to the access keys in order to access the private repo's held in bitbucket which are installed via NPM.

Is there a flag we can pass via the yml flag which will start the docker container with access to the access keys?

[Leonardo M]

Hey, no problem!

I had a talk with one of my senior engineers here at Atlassian, and checking this again, we believe the issue to be related to your image not having the known hosts file available when you spin up the image for docker-in-docker.

See, as you may know, Pipelines by default already has Bitbucket added to the known hosts file. The problem here, however, is that when you spin up Docker-in-Docker within your build, it will not have that file present on your container.

Because of this, if you try to reach Bitbucket using the Docker container, it will ask you to add Bitbucket to the known hosts file inside the container, as it is not there when you spin up the Docker-in-Docker container.

Since Pipelines is not interactive, it will not allow you to proceed, as the git clone command will send you a prompt to add Bitbucket to the known hosts file, like this:

git clone git@bitbucket.org:XXXXXX/XXXXX.git
Cloning into 'XXXXXX'...
The authenticity of host 'bitbucket.org (18.205.93.2)' can't be established.
RSA key fingerprint is X.
Are you sure you want to continue connecting (yes/no)? no
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.

The best approach you can take in this case is to add the known hosts file from your Pipelines build in your Docker container. Adding these commands should fix the issue for you:

run mkdir ~/.ssh
COPY /root/.ssh/known_host ~/.ssh/

Cheers!

[William Bailey]

Thanks for this @Leonardo M . I have added the commands to the top of my dockerfile which now looks like this - 

-------------------------------------------------------------------------------------

FROM node:latest

MAINTAINER 

run mkdir ~/.ssh
COPY /root/.ssh/known_host ~/.ssh/

#Set working directory in the container
WORKDIR /Users/xxxxxx/Repositories/xxxxxxxxx

# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./

#Install local packages listed in package.json
RUN npm install

# Bundle app source
COPY . .

#Export port 8080
EXPOSE 8080

#Set command to run when the container is started
CMD [ "npm", "run", "start", "test" ]

-------------------------------------------------------------------------------------

The pipelines.yml file looks like this -

image: atlassian/default-image:2
pipelines:
default:
- step:
name: NewHomePage
script:
- git config --global url."git@bitbucket.org:".insteadOf "https://bitbucket.org/"
- docker build --tag=build5 .
- docker run -p build5
services:
- docker
caches:
- docker # adds docker layer caching

-------------------------------------------------------------------------------------

This is the result when running the pipeline.

Can you see anything obvious that may be causing it to still fail?

Your help is greatly appreciated - Thank you! :)

[William Bailey]

Hi @Leonardo M - have you been able to take another look at this yet? 

Thanks! 

[Dina]

Is there any update on this issue? Wha is the final resolution?