SFTP-deploy pipe creates build folder on remote and fails after 1st deployment

[wiegert]

So I'm trying to set up automated deployment to our LAMP server using Bitbucket Pipelines and the sftp-deploy pipe. Here is my bitbucket-pipelines.yml:

image: atlassian/default-image:2

pipelines:
  branches:
    master:
      - step:
          script:
            - pipe: atlassian/sftp-deploy:0.3.1
              variables:
                USER: $SFTP_USER
                SERVER: $REMOTE_HOST
                REMOTE_PATH: $REMOTE_PATH
                LOCAL_PATH: $BITBUCKET_CLONE_DIR
                DEBUG: 'true'
          deployment: 'production'

Two problems:

1. I'm trying to deploy my files to a public_html folder, but the pipe creates a 'build' folder inside it and puts the files in there. That is not what I want, the files must be direct children of the public_html folder. How can I change this behavior?
2. Apart from the creation of the build folder, the first run of this pipe works fine, but on successive runs, it fails. It keeps throwing errors like this:

remote open("<my_server>/public_html/build/.git/objects/40/002c7bb269367f3ae0966cd9ffaf96f80d66e3"): Permission denied 

File permissions can't be the problem, the user I login with owns the files the script is trying to open. 

Can anyone help me out?

[wiegert · Accepted solution]

Hi Steven,

Thank you for your reply.

Meanwhile, I have continued my search for a solution. I found one in using git-ftp instead of the sftp-deploy pipe. My bitbucket-pipelines.yml now looks like this:

image: wagnerstephan/bitbucket-git-ftp:latest

pipelines:
custom: # Pipelines that are triggered manually via the Bitbucket GUI
Initialize master: # -- To be run manually after the first push to the master branch
- step:
script:
- git config git-ftp.url $MASTER_HOST
- git config git-ftp.user $MASTER_USER
- git config git-ftp.password $MASTER_PASSWORD
- git config git-ftp.remote-root $MASTER_ROOT
- git ftp init -v
branches: # Automated triggers on commits to branches
master: # -- When committing to master branch
- step:
deployment: production
script:
- git config git-ftp.url $MASTER_HOST
- git config git-ftp.user $MASTER_USER
- git config git-ftp.password $MASTER_PASSWORD
- git config git-ftp.remote-root $MASTER_ROOT
- git ftp push

(Somehow the editor breaks my indentation)

This works for me. Includes a manual step to init the remote repo and an automated one that pushes successive commits.

Thanks for your help anyway :)

Wiegert

[raschidjfr]

I solved the first issue by adding an asterisk at the end of local path:

From:

LOCAL_PATH: build/foo

to:

LOCAL_PATH: build/foo/*

For the second issue... I started noticing that same permission problem after upgrading `sftp-deploy` from `0.4.1` to `0.5.0`. I'm not sure yet this is the problem, I'm missing some testing. Just for the record, 'cuase I'm still having it.

[raschidjfr]

EDIT: turns out to be a server issue I think. I need to apply recursively write permissions to all files and folders on remote target path before running this pipe.

This didn't happen before so I don't know what's causing the problem but at least got a workaround.

[snorbik]

I was solving same issue with atlassian/ftp-deploy.

What helped me, was erasing slash from the end of REMOTE_PATH string

Basically I have changed

/this/was/my/path/ 

to

/this/was/my/path

Now it works great. 

[Jan Guardian]

Steven,

I have the same problem. I'm trying to upload the contents of my repo to my server via sftp-deploy. For some reason, the Bitbucket sftp-deploy creates a /build folder inside the specified remote_path and uploads everything there instead of uploading everything to the remote_path.

[Steven Vaccarella]

Which version of the sftp-deploy pipe are you using? I just had a quick look at the latest version and I can't see an obvious reason that it would be doing this. Does your source directory structure contain a directory called "build"?

[Jan Guardian]

It's pipe: atlassian/sftp-deploy:0.3.1

The local path is $BITBUCKET_CLONE_DIR

The remote path is a string like '/remote/path/' and does not contain 'build' in it

[wiegert]

For what it's worth: my problem was exactly the same, I didn't have a build folder in my source directory, my remote path didn't contain 'build', and I used $BITBUCKET_CLONE_DIR as my local path.

[Oleksandr Kyrdan]

Hi Jan, Hi Wiegert,

you can provide '/local/path/build/*' with  trailing asterisks symbol (*) as a LOCAL_PATH variable to deploy only files from your build directory to the remote.

[Steven Vaccarella]

... or more specifically in your case:

    LOCAL_PATH: '${BITBUCKET_CLONE_DIR}/*'

[Steven Vaccarella]

Hi Wiegert,

It's hard to be sure from the information you've provided, but it looks like you don't really want to deploy your entire build directory (including the .git subdirectory) but just a subdirectory that contains the website content. Is that correct? If so, I'd try changing the LOCAL_PATH and REMOTE_PATH parameters to point to the most specific directory that you're trying to deploy.

With any luck that might fix the file permission problem too if it's somehow related to the files in the ".git" subdirectory and you're no longer trying to deploy that directory.

If I haven't understood the problem correctly, could you provide a bit more information, particularly what your directory structure looks like, which part of that directory structure you're trying to deploy and what your REMOTE_PATH environment variable looks like (with any sensitive information redacted). 

Cheers,
Steven

[Shabeena Begum Syed]

it's basically comes with unnecessary trailing slashes, just keep without trailing slash in remote path, it works. It worked for me.