persist repository changes between steps

[Martí Bastida Comas]

I have a pipelines that publishes a python package to a private PYPI.

The below code works perfectly:

pipelines:
branches:
development:
- step:
runs-on: 
- 'self.hosted'
- 'linux'

name: Bump version
deployment: Test
services:
- docker
caches:
- docker
script:
# bump package version
- echo "__version__ = '1.0.${BITBUCKET_BUILD_NUMBER}'" > gateway/version.py
- echo "__version__ = '1.0.${BITBUCKET_BUILD_NUMBER}'"

# See more details https://bitbucket.org/atlassian/pypi-publish/src/master
- pipe: atlassian/pypi-publish:0.3.1
variables:
PYPI_USERNAME: $pypi_username
PYPI_PASSWORD: $pypi_password
REPOSITORY: $pypiRepo

It basically changes the version on the version.py files and publishes to PyPI via a pipe.

However, if I want to put the pipe part on a step (because it will be done both from master and development branch, but with diffrent steps before publishing) I see that the changes are not persisted:

definitions:
steps:
- step: &publish
runs-on: 
- 'self.hosted'
- 'linux'
services:
- docker
caches:
- docker
name: Publish to private PyPI
script:
# See more details https://bitbucket.org/atlassian/pypi-publish/src/master
- pipe: atlassian/pypi-publish:0.3.1
variables:
PYPI_USERNAME: $pypi_username
PYPI_PASSWORD: $pypi_password
REPOSITORY: $pypiRepo
- step: &getdependencies
name: Download project dependencies from PyPI
script:
- python3 -m pip download -r requirements.txt -d dist/

pipelines:
branches:
development:
- step:
runs-on: 
- 'self.hosted'
- 'linux'

name: Bump version
deployment: Test
services:
- docker
caches:
- docker
script:
# bump package version
- echo "__version__ = '1.0.${BITBUCKET_BUILD_NUMBER}'" > gateway/version.py
- echo "__version__ = '1.0.${BITBUCKET_BUILD_NUMBER}'"

-step: *publish

I have come to this conclusion because version.py by default has version 1.0.0 (This is the one that is on the repository by default, before changing anything). If the pipe runs well the version pushed to PyPI ends up being 1.0.100 (for example, I am at build number 109 to be precise). But instead in the second yml the package published has version 1.0.0 (the "default" one)

Am I doing something wrong?

This is to be expected? If so, is there any workaround?

[Theodora Boudale · Accepted solution]

Hi @Martí Bastida Comas and welcome to the community!

For every step of a Pipelines build, a Docker container starts, the repo is cloned in that container (unless you disable clone), then the commands of that step's script are executed, and then the container gets destroyed.

If you modify a repository file during a Pipelines build step, this change occurs in the clone inside the Docker container for that step.
When that step finishes, that Docker container and that clone get destroyed.

A new Docker container will then start for the second step, the repo will get cloned again from Bitbucket and gateway/version.py will have the content that it has in the Bitbucket repo, on the commit the build is running for.

If you need to include in the deployment a repository file that is modified during the build, the pipe command will need to be in the same step that modifies this file.

Artifacts can be used in Bitbucket Pipelines to pass files generated during one step in subsequent steps, however, this is useful for new files generated during the build (not for repo files).

Kind regards,
Theodora