How do I persist system dependencies between Pipeline steps?

[Giampaolo Falqui]

I have learned that I can use artifacts in order to persist specific directories for the following steps, however what if I want to install system dependencies and application dependencies in two different steps?

Example:

# This is a sample build configuration for PHP.
# Check our guides at https://confluence.atlassian.com/x/e8YWN for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: php:7-fpm
pipelines:
  branches:
    '{staging,develop,master}':
      - step:
          name: Install system dependencies
          script:
            - apt-get update && apt-get install -qy zip unzip git curl libmcrypt-dev mysql-client
            - yes | pecl install mcrypt-1.0.1
            - docker-php-ext-install pdo_mysql
            - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
      - step:
          name: Install application dependencies
          caches:
            - composer
          script:
            - composer install
            - ln -s .env.pipelines .env
            - export CACHE_DRIVER=array
            - export DB_CONNECTION=sqlite
            - export QUEUE_DRIVER=array
            - php artisan jwt:secret
      - step:
          name: Test Suite
          script:
            - ln -s .env.pipelines .env
            - export CACHE_DRIVER=array
            - export DB_CONNECTION=sqlite
            - export QUEUE_DRIVER=array
            - php artisan jwt:secret
            - php artisan serve &
            - sleep 5
            - ./vendor/bin/phpunit 
            - curl -vk http://localhost:8000

However, once I reach the install application dependencies step, it says that composer does not exist.

[StannousBaratheon · Accepted solution]

Hi @Giampaolo Falqui

Each step in Pipelines is a new environment. It's equivalent to starting a new Docker container using the image referenced by the `image` tag (in this example the php:7-fpm image).

You can install system dependencies in the same step that they're used however this is not the recommended approach. Instead we encourage you to build your own Docker image which has the system dependencies already installed. This way your build environment is already set up in the image and you will benefit from Docker image caching that is built in to Pipelines.

For example:

* Create a Dockerfile that is based on the php:7-fpm image and installs the additional dependencies:

FROM php:7-fpm

RUN apt-get update && apt-get install -qy zip unzip git curl libmcrypt-dev mysql-client
RUN yes | pecl install mcrypt-1.0.1
RUN docker-php-ext-install pdo_mysql
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

* Build this image and push it to Dockerhub:

docker build -t your_dockerhub_username/your_image_name .
docker push your_dockerhub_username/your_image_name

* Use this new image in your Bitbucket pipelines yml file:

image: your_dockerhub_username/your_image_name
pipelines:
  branches:
    '{staging,develop,master}':
      - step:
          name: Install application dependencies
          ...

I hope this helps.

Sam

[Giampaolo Falqui]

Yes, thank you very much!

[Anatoli Babenia]

And to make sure this image doesn't get outdated with security vulnerabilities, you need to setup another CI.