Bitbucket Pipelines with pytest XML report

[glinskyc]

Upon running a Bitbucket Pipeline with the following:

• python3.6 environement
• pytest to run tests
• pytest-cov plugin used to generate XML coverage report via pytest option:

  pytest --cov=<my_project> --cov-report=xml:./test-reports/coverage.xml

As you can see from the pytest option, the XML report is saved to "test-reports" folder and the file is named "coverage.xml".

Upon tests failing, the Bitbucket Pipeline says it finds the test report during the pipeline teardown, however it does not display any information on a "Test" tab as explained here:

https://confluence.atlassian.com/bitbucket/test-reporting-in-pipelines-939708543.html

Here's my teardown output in the Bitbucket Pipeline:

Skipping cache upload for failed step
Searching for test report files in directories named [test-results, failsafe-reports, test-reports, surefire-reports] down to a depth of 4
Found matching test report file /opt/atlassian/pipelines/agent/build/test-reports/coverage.xml
Finished scanning for test reports. Found 1 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

So my question is why isn't the XML test report (which is supposedly found) not being used by the pipeline to provide me with a proper test summary?

Thanks in advance for help,

Chad

[Graham Gatus]

Hi @glinskyc

The test-reports feature in pipelines requires the xml test reports to be in Junit format.

Having a skim through the pytest docs at https://docs.pytest.org/en/latest/usage.html, it looks like you should be able to do this with the 'junitxml' flag:

pytest --junitxml=./test-reports/coverage.xml 

 Thanks,

Graham

[glinskyc]

Thanks Graham, I'll give that a try.

[glinskyc]

@Graham Gatus, that works perfectly.  Thanks again!

As a note to anyone else that may encounter this, the solution here means that pytest-cov is not used for Bitbucket Pipelines, but rather the built-in pytest JUnitXML report is used as pointed out by Graham above.

Therefore, I opted to include the following pytest option (notice junit.xml naming) in case the coverage XML report is still desired.  This is simply to avoid confusion if an XML coverage report is also created.

pytest --junitxml=./test-reports/junit.xml