Bitbucket does not detect CTest reports in JUnit format

[Francisco Perez Moreno]

CTest, as part of CMake, is a test runner which generates xml reports. Starting with the 3.21 version, they support JUnit generation.

However, this is not detected by bitbucket, in which after a pipeline runs a job with a ctest command on it it reports 0 detected tests. When using other test runners (like GTest) on the same test code, bitbucket correctly reports the number of tests.

How to reproduce:

1. Create a git repository within bitbucket.
2. Create a file named main.cpp within the git repository's root folder, with these contents (this is just a minimal example taken from https://stackoverflow.com/a/21688776):

#include <cstdlib>
int main() {
    std::exit(-1);
}

1. Create another file named CMakeLists.txt in the same folder with these contents (taken from the previous example from https://stackoverflow.com/a/21688776):

cmake_minimum_required(VERSION 3.21)
project(BitbucketExample)
enable_testing()
add_executable(main main.cpp)
add_test(test1 main)
add_test(test2 main)
set_tests_properties(test2 PROPERTIES WILL_FAIL ON

1. Create a bitbucket-pipelines.yml file in the same folder with these contents. The docker image is irrelevant, as long as it has some build-essential packages (C++ compiler) and a "new" cmake version (>=3.21):

image:
    name: feabhas/gcc11-cmake:latest # this comes from https://hub.docker.com/r/feabhas/gcc11-cmake/

options:
    docker: true

clone:
    depth: full

pipelines:
    default:
        - step:
            name: Build and check the pipeline unit tests reports
                - mkdir buildForCTest
                - cd buildForCTest
                - cmake ..
                - make -j8
                - ctest --output-junit test-results/unit_tests_results.xml

1.  Push this repository onto the bitbucket's remote. Bit bucket should start a new pipeline job.
2. Upon the pipeline ending, it reports 0 tests on the job, even though some unit tests are generated within a folder whitelisted to detect unit tests.

I tried to be more or less thorough, but I likely missed some steps. In that case I'll be happy to help :)

Best regards and have a nice day

[Norbert Csupka]

Hello Francisco,

Thank you for contacting Atlassian Community, my name is Norbert and I'm a Bitbucket Cloud Support Engineer, it's nice to meet with you. Welcome to the Atlassian Community!

This is interesting, please see my Pipelines configuration and the "Build Teardown" section, once the build has finished, it found the test result:

Can you try to compare your bitbucket-pipelines.yml file and my bitbucket-pipelines.yml file and let me know if there's any difference between those builds?

I'm looking forward to hear from you Francisco.

Have a great day!

Best Regards,
Norbert
Atlassian Bitbucket Cloud Support