How can I make the build green, with quarantined test?

[Casey Harford]

We just upgraded to Bamboo 4.2.1 and I have a build which runs a set of Junit tests. One of these tests is failing, so I quarantined it temporarily. I've re-run the build, but in the Dashboard view, it's still showing as red(failing) but the test column says 8 passed. It doesn't show any failures. How can I make the build show green, even with quarantined tests?

[Przemek Bruski · Accepted solution]

It should be green if the only failing test is quarantined ant the build has completed successfully. If it's still red, please open an issue at support.atlassian.com

[Casey Harford]

I'm running the test with maven, and since it's failing in the build log, it's exiting with an error status. I suspect this is why the build is showing red.

[Casey Harford]

I ended up finding a work around to my issue. In Bamboo, I use a mvn command to launch tests. I appended || true to the command, which resolves the build color issue.

Example:

mvn clean test [arguments] || true

[Brent Villalobos]

I have a similar problem. Our build process does both the build and test in a single process. If there is a build or test failure, the process returns a non-zero return code. And so, even if I quarantine failing tests, the job is still recorded as a failure. However, if I added a "|| true", then that would pave over all errors, including build errors. That just flips the problem from good builds being marked as failed to bad builds being marked as successful.

Is there a standard or best practice on what a test running should return when there are test failures? One workaround that I can think of is that the build/test process returns a specific code on test failure that a Bamboo wrapper could then check for, return 0, and let the JUnit parser do its thing. Seems hacky to me.

[bgreen-dt]

Found this via searching; if anyone's tempted to go down the same path, I suggest doing a testless maven build, then doing a separate maven step that runs the tests which you can || with true.