How can I remove a failed Jenkins build from a Stash pull request?

[Brandon Barlow]

Several times, our developers have cloned a Stash repo's corresponding Jenkins project when standing up a new software component's continuous integration cycle. Our Jenkins projects are configured to notify Stash upon completion. However, when cloning a Jenkins project, Jenkins often (always?) immediately runs a build, before the developer even finishes customizing the new project and saving the Jenkins project configuration. This results in pull requests on the original Stash repo getting stuck with a failed build from the new project. As far as I can tell, the only way to allow such a pull request to be merged is to have an admin uncheck the 'Require N or more successful builds' setting in the repo properties. I plan on following up on the Jenkins side to investigate why cloned projects kick off builds immediately, but the question remains - how can I remove a failed Jenkins build from a Stash pull request?

[Mike Friedrich]

I have the same issue.

When looking at the build status keys Jenkins uses in the notifier (via REST API), i find the reason for this. The key contains the project name and jenkins root URL.

When a project is renamed or cloned in Jenkins, they do not match any more. Same if the server is renamed, relocated or if the configuration is tranferred to another Jenkins instance.

==> We definiely need at least a REST API for delete.

[Robert D]

The situation that affected us:

1. Create a branch on a production repo, and a Pull Request for that branch back to master
2. Jenkins automatically creates a pipeline build for that branch, which succeeds
3. Create a fork of the production repo, and create unrelated branches for other work
4. Set up pipeline builds for the new fork
5. A mistake during configuration causes all branches on fork to build, including all branches from the origin repo
6. These builds fail or get cancelled
7. Fix the configuration error, which removes the forked version of these branch builds

At this point the pull request now has two builds, a successful one from the production repo, and a failed one from the forked repo, and the PR cannot be merged if the 'require successful builds' is enabled.

Having a way to delete this false failure would be nice.

[Maurice Williams]

Hey Robert - did you ever find a way to deal with this? I'm in the same situation that you described.

Aside from temporarily remove the "require successful builds" check I'm not sure what other course of action there is  at this time.

[Robert D]

Hi Maurice, unfortunately not, at least not that I can recall.  We just ended up disabling the option when having to merge each of these PR's, and then re-enabling it.

[cofarrell]

Hi Travis,

Have you checked the 'Keep repeated builds in Stash'?

That means the 'key' is going to have a build number - which means it will _always_ be unique.

https://github.com/jenkinsci/stashnotifier-plugin/blob/develop/src/main/java/org/jenkinsci/plugins/stashNotifier/StashNotifier.java#L607

This will result in separate builds in Stash every time - which is _not_ what you want.

Cheers,

Charles

[srinath45]

You cannot delete those builds. But, here we have two options to solve this issue.

1. If a branch failed and still exists on Jenkins:-

Use the replay option and put some dummy code over there like "echo $something", this will make the build to automatically pass. So, Stash will allow you to merge the PR.

2. If a branch is deleted on Jenkins:-

Recreate the branch with same name and use the same process specified in step 1, it will make the build pass.

[Lewox]

Another way to get stuck with this issue is to have two instances of Jenkins notifying the same instance of Bitbucket Server / Stash.

If one of them fails, you cannot merge it, and you cannot remove the failed build neither.

Regards

[flemmon]

Just ran into this issue here. It's three years later and Atlassian still doesn't have a fix for this? What's with that?!

Atlassian, please direct some attention to https://jira.atlassian.com/browse/BSERV-3203!!

[Alastair Bain]

We had a problem where a broken Jenkins notifier plugin was causing incorrect Jenkins jobs to run, resulting in failed builds which couldn't be removed. I ended up manually posting fake "pass" results to overwrite the fails via the REST API.

[Deleted user]

Can you describe how you did that?

[Travis Stevens]

That is correct, you can _never_ merge if a build fails once. The only use case I can think of is if one would want the build to succeed in multiple environments before merging. However, I would expect my case to be more prevelant where the build fails because of environmental reasons and we want to build it again. However, there does not seem to be away to configure it my way.

[cofarrell]

Travis,

That sounds like a massive bug in the Jenkins plugin - the build integration was designed so that would only care about the latest build result. Otherwise it will result in the problems you're currently facing.

Charles

[Brandon Barlow]

Travis - I was unable to replicate your issue. Can you post an explicit list of steps to reproduce?

[Travis Stevens]

Actually, I just remembered that I did get Jenkins to rebuild a failed build. Jenkins didn't update the build status, but added a new successful one. Unfortunatley, there is no way to change the setting which is that you can not merge if ANY build has failed.

[cofarrell]

Hi Travis,

Did you manage to get anywhere with this?

Are you saying every build in Jenkins results in a _new_ build result in Stash? That's not how the feature was intended to work, and would mean that if the build ever fails you would _never_ be able to merge if you have that setting enabled.

The other short-term option BTW, would be to fake a successful build status via REST so that you continue. This isn't going to scale though if the failed build happens on every pull request though.

Cheers,

Charles

[Travis Stevens]

Jenkins should rebuild the git commit if triggered manually, but it doesn't. I agree that Jenkins should be fixed.

[Travis Stevens]

As a really hacky workaround (and probably not recommended) you can edit the stash database. In my environment there is a table called AO_CFE8FA_BUILD_STATUS. I issued this command in mysql to change the bulid status:

update AO_CFE8FA_BUILD_STATUS set STATE='SUCCESSFUL' where url = 'http://yam:8080/job/GCSI%20Stash%20Verifier/222/';

[cofarrell]

While we don't recommend this (and please backup before you ever modify the DB directly), there isn't really any other way to remove the build statuses. They were intended to only ever be added or updated with a newer status.

We could probably add a REST endpoint for deleting them if it really becomes a problem, but personally it sounds like fixing Jenkins might be the better way to go. I'm surprised it's not disabling a cloned build by default, or something like that?

Charles

[Alexey_Efimov]

Charles, REST is already exists for this :) Not for deleting, but for update.

POST /rest/build-status/1.0/commits/{changesetId}

[cofarrell]

Hi Alexey,

That's right. You can definitely set the status with a POST, but you can't (currently) delete the status entirely.

See above though - the problem in this case seems to be the configuration of the Jenkins plugin.

Cheers,

Charles

[CelsoA]

Hi there,

In newer Stash versions you can change the destination branch if that helps. :)

There is an imprivement request related to it here:

https://jira.atlassian.com/browse/STASH-3203

For more information on how Atlassian implements new features and improvements please see the following document:

http://confluence.atlassian.com/display/DEV/Implementation+of+New+Features+Policy

My apologies that I don't have a solution for you at this time.

Regards,

Celso Yoshioka