502 Proxy Error (Only on some Issues)

[Wilhelm Krüger]

Hello Guys,

we have some proxy issues on our JIRA. When browsing in Project XY there appears the following error:

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET/browse/XY-21.

Reason: Error reading from remote server

A piece of the error.log

[Mon Sep 17 11:45:10.269626 2012] [proxy:error] [pid 17364] [client xxx.xx.xx.xx:24971] AH00898: Error reading from remote server returned by /browse/XY-21, referer: https://xxxxxx/browse/XY-21

The strange thing is that this error only occurs on some special issues on this project. Other projects or issues are not affected by this problem.

Does anyone of you have an idea?

Regards,

Wilhelm

[Wilhelm Krüger · Accepted solution]

Apparently there is a bug in JIRA - JRA-30068

We have already tested the JIRA 5.2 EAP and can confirm that this bug is fixed in JIRA 5.2

[Vinay Sharma]

Does anyone got solution to this problem?

[Nic Brough -Adaptavist-]

Posting on a 3 year old thread often doesn't do much as things have often moved on. But the *generic* answers here are still valid - you are going to need to establish the pattern of when it occurs and read the logs to see what errors you might be getting.

[Nic Brough -Adaptavist-]

What is "special" about these issues? What is different between them and the ones that work?

What does your Jira application log say?

[Wilhelm Krüger]

I don't really know, i can't see any difference between those issues and the others.
And it only affects about three issues, all other issues are ok.

Some more information:

I was able to clone the issue by using the issue navigator, but i wasn't able to clone the attachments. So there might be some inconsistance in the database?!

I found those Information in the catalina.log (Is this similar to the application.log? If not where do I find it?)

Sep 17, 2012 11:23:45 AM com.sun.jersey.spi.container.servlet.WebComponent filterFormParameters
WARNING: A servlet POST request, to the URI https://xxxxxxxxx/rest/gadget/1.0/login, contains form parameters in the request body but
the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @Fo
rmParam will work as expected. Resource methods consuming the request body by other means will not work as expected.
Sep 17, 2012 11:49:46 AM com.sun.jersey.spi.container.servlet.WebComponent filterFormParameters
WARNING: A servlet POST request, to the URI https://xxxxxxxxx/rest/gadget/1.0/login, contains form parameters in the request body but
the request body has been consumed by the servlet or a servlet filter accessing the request parameters. Only resource methods using @Fo
rmParam will work as expected. Resource methods consuming the request body by other means will not work as expected.

[Wilhelm Krüger]

part of the access.log

xxx.xxx.xx.xxx - - [17/Sep/2012:11:44:10 +0000] "GET /browse/XY-21 HTTP/1.1" 502 403 "https://xxxxxxxxxx/browse/XY-21" "Mozilla/5.0 (Maci
ntosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1"

[Nic Brough -Adaptavist-]

Not sure, those errors are more likely to be gadget or report errors, possibly caused by the same issues, but not the root cause of the problem.

Catalina.out is a good place to look, but I'd also check the application log (rather than random guesswork starting from "it's usually, but not always, jira-application.log"), go to Admin -> System information, that will tell you where the app log is)

As a hunch, can you compare the fields on the issues? For example, if you have a custom field called "Fred" and it's blank for working issues, and filled in with something on the broken ones? Are they all the same priority? Etc

[Wilhelm Krüger]

Those are the logs I find on our server:

access.log
atlassian-jira-security.log
atlassian-jira-slow-queries.log
atlassian-jira.log
catalina.log
wrapper.log

APACHE:

access.log
error.log

I'm not able to open those issues in their default view. But I can say that there are no costum fields added for this project. Those issues all have the status "open" and resolution "unresolved". Issuetype is "task" or "subtask"

[Nic Brough -Adaptavist-]

atlassian-jira.log is the application log you want.

The question about fields is a bit of a guess, because I've seen custom fields that cause this error when they try to render the issue view (my code was awful!)

Another thought though - I've also seen this happen when there's a huge number of comments on an issue - could that have happened? My favourite trick - if you allow "create or comment by email", some humans seem to think setting "autoreply to every email I get with an out-of-office" is a sane thing to do - their colleagues quickly get annoyed (we only need one response), but it's also possible to set up a comment loop - their dumb autoresponse triggers a comment, which goes to them, which triggers an autoresponse, etc. 32,000 comments on an issue does make the display fail...

[Nic Brough -Adaptavist-]

Good to rule the email out.

Cloning doesn't take comments, it's only the issue data it copies, so you could still have the comment problem. Having said that, it's unlikely, because I've seen issues with a few hundred comments working fine, it's only when you get into silly numbers that problem occurs, and humans generally wont get that far!

The errors in your log excerpt don't seem to be related to viewing issues, they seem to be to do with a broken hipchat configuration, and a broken attachment alongside an event or notification. I don't think they would break the simple issue view.

I think you need to do some careful logging here - I notice that the times on all your logs are quite significantly different. I would

1. get the url for a broken issue ready to load but don't actually hit return
2. start tailing the application log
3. start tailing the error.log
4. start tailing the access.log
5. hit return in the browser
6. Read all three logs to see what warnings and errors come out at the time you hit view. Bear in mind that some logs won't actually catch any warnings or errors.

[Wilhelm Krüger]

I extracted a small part of the atlassian-jira.log.

We don't allow any comments created by e-mail. It's also not allowed to create issues by e-mail.
In addition to that there are no comments on the cloned issues, so I think there aren't anyones at the original.

(atlassian-jira.txt)

[Wilhelm Krüger]

Those issues contain about 50-70 comments.

how is it possible that jira complains about hipchat configuration despite we aren't using it.

I have taken your advice and attached all logs but there seems to be nothing suspicious:

(access_apache.log)

(access_jira.log)

(atlassian-jira.log)

(error_apache.log)

[Wilhelm Krüger]

Those issues contain about 50-70 comments.

how is it possible that jira complains about hipchat configuration despite we aren't using it.

I have taken your advice and attached all logs but there seems to be nothing suspicious:

(access_apache.log)

(access_jira.log)

(atlassian-jira.log)

(error_apache.log)