Recently I have come across a sporadic issue that is causing a Jira Server webGUI to not resolve.
This system is running NGINX.
I have checked all running services to ensure that all services are running when the system ends up in this state.
I am unable to locate the cause of the issue. Could someone please point me in the right direction to what is causing this issue?
A simple reboot brings it all back up on line.
Hi Kale,
There are many, many possibilities. Let's try to gather more information to see if we can narrow them down!
Some general thoughts:
proxy_read_timeout 60s;
I think possibly the most likely scenario (without additional information) is that your Java heap for Jira may be undersized, it's using the CMS garbage collector, and what you're seeing is the JVM trying (unsuccessfully) to complete a full garbage collection. You could confirm this by looking at the GC logs which should be in <jira's install directory>/logs. A tool like GCViewer would be helpful to analyze your GC logs if you're not familiar with Java's Garbage Collection mechanisms. I would be interested in knowing the following related to this theory:
Finally, I would just mention that restarting the services individually (Jira and nginx) rather than doing a full system reboot is going to help you narrow down where the cause of this is. If possible, eliminate variables by only changing one thing at once.
Cheers,
Daniel
Hi @Daniel Eads
Thanks for this detailed response.
It has been some time since I posted this and as per my message it is sporadic and well touch wood it has not yet had a repeat issue as yet. But I will still dive in and investigate your comments above and see what I find.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
To answer your questions:
To answer your questions below about memory and GC, I will investigate this shortly.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I am currently in the process of working out how to work the GC Viewer app works.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Thanks for the extra info! Given all the above (particularly the default heap application of 1820mb), I think the most likely scenario is still the slowdown being caused by a full garbage collection. Since you mentioned it's been some time since this slowdown has happened, it's possible your GC logs have rolled over and wouldn't show the last memory exhaustion.
As a preventative measure, you might consider increasing your Xmx value by 512mb if there's enough free RAM on your server to do so (and still provide the base operating system with enough RAM to carry out its operations safely) - for example you would probably want to ensure the server had at least 6GB of RAM total if Jira and nginx are the only applications running on it, and increase the Xmx value to 2332mb.
We've got guidance on increasing your heap size on this article . Setting the Xmx (maximum heap size) and Xms (minimum heap size) to the same value can also help reduce the time it takes for a GC to complete.
Since you've not noticed this issue in some months, I'm thinking it's likely that just a preventative boost of the heap size may stave off issues for you going forward. By all means, keep learning about the GC allocation so this is less likely to be a mystery - but from what you've seen so far, I think tuning the Xmx/Xms slightly might be a quick way to ensure stability in the coming months.
Cheers,
Daniel
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.