JIRA wouldn't stop and now it won't start

[Whitni Smith]

On Linux, standalone install. 

I went to stop JIRA and it wouldn't stop. So after about 10min I killed the process and now I cannot get it to start back up. 

[Nic Brough -Adaptavist-]

What does the log say the problem is with restarting?

[Whitni Smith]

from catalina.out just shy of an hour ago, the last log input

08-Jan-2018 17:26:18.834 WARNING [Thread-4] org.apache.tomcat.util.net.AbstractEndpoint.shutdownExecutor The executor associated with thread pool [http-nio-8080] has not fully shutdown. Some application threads may still be running. 

[Nic Brough -Adaptavist-]

Ok, that's vague (Tomcat's fault, not yours).  Could you have a look at the application log - <jira home>/logs/atlassian-jira.log by default.

[Whitni Smith]

The last entry (full) is: 

2018-01-08 17:26:13,652 localhost-startStop-2 ERROR [o.a.c.c.C.[Catalina].[localhost].[/]] Exception sending context destroyed event to listener instance of class com.atlassian.jira.startup.LauncherContextListener
java.lang.NoClassDefFoundError: org/codehaus/groovy/runtime/GroovyCategorySupport$1
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at java.lang.Class.getCanonicalName(Class.java:1392)
at com.atlassian.threadlocal.BruteForceThreadLocalCleanup.getPrettyClassName(BruteForceThreadLocalCleanup.java:257)
at com.atlassian.threadlocal.BruteForceThreadLocalCleanup.checkThreadLocalMapForLeaks(BruteForceThreadLocalCleanup.java:181)
at com.atlassian.threadlocal.BruteForceThreadLocalCleanup.cleanUpThreadImpl(BruteForceThreadLocalCleanup.java:135)
at com.atlassian.threadlocal.BruteForceThreadLocalCleanup.cleanupThreadLocals(BruteForceThreadLocalCleanup.java:94)
at com.atlassian.threadlocal.BruteForceThreadLocalCleanup.cleanUp(BruteForceThreadLocalCleanup.java:63)
at com.atlassian.jira.startup.DefaultJiraLauncher.cleanupThreadLocals(DefaultJiraLauncher.java:285)
at com.atlassian.jira.startup.DefaultJiraLauncher.cleanupAfterOurselves(DefaultJiraLauncher.java:250)
at com.atlassian.jira.startup.DefaultJiraLauncher.stop(DefaultJiraLauncher.java:231)
at com.atlassian.jira.startup.LauncherContextListener.contextDestroyed(LauncherContextListener.java:195)
... 5 filtered
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: Unable to load class 'org.codehaus.groovy.runtime.GroovyCategorySupport$1' because the bundle wiring for com.onresolve.jira.groovy.groovyrunner is no longer valid.
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1494)
at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75)
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 22 more
~

[Whitni Smith]

and the one immediately before that

2018-01-08 17:26:08,513 Gemini Blueprint context shutdown thread1 WARN wwatkins 831x27276x1 jtwxjc 10.66.59.80 /rest/plugins/1.0/installed-marketplace [o.s.b.factory.support.DisposableBeanAdapter] Invocation of destroy method failed on bean with name 'eventTrackingManager': java.lang.RuntimeException: Retry failed; interrupted while waiting

[Nic Brough -Adaptavist-]

I think we may be talking too fast - those still look like catalina.out errors.  Is that right?   Or are they in atlassian-jira.log?

[Whitni Smith]

nope, they are in the atlassian-jira.log file. 

[Nic Brough -Adaptavist-]

Bother, that makes it a bit harder to diagnose, but it's good information.

So, your Jira is not starting.   Could you tell us what you get when you visit the url it should be running on in your browser?

Also, can you read back through your log file?  This is a pain for most humans, but we need to be a bit clever - we need to look for the point of failure.  The errors you have shown us are either warnings (which won't stop the systems) or errors that are a consequence of other errors.  We need the root errors, not the fallout.

[Whitni Smith]

Err_connection_refused -- I checked and both httpd and mariadb are running so not sure why connection is being refused. 

The main error that shows up in the log has to do with a plugin, I noticed it this afternoon and it fails every minute, or at least reports the error every minute 

2018-01-08 00:35:00,003 Caesium-1-1 ERROR anonymous Active-Directory-Attributes-Sync [c.a.jira.service.ServiceRunner] An error occurred while trying to run service 'Active-Directory-Attributes-Sync'. service proxy has been destroyed

This error occurred also

2018-01-08 17:25:57,573 localhost-startStop-2 ERROR [o.e.g.b.e.i.util.concurrent.RunnableTimedExecution] Closing runnable for context NonValidatingOsgiBundleXmlApplicationContext(bundle=com.atlassian.jira.ext.charting.jira-charting-plugin, config=osgibundle:/META-INF/spring/*.xml) did not finish in 10000ms; consider taking a snapshot and then shutdown the VM in case the thread still hangs

These are the only two errors in today's error file. 

UPDATE to say, these errors occur multiple times, but they are the only two errors. 

[Nic Brough -Adaptavist-]

Mariadb is not supported, what happens if you try setting it up with a supported database?

[Whitni Smith]

Our Service Desk has been set up and running fine for the last year, we've rebooted it multiple times and have not had any error. This isn't a new install so "setting it up with a supported database" isn't the answer, especially given that we've not had any issues with it since start date.  It's not a problem with the db. Let's look for another solution to try. 

[Nic Brough -Adaptavist-]

You are getting errors which are quite vague in what they are saying and could be caused by all sorts of things.

Including an unsupported database.

It's utterly irrelevant how it has been working.  You now have an issue with it, and there is no way to rule out "your database is broken", until you start running it on a supported database.

You say > It's not a problem with the db.

You need to prove that.  You cant say it is not a problem with the db when you are using an unsupported one.  Because it very well could be.

You say > Let's look for another solution to try. 
  Let's get you moved to a supportable system before looking to new solutions.

[Whitni Smith]

I was able to determine the root of the problem, which is the check_java.sh on startup wasn't registering the correct JVM that is installed as being accurate and a line in the check_java.sh said for it to exit if it didn't match. The jvm is accurate, I commented out that line in the .sh file and will need to determine why it isn't registering the correct jvm version. 

I used part of the error code to determine that it was a java issue and deducted from that. 

While our Service Desk is finally back up and running, I still have this problem to be solved.