(ubuntu x64 - jira 5.2.2) JIRA startup failed, JIRA has been locked

[nkaravias]

Hello,

I'm trying to install Jira 5.2.2 on an ubuntu 12.04 vm. There is no firewall running (Iptables -L shows no rules applied) and I have a properly configured mysql (5.5) database running and accepting connections properly.

After doing a custom install (Jira installed directory is /var/atlassian/jira) I create the database on mysql without issues and the admin account. I am then routed normally to the Jira dashboard as if there is no problem with my installation, As soon as I perform a reboot jira fails to start and I am greeted by the following message:

JIRA Startup Failed

You cannot access JIRA at present. Look at the table below to identify the reasons

Description The following plugins are required by JIRA, but have not been started: Atlassian JIRA - Plugins - Project Config Plugin (com.atlassian.jira.jira-project-config-plugin) Atlassian OAuth Admin Plugin (com.atlassian.oauth.admin) Embedded Gadgets Plugin (com.atlassian.gadgets.embedded) Gadget Dashboard Plugin (com.atlassian.gadgets.dashboard) Atlassian JIRA - Plugins - Look And Feel Logo Upload Plugin (com.atlassian.jira.lookandfeel) Gadget Spec Publisher Plugin (com.atlassian.gadgets.publisher) Atlassian OAuth Service Provider Plugin (com.atlassian.oauth.serviceprovider) Opensocial Plugin (com.atlassian.gadgets.opensocial) Gadget Directory Plugin (com.atlassian.gadgets.directory) Atlassian JIRA - Plugins - Gadgets Plugin (com.atlassian.jira.gadgets)

I can provide my catalina.out log () for anyone willing to have a look. I haven't found any other solution online.

P.S My catalina.out ( http://pastebin.com/GyXhCkvr ) and a second one after chowning /var/atlassian to jira ( http://pastebin.com/EzefjXrW ).

Thanks,

Nikolas

[Andy Brook [Plugin People] · Accepted solution]

The subject question is generally because the JIRA process wass killed or there was a crash, perhaps an Out Of Memory problem. However, it is also possible there is still a JIRA process running (in some form, good to check).

The result of these situations is a lockfile: JIRA_HOME/.jira-home.lock Removing the lockfile allows the startup process to continue.

@volker, there isn't enough info to go on here. Likely memory will be your problem, check the atlassian logifles, specifically for out of memory or simialr stack traces. It would also be useful to run 'top' on the machine during startup, watching your available memory....

[nkaravias]

Thanks for the feedback Andy. However i don't thing this is the case. If i just remove the lock file you correctly mentioned, and then restart jira again, the same issue in this ticket appears.

How can JIRA be crashing when this is a vanilla install and this happens if I just restart the application once post-install? There is no security software running (like SElinux on redhat - i'm on ubuntu for this) and iptables -L shows nothing configured in regards to routing.

From what I can see there is only one java process running:

@v-webjira04:~$ ps -A | grep java

3094 pts/1 00:04:18 java

Posting a top output. Clearly this configuration should be able to keep a vanilla jira 5.2 up. As you can see there's plenty of memory available (while I can agree that the CPU is pegged):

top - 16:38:37 up 2:19, 2 users, load average: 0.66, 0.56, 0.86

Tasks: 69 total, 1 running, 68 sleeping, 0 stopped, 0 zombie

Cpu(s): 88.4%us, 10.9%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.7%si, 0.0%st

Mem: 2051688k total, 750564k used, 1301124k free, 35468k buffers

Swap: 2622460k total, 0k used, 2622460k free, 437932k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

2899 jira 20 0 1648m 139m 13m S 98.4 6.9 0:19.40 java

850 mysql 20 0 873m 59m 8116 S 0.3 3.0 0:50.50 mysqld

2153 nkaravia 20 0 73360 1668 868 S 0.3 0.1 0:01.78 sshd

Added catalina.out on pastebin ( http://pastebin.com/GyXhCkvr )

[Justin Alex Paramanandan]

Okay, mate. Thanks for the input.

Can you attempt to delete the following from your end:

• /var/atlassian/application-data/jira/plugins/.bundled-plugins
• /var/atlassian/application-data/jira/plugins/.osgi-plugins
• /var/atlassian/application-data/jira/plugins/.bundled-plugins
• /var/atlassian/jira/work
• /var/atlassian/jira/logs/catalina.out

Also, just do a round of check to see whether the user executing /var/atlassian/jira/bin/startup.sh has the correct permissions? Sometimes, permissions can play a big role in whether your instance starts up properly or not. Things such as root is able to start off everything, but since I see the user jira there, I would just like you to ensure that all is well set-up. ;)

If this doesn't work, can you attach your /var/atlassian/jira/logs/catalina.out for us to look at it further?

[nkaravias]

I chowned everything for jira:jira (you were right, since I installed as sudo - according to instructions - some folders got assigned to root), removed the files mentioned and did a clean restart as jira user.

The same issue appeared again.

Here's a clean catalina.out (with only the recent jira startup): http://pastebin.com/EzefjXrW - (for some reason I can't upload here).

[Justin Alex Paramanandan]

Cool, got to see that there are some lesser plugins that have yet to startup.

Can you try to run some SQL queries that are provided here: https://confluence.atlassian.com/display/JIRAKB/JIRA+does+not+Start+Due+to+Disabled+Bundled+Plugin

Let us know how this goes for you.

[nkaravias]

I already tried this on my RedHat installation last week. Nothing changed. (yes i've googled this issue a lot since I need to upgrade to 5.2 on a production box eventually).

When I run the query on my current Ubuntu setup it seems it's ok by default (propertyvalue='true')

mysql> SELECT * FROM propertyentry e JOIN propertystring s ON e.id=s.id WHERE e.property_key LIKE '%plugin.state%';

+-------+-----------------+-----------+---------------------------------------------------------------------------------------+--------------+-------+---------------+

| ID | ENTITY_NAME | ENTITY_ID | PROPERTY_KEY | propertytype | ID | propertyvalue |

+-------+-----------------+-----------+---------------------------------------------------------------------------------------+--------------+-------+---------------+

| 11203 | jira.properties | 1 | jira.plugin.state-.com.atlassian.jira.welcome.jira-welcome-plugin:show-whats-new-flag | 5 | 11203 | true |

1 row in set (0.00 sec)

Running the update doesn't change anything obviously. And yes restarting jira didn't change much. Tried to restart the whole vm but in vain, same error.

Thanks again for your time!

[nkaravias]

I added another core to my VM and there are no errors anymore. I will attempt to install a plugin from the market and see how things go. There's a good chance Jira was getting locked because of insufficient CPU. However this wasn't logged in a readable sense and I might have missed it. I'll keep this ticket updated (and might be closing it soon).

Thanks,

Nikolas

Edit: Yeah that was it. Kind of annoying but i'm all good now (except a groovy pluging problem, but that's another story. Closing.

[Andy Brook [Plugin People]]

Er, just for the record, insufficient CPU is not going to cause this problem, Im pretty sure it will have been memory. The reasons a clean restart after your CPU change will have helped is that other processes running may have been cleared. You can avoid this kind of problem by pre-allocating the heap you are providing to JIRAs JVM, good practice for a dedicated instance if you are running close to memory limits.

ref: http://docs.oracle.com/javase/6/docs/technotes/tools/solaris/java.html

If you set -Xms (the initial memory heap) and -Xmx (the maximum memory heap) you will guarantee that on startup, the JVM reserves its 'working' heap in full. Of course, there are additional heaps that can also be set, stack, permgen (depending on platform).

[nkaravias]

1. JIRA_HOME = /var/atlassian/application-data/jira

2. Results of ls -lah with jira shutdown (if i start it you can just add the .jira.home.lock file):

@v-webjira04:/var/atlassian/application-data/jira# ls -lah

total 44K

drwx------ 10 jira root 4.0K Dec 14 16:30 .

drwxr-xr-x 3 root root 4.0K Dec 13 16:07 ..

drwxrwxr-x 3 jira jira 4.0K Dec 14 16:30 caches

drwxrwxr-x 3 jira jira 4.0K Dec 13 16:08 data

-rw-rw-r-- 1 jira jira 1.1K Dec 13 16:17 dbconfig.xml

drwxrwxr-x 2 jira jira 4.0K Dec 13 20:09 export

drwxrwxr-x 4 jira jira 4.0K Dec 13 16:08 import

drwxrwxr-x 2 jira jira 4.0K Dec 13 16:09 log

drwxrwxr-x 2 jira jira 4.0K Dec 13 16:23 monitor

drwxrwxr-x 5 jira jira 4.0K Dec 13 16:08 plugins

drwxrwxr-x 3 jira jira 4.0K Dec 13 16:08 tmp

Thanks

[Volker Demel]

Hi Nikolas,

I have the same problem. My first install of Jira.

I am on a m1.small instance on ec2 running Ubuntu 12.04.1 LTS

Linux ip-10-224-67-182 3.2.0-34-virtual #53-Ubuntu SMP Thu Nov 15 11:08:40 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Database is Postgresql 8.4

Install went thru w/o probs, but the first user login failed with

Das gebündelte Gadget-Dashboard-Plugin ist nicht verfügbar

(the bundled Gadget-Dashboard-Plugin is not available)

(atlassian-jira.log)

Can anyone please assit?

Thanks

Volker

[Justin Alex Paramanandan]

1. Where is the location of the <JIRA_HOME> (since <JIRA_INSTALL> is at /var/atlassian/jira)?
2. Can you do an "ls -lah" for <JIRA_HOME> to list down the contents of the directory?