Confluence on Debain 9 with MySQL 5.7 crashes after installation

[Hendrik]

I have installed Confluence on Debian 9 and it fails to start after successful installation with: confluence.lifecycle.core:dblowercheck out of memory error - unable to create new native thread.

My setup:

- Virtual server env with debian 9
- MySQL 5.7
- PHP 7.2
- Confluence 6.12.0
- Mysql driver 5.1.47
- Jira Software 7.12.3

The virtual Server has 6 vCPU cores and 12GB ram.

I followed this guide for MySQL server Installation and in addition, I followed this guide to Memory

Jira is running on the same server without any issues.

[Jochen]

Hi, after a lot of trial and error and research I found a solution based on this Bitbucket knowledge base entry!

The systemd value for DefaultTasksMax was set too low in our hosting environment.

Check current setting (Debian 9):

$ systemctl show --property DefaultTasksMax

This was set to 165 on our system. A more common default seems to be 512 which is what I used.

Set a new value by editing /etc/systemd/system.conf and change/add the line:

DefaultTasksMax=512

 After saving the file reload systemd for changes to take effect:

$ systemctl daemon-reload

With the new setting Confluence finally startet up without crashing.

Keep in mind that this is my first attempt at changing the setting. I don't know yet if 512 will work for long-term operation of Confluence but at least it starts up now without running into "java.lang.OutOfMemoryError: unable to create new native thread".

Hope this helps...

[Jochen]

Hi Hendrik,

Did you find a solution yet?

I have exactly the same issue with Confluence 6.13 on a very similar setup. The only other differences are that we use Jira 7.13 (which is running fine) and don't have PHP installed. But I don't think those differences matter with regard to the out of memory error.

I tried everything that has been recommended in the documentaion I found so far including xms/xmx adjustments (currently at 1.5GB each, also tried 1 and 2 GB) and increasing ulimits (started with the recommended 4096/8192 hard/soft limits for nofile and nproc, currently at 8192/16384).

Your hosting specs look familiar. You don't happen to be hosted on a "v40" Linux setup at a well known german hoster?

I agree that it is probably not a high performance setup but it should be sufficient for a small team. At least Confluence should start without crashing.

[Josh Steckler]

Remember that Confluence also has a separate heap setting for Synchrony (collaborative editing) - perhaps that 's what is causing the issue?

[Jochen]

Thanks Josh, I actually did not know that.

I added the following setting to /opt/atlassian/confluence/bin/setenv.sh but it did not change the behaviour.

CATALINA_OPTS="-Dsynchrony.memory.max=2g ${CATALINA_OPTS}"

Even with the Jira instance stopped I run into the error "java.lang.OutOfMemoryError: unable to create new native thread" when starting Confluence.

 Any recommendations?

[Josh Steckler]

You need to total up how much memory each app's maximum memory is set to and make sure it's lower than 12gb.

Running Jira, Confluence and the database all on one server is probably not going to perform well.

[Hendrik]

Thank you for your answer.

Total consumption is at 10. I know that this Setup is not high Performance but same configuration works well with Debian 8.

[Hendrik]

The Installation fails even if the Server has been reinstalled and only Confluence is installed.

Any idea?

[Josh Steckler]

Still with the out of memory error?

It may be necessary to use lower limits. Leaving 2/10 for the operating system may not be enough. 

[Hendrik]

Hi Josh,

thanks for your answer. The Basic System consumes less then 2GB Memory (including Debian, mysql Nothing more is processed). I cannot find any other Linux or mysql related issues.

[Josh Steckler]

Can you post your Xms and Xmx settings from both Jira and Confluence, and your memory settings from your mysql config file?

Also what version of Java are you using (and is it oracle java or openjdk)?