Forums

Articles
Create
cancel
Showing results for 
Search instead for 
Did you mean: 

java.lang.OutOfMemoryError: unable to create new native thread

Kosta September 9, 2018

Hi,

on startup of Confluence I'm getting a lot of OutOfMemory messages.

Setting CATALINA_OPTS="-Xms2048m -Xmx2048m -XX:+UseG1GC ${CATALINA_OPTS}" didn't solve the problem.

--------------------------

02-Sep-2018 21:15:14.916 INFORMATION [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8090"]
02-Sep-2018 21:15:14.999 INFORMATION [main] org.apache.catalina.startup.Catalina.start Server startup in 210098 ms
Exception in thread "Dispatch-Thread-for-com.mchange.v2.async.ThreadPerTaskAsynchronousRunner@49a60313" java.lang.OutOfMemoryError: unable to create new native th
read
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717)
        at com.mchange.v2.async.ThreadPerTaskAsynchronousRunner$DispatchThread.run(ThreadPerTaskAsynchronousRunner.java:214)
Exception in thread "C3P0PooledConnectionPoolManager[identityToken->ntek999x1778udb1lzcqhe|3dffa4e1]-AdminTaskTimer" java.lang.OutOfMemoryError: unable to create
new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner.recreateThreadsAndTasks(ThreadPoolAsynchronousRunner.java:535)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner.access$400(ThreadPoolAsynchronousRunner.java:49)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector.run(ThreadPoolAsynchronousRunner.java:825)
        at java.util.TimerThread.mainLoop(Timer.java:555)
        at java.util.TimerThread.run(Timer.java:505)
Exception in thread "C3P0PooledConnectionPoolManager[identityToken->ntek999x1778udb1lzcqhe|71b539b]-AdminTaskTimer" java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner.recreateThreadsAndTasks(ThreadPoolAsynchronousRunner.java:535)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner.access$400(ThreadPoolAsynchronousRunner.java:49)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector.run(ThreadPoolAsynchronousRunner.java:825)
        at java.util.TimerThread.mainLoop(Timer.java:555)
        at java.util.TimerThread.run(Timer.java:505)
Exception in thread "C3P0PooledConnectionPoolManager[identityToken->ntek999x1778udb1lzcqhe|12e44862]-AdminTaskTimer" java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:717)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner.recreateThreadsAndTasks(ThreadPoolAsynchronousRunner.java:535)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner.access$400(ThreadPoolAsynchronousRunner.java:49)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector.run(ThreadPoolAsynchronousRunner.java:825)
        at java.util.TimerThread.mainLoop(Timer.java:555)
        at java.util.TimerThread.run(Timer.java:505)

--------------------------

 

2 answers

0 votes
Igor M.
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 9, 2018

Hello,

Could you clarify which version of OS you are using?

There is an issue with the certain version of OS that can result in similar failures you described. The issue is outlined in Confluence crashes and throws 'OutOfMemory: unable to create new native thread' error KB.

Can you try the workaround from KB?

  • Edit the $CONFLUENCE_INSTALL/bin/setenv.sh file and add the below to the top of the file:
ulimit -u 4096
ulimit -n 4096


These values may be different depending upon the operating system used, consult with your System Administrator / hosting provider about the most appropriate limits to set.

  • Restart confluence.
Kosta September 10, 2018

Hi Igor,

I'm running Confluence on Ubuntu 18.04.1 LTS (Bionic Beaver).

It seems that there is no ulimit -u on user level. Only root can do.

root@h2798701:~# ulimit -u 4096
root@h2798701:~# su - confluence
No directory, logging in with HOME=/
$ ulimit -u 4096
-su: 1: ulimit: Illegal option -u
$ ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        8192
coredump(blocks)     0
memory(kbytes)       unlimited
locked memory(kbytes) 16384
process              8192
nofiles              4096
vmemory(kbytes)      unlimited
locks                unlimited
rtprio               0

Since setenv.sh is executed as user confluence it gives an illegal option error.

So I increased "process" to 8192 in limits.conf. But it didn't change the behavior.

Igor M.
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 10, 2018

Can you also increase maxThreads in <conf-install>\conf\server.xml for the connector used, I would recommend setting it to 100 while hibernate.c3p0.max_size from <confluence-home>/confluence.cfg.xml should be set to at least 120 (10 counts of 25% higher rule)

Kosta September 10, 2018

No success.

Right now I'm facing

-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable

errors when I try to login. A jira running on the same box is now facing also OutOfMemory unable to create new native thread errors. 

A nextcloud installation seems not affected.

Kosta September 12, 2018

@Igor M_Hi, any other hind what could block Confluence from starting and consume all resources on the machine?

Igor M.
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 17, 2018

@Konstantinos Sianavas considering that both Confluence and Jira are facing the same problem and these do not share memory or processing power in any way, this would indicate a problem with the actual server, I would go back to the problem with thread allocation, for example, this ubuntu KB discusses the same topic 

0 votes
Craig Castle-Mead
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
September 9, 2018

Hi Konstantinos,

How much spare RAM does the server have you're trying to start Confluence on?

If this is a Linux box "free -m" should give you the answer.

CCM

Kosta September 9, 2018
:/opt/atlassian/confluence/bin# free -m
              total        used        free      shared  buff/cache   available
Mem:          12288        1234       10894          14         159        6135
Swap:             0           0           0

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events