Jira Software Stuck on Database Setup Page

[gf]

Hello, 

I have successfully installed Jira Software and created database as guided buy Atlassian. I also added the MySQL connector required.

For some reason, I seem to be getting stuck on the database setup page. It shows connectivity and 175 tables are created but I can't get past this page and it's been an hour. All I get is a spinning wheel and "Please wait while the database is set up. This may take a minute..."

I initially thought it was an error I made and uninstalled about 3 times. With fresh installs, still experiencing the same issue. It is frustrating, please help.

OS is CentOS 7
4TB Storage, 2GB RAM with 6GB Swap

MySQL Connector Version - Java-8.0.17
JIRA Software - 8.3.1

[brbojorque]

Looking at the problem it is obvious that there is an infrastructure issue here.

You are running JIRA using 2GB RAM, the minimum requirements are 6GB RAM or even higher depending on the users as per https://confluence.atlassian.com/doc/server-hardware-requirements-guide-30736403.html

Please upgrade the infrastructure, make sure before you connect the database the JVM values are updated. xms = 1g and xmx = 3g

[gf]

Hello @brbojorque ,

infrastructure had been upgraded since the initial post with additional 6GB RAM space added. Still, the same issue occur.

Deleted MariaDB and installed MySQL, now stuck on

'Database: We've found an error in MySQL supported version!

The database setup is not supporting utf8mb4'

I never imagined i'd need to put in this much work to get this running. It is still not running.

[brbojorque]

It is very easy to fix, just drop the database and create a new one using the link below.

https://confluence.atlassian.com/kb/how-to-fix-the-collation-and-character-set-of-a-mysql-database-744326173.html

Or just use Postgres, its much easier just follow the steps below.

https://confluence.atlassian.com/conf64/database-setup-for-postgresql-936511499.html

[gf]

Hello @brbojorque Already done all that as well.

[brbojorque]

Hi @gf apologies the thread is too long I did not have the chance to read. 

Let me know if I can help in any way.

[gf]

Don't apologise @brbojorque I appreciate your kind assistance.

I have dropped the DB to see if anything changes and I seem to be seeing something that never showed before. 

Tables are being created as above but with progress bar on the startup screen moving with a message 'Loading Jira apps ...'

It never moved before, it always remained stuck for hours. I will update you with the progress made.

[gf]

@brbojorque  After waiting a while, i get the following of on the start up screen;

[brbojorque]

Hi @gf ,

So it looks like you need to have an updated MySQL Driver for 5.7 version and enable the utf8mb4 in mysqld configuration.

[mysqld]
...
character_set_server=utf8mb4

Download Driver

• Download the driver:
  • https://dev.mysql.com/downloads/connector/j/5.1.html
• Copy the driver in <install>/lib
• Restart JIRA

For reference please read more.

https://confluence.atlassian.com/adminjiraserver/connecting-jira-applications-to-mysql-5-7-966063305.html

[brbojorque]

Or just use postgresql and make your life easier :)

Don't forget to create the database with the right collation based on the documentation.

[gf]

I used PostgreSQL yesterday and it was all so easy till I started facing permission issues at setup. This then angered me enough to remove both Postgres & Maria with the consequence of me losing all my other DBs 

I'm making progress with MySQL now with the following you stated in link above;

https://confluence.atlassian.com/kb/how-to-fix-the-collation-and-character-set-of-a-mysql-database-744326173.html

Though I followed it before and didn't get anywhere, it is because they missed out a semicolon ';' in the KB/guide.

ALTER DATABASE yourDB CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;

SELECT CONCAT('ALTER TABLE ',  table_name, ' CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;')
FROM information_schema.TABLES AS T, information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` AS C
WHERE C.collation_name = T.table_collation
AND T.table_schema = 'jiradb'
AND
(
    C.CHARACTER_SET_NAME != 'utf8mb4'
    OR
    C.COLLATION_NAME != 'utf8mb4_bin'
);

However, I've had to let Jira do the auto setup to for it to work and then change licence. Exhausting!

I was told all these were partly due to hardware limitations but majorly, I feel it lies with limited compatibility issues.

[brbojorque]

Hi @gf , you are right, there a lot of complexities with setting up JIRA there quite few preparations you need to do and documentations to be read.

This is not to mention the maintenance of the systems and the overall health of the platform. 

If you are familiar with AWS then this quick start templates should probably help you with the infrastructure and setup.

https://aws.amazon.com/quickstart/architecture/jira/

[gf]

Hello @brbojorque @Alexis Robert Thanks to you both.

My final solution has been to remove Jira Software 8.x and install 7.x which is much lighter on RAM and infrastructure.

It's been a journey. Many thanks once again.

[Alexis Robert]

Hi @gf , 

it might be a permissions issue on the database, can you have a look at the logs in JIRA_INSTALL_DIR/log/catalina.out and see if you have any errors related to the database ? 

[gf]

Hello @Alexis Robert many thanks for your response. I have had a look at the logs but I'm not tech savvy, I only know how to follow guidelines for now so the content doesn't make any sense to me.

Shall I attach the log on here for your perusal?

[Alexis Robert]

Hi @gf , 

if you open the log files, can you search for lines starting with ERROR ? You can then copy these lines here.

[gf]

Hello @Alexis Robert

the content of the logs are as below;

06-Aug-2019 11:52:52.531 INFO [http-nio-8080-exec-15] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:430)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:684)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)

06-Aug-2019 11:52:52.531 INFO [http-nio-8080-exec-16] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:430)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:684)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)

[Alexis Robert]

Hi @gf , 

these log lines start with "INFO" as you can see, we need the ones starting with "ERROR" to get a more detailed view of what is going wrong.

[gf]

Hello @Alexis Robert there are no lines starting with 'Error' in the catalina.out file

The only ones in there are the ones with error in between the lines similar to above.

Is there no facility for a screen share or remote assistance? I urgently need a fix on this matter.

[gf]

All the other bits of the log say 'SEVERE' not 'ERROR' - Please see below;

05-Aug-2019 20:40:45.629 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [org.picocontainer.DefaultPicoContainer.IntoThreadLocal] (value [org.picocontainer.DefaultPicoContainer$IntoThreadLocal@6cf3cfbc]) and a value of type [java.lang.Class] (value [class org.picocontainer.ComponentAdapter$NOTHING]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
05-Aug-2019 20:40:45.664 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@60865b]) and a value of type [com.atlassian.jira.tenancy.JiraTenantImpl] (value [JiraTenantImpl{id='fakeTenantId'}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
05-Aug-2019 20:40:45.666 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1ac310e7]) and a value of type [com.atlassian.jira.web.filters.ThreadLocalQueryProfiler] (value [com.atlassian.jira.web.filters.ThreadLocalQueryProfiler@6b8cfcdb]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
05-Aug-2019 20:40:45.672 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [webwork.action.ActionContext$1] (value [webwork.action.ActionContext$1@19705039]) and a value of type [webwork.action.ActionContext] (value [webwork.action.ActionContext@1da65f12]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
05-Aug-2019 20:40:45.685 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@60865b]) and a value of type [com.atlassian.jira.tenancy.JiraTenantImpl] (value [JiraTenantImpl{id='fakeTenantId'}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
05-Aug-2019 20:40:45.690 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [webwork.action.ActionContext$1] (value [webwork.action.ActionContext$1@19705039]) and a value of type [webwork.action.ActionContext] (value [webwork.action.ActionContext@711c5aa2]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
05-Aug-2019 20:40:45.694 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@60865b]) and a value of type [com.atlassian.jira.tenancy.JiraTenantImpl] (value [JiraTenantImpl{id='fakeTenantId'}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.

So i uninstalled and reinstalled, now the only ERROR is;

2019-08-06 18:15:14,581 http-nio-8080-exec-1 ERROR anonymous 1095x1x1 - 86.169.51.64 / [c.atlassian.instrumentation.Instrument] Unable to snapshot thread local operations (implementation of OpTimerFactory is not a ThreadLocalOpTimerFactory): null

[gf]

/opt/atlassian/jira/logs/atlassian-jira-gc-2019-08-07_18-04-05.log.0.current
OpenJDK 64-Bit Server VM (25.202-b08) for linux-amd64 JRE (1.8.0_202-b08), built on Jan 22 2019 13:35:07 by "jenkins" with gcc 4.8.2 20140120 (Red Hat 4.8.2-15)
Memory: 4k page, physical 1872268k(1038084k free), swap 2097148k(1174116k free)
CommandLine flags: -XX:GCLogFileSize=20971520 -XX:InitialCodeCacheSize=33554432 -XX:InitialHeapSize=402653184 -XX:MaxHeapSize=2147483648 -XX:NumberOfGCLogFiles=5 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCCause -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=536870912 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseGCLogFileRotation -XX:+UseParallelGC 
2019-08-07T18:04:16.668+0100: 11.350: [GC (Allocation Failure) [PSYoungGen: 98304K->11538K(114688K)] 98304K->11618K(376832K), 0.0658262 secs] [Times: user=0.16 sys=0.03, real=0.07 secs] 
2019-08-07T18:04:23.324+0100: 18.005: [GC (Allocation Failure) [PSYoungGen: 109842K->11350K(114688K)] 109922K->11502K(376832K), 0.0564180 secs] [Times: user=0.13 sys=0.03, real=0.05 secs] 
2019-08-07T18:04:29.957+0100: 24.638: [GC (Allocation Failure) [PSYoungGen: 109654K->11399K(114688K)] 109806K->11559K(376832K), 0.0410507 secs] [Times: user=0.12 sys=0.00, real=0.04 secs] 
2019-08-07T18:04:36.980+0100: 31.662: [GC (Allocation Failure) [PSYoungGen: 109703K->12166K(114688K)] 109863K->12326K(376832K), 0.0505078 secs] [Times: user=0.16 sys=0.00, real=0.05 secs] 
2019-08-07T18:04:40.376+0100: 35.057: [GC (Allocation Failure) [PSYoungGen: 110470K->12500K(114688K)] 110630K->12668K(376832K), 0.0532886 secs] [Times: user=0.15 sys=0.01, real=0.05 secs] 
2019-08-07T18:04:45.483+0100: 40.164: [GC (Allocation Failure) [PSYoungGen: 110804K->12779K(117248K)] 110972K->13447K(379392K), 0.0793521 secs] [Times: user=0.24 sys=0.02, real=0.08 secs] 
2019-08-07T18:04:50.670+0100: 45.351: [GC (Allocation Failure) [PSYoungGen: 117227K->4640K(118272K)] 117895K->15070K(380416K), 0.1549507 secs] [Times: user=0.43 sys=0.05, real=0.16 secs] 
2019-08-07T18:04:56.171+0100: 50.852: [GC (Allocation Failure) [PSYoungGen: 109088K->4207K(222720K)] 119518K->15813K(484864K), 0.0463418 secs] [Times: user=0.11 sys=0.03, real=0.05 secs] 
2019-08-07T18:05:07.400+0100: 62.081: [GC (Allocation Failure) [PSYoungGen: 212079K->8088K(223744K)] 223685K->20974K(485888K), 0.0839893 secs] [Times: user=0.08 sys=0.03, real=0.09 secs] 
2019-08-07T18:05:14.752+0100: 69.434: [GC (Allocation Failure) [PSYoungGen: 215960K->13015K(348160K)] 228846K->26381K(610304K), 0.0845298 secs] [Times: user=0.21 sys=0.05, real=0.09 secs] 
2019-08-07T18:05:20.049+0100: 74.730: [GC (Metadata GC Threshold) [PSYoungGen: 167644K->4466K(348672K)] 181010K->22649K(610816K), 0.0486149 secs] [Times: user=0.11 sys=0.04, real=0.05 secs] 
2019-08-07T18:05:20.098+0100: 74.779: [Full GC (Metadata GC Threshold) [PSYoungGen: 4466K->0K(348672K)] [ParOldGen: 18182K->17974K(262144K)] 22649K->17974K(610816K), [Metaspace: 20685K->20685K(1069056K)], 0.3977013 secs] [Times: user=1.18 sys=0.13, real=0.40 secs] 
2019-08-07T18:05:38.894+0100: 93.575: [GC (Allocation Failure) [PSYoungGen: 332288K->15167K(477696K)] 350262K->33150K(739840K), 0.1196809 secs] [Times: user=0.20 sys=0.04, real=0.12 secs] 
2019-08-07T18:05:40.035+0100: 94.716: [GC (System.gc()) [PSYoungGen: 28925K->224K(462848K)] 46907K->27072K(724992K), 0.0680786 secs] [Times: user=0.20 sys=0.04, real=0.06 secs] 
2019-08-07T18:05:40.103+0100: 94.785: [Full GC (System.gc()) [PSYoungGen: 224K->0K(462848K)] [ParOldGen: 26848K->24145K(262144K)] 27072K->24145K(724992K), [Metaspace: 31840K->31840K(1079296K)], 0.6087654 secs] [Times: user=1.99 sys=0.03, real=0.60 secs] 
​

[Alexis Robert]

Hi @gf , 

the issue is pretty clear: you don't have enough memory on your server, leading to these errors (GC Allocation Failure means the system can't allocate memory to Jira).

You need to increase the server memory from 2GB to 4GB and try again the installation.

[gf]

Hello @Alexis Robert 

the memory has been increased to about 6GB but yet, the same issue occurs. See below;

OpenJDK 64-Bit Server VM (25.202-b08) for linux-amd64 JRE (1.8.0_202-b08), built on Jan 22 2019 13:35:07 by "jenkins" with gcc 4.8.2 20140120 (Red Hat 4.8.2-15)
Memory: 4k page, physical 1872268k(124144k free), swap 6097144k(6037488k free)
CommandLine flags: -XX:GCLogFileSize=20971520 -XX:InitialCodeCacheSize=33554432 -XX:InitialHeapSize=402653184 -XX:MaxHeapSize=2147483648 -XX:NumberOfGCLogFiles=5 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCCause -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=536870912 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseGCLogFileRotation -XX:+UseParallelGC 
2019-08-10T17:28:22.909+0100: 12.478: [GC (Allocation Failure) [PSYoungGen: 98304K->11517K(114688K)] 98304K->11597K(376832K), 0.1131190 secs] [Times: user=0.22 sys=0.05, real=0.11 secs] 
2019-08-10T17:28:28.739+0100: 18.308: [GC (Allocation Failure) [PSYoungGen: 109821K->11211K(114688K)] 109901K->11299K(376832K), 0.0623749 secs] [Times: user=0.17 sys=0.04, real=0.06 secs] 
2019-08-10T17:28:32.137+0100: 21.706: [GC (Allocation Failure) [PSYoungGen: 109515K->11503K(114688K)] 109603K->11599K(376832K), 0.0433805 secs] [Times: user=0.13 sys=0.00, real=0.04 secs] 
2019-08-10T17:28:35.278+0100: 24.848: [GC (Allocation Failure) [PSYoungGen: 109807K->12610K(212992K)] 109903K->12778K(475136K), 0.0553759 secs] [Times: user=0.13 sys=0.04, real=0.06 secs] 
2019-08-10T17:28:42.993+0100: 32.562: [GC (Allocation Failure) [PSYoungGen: 209218K->14171K(212992K)] 209386K->14347K(475136K), 0.1111522 secs] [Times: user=0.31 sys=0.06, real=0.11 secs] 
2019-08-10T17:28:50.773+0100: 40.342: [GC (Allocation Failure) [PSYoungGen: 210779K->14327K(405504K)] 210955K->16215K(667648K), 0.1001112 secs] [Times: user=0.23 sys=0.05, real=0.11 secs] 
2019-08-10T17:29:11.734+0100: 61.371: [GC (Allocation Failure) [PSYoungGen: 405495K->16376K(407552K)] 407383K->27516K(669696K), 0.7074021 secs] [Times: user=0.23 sys=0.07, real=0.70 secs] 
2019-08-10T17:29:21.034+0100: 70.604: [GC (Metadata GC Threshold) [PSYoungGen: 189557K->9906K(676864K)] 200697K->21334K(939008K), 0.0749521 secs] [Times: user=0.20 sys=0.04, real=0.07 secs] 
2019-08-10T17:29:21.110+0100: 70.679: [Full GC (Metadata GC Threshold) [PSYoungGen: 9906K->0K(676864K)] [ParOldGen: 11428K->18512K(262144K)] 21334K->18512K(939008K), [Metaspace: 20681K->20681K(1069056K)], 0.3215499 secs] [Times: user=0.85 sys=0.06, real=0.32 secs] 
2019-08-10T17:29:47.501+0100: 97.079: [GC (Allocation Failure) [PSYoungGen: 654848K->18054K(673280K)] 673360K->36574K(935424K), 0.5407929 secs] [Times: user=0.21 sys=0.05, real=0.54 secs] 
2019-08-10T17:30:35.779+0100: 145.356: [GC (Metadata GC Threshold) [PSYoungGen: 202782K->13817K(676864K)] 221302K->32345K(939008K), 0.1936948 secs] [Times: user=0.15 sys=0.01, real=0.20 secs] 
2019-08-10T17:30:35.988+0100: 145.558: [Full GC (Metadata GC Threshold) [PSYoungGen: 13817K->0K(676864K)] [ParOldGen: 18528K->28897K(262144K)] 32345K->28897K(939008K), [Metaspace: 34426K->34426K(1081344K)], 0.3905433 secs] [Times: user=0.81 sys=0.03, real=0.39 secs] 
2019-08-10T17:32:17.658+0100: 247.227: [GC (Allocation Failure) [PSYoungGen: 653824K->22004K(675840K)] 682721K->59780K(937984K), 0.2392765 secs] [Times: user=0.31 sys=0.03, real=0.24 secs] 
2019-08-10T17:41:05.988+0100: 775.558: [GC (Metadata GC Threshold) [PSYoungGen: 632840K->17961K(667648K)] 670615K->72188K(929792K), 0.3674903 secs] [Times: user=0.84 sys=0.10, real=0.37 secs] 
2019-08-10T17:41:06.357+0100: 775.927: [Full GC (Metadata GC Threshold) [PSYoungGen: 17961K->0K(667648K)] [ParOldGen: 54227K->59484K(262144K)] 72188K->59484K(929792K), [Metaspace: 56657K->56602K(1101824K)], 1.5454973 secs] [Times: user=4.00 sys=0.08, real=1.55 secs] 
2019-08-10T17:41:39.900+0100: 809.469: [GC (Allocation Failure) [PSYoungGen: 644608K->12938K(632832K)] 704092K->72431K(894976K), 0.0953101 secs] [Times: user=0.30 sys=0.01, real=0.10 secs] 
2019-08-10T17:47:42.254+0100: 1171.824: [GC (Allocation Failure) [PSYoungGen: 632458K->16599K(612352K)] 691951K->81785K(874496K), 0.3948814 secs] [Times: user=0.44 sys=0.05, real=0.39 secs] 
2019-08-10T17:54:34.750+0100: 1584.319: [GC (Allocation Failure) [PSYoungGen: 612055K->1712K(574464K)] 677241K->78592K(836608K), 1.3012047 secs] [Times: user=1.15 sys=0.25, real=1.30 secs] 

[Alexis Robert]

Hi @gf , 

can you try changing the JVM memory parameters for Jira ? Stop Jira and follow these insctructions : 

1. To increase heap space memory in Linux installations:
   In your <JIRA application installation directory>/bin open the setenv.sh file.
2. Find the sections JVM_MINIMUM_MEMORY= and JVM_MAXIMUM_MEMORY=
3. Change both values to 2048m

 Start Jira again and let me know if you still have issues

[gf]

Hello @Alexis Robert 

done as advised but still experiencing the same issue. :(  This is a terrible experience. I had so much hope in Jira as it looked refined. It has taken 7 days and I'm still unable to use the software with minor projects pending.

OpenJDK 64-Bit Server VM (25.202-b08) for linux-amd64 JRE (1.8.0_202-b08), built on Jan 22 2019 13:35:07 by "jenkins" with gcc 4.8.2 20140120 (Red Hat 4.8.2-15)
Memory: 4k page, physical 1872268k(969372k free), swap 6097144k(5133232k free)
CommandLine flags: -XX:GCLogFileSize=20971520 -XX:InitialCodeCacheSize=33554432 -XX:InitialHeapSize=2147483648 -XX:MaxHeapSize=2147483648 -XX:NumberOfGCLogFiles=5 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCCause -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=536870912 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseGCLogFileRotation -XX:+UseParallelGC 
2019-08-12T12:15:50.025+0100: 26.806: [GC (Allocation Failure) [PSYoungGen: 524800K->19752K(611840K)] 524800K->19904K(2010112K), 0.0991870 secs] [Times: user=0.21 sys=0.05, real=0.10 secs] 
2019-08-12T12:16:09.219+0100: 46.001: [GC (Allocation Failure) [PSYoungGen: 544552K->23665K(611840K)] 544704K->23825K(2010112K), 0.1610852 secs] [Times: user=0.50 sys=0.07, real=0.17 secs] 
2019-08-12T12:16:20.996+0100: 57.778: [GC (Metadata GC Threshold) [PSYoungGen: 385530K->25670K(611840K)] 385690K->25854K(2010112K), 0.1247635 secs] [Times: user=0.39 sys=0.01, real=0.13 secs] 
2019-08-12T12:16:21.122+0100: 57.903: [Full GC (Metadata GC Threshold) [PSYoungGen: 25670K->0K(611840K)] [ParOldGen: 184K->25230K(1398272K)] 25854K->25230K(2010112K), [Metaspace: 20613K->20613K(1069056K)], 0.3350832 secs] [Times: user=0.85 sys=0.13, real=0.33 secs] 
2019-08-12T12:16:35.059+0100: 71.840: [GC (Allocation Failure) [PSYoungGen: 524800K->16800K(611840K)] 550030K->42039K(2010112K), 0.0729497 secs] [Times: user=0.15 sys=0.00, real=0.08 secs] 
2019-08-12T12:16:43.407+0100: 80.189: [GC (Metadata GC Threshold) [PSYoungGen: 292471K->14114K(611840K)] 317709K->39361K(2010112K), 0.0657202 secs] [Times: user=0.22 sys=0.01, real=0.07 secs] 
2019-08-12T12:16:43.474+0100: 80.255: [Full GC (Metadata GC Threshold) [PSYoungGen: 14114K->0K(611840K)] [ParOldGen: 25246K->29593K(1398272K)] 39361K->29593K(2010112K), [Metaspace: 34442K->34442K(1081344K)], 0.3630283 secs] [Times: user=1.14 sys=0.01, real=0.36 secs] 
2019-08-12T12:17:07.152+0100: 103.934: [GC (Allocation Failure) [PSYoungGen: 524800K->26924K(664576K)] 554393K->56526K(2062848K), 0.2032977 secs] [Times: user=0.52 sys=0.08, real=0.20 secs] 
2019-08-12T12:17:55.067+0100: 151.848: [GC (Metadata GC Threshold) [PSYoungGen: 600552K->31714K(667136K)] 630153K->66832K(2065408K), 0.2408229 secs] [Times: user=0.54 sys=0.13, real=0.24 secs] 
2019-08-12T12:17:55.308+0100: 152.090: [Full GC (Metadata GC Threshold) [PSYoungGen: 31714K->0K(667136K)] [ParOldGen: 35117K->62501K(1398272K)] 66832K->62501K(2065408K), [Metaspace: 56549K->56535K(1101824K)], 1.1391777 secs] [Times: user=3.17 sys=0.38, real=1.14 secs] 
2019-08-12T12:21:37.285+0100: 374.066: [GC (Allocation Failure) [PSYoungGen: 635392K->20478K(655872K)] 697893K->82996K(2054144K), 0.2565372 secs] [Times: user=0.24 sys=0.01, real=0.26 secs] 

[Alexis Robert]

are you using the JRE binary bundled with your linux distribution or the one bundled with the Jira installer ? 

Atlassian only supports AdoptOpenJDK or Oracle JRE as mentioned here : https://confluence.atlassian.com/adminjiraserver/supported-platforms-938846830.html

[gf]

Hello,

I'm, using the one bundled with the installer.

Also i use MySQL with MariaDB. Could this be an issue? Though 175 tables are successfully written to DB.

[Alexis Robert]

Well MariaDB is not officially supported, and the logs you're showing do not point to a DB issue but you should still try to deploy Jira with the recommended versions if possible.

[gf]

I feel like I have exhausted every avenue and don't know what else I could try. Could this be the result of a bug?

This is the latest log:

OpenJDK 64-Bit Server VM (25.202-b08) for linux-amd64 JRE (1.8.0_202-b08), built on Jan 22 2019 13:35:07 by "jenkins" with gcc 4.8.2 20140120 (Red Hat 4.8.2-15)
Memory: 4k page, physical 1872268k(695740k free), swap 6097144k(5088264k free)
CommandLine flags: -XX:GCLogFileSize=20971520 -XX:InitialCodeCacheSize=33554432 -XX:InitialHeapSize=2147483648 -XX:MaxHeapSize=2147483648 -XX:NumberOfGCLogFiles=5 -XX:-OmitStackTraceInFastThrow -XX:+PrintGC -XX:+PrintGCCause -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:ReservedCodeCacheSize=536870912 -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseGCLogFileRotation -XX:+UseParallelGC 
2019-08-12T15:48:54.448+0100: 41.273: [GC (Allocation Failure) [PSYoungGen: 524800K->19880K(611840K)] 524800K->20032K(2010112K), 0.1966043 secs] [Times: user=0.33 sys=0.10, real=0.20 secs] 
2019-08-12T15:49:17.297+0100: 64.121: [GC (Allocation Failure) [PSYoungGen: 544680K->24351K(611840K)] 544832K->24511K(2010112K), 0.1583792 secs] [Times: user=0.35 sys=0.09, real=0.16 secs] 
2019-08-12T15:49:30.173+0100: 76.998: [GC (Metadata GC Threshold) [PSYoungGen: 406955K->26860K(611840K)] 407115K->27036K(2010112K), 0.1399459 secs] [Times: user=0.37 sys=0.03, real=0.14 secs] 
2019-08-12T15:49:30.314+0100: 77.138: [Full GC (Metadata GC Threshold) [PSYoungGen: 26860K->0K(611840K)] [ParOldGen: 176K->26332K(1398272K)] 27036K->26332K(2010112K), [Metaspace: 20678K->20678K(1069056K)], 0.4415408 secs] [Times: user=0.85 sys=0.28, real=0.44 secs] 
2019-08-12T15:49:54.233+0100: 101.058: [GC (System.gc()) [PSYoungGen: 330743K->13095K(611840K)] 357075K->39436K(2010112K), 0.0683553 secs] [Times: user=0.21 sys=0.00, real=0.07 secs] 
2019-08-12T15:49:54.302+0100: 101.127: [Full GC (System.gc()) [PSYoungGen: 13095K->0K(611840K)] [ParOldGen: 26340K->28036K(1398272K)] 39436K->28036K(2010112K), [Metaspace: 30083K->30083K(1077248K)], 0.6190763 secs] [Times: user=1.82 sys=0.08, real=0.62 secs] 
2019-08-12T15:51:29.983+0100: 196.808: [GC (Allocation Failure) [PSYoungGen: 524800K->11796K(611840K)] 552836K->39841K(2010112K), 0.2087734 secs] [Times: user=0.15 sys=0.01, real=0.21 secs] 
2019-08-12T15:56:56.333+0100: 523.158: [GC (Allocation Failure) [PSYoungGen: 536596K->6852K(666112K)] 564641K->34904K(2064384K), 0.0830882 secs] [Times: user=0.11 sys=0.01, real=0.08 secs] 
2019-08-12T16:02:23.035+0100: 849.860: [GC (Allocation Failure) [PSYoungGen: 585924K->9939K(611840K)] 613976K->37999K(2010112K), 0.1168539 secs] [Times: user=0.12 sys=0.02, real=0.12 secs] 
2019-08-12T16:10:10.974+0100: 1317.799: [GC (Allocation Failure) [PSYoungGen: 589011K->7735K(666624K)] 617071K->35803K(2064896K), 0.1138867 secs] [Times: user=0.13 sys=0.02, real=0.11 secs] 
2019-08-12T16:18:07.110+0100: 1793.945: [GC (Allocation Failure) [PSYoungGen: 642103K->7572K(666624K)] 670171K->35648K(2064896K), 0.4461006 secs] [Times: user=0.18 sys=0.04, real=0.44 secs] 
2019-08-12T16:26:09.848+0100: 2276.676: [GC (Allocation Failure) [PSYoungGen: 641940K->7813K(667648K)] 670016K->35897K(2065920K), 0.1160348 secs] [Times: user=0.18 sys=0.01, real=0.12 secs] 
2019-08-12T16:34:08.517+0100: 2755.343: [GC (Allocation Failure) [PSYoungGen: 643717K->7591K(667136K)] 671801K->35683K(2065408K), 0.1453826 secs] [Times: user=0.11 sys=0.01, real=0.15 secs] 
2019-08-12T16:42:13.949+0100: 3240.773: [GC (Allocation Failure) [PSYoungGen: 643495K->7671K(669184K)] 671587K->35771K(2067456K), 0.0495611 secs] [Times: user=0.11 sys=0.00, real=0.05 secs] 

[Alexis Robert]

there's clearly an issue with available memory on this server as the error is very clear : the java process can't allocate memory to get Jira running.

You should definitely investigate more on this front.

[gf]

Very strange indeed. Is there nothing else you can recommend I try please? Also, this community seems to quiet. It's just been you assisting with this and it makes me wonder. It's called a community right? Where is everyone else with ideas and solutions? :(

[gf]

Hello @Alexis Robert found this link with a solution;

https://community.atlassian.com/t5/Jira-questions/Java-GC-blocks-Jira/qaq-p/122767#U1152400

Here is the proposal from Atlassian support. It' slightly different:

Based on the Database statistic, it seems that this is a medium size instance. Based on the GCViewer image as per attached in the screenshot GClog, it seems that the timespent for the GC is really high for on the 27th July 2012. We also notice that the heap usage was high during the day. Since the time taken for the GC full GC is high, this might be cause by the size of the heap that Full GC need to clear is very large.

Therefore, we would like to recommend to adjust the current JVM input parameter to the following. In Linux, please edit the <tt>JIRA_Install/bin/setenv.sh</tt>:

• Use a parallel collector: <tt>-XX:+UseParallelOldGC</tt>. This will ensure that GC will perform the Garbage Collection more frequently. But, every collection should be small thus the lag time should be reduce.
• Disable remote clients from triggering a full GC event <tt>-XX:+DisableExplicitGC</tt>
• Set the Young space up to 30-40% of the overall heap. Eg: <tt>-XX:NewSize=1229m</tt>
• optionally, reduce the size of the -Xmx to 3GB because as a benchmark we allocate 1 GB of heap to instance with 100,000 of issues. Reducing the -Xmx will reduce the amount of heap that GC need to clear for each Full GC.

Do let us know how it goes.

But the issue is, I don't know how to implement the solution.

Any ideas?