Jira is Unable to start after upgrade to 8.20.3

[Seemi Anwar]

Hi Experts,

My Jira is hosted on AWS Ec2 machine. I have upgraded my Jira from 8.16.0 to latest version 8.20.3. It has been upgraded successfully without throwing any error. But i cannot access Jira on my browser. On browser it throws the following error: 

Database: JIRA couldn't connect to your database

JIRA failed to establish a connection to your database.
This could be because:

• Your database isn't running
• The configuration of your dbconfig.xml file is incorrect (user, password, or database URL etc.)
• There is a network issue between JIRA and your database (e.g. firewall, database doesn't allow remote access etc.)

The current status of Jira is :

● jira.service
Loaded: loaded (/etc/init.d/jira; generated)
Active: active (exited) since Wed 2021-12-29 22:19:56 UTC; 12h ago
Docs: man:systemd-sysv-generator(8)
Process: 1574 ExecStart=/etc/init.d/jira start (code=exited, status=0/SUCCESS)

Dec 29 22:19:55 ip-172-31-12-84 jira[1574]: Atlassian Jira
Dec 29 22:19:55 ip-172-31-12-84 jira[1574]: Version : 8.20.3
Dec 29 22:19:55 ip-172-31-12-84 jira[1574]: [19B blob data]
Dec 29 22:19:55 ip-172-31-12-84 jira[1574]: If you encounter issues starting or stopping Jira, please see the Troubleshooting guide at https://docs.atlassian.com/jira/j
Dec 29 22:19:55 ip-172-31-12-84 jira[1574]: Server startup logs are located in /opt/atlassian/jira/logs/catalina.out
Dec 29 22:19:56 ip-172-31-12-84 jira[1574]: Existing PID file found during start.
Dec 29 22:19:56 ip-172-31-12-84 jira[1574]: Removing/clearing stale PID file.
Dec 29 22:19:56 ip-172-31-12-84 jira[1574]: Tomcat started.
Dec 29 22:19:56 ip-172-31-12-84 runuser[1591]: pam_unix(runuser:session): session closed for user jira
Dec 29 22:19:56 ip-172-31-12-84 systemd[1]: Started jira.service.

I think there is a user or a permission issue.

Can anyone guide me please?

Logs are below.

1- atlassian-jira.log

2021-12-29 22:08:30,637+0000 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]

****************
Jira starting...
****************

2021-12-29 22:08:30,784+0000 JIRA-Bootstrap INFO [c.a.jira.startup.JiraStartupLogger]

___ Environment _____________________________

JIRA Build : 8.20.3#820003-sha1:e9c1f3b400a5545d56ec3d8e0414de7a0e75ddb3
Build Date : Thu Dec 16 00:00:00 UTC 2021
JIRA Installation Type : Standalone
Application Server : Apache Tomcat/8.5.65 - Servlet API 3.1
Java Version : 11.0.11 - AdoptOpenJDK
Current Working Directory : /opt/atlassian/jira/work
Maximum Allowable Memory : 2048MB
Total Memory : 384MB
Free Memory : 214MB
Used Memory : 170MB
Memory Pool: CodeHeap 'non-nmethods' : CodeHeap 'non-nmethods': init = 5828608(5692K) used = 1174272(1146K) committed = 5828608(5692K) max = 5828608(5692K)
Memory Pool: Metaspace : Metaspace: init = 0(0K) used = 28357072(27692K) committed = 29278208(28592K) max = -1(-1K)
Memory Pool: CodeHeap 'profiled nmethods' : CodeHeap 'profiled nmethods': init = 33554432(32768K) used = 8135808(7945K) committed = 33554432(32768K) max = 265519104(259296K)
Memory Pool: Compressed Class Space : Compressed Class Space: init = 0(0K) used = 3085520(3013K) committed = 3407872(3328K) max = 1073741824(1048576K)
Memory Pool: G1 Eden Space : G1 Eden Space: init = 20971520(20480K) used = 140509184(137216K) committed = 236978176(231424K) max = -1(-1K)
Memory Pool: G1 Old Gen : G1 Old Gen: init = 381681664(372736K) used = 22020096(21504K) committed = 148897792(145408K) max = 2147483648(2097152K)
Memory Pool: G1 Survivor Space : G1 Survivor Space: init = 0(0K) used = 16777216(16384K) committed = 16777216(16384K) max = -1(-1K)
Memory Pool: CodeHeap 'non-profiled nmethods' : CodeHeap 'non-profiled nmethods': init = 33554432(32768K) used = 1640192(1601K) committed = 33554432(32768K) max = 265523200(259300K)
JVM Input Arguments : --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED -Djava.util.logging.config.file=/opt/atlassian/jira/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms384m -Xmx2048m -XX:InitialCodeCacheSize=32m -XX:ReservedCodeCacheSize=512m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Dorg.dom4j.factory=com.atlassian.core.xml.InterningDocumentFactory -XX:-OmitStackTraceInFastThrow -Djava.locale.providers=COMPAT -Datlassian.plugins.startup.options= -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Xlog:gc*:file=/opt/atlassian/jira/logs/atlassian-jira-gc-%t.log:tags,time,uptime,level:filecount=5,filesize=20M -XX:+ExplicitGCInvokesConcurrent -Dignore.endorsed.dirs= -Dcatalina.base=/opt/atlassian/jira -Dcatalina.home=/opt/atlassian/jira -Djava.io.tmpdir=/opt/atlassian/jira/temp
Java Compatibility Information : JIRA version = 8.20.3, Java Version = 11.0.11

2-atlassian-jira-gc-2021-12-29_22-36-44.log

[2021-12-29T22:36:44.090+0000][0.004s][info][gc,heap] Heap region size: 1M
[2021-12-29T22:36:44.095+0000][0.009s][info][gc ] Using G1
[2021-12-29T22:36:44.095+0000][0.009s][info][gc,heap,coops] Heap address: 0x0000000080000000, size: 2048 MB, Compressed Oops mode: 32-bit
[2021-12-29T22:36:45.003+0000][0.917s][info][gc,start ] GC(0) Pause Young (Normal) (G1 Evacuation Pause)
[2021-12-29T22:36:45.003+0000][0.917s][info][gc,task ] GC(0) Using 2 workers of 2 for evacuation
[2021-12-29T22:36:45.012+0000][0.926s][info][gc,phases ] GC(0) Pre Evacuate Collection Set: 0.0ms
[2021-12-29T22:36:45.012+0000][0.926s][info][gc,phases ] GC(0) Evacuate Collection Set: 8.4ms
[2021-12-29T22:36:45.013+0000][0.926s][info][gc,phases ] GC(0) Post Evacuate Collection Set: 0.7ms
[2021-12-29T22:36:45.013+0000][0.926s][info][gc,phases ] GC(0) Other: 0.4ms
[2021-12-29T22:36:45.013+0000][0.926s][info][gc,heap ] GC(0) Eden regions: 19->0(16)
[2021-12-29T22:36:45.013+0000][0.926s][info][gc,heap ] GC(0) Survivor regions: 0->3(3)
[2021-12-29T22:36:45.013+0000][0.926s][info][gc,heap ] GC(0) Old regions: 0->1
[2021-12-29T22:36:45.013+0000][0.926s][info][gc,heap ] GC(0) Humongous regions: 0->0
[2021-12-29T22:36:45.013+0000][0.926s][info][gc,metaspace ] GC(0) Metaspace: 13336K->13336K(1060864K)
[2021-12-29T22:36:45.013+0000][0.927s][info][gc ] GC(0) Pause Young (Normal) (G1 Evacuation Pause) 19M->3M(384M) 9.707ms
[2021-12-29T22:36:45.013+0000][0.927s][info][gc,cpu ] GC(0) User=0.02s Sys=0.00s Real=0.01s
[2021-12-29T22:36:45.376+0000][1.290s][info][gc,start ] GC(1) Pause Young (Normal) (G1 Evacuation Pause)
[2021-12-29T22:36:45.376+0000][1.290s][info][gc,task ] GC(1) Using 2 workers of 2 for evacuation
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,phases ] GC(1) Pre Evacuate Collection Set: 0.0ms
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,phases ] GC(1) Evacuate Collection Set: 8.4ms
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,phases ] GC(1) Post Evacuate Collection Set: 1.1ms
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,phases ] GC(1) Other: 0.2ms
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,heap ] GC(1) Eden regions: 16->0(16)
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,heap ] GC(1) Survivor regions: 3->3(3)
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,heap ] GC(1) Old regions: 1->4
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,heap ] GC(1) Humongous regions: 0->0
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,metaspace ] GC(1) Metaspace: 17220K->17220K(1064960K)
[2021-12-29T22:36:45.386+0000][1.300s][info][gc ] GC(1) Pause Young (Normal) (G1 Evacuation Pause) 19M->6M(384M) 9.819ms
[2021-12-29T22:36:45.386+0000][1.300s][info][gc,cpu ] GC(1) User=0.01s Sys=0.01s Real=0.01s
[2021-12-29T22:36:45.582+0000][1.495s][info][gc,start ] GC(2) Pause Young (Normal) (G1 Evacuation Pause)
[2021-12-29T22:36:45.582+0000][1.496s][info][gc,task ] GC(2) Using 2 workers of 2 for evacuation
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,phases ] GC(2) Pre Evacuate Collection Set: 0.0ms
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,phases ] GC(2) Evacuate Collection Set: 6.9ms
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,phases ] GC(2) Post Evacuate Collection Set: 7.6ms
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,phases ] GC(2) Other: 0.2ms
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,heap ] GC(2) Eden regions: 16->0(26)
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,heap ] GC(2) Survivor regions: 3->3(3)
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,heap ] GC(2) Old regions: 4->6
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,heap ] GC(2) Humongous regions: 2->2
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,metaspace ] GC(2) Metaspace: 18265K->18265K(1067008K)
[2021-12-29T22:36:45.596+0000][1.510s][info][gc ] GC(2) Pause Young (Normal) (G1 Evacuation Pause) 24M->10M(384M) 14.863ms
[2021-12-29T22:36:45.596+0000][1.510s][info][gc,cpu ] GC(2) User=0.03s Sys=0.00s Real=0.02s
[2021-12-29T22:36:45.753+0000][1.667s][info][gc,start ] GC(3) Pause Young (Normal) (G1 Evacuation Pause)

[Nic Brough -Adaptavist-]

That's a generic error which means a file could not be opened.  If there is a problem opening a file, that would mean the installation or, if you are using h2, the database files are broken, and it's failing because of them.

I would want to read the log for more errors, but given this one, I think you might have to install a new one and restore data from a backup.

[Seemi Anwar]

@Nic Brough -Adaptavist- i found an error in the log. does that help you in  identifying the problem for me?

java.io.IOError: java.io.IOException: Input/output error
at java.io.Console.readLine(Console.java:254)
at com.install4j.runtime.installer.helper.console.Console16Access.readLine(Console16Access.java:40)
at com.install4j.runtime.installer.helper.console.ConsoleImpl.readLine(ConsoleImpl.java:508)
at com.install4j.runtime.installer.helper.console.ConsoleImpl.askOption(ConsoleImpl.java:274)
at com.install4j.runtime.installer.helper.console.ConsoleImpl.askOption(ConsoleImpl.java:258)
at com.install4j.runtime.installer.helper.console.ConsoleImpl.askYesNo(ConsoleImpl.java:206)
at com.install4j.runtime.beans.formcomponents.CheckboxComponent.handleConsole(CheckboxComponent.java:181)
at com.install4j.runtime.installer.frontend.FormPanel.handleConsole(FormPanel.java:170)
at com.install4j.runtime.beans.formcomponents.FormEnvironmentImpl.handleConsole(FormEnvironmentImpl.java:245)
at com.install4j.runtime.beans.screens.FormScreen.handleConsole(FormScreen.java:164)
at com.install4j.runtime.installer.frontend.headless.ConsoleScreenExecutor.handleScreen(ConsoleScreenExecutor.java:53)
at com.install4j.runtime.installer.frontend.headless.AbstractHeadlessScreenExecutor.execute(AbstractHeadlessScreenExecutor.java:96)
at com.install4j.runtime.installer.controller.Controller$1.run(Controller.java:155)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.io.IOException: Input/output error
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:255)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.Console$LineReader.read(Console.java:437)
at java.io.Console.readline(Console.java:376)
at java.io.Console.readLine(Console.java:250)
... 26 more

System properties:

[Nic Brough -Adaptavist-]

None of the logs in here are showing the actual failure to start up, so we can't tell you anything about the real problem.  I would want to know what happens later in the Atlassian-jira.log - there should be an error in there telling you what the real error is.

However, you mention that you're using the embedded h2 database.  There is a good chance that the error is that the database has failed you.

H2 is only suitable for dev/test/demo environments that are a) small and b) not going to be used for a prolonged period.  H2 fails catastrophically and without warning, especially when it is put under strain (one example of strain is going through an upgrade) 

Anyway, check your log for the actual failure error!

[Seemi Anwar]

Thank for the response @KAGITHALA BABU ANVESH , i am using default H2 database here is details from dbconfig.xml file

<?xml version="1.0" encoding="UTF-8"?>

<jira-database-config>
<name>defaultDS</name>
<delegator-name>default</delegator-name>
<database-type>h2</database-type>
<schema-name>PUBLIC</schema-name>
<jdbc-datasource>
<url>jdbc:h2:file:/var/atlassian/application-data/jira/database/h2db</url>
<driver-class>org.h2.Driver</driver-class>
<username>sa</username>
<password></password>
<pool-min-size>20</pool-min-size>
<pool-max-size>20</pool-max-size>
<pool-max-wait>30000</pool-max-wait>
<min-evictable-idle-time-millis>4000</min-evictable-idle-time-millis>
<time-between-eviction-runs-millis>5000</time-between-eviction-runs-millis>
<pool-max-idle>20</pool-max-idle>
<pool-remove-abandoned>true</pool-remove-abandoned>
<pool-remove-abandoned-timeout>300</pool-remove-abandoned-timeout>
</jdbc-datasource>
</jira-database-config>
~

i have also checked mysql version and the databases which my machine is using, can you please find the attached information and guide me.

[KAGITHALA BABU ANVESH]

Hello @Seemi Anwar ,

Can you please check the below link, regarding the supported software's for 8.20.X versions

https://confluence.atlassian.com/adminjiraserver0820/supported-platforms-1095776511.html 

is your current DB or version is compatible or not, If not you can check with your DB team to upgrade DB as per the above link.

Thanks