Failure to startup JIRA aftering updating to JIRA 9.0 and MySQL 5.7

[Christian Pickman]

The other day I decided to update the installed version of JIRA from 8.22.3 to JIRA 9.0.0. Also, since I was getting warnings about not have a supported version of MySQL for a number of months, I also decided to update MySQL from 5.6 (unsupported) to 5.7 (Support). After running the update to JIRA from 8.22.3 to 9.0.0 I get the following message when JIRA starts up:

Database: JIRA couldn't connect to your database

JIRA failed to establish a connection to your database.
This could be because:

• Your database isn't running
• The configuration of your dbconfig.xml file is incorrect (user, password, or database URL etc.)
• There is a network issue between JIRA and your database (e.g. firewall, database doesn't allow remote access etc.)

There are several other solutions you can try, review our documentation and see what works for you.

I've tried various possible solutions to resolve the issue but no luck. Here's the setup information I have when I user the 'version.sh' script:

Server startup logs are located in /opt/atlassian/jira/logs/catalina.out
Using CATALINA_BASE: /opt/atlassian/jira
Using CATALINA_HOME: /opt/atlassian/jira
Using CATALINA_TMPDIR: /opt/atlassian/jira/temp
Using JRE_HOME: /opt/atlassian/jira/jre/
Using CLASSPATH: /opt/atlassian/jira/bin/bootstrap.jar:/opt/atlassian/jira/bin/tomcat-juli.jar
Using CATALINA_OPTS:
Using CATALINA_PID: /opt/atlassian/jira/work/catalina.pid
NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
Server version: Apache Tomcat/9.0.62
Server built: Mar 31 2022 14:34:15 UTC
Server number: 9.0.62.0
OS Name: Linux
OS Version: 2.6.32-754.35.1.el6.x86_64
Architecture: amd64
JVM Version: 11.0.13+8
JVM Vendor: Eclipse Adoptium

...everything looks correct, so I must be missing something or need to update my configuration for it to load my instance of JIRA correctly like it did before the updating from version 8.x to 9.x. Any help would be greatly appreciated. Thank you in advance.

[Mirek]

Maybe you can start troubleshooting by looking at this KB article and let us know what is the result of each check

https://confluence.atlassian.com/jirakb/resolve-database-connection-errors-during-jira-server-startup-872262124.html

[Christian Pickman]

Greetings, and thank you for this referenced article. I tried these steps before and tried them again and this is what I encountered.

1. Performed a ping of localhost, 127.0.0.1 and the static IP address and no issues encountered. No dropped packets.

2. Ran nmap and it would appear there's no issue with the ports assigned for accessing the instance of JIRA (8081) as well as for MySQL (3306):

Nmap scan report for localhost (127.0.0.1)
Host is up (0.0000090s latency).
Not shown: 985 closed ports
PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
80/tcp open http
111/tcp open rpcbind
143/tcp open imap
443/tcp open https
465/tcp open smtps
587/tcp open submission
631/tcp open ipp
993/tcp open imaps
3052/tcp open powerchute
3306/tcp open mysql
6547/tcp open powerchuteplus
8081/tcp open blackice-icecap
10000/tcp open snet-sensor-mgmt

Nmap done: 1 IP address (1 host up) scanned in 0.09 seconds

3. The KB article references using Postgres to determine permissions are correct or not for the dbconfig.xml file. However, I am using MySQL and I am not seeing a method to check permissions on the dbconfig.xml file. The port referenced in my dbconfig.xml file is 3306 for mysql which it has always been, so I don't believe it's a port issue especially since nmap confirms that port 3306 is open.

Any other suggestions would be greatly appreciated. Thanks.

[chihara]

Check dbconfig.xml.

<database-type>mysql57</database-type>

[Christian Pickman]

When I update the <database-type> with 'mysql57' instead of just 'mysql', I get this error on reload of JIRA:

Database: We've found an error in MySQL supported version!

The database type in your dbconfig.xml is set up to MySQL 5.7 and your MySQL version is different. Consider using MySQL 5.7 database type instead
See our documentation for more information on changing database type.

...though when I do a 'mysql -V' at the command line I get this:

mysql Ver 14.14 Distrib 5.7.38, for Linux (x86_64) using EditLine wrapper

Any thoughts on why I get this message from JIRA when I use the <database-type>mysql57</database-type> tag in the dbconfig.xml file? Thanks

[chihara]

1. backup current dbconfig.xml (cp dbconfig.xml dbconfig.xml-original)
2. set JRE_HOME
3. run bin/config.sh to reconfigure DB settings and save
4. then check updated dbconfig.xml
5. start Jira to check any errors

[Christian Pickman]

Speaking out of ignorance here, but how would I go about to 'set JRE_HOME? Thank you for your posting.

[chihara]

Where do you install Java and which java do you use?

If you use Jira installer package, you can find $JIRA_INSTALL_DIRECTORY/jre. Then set JRE_HOME="$JIRA_INSTALL_DIRRECTORY/jre".

[Christian Pickman]

Using 'find $JIRA_INSTALL_DIRECTORY/jre' from the command line, I get:

find: `/jre': No such file or directory

Using java -version, I get:

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

[chihara]

Where do you install you Jira?

Where do you install Java?

If you use Oracle Java and installed it using RPM package, check /usr/java/XXXXXX.

Then JRE_HOME= /usr/java/XXXXX/jre

Where do you install you Jira?

[Christian Pickman]

Jira is installed in:

/opt/atlassian/jira/bin

/var/atlassian/application-data/jira

...so it's the typical Jira installation from what I've read. As for the java installation it would appear that it is in /usr/java with this output for using ls:

drwxr-xr-x 3 root root 4096 Apr 24 2015 .
drwxr-xr-x. 16 root root 4096 Apr 10 2015 ..
lrwxrwxrwx 1 root root 16 Apr 24 2015 default -> /usr/java/latest
drwxr-xr-x 7 root root 4096 Apr 24 2015 jre1.8.0_45
lrwxrwxrwx 1 root root 21 Apr 24 2015 latest -> /usr/java/jre1.8.0_45

...I updated java for the last supported version since I am running on the last version of Centos 6 (6.10).

[chihara]

If you are using bash, then

export JRE_HOME="/usr/java/ltest"

/opt/atlassian/jira/bin/config.sh &

[Christian Pickman]

OK. I did that and corrected what you wrote so the command I issued was:

export JRE_HOME="/usr/java/latest"

Then I did /opt/atlassian/jira/bin/config.sh and updated it. However, now when I start Jira, I get this message:

Database: We've found an error in MySQL supported version!

The database setup is not supporting utf8mb4
See our documentation for more information on setting up MySQL 5.7

...my guess would be the jiradb schema needs to be set to utf8mb4? If so, how would I go about that? I did update my.cnf file with the following:

character-set-server=utf8mb4
collation-server=utf8mb4_bin
default-storage-engine=INNODB
max_allowed_packet=256M
innodb_log_file_size=2GB
transaction-isolation=READ-COMMITTED
binlog_format=row
log_bin_trust_function_creators = 1
optimizer_switch = derived_merge=off

...but it made no difference and I still received the 'Database: We've found an error in MySQL supported version!' error. Thanks for your continued help and hand holding. :)

[chihara]

Check if DB & tables & colums are using UTF8MB4 & UTF8MB4_BIN as this Atlassian's guide.

[Christian Pickman]

Thanks for the link to the Atlassian guide. We will have to execute a ton of individual MySQL commands to convert to UTF8MB4 & UTF8MB4_BIN with the existing jiradb file that we are using. I've already made a backup of the jiradb in its current state. Very appreciative of all your help on this matter.

[omar caballero]

@Christian Pickman  did you ever get this resolved? If so, how? I'm in exactly the same situation.

Seems like the upgrade doesn't like the encrypted PW that's in the dbconfig.xml. Looking at the logs I get:

2022-07-16 12:42:05,341-0500 JIRA-Bootstrap INFO [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /var/data/atlassian/jira/dbconfig.xml
2022-07-16 12:42:05,374-0500 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigHandler] Trying to get encrypted password from xml and decrypt it
2022-07-16 12:42:05,375-0500 JIRA-Bootstrap INFO [c.a.j.config.database.DatabaseConfigHandler] Database password decryption not performed.
2022-07-16 12:42:05,513-0500 main WARN [o.a.catalina.util.SessionIdGeneratorBase] Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [558] milliseconds.
2022-07-16 12:42:05,850-0500 JIRA-Bootstrap ERROR [c.a.config.bootstrap.DefaultAtlassianBootstrapManager] Could not successfully test your database:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Thanks.

[Christian Pickman]

Not as of yet. We need to work on updating our current Jira database since it's not supporting utf8mb4. We are hoping we can somehow automate this task to get our instance of Jira accessible again.