9.4.0 to 9.12.1 Jira Server upgrade. Jira doesn't start

[Sue Bridges]

As usual, all appeared to go OK in the upgrade - but then the installer is quite brief.  However, after checking the Jira Windows Service settings (something I do from old) and then clicking OK to start Jira to complete the upgrade installation, the Browser remains blank with errors.jsp.  This is connecting to an Oracle DB which is fine.

I stopped the Jira Service, checked the Jira account is OK in Oracle and restarted the Jira Service, but the Browser is the same.  I am not sure where to look in the Atlassian logs,  I removed and reinstalled the Windows Service with service.bat but errors page persists.  Please advise on how I can proceed.

I did check the basic log, and starting from the bottom of the atlassian-jira.log file I search on "fail" upwards to find the last one.

2024-01-05 11:37:53,931+0000 JIRA-Bootstrap ERROR [c.a.jira.startup.DefaultJiraLauncher] JIRA has failed to start because of the following errors: [(Event: Level = (EventLevel: fatal) , Key = (EventType: startup-unexpected) , Desc = We couldn't start JIRA , Exception = An error occurred while trying to start JIRA. We can't give you any more detail right now, we suggest checking the logs for more detail and contacting our support team.<br/>See our documentation for more information on contacting our support team and creating a support zip.), (Event: Level = (EventLevel: fatal) , Key = (EventType: startup-unexpected) , Desc = We couldn't start JIRA , Exception = An error occurred while trying to start JIRA. We can't give you any more detail right now, we suggest checking the logs for more detail and contacting our support team.<br/>See our documentation for more information on contacting our support team and creating a support zip.), (Event: Level = (EventLevel: fatal) , Key = (EventType: database) , Desc = Could not execute health check, DatabaseConfigurationManager not available. , Exception = ]
2024-01-05 11:37:53,931+0000 JIRA-Bootstrap INFO [c.a.jira.startup.LauncherContextListener] Startup is complete. Jira is ready to serve.

[Sue Bridges]

Using the two docs above - I'm sure dbconfig.xml is OK - DB config and setup is unchanged and running with all accounts able to log in, and this file dbconfig contents is the same as the 9.4.0 one.  I have run config.bat and it all looks as it used to. 

Plugins - well always potential, but in my pre upgrade checks I have the following:

Incompatible but Disabled - Automation Lite

Compatible and Enabled - Automation and Atlassian Troubleshooting and Support Tools

Unknown and Enabled - Advanced Roadmaps for Jira

We can't get a Support contract for this small Server DB so how do I contact Support please?

I stopped Jira, renamed the installed_plugins directory as directed, and then started Jira.  Same errors.jsp Browser.  It has created a new (empty) installed_plugins directory though.

[Sue Bridges]

I'm leaning toward plugin errors.  I don't know what I am looking for in the atlassian-jira.log but I do have this

2024-01-05 14:01:05,830+0000 JIRA-Bootstrap ERROR [c.a.j.component.pico.ComponentManager] Error occurred while initializing component 'com.atlassian.jira.plugin.JiraPluginPersistentStateStore'.
com.atlassian.jira.exception.DataAccessException: org.ofbiz.core.entity.GenericDataSourceException: SQL Exception while executing the following:SELECT pluginkey, pluginenabled FROM pluginstate WHERE (pluginkey LIKE ? ) (ORA-00904: "PLUGINKEYIKE": invalid identifier
)

[Sue Bridges]

Thanks, but I have no "Truncated class file" error in my Atlassian-Jira.log.  Suspect it's plugins as above.

[Sue Bridges]

The article did mention using the installer and upgrading the "dodgy upgrade" again.  Would that work?

[Nic Brough -Adaptavist-]

I am sorry that the answer you got before was misleading - it was generated by an AI that has no understanding of your question or Atlassian software.

The most likely problem here is that Jira has not been able to make changes to the database that it needs to during the upgrade.

If that is the case, you have not lost anything, but the solution is to do most of the upgrade again.  I would stop the currently running Jirs and take a full backup of the database right now, before trying to work out what the problem is.

If my guess is right, then I would change the permissions in the database such that Jira can do everything.  Then restart it, it should detect a DB that needs updating and do those updates from when it could not.

[Sue Bridges]

Hi, the Jira account should have all the permissions required for the Oracle DB.  Nothing has changed from before in 9.4.0 and it doesn't have a problem normally. 

I don't get as far as logging into Jira for it to say that an upgrade is required.  Starting the Windows Jira Server, it errors in the Jira startup process somewhere...

If I understand you correctly, after a new DB backup, should I just run the installer again and select upgrade?

[Nic Brough -Adaptavist-]

Yes, that is what I would try next.

The error message you have looks like a failure to read the database, which suggests it might not be updated by your upgrade correctly, so I would want to try the upgrade again, and read the logs for that as it runs.

[Sue Bridges]

Hi, I ran the upgrade again (over the top of the failed one) and the main Oracle error I see in the log is ORA-00904: "PLUGINKEYIKE".  Same outcome anyway, Jira doesn't start and the browser says error.jsp.  There are a couple of other issues on here with that error but no resolution posted either.

[Nic Brough -Adaptavist-]

It is the same underlying problem I'm afraid - your database is not in the same shape that your Jira is expecting.

An upgrade will try to change the shape to the new version, but it looks like your upgrade can not do it.

I would go back again, install the version of Jira you had before upgrade, point it to the database and test that it all works, then run the upgrade, watching the logs very carefully for any database errors.  The most likely error in this case is going to be that the Jira user does not have the permission to alter tables.

[Sue Bridges]

Hi, yes, I've rolled back OK.

I don't know what changes Jira is making to the DB unfortunately, but I noticed some other people got the same ORA error as me during the process, though with no solution posted unfortunately.  I am going to remove a couple of apps (Roadmaps and Automation Lite) and try again with the upgrade.

[Sue Bridges]

Upgraded again, after uninstalling a couple of apps and updating the Cloud Migration one.  It just doesn't upgrade properly I think

ORA-00904: "PLUGINKEYIKE": invalid identifier

[Jens de Waard]

We ran into the PLUGINKEYIKE exception after upgrading to 9.12.2. We were able to solve it by replacing ojdbc7.jar with ojdbc8.jar (version 19.3.0.0) in the lib/ subdirectory of the installation directory.