JIRA is not accessable after WinServer 2016 Security Updates

[Martin Zennig]

I've installed the latest WinServer 2016 Security Updates. After the nessesary restart, Jira doesn't work correctly. I can't access, localy nor over hostname.

Our Databaseserver was updated at the same time.

[Martin Zennig]

The Last Log Messages are:

    ************************************************************************************
    JIRA 7.10.1 build: 710002 started. You can now access JIRA through your web browser.
    ************************************************************************************
    
2018-06-21 11:38:04,825 JIRA-Bootstrap INFO      [c.a.plugin.manager.DefaultPluginManager] Plugin system lateStartup begun
2018-06-21 11:38:04,841 JIRA-Bootstrap INFO      [c.a.plugin.manager.DefaultPluginManager] Plugin system lateStartup ended
2018-06-21 11:38:04,841 JIRA-Bootstrap DEBUG      [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.plugins.atlassian-whitelist-core-plugin]
2018-06-21 11:38:04,903 JIRA-Bootstrap INFO      [c.a.jira.startup.JiraStartupLogger]
    
    ___ Plugin System Started _________________
    
2018-06-21 11:38:04,950 JIRA-Bootstrap INFO      [c.a.jira.upgrade.UpgradeScheduler] Scheduling upgrades to run in 1 minute(s)
2018-06-21 11:38:05,125 JIRA-Bootstrap INFO      [c.a.j.c.embedded.ofbiz.IndexedUserDao] Reindex all users took: 130,5 ms
2018-06-21 11:38:05,138 WaitForAO DEBUG      [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.jira.plugins.jira-dvcs-connector-plugin]
2018-06-21 11:38:05,138 JIRA-Bootstrap DEBUG      [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.labs.hipchat.hipchat-for-jira-plugin]
2018-06-21 11:38:05,153 JIRA-Bootstrap INFO      [c.a.j.p.hipchat.service.HipChatPluginLauncher] HipChat Plugin launched successfully
2018-06-21 11:38:05,153 JIRA-Bootstrap INFO      [c.a.j.p.h.service.scheduling.AbstractTaskScheduler] On plugin event: com.atlassian.jira.plugins.hipchat.model.event.PluginStartedEvent
2018-06-21 11:38:05,153 JIRA-Bootstrap INFO      [c.a.j.p.h.service.scheduling.AbstractTaskScheduler] Registering a new hipchatRefreshConnectionStatusJob task.
2018-06-21 11:38:05,153 JIRA-Bootstrap INFO      [c.a.j.p.h.service.scheduling.AbstractTaskScheduler] On plugin event: com.atlassian.jira.plugins.hipchat.model.event.PluginStartedEvent
2018-06-21 11:38:05,153 JIRA-Bootstrap INFO      [c.a.j.p.h.service.scheduling.AbstractTaskScheduler] Registering a new hipchatInstallGlancesJob task.
2018-06-21 11:38:05,153 JIRA-Bootstrap INFO      [c.a.j.p.h.service.scheduling.AbstractTaskScheduler] On plugin event: com.atlassian.jira.plugins.hipchat.model.event.PluginStartedEvent
2018-06-21 11:38:05,153 JIRA-Bootstrap INFO      [c.a.j.p.h.service.scheduling.AbstractTaskScheduler] Registering a new hipchatUpdateGlancesDataJob task.
2018-06-21 11:38:08,419 JIRA-Bootstrap DEBUG      [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.pyxis.greenhopper.jira]
2018-06-21 11:38:08,450 JIRA-Bootstrap INFO      [c.a.j.p.e.bamboo.service.PlanStatusUpdateServiceImpl] PlanStatusUpdateJob scheduled to run every 60 seconds
2018-06-21 11:38:08,856 JIRA-Bootstrap DEBUG      [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.jira.plugins.webhooks.jira-webhooks-plugin]
2018-06-21 11:38:08,950 JIRA-Bootstrap INFO      [c.a.jira.scheduler.JiraSchedulerLauncher] Starting the JIRA Scheduler....
2018-06-21 11:38:08,981 JIRA-Bootstrap INFO      [c.a.jira.scheduler.JiraSchedulerLauncher] JIRA Scheduler started.
2018-06-21 11:38:09,028 JIRA-Bootstrap INFO      [c.a.jira.startup.LauncherContextListener] Memory Usage:
    ---------------------------------------------------------------------------------
      Heap memory     :  Used:  432 MiB.  Committed:  611 MiB.  Max:  747 MiB
      Non-heap memory :  Used:  234 MiB.  Committed:  255 MiB.  Max: 1264 MiB
    ---------------------------------------------------------------------------------
      TOTAL           :  Used:  666 MiB.  Committed:  866 MiB.  Max: 2011 MiB
    ---------------------------------------------------------------------------------
2018-06-21 11:38:09,106 Caesium-1-4 INFO      [c.a.jira.startup.CacheWarmerLauncher] Warming up 3 caches.
2018-06-21 11:38:09,122 Caesium-1-4 INFO      [c.a.jira.i18n.I18nWarmer] Init i18n cache
2018-06-21 11:38:09,122 ForkJoinPool.commonPool-worker-2 INFO      [c.a.j.cache.soy.SoyCacheWarmer] Warming 41 soy modules
2018-06-21 11:38:10,356 Caesium-1-3 INFO      [c.a.j.index.ha.DefaultIndexRecoveryManager] Latest index date: {2018-06-20 13:26:17}, Latest DB date: {2018-06-20 13:26:17}
2018-06-21 11:38:18,447 Modification Check:thread-1 INFO      [c.a.jira.startup.JiraStartupLogger]
    
    ___ Modifications ___________________________
    
         Modified Files                                : jira-application.properties
         Removed Files                                 : None
    
2018-06-21 11:38:18,510 ForkJoinPool.commonPool-worker-2 INFO      [c.a.j.cache.soy.SoyCacheWarmer] Warmed 41 soy module(s) in 9388 ms, 0 module(s) were not present or disabled, 0 failed.
2018-06-21 11:38:21,338 Caesium-1-3 DEBUG      [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.plugins.base-hipchat-integration-plugin-api]
2018-06-21 11:38:23,510 Caesium-1-4 INFO      [c.a.jira.i18n.I18nWarmer] Initialised i18n cache in 14388
2018-06-21 11:38:23,510 Caesium-1-4 INFO      [c.a.jira.startup.CacheWarmerLauncher] Warmed cache(s) in 14404 ms.
2018-06-21 11:38:24,353 Caesium-1-4 INFO      [c.a.j.p.h.service.connect.InstallGlancesJobHandler] Running InstallGlancesJobHandler...
2018-06-21 11:38:24,353 Caesium-1-4 INFO      [c.a.j.p.h.service.connect.InstallGlancesJobHandler] There is no link to HipChat, no need to install glances.
2018-06-21 11:38:29,042 Caesium-1-2 INFO      [c.a.j.p.h.service.ping.RefreshConnectionStatusJobHandler] Running RefreshConnectionStatusJobHandler...
2018-06-21 11:38:59,847 Caesium-1-1 WARN      [c.a.a.c.hash.reader.RemoteHashingInstructionsReader] Unable to read remote instructions with key 'uid.onewayhash'.
2018-06-21 11:38:59,847 Caesium-1-1 WARN      [c.a.a.client.hash.BcryptAnalyticsEmailHasher] No instructions for hashing could be found.
2018-06-21 11:39:04,953 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeScheduler] Running scheduled upgrades
2018-06-21 11:39:05,000 Caesium-1-2 INFO      [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has started
2018-06-21 11:39:05,235 Caesium-1-2 INFO      [c.a.j.upgrade.util.BuildNumberDao] Setting current build number to 710002
2018-06-21 11:39:05,250 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeIndexManager] There is no reindex requests of type [IMMEDIATE, DELAYED] so none will be run
2018-06-21 11:39:05,250 Caesium-1-2 INFO      [c.a.j.upgrade.util.BuildNumberDao] Setting current version to 7.10.1
2018-06-21 11:39:05,250 Caesium-1-2 INFO      [c.a.j.upgrade.util.BuildNumberDao] Setting downgrade version to 7.1.2
2018-06-21 11:39:05,281 Caesium-1-2 INFO      [c.a.jira.upgrade.LoggingUpgradeService] run upgrades has finished successfully, and took 281 milliseconds to process.
2018-06-21 11:39:05,281 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeScheduler] JIRA upgrades completed successfully
2018-06-21 11:39:05,281 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeScheduler] Plugins upgrades completed successfully
2018-06-21 11:39:05,281 Caesium-1-2 INFO      [c.a.jira.upgrade.UpgradeIndexManager] Reindexing is not allowed after this upgrade and there is no immediate reindex requests
2018-06-21 11:39:08,941 Caesium-1-3 DEBUG ServiceRunner     [c.a.activeobjects.osgi.ActiveObjectsServiceFactory] getService bundle [com.atlassian.troubleshooting.plugin-jira]
2018-06-21 11:39:38,828 UpmScheduler:thread-1 WARN anonymous     [c.a.upm.pac.PacClientImpl] Error when querying application info from MPAC: com.atlassian.marketplace.client.MpacException: org.apache.http.conn.ConnectTimeoutException: Connect to marketplace.atlassian.com:443 [marketplace.atlassian.com/34.196.173.143, marketplace.atlassian.com/52.2.89.223, marketplace.atlassian.com/107.23.25.67] failed: connect timed out

[Sloan]

Check these things first:

• Is the Jira Service running?
• Can you find a process related to jira in the Task Manager?
• Have you checked the Application Logs of your Jira, do they tell you something? You find them at <Jira-data-directory>/log/atlassian-jira.log

[Martin Zennig]

Service is running and visible in task manager.

It seems that jira is starting and running correctly according to the log. The last step after start up is a message that the healthcheck fails (the server has no internet access)

[Sloan]

No internet access is no big deal, you can ignore that warning.

Have you tried accessing jira on the Server via http://localhost:8080?

[Martin Zennig]

yes, without success.

there is no java process in the task manager... is that normal?

[Sloan]

@Martin Zennig On Linux and mac, yes, there is a java process. I think the same goes for Windows. I have not Windows environment to test this rn, sorry! But Jira depends on Java, on every platform.

Restart the Jira service if not done already. If Java would cause you problems, you would see anomalies in your log.

Maybe try troubleshooting with the help of this knowledgebase Article: How to check the JIRA application is running

Do you know what exactly the Windows Updates changed?

[Martin Zennig]

The Jira log seems to be normal.

There occured an error during the update. Therefore windows did a rollback and we have to restart. After this restart Jira doesn't work correctly. Then windows did the update successfully.

I have no idea where the error is.

[PJ Wysota]

And you run JIRA on Apache/Tomcat or IIS/Tomcat? If everything else is running, I would check these as well.

[Martin Zennig]

I Apache/Tomcat. It's the default setting via installer by atlassian. I've figured out port 8080 is blocked by svchost.exe. If i run Jira on Port 8081 everything works.

Strange that it doesn't appear as an error in the log.

[Sloan]

@Martin Zennig Good you found the culprit. I think jira doesn't check if it is reachable on it's standard port just during the startup. 

svchost.exe is the program managing services in windows. I wonder what service is blocking this port. Maybe there is a second defective Jira service running?

You can run netstat -ano and look for the service running on port 8080 and then write down the PID.

With the command tasklist you get a list of all services/tasks with their pid.

[Martin Zennig]

that is exactly what I did.

[PJ Wysota]

Try to check running services - if there are no duplicated JIRA/Catalina service. If there are it might be the cause.

Quick fix attempt - kill java process in Task manager , and then start JIRA service from power shell. 

And then review config if everything is OK.

[Martin Zennig]

There are no duplicated services. And there is no java process while Jira is running. Is that a nomal behavior?