Problems with customize-application-navigator after 7.2.4->7.11.2

[Robert]

I am getting a strange error post upgrade from 7.2.4->7.11.2, (Oracle and Linux) .  I am no longer able to manage application navigator.  The link work but when I go to configure i get directed to https://<MYURLHERE>/plugins/servlet/customize-application-navigator and i get a 500 error.  BTW everything else is working and I do not see anything jumping out in catalina.out.....it's really strange.   Full error below, with <MYURLHERE> is replacement for real url.  

It works in my test environment.......confused.....thoughts. 

echnical details
Log's referral number: 9e976e67-f9ea-498a-b1ca-f6a536670ba8

Cause
Referer URL: https://<MYURLHERE>/plugins/servlet/applications/versions-licenses

java.lang.RuntimeException: Error getting login uri. LoginUrl = ://<MYURLHERE>/okta_login.jsp?RelayState=${originalurl}, ReturnUri = https://<MYURLHERE>/plugins/servlet/customize-application-navigator
java.lang.RuntimeException: Error getting login uri. LoginUrl = ://<MYURLHERE>/okta_login.jsp?RelayState=${originalurl}, ReturnUri = https://<MYURLHERE>/plugins/servlet/customize-application-navigator
at com.atlassian.sal.core.auth.SeraphLoginUriProvider.getLoginUriForRole(SeraphLoginUriProvider.java:51) [?:?]
at com.atlassian.sal.core.auth.SeraphLoginUriProvider.getLoginUri(SeraphLoginUriProvider.java:33) [?:?]
at com.atlassian.sal.core.auth.SeraphLoginUriProvider.getLoginUri(SeraphLoginUriProvider.java:28) [?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [?:1.8.0_102]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [?:1.8.0_102]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56) [?:?]
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60) [?:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70) [?:?]
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53) [?:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57) [?:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) [spring-aop-4.2.5.RELEASE.jar:4.2.5.RELEASE]
at com.sun.proxy.$Proxy2205.getLoginUri(Unknown Source) [?:?]
at com.atlassian.plugins.custom_apps.security.authentication.LoginPage.getRedirectUrl(LoginPage.java:25) [?:?]
at com.atlassian.plugins.custom_apps.ui.AdminServlet.doGet(AdminServlet.java:57) [?:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) [servlet-api.jar:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [servlet-api.jar:?]
at com.atlassian.plugin.servlet.DelegatingPluginServlet.service(DelegatingPluginServlet.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [servlet-api.jar:?]
at com.atlassian.plugin.servlet.ServletModuleContainerServlet.service(ServletModuleContainerServlet.java:45) [atlassian-plugins-servlet-4.5.1.jar:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) [servlet-api.jar:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [catalina.jar:8.5.29]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.29]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-websocket.jar:8.5.29]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.29]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.29]
at com.atlassian.jira.web.filters.JiraLastFilter.lambda$doFilter$0(JiraLastFilter.java:39) [classes/:?]
at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:74) [classes/:?]
at com.atlassian.jira.web.filters.JiraLastFilter.doFilter(JiraLastFilter.java:36) [classes/:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.29]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.29]
at com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:37) [atlassian-core-5.0.9.jar:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:8.5.29]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:8.5.29]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:39) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.jira.tzdetect.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:77) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.jira.baseurl.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:34) [?:?]
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32) [atlassian-core-5.0.9.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.servicedesk.internal.web.ExternalCustomerLockoutFilter.doFilter(ExternalCustomerLockoutFilter.java:56) [?:?]
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32) [atlassian-core-5.0.9.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.mywork.client.filter.ServingRequestsFilter.doFilter(ServingRequestsFilter.java:37) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.analytics.client.filter.UniversalAnalyticsFilter.doFilter(UniversalAnalyticsFilter.java:92) [?:?]
at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:39) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21) [?:?]
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32) [atlassian-core-5.0.9.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFixupFilter.doFilter(PrettyUrlsSiteMeshFixupFilter.java:32) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:55) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:79) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:51) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.labs.botkiller.BotKillerFilter.doFilter(BotKillerFilter.java:36) [?:?]
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:64) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:37) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:70) [atlassian-plugins-servlet-4.5.1.jar:?]
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:58) [atlassian-plugins-servlet-4.5.1.jar:?]
*****repeating section remove***************
Caused by: java.net.URISyntaxException: Expected scheme name at index 0: ://<MYURLHERE>/okta_login.jsp?RelayState=https%3A%2F%2F<MYURLHERE>%2Fplugins%2Fservlet%2Fcustomize-application-navigator
at java.net.URI$Parser.fail(URI.java:2848) [?:1.8.0_102]
at java.net.URI$Parser.failExpecting(URI.java:2854) [?:1.8.0_102]
at java.net.URI$Parser.parse(URI.java:3046) [?:1.8.0_102]
at java.net.URI.<init>(URI.java:588) [?:1.8.0_102]
at com.atlassian.sal.core.auth.SeraphLoginUriProvider.getLoginUriForRole(SeraphLoginUriProvider.java:49) [?:?]
... 291 more

[Andy Heinzer]

It looks like your instance is using some kind of okta integration for authentication:

java.lang.RuntimeException: Error getting login uri. LoginUrl = ://<MYURLHERE>/okta_login.jsp?RelayState=${originalurl}, ReturnUri = https://<MYURLHERE>/plugins/servlet/customize-application-navigator

that okta_login.jsp page seems to be a factor here, but I'm not sure how this would be configured in your environment.  Okta integration is not something jira does natively, yet, so you probably have some plugin to Jira to handle that.  

I am thinking that to reach this page you do have to be a Jira admin.  So if you didn't have okta, you would probably see the secure administrator prompt, that asks you for your admin password once more (even though you are already logged into Jira).  In you case it looks like your Okta integration can't redirect here for some reason.

One way to troubleshoot this would be to remove the okta integration.  But another way you might try here would be to disable this secure administrator session feature in Jira.  Steps on how to do that are in Configuring secure administrator sessions.  From that page:

Disabling secure administrator sessions

Secure administrator sessions (i.e. password confirmation before accessing administration functions) are enabled by default. If this causes issues for your Jira instance (e.g. if you are using a custom authentication mechanism), you can disable this feature by specifying the following line in your jira-config.properties file:

jira.websudo.is.disabled = true

 You will need to restart your Jira server for this setting to take effect.

[Robert]

We have a non-prod instance, same version, etc that is working normally that is configured thru OKTA so I do not believe that this is related to OKTA.  

Any other thoughts? 

[Andy Heinzer]

Did you try to disable the secure administrator sessions?  Did that make any difference?

Given the code packages being called in your stacktrace, it's quite clear to me there is a problem with authentication.