Having Confluence-Postgres issues after pg_upgrade from 9.3 to 11.15

[Frank Campagna]

I'm upgrading our Confluence 7.16 instance from Postgres 9.3 to Postgres 11.15 and can't seem to get Confluence to connect to the database.  I can connect to the confluencedb through psql with no problem.  I don't think the problem is my confluence.cfg.xml file.  I am getting a 500 error, when looking at the catalina.out log this is what I see (below).  Any ideas?  Thx.

23-Mar-2022 17:13:45.143 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [24178] milliseconds
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.atlassian.hibernate.adapter.proxy.BytecodeProviderImpl_ImplementV2Proxy (file:/opt/atlassian/confluence/confluence/WEB-INF/lib/hibernate.adapter-1.0.3.jar) to field java.lang.reflect.Field
.modifiers
WARNING: Please consider reporting this to the maintainers of com.atlassian.hibernate.adapter.proxy.BytecodeProviderImpl_ImplementV2Proxy
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
23-Mar-2022 17:14:10.165 SEVERE [http-nio-8080-exec-2] org.apache.catalina.core.StandardHostValve.custom Exception Processing ErrorPage[errorCode=500, location=/500page.jsp]
org.apache.jasper.JasperException: An exception occurred processing [500page.jsp] at line [187]

184: <h3>Referer URL</h3>
185: <p><%= request.getHeader("Referer") != null ? HtmlUtil.htmlEncode(request.getHeader("Referer")) : "Unknown" %></p>
186:
187: <% if ((sysInfoService != null) && !ExceptionMonitorPredicates.shortCircuitRequestTester().test(request) && sysInfoService.isShowInfoOn500()) {%>
188: <h3>Confluence Application Information</h3>
189: <h4>Build Information</h4>
190: <p>

Stacktrace:
at org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:610)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:489)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:63)
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129)
at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77)
at com.atlassian.confluence.util.profiling.ProfilingSiteMeshFilter.doFilter(ProfilingSiteMeshFilter.java:50)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.confluence.web.filter.HttpSessionRegistrarFilter.doFilter(HttpSessionRegistrarFilter.java:49)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.confluence.web.filter.ConfluenceSecurityFilter.doFilter(ConfluenceSecurityFilter.java:27)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:148)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.johnson.plugin.servlet.filter.JohnsonServletFilterModuleContainerFilter.doFilter(JohnsonServletFilterModuleContainerFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.confluence.impl.servlet.HibernateGetConnectionFilter.doFilter(HibernateGetConnectionFilter.java:50)
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:67)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:51)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:63)
at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:711)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:385)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:313)
at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:403)
at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:249)
at org.apache.catalina.core.StandardHostValve.throwable(StandardHostValve.java:344)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:895)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1722)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.atlassian.confluence.tenant.VacantException: Confluence is vacant, a call to tenanted [public abstract org.hibernate.Session org.hibernate.SessionFactory.getCurrentSession() throws org.hibernate.HibernateExc
eption] is not allowed.
at com.atlassian.confluence.impl.tenant.TenantedProxyFactory.lambda$throwVacantExceptionOnMethodCall$3(TenantedProxyFactory.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy82.getCurrentSession(Unknown Source)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy82.getCurrentSession(Unknown Source)
at org.springframework.orm.hibernate5.HibernateTemplate.doExecute(HibernateTemplate.java:352)
at org.springframework.orm.hibernate5.HibernateTemplate.execute(HibernateTemplate.java:320)
at com.atlassian.confluence.setup.bandana.persistence.dao.hibernate.HibernateConfluenceBandanaRecordDao.getRecord(HibernateConfluenceBandanaRecordDao.java:48)
at com.atlassian.confluence.setup.bandana.ConfluenceDaoBandanaPersister.retrieve(ConfluenceDaoBandanaPersister.java:29)
at com.atlassian.confluence.impl.bandana.ReadThroughCachingBandanaPersister.lambda$retrieve$0(ReadThroughCachingBandanaPersister.java:84)
at com.atlassian.confluence.impl.cache.ReadThroughAtlassianCache.lambda$get$0(ReadThroughAtlassianCache.java:76)
at com.atlassian.confluence.cache.ehcache.DefaultConfluenceEhCache$ValueWrapperSupplier.get(DefaultConfluenceEhCache.java:228)
at com.atlassian.confluence.cache.ehcache.DefaultConfluenceEhCache$ValueWrapperSupplier.get(DefaultConfluenceEhCache.java:217)
at com.atlassian.cache.ehcache.ReferenceCacheLoader.load(ReferenceCacheLoader.java:31)
at com.atlassian.cache.ehcache.wrapper.ValueProcessorEhcacheLoaderDecorator.load(ValueProcessorEhcacheLoaderDecorator.java:30)
at net.sf.ehcache.Cache.loadValueUsingLoader(Cache.java:3383)
at net.sf.ehcache.Cache.getWithLoader(Cache.java:1860)
at com.atlassian.cache.ehcache.DelegatingCache.get(DelegatingCache.java:128)
at com.atlassian.confluence.cache.ehcache.DefaultConfluenceEhCache.get(DefaultConfluenceEhCache.java:47)
at com.atlassian.confluence.cache.ConfluenceMonitoringCache.get(ConfluenceMonitoringCache.java:57)
at com.atlassian.confluence.cache.TransactionalCacheFactory$TransactionalCache.get(TransactionalCacheFactory.java:343)
at com.atlassian.confluence.impl.cache.ReadThroughAtlassianCache.get(ReadThroughAtlassianCache.java:88)
at com.atlassian.confluence.impl.cache.ReadThroughCache.get(ReadThroughCache.java:28)
at com.atlassian.confluence.impl.bandana.ReadThroughCachingBandanaPersister.retrieve(ReadThroughCachingBandanaPersister.java:82)
at com.atlassian.confluence.impl.bandana.ReadThroughCachingBandanaPersister.retrieve(ReadThroughCachingBandanaPersister.java:75)
at com.atlassian.confluence.impl.bandana.TransactionalBandanaPersister.retrieve(TransactionalBandanaPersister.java:28)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy85.retrieve(Unknown Source)
at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:32)
at com.atlassian.bandana.DefaultBandanaManager.getValue(DefaultBandanaManager.java:24)
at com.atlassian.confluence.impl.settings.BandanaGlobalSettingsManager.getGlobalSettings(BandanaGlobalSettingsManager.java:43)
at com.atlassian.confluence.impl.settings.DelegatingLegacySettingsManager.getGlobalSettings(DelegatingLegacySettingsManager.java:57)
at com.atlassian.confluence.status.service.DefaultSystemInformationService.isShowInfoOn500(DefaultSystemInformationService.java:326)
at org.apache.jsp._500page_jsp._jspService(_500page_jsp.java:383)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:466)
... 69 more