Forums
Home Product Q&A Discussion groups Articles Ask a question Search
Learning
Events
Champions

Forums

Articles
Create
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Product Q&A
See all
Community resources
Top groups
Explore all groups
Community resources
Learn
Community resources
Events
Community resources

Confluence Fails to load after DB Update

Andy
Andy June 19, 2021

I am running CentOS 7, and uninstalled my old version of postgresql after backing my db up.  I am now running version 11, and I did a restore of the db.  When I attempt to visit confluence in my browser, this is what I am seeing.  

https://ibb.co/XkFFMP0

Update

Refreshing the page now shows this:

HTTP Status 500 – Internal Server Error

Type Exception Report

Message java.io.IOException: java.lang.reflect.InvocationTargetException

Description The server encountered an unexpected condition that prevented it from fulfilling the request.

Exception

java.lang.IllegalStateException: java.io.IOException: java.lang.reflect.InvocationTargetException
	org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(AbstractSingleArchiveResourceSet.java:101)
	org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:256)
	org.apache.catalina.webresources.StandardRoot.getResourceInternal(StandardRoot.java:282)
	org.apache.catalina.webresources.Cache.getResource(Cache.java:62)
	org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:217)
	org.apache.catalina.webresources.StandardRoot.getClassLoaderResource(StandardRoot.java:226)
	org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2303)
	org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:865)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1334)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1188)
	com.atlassian.confluence.tenant.TenantGateFilter.doFilter(TenantGateFilter.java:38)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.MobileAppRequestFilter.doFilter(MobileAppRequestFilter.java:36)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.internal.web.filter.spring.IgnoreWebAsyncManagerFilter.doFilter(IgnoreWebAsyncManagerFilter.java:59)
	com.atlassian.confluence.web.filter.validateparam.RequestParamValidationFilter.doFilter(RequestParamValidationFilter.java:51)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.filter.TranslationModeFilter.doFilter(TranslationModeFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.plugin.servlet.filter.ActionContextCleanUp.doFilter(ActionContextCleanUp.java:71)
	com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:58)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:59)
	com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
	com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
	com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:37)
	com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
	com.atlassian.confluence.internal.diagnostics.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:35)
	com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

Root Cause

java.io.IOException: java.lang.reflect.InvocationTargetException
	org.apache.tomcat.util.compat.Jre9Compat.jarFileNewInstance(Jre9Compat.java:236)
	org.apache.tomcat.util.compat.JreCompat.jarFileNewInstance(JreCompat.java:162)
	org.apache.catalina.webresources.AbstractArchiveResourceSet.openJarFile(AbstractArchiveResourceSet.java:307)
	org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(AbstractSingleArchiveResourceSet.java:97)
	org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:256)
	org.apache.catalina.webresources.StandardRoot.getResourceInternal(StandardRoot.java:282)
	org.apache.catalina.webresources.Cache.getResource(Cache.java:62)
	org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:217)
	org.apache.catalina.webresources.StandardRoot.getClassLoaderResource(StandardRoot.java:226)
	org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2303)
	org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:865)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1334)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1188)
	com.atlassian.confluence.tenant.TenantGateFilter.doFilter(TenantGateFilter.java:38)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.MobileAppRequestFilter.doFilter(MobileAppRequestFilter.java:36)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.internal.web.filter.spring.IgnoreWebAsyncManagerFilter.doFilter(IgnoreWebAsyncManagerFilter.java:59)
	com.atlassian.confluence.web.filter.validateparam.RequestParamValidationFilter.doFilter(RequestParamValidationFilter.java:51)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.filter.TranslationModeFilter.doFilter(TranslationModeFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.plugin.servlet.filter.ActionContextCleanUp.doFilter(ActionContextCleanUp.java:71)
	com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:58)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:59)
	com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
	com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
	com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:37)
	com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
	com.atlassian.confluence.internal.diagnostics.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:35)
	com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

Root Cause

java.lang.reflect.InvocationTargetException
	java.base/jdk.internal.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
	java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
	org.apache.tomcat.util.compat.Jre9Compat.jarFileNewInstance(Jre9Compat.java:233)
	org.apache.tomcat.util.compat.JreCompat.jarFileNewInstance(JreCompat.java:162)
	org.apache.catalina.webresources.AbstractArchiveResourceSet.openJarFile(AbstractArchiveResourceSet.java:307)
	org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(AbstractSingleArchiveResourceSet.java:97)
	org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:256)
	org.apache.catalina.webresources.StandardRoot.getResourceInternal(StandardRoot.java:282)
	org.apache.catalina.webresources.Cache.getResource(Cache.java:62)
	org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:217)
	org.apache.catalina.webresources.StandardRoot.getClassLoaderResource(StandardRoot.java:226)
	org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2303)
	org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:865)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1334)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1188)
	com.atlassian.confluence.tenant.TenantGateFilter.doFilter(TenantGateFilter.java:38)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.MobileAppRequestFilter.doFilter(MobileAppRequestFilter.java:36)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.internal.web.filter.spring.IgnoreWebAsyncManagerFilter.doFilter(IgnoreWebAsyncManagerFilter.java:59)
	com.atlassian.confluence.web.filter.validateparam.RequestParamValidationFilter.doFilter(RequestParamValidationFilter.java:51)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.filter.TranslationModeFilter.doFilter(TranslationModeFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.plugin.servlet.filter.ActionContextCleanUp.doFilter(ActionContextCleanUp.java:71)
	com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:58)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:59)
	com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
	com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
	com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:37)
	com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
	com.atlassian.confluence.internal.diagnostics.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:35)
	com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

Root Cause

java.io.FileNotFoundException: /opt/atlassian/confluence/confluence/WEB-INF/lib/atlassian-http-2.0.8.jar (Too many open files)
	java.base/java.io.RandomAccessFile.open0(Native Method)
	java.base/java.io.RandomAccessFile.open(Unknown Source)
	java.base/java.io.RandomAccessFile.<init>(Unknown Source)
	java.base/java.io.RandomAccessFile.<init>(Unknown Source)
	java.base/java.util.zip.ZipFile$Source.<init>(Unknown Source)
	java.base/java.util.zip.ZipFile$Source.get(Unknown Source)
	java.base/java.util.zip.ZipFile$CleanableResource.<init>(Unknown Source)
	java.base/java.util.zip.ZipFile$CleanableResource.get(Unknown Source)
	java.base/java.util.zip.ZipFile.<init>(Unknown Source)
	java.base/java.util.zip.ZipFile.<init>(Unknown Source)
	java.base/java.util.jar.JarFile.<init>(Unknown Source)
	java.base/jdk.internal.reflect.GeneratedConstructorAccessor7.newInstance(Unknown Source)
	java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
	org.apache.tomcat.util.compat.Jre9Compat.jarFileNewInstance(Jre9Compat.java:233)
	org.apache.tomcat.util.compat.JreCompat.jarFileNewInstance(JreCompat.java:162)
	org.apache.catalina.webresources.AbstractArchiveResourceSet.openJarFile(AbstractArchiveResourceSet.java:307)
	org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.getArchiveEntry(AbstractSingleArchiveResourceSet.java:97)
	org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:256)
	org.apache.catalina.webresources.StandardRoot.getResourceInternal(StandardRoot.java:282)
	org.apache.catalina.webresources.Cache.getResource(Cache.java:62)
	org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:217)
	org.apache.catalina.webresources.StandardRoot.getClassLoaderResource(StandardRoot.java:226)
	org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2303)
	org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:865)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1334)
	org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1188)
	com.atlassian.confluence.tenant.TenantGateFilter.doFilter(TenantGateFilter.java:38)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.MobileAppRequestFilter.doFilter(MobileAppRequestFilter.java:36)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.internal.web.filter.spring.IgnoreWebAsyncManagerFilter.doFilter(IgnoreWebAsyncManagerFilter.java:59)
	com.atlassian.confluence.web.filter.validateparam.RequestParamValidationFilter.doFilter(RequestParamValidationFilter.java:51)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.filter.TranslationModeFilter.doFilter(TranslationModeFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.plugin.servlet.filter.ActionContextCleanUp.doFilter(ActionContextCleanUp.java:71)
	com.atlassian.confluence.web.filter.LanguageExtractionFilter.doFilter(LanguageExtractionFilter.java:39)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.impl.vcache.VCacheRequestContextFilter.doFilter(VCacheRequestContextFilter.java:58)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.LoggingContextFilter.doFilter(LoggingContextFilter.java:33)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.util.RequestCacheThreadLocalFilter.doFilter(RequestCacheThreadLocalFilter.java:59)
	com.atlassian.confluence.web.filter.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:54)
	com.atlassian.confluence.web.filter.ResponseOutputStreamFilter.doFilter(ResponseOutputStreamFilter.java:25)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.confluence.web.ConfluenceJohnsonFilter.doFilter(ConfluenceJohnsonFilter.java:30)
	com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:37)
	com.atlassian.confluence.servlet.FourOhFourErrorLoggingFilter.doFilter(FourOhFourErrorLoggingFilter.java:64)
	com.atlassian.confluence.internal.diagnostics.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:35)
	com.atlassian.confluence.web.filter.DebugFilter.doFilter(DebugFilter.java:46)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

Note The full stack trace of the root cause is available in the server logs.

Apache Tomcat/9.0.33

I did run the initdb command. 

Answer
Watch
Like Be the first to like this
875 views

1 answer

0 votes
Nic Brough -Adaptavist-
Nic Brough -Adaptavist-
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
Get recognized
June 19, 2021

Ok, the core of the problem is "Jira cannot connect to the database", but you've given us enough information to track that down to why - it can't actually open the database driver.

java.io.FileNotFoundException: /opt/atlassian/confluence/confluence/WEB-INF/lib/atlassian-http-2.0.8.jar (Too many open files)

is not 100% clear - what it says is "file not found", but what it means is "I can't open a file", not that the file is not there. 

The "too many open files" is the culprit, that means the OS is limited to only opening a certain number of files (by the current user) and you have run into that limit.  It is a bit archaic, one of those things that really should just go away as it is mostly pointless as hardware and software have evolved way past it (a bit like Windows for example)

Anyway, the history of it is meh.  There's a fix written up at https://confluence.atlassian.com/confkb/too-many-open-files-error-659784628.html - it boils down to "change the ulimit for the user and restart the application"

Reply

Suggest an answer

Log in or Sign up to answer
Confluence
Confluence
TAGS
AUG Leaders

Atlassian Community Events

    See all events
    Community
    Forums
    FAQs Forums guidelines
    Learning
    About learning Resources
    Events
    Champions
    Copyright © 2025 Atlassian
    Privacy Policy Terms Security
    Welcome illustration

    Welcome to the new Atlassian Community

    Online forums and learning are now in one easy-to-use experience.

    By continuing, you accept the updated Community Terms of Use and acknowledge the Privacy Policy. Your public name, photo, and achievements may be publicly visible and available in search engines.