Upgrade from 7.4 to 7.13 fails with out of memory error

[Martin Begley]

We have an existing Confluence Server V7.4.0 installation. I am trialling the upgrade to V7.13.0 and have encountered a problem.

I am carrying out the upgrade using the manual method, i.e. by restoring the archive manually.

After editing the various required files (such as setenv.sh) to put our customisations back in I started up Confluence and proceeded to watch the output from atlassian-confluence.log. 

It started processing our database tables and all was well until it hit the BODYCONTENT table. It spent some time on that table and eventually spat out the following errors:

2021-08-23 12:42:59,822 ERROR [httpclient-io:thread-1] [impl.nio.client.InternalHttpAsyncClient] run I/O reactor terminated abnormally
org.apache.http.nio.reactor.IOReactorException: I/O dispatch worker terminated abnormally
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:356)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space

Because it had been taking a while to process BODYCONTENT I had a look in the database (MySQL) to see what it was doing - it was running the query "SELECT * FROM BODYCONTENT"

Our BODYCONTENT table is fairly large - it contains 2.47 million rows, so it is not surprising that it ran out of memory - and the size of the table on disk is 57 Gb.

Our Java heap settings are ""-Xms16384m -Xmx16384m"

While I could try increasing the heap sizes (the server has 32 Gb or RAM) I'm not convinced that it would work as the process seems to be trying to read the entire table into memory.

Does anyone have any ideas about this? Has anyone successfully upgraded a Confluence installation of this size or larger?

[Alexis Robert]

Hi @Martin Begley , 

the easiest way to upgrade Confluence is not to restore an XML backup, it would be to start your new Confluence with the same CONFLUENCE_HOME against the current production database : this way the upgrade will perform the needed databases updates and will not require you to load a huge XML file.

Let me know if this helps, 

--Alexis

[Martin Begley]

Hi, perhaps I wasn't clear enough in my description - I am not restoring an XML backup. When I said "I am carrying out the upgrade using the manual method, i.e. by restoring the archive manually.", what I meant was that I have copied down the "atlassian-confluence-7.13.0.tar.gz" file and have unzipped that manually rather than using the installer upgrade method.

I am carrying out the upgrade against our existing CONFLUENCE_HOME and the existing database.

It is during the database updates that it fails with the out of memory error.

[Martin Begley]

It turns out that it was failing during the creation of the recovery file. Disabled this and it is now getting further.