Restore backup of cloud instance on staging on-premises server

[Sravya Vuggina]

Our team currently uses JIRA cloud instance and is planning on setting up on prem server in near future. I am performing POC to migrate data of certain projects from the backup file of cloud instance. Based on some readings I did, all the plugins currently running on cloud instance must be present on server as well. However, few plugins present are specific to cloud instance hence I failed to import. 

A colleague recommended to use "Restore System" instead of Import Project. This time I have not installed plug ins on new server. After restoring I lost connection with database. JIRA Configuration Tool says connection successful but h2 console displays otherwise. 

Any help in migrating data is much appreciated! 

[Mike Rathwell]

HI @Sravya Vuggina,

I can see just a couple of things that you might need to look at, especially if you're wanting to do a solid POC.

First one up is, especially depending on the size of your instance is to grow your Xms and Xmx to give the JVM enough room to work. If it's not too big of a system, just set 'em both at 2048m.

Second... the H2 is good to stand up an empty one but probably won't give you reasonable performance OR a good POC. Toss in a PostgreSQL server (I'm suggesting 9.5 as it has been supported for a long time and will be for some time but 9.6 if you're starting the latest version of Jira). The PostgreSQL drivers are included with the base Jira install. 

You're probably going to want to install all the plugins you CAN as there may be some startup issues without depending on how "invasive" they are.

Build it up all empty and THEN try the restore.

Give 'er a shot

[Andy Heinzer]

I agree, the H2 database should not be used for importing existing data.  It's much better to use a supported database from the supported platforms page instead.

[Sravya Vuggina]

Thanks much @Mike Rathwell! Will try the recommended steps.

The zipped backup file is about 100GB (including attachments), so I am assuming my very first step should be to use a supported database. 

[Mike Rathwell]

Yup @Sravya Vuggina, that is NOT a trivial restore you are doing. That is going to kill both JVM memory if not big enough AND the DB. 

I notice you are on Windows from your screenshots. IF you're more comfortable with it, you could use MS SQL server but I, personally, don't like it a lot given performance hits based on how it locks records for write (often locks the whole table). PgSQL has been a winner for me for years now (and have used it on Windows server in a dark period where I was forced to use that).

[Mike Rathwell]

Also, if you've not found it yet, here are the instructions for setting JVM settings if you're running this as a Windows Service

https://confluence.atlassian.com/adminjiraserver073/increasing-jira-application-memory-861253796.html

[Sravya Vuggina]

I have switched database to MSSQL 2012 (just a comfortable choice) and noticed an interesting thing. Data got imported to few tables and not all (when alphabetically sorted data exists up to jiraissue table, rest of the tables are empty). 

I have tried to login with my credentials and another admin credentials but it continues to pop up below message. Wonder if JIRA is unable to identify login as data pertaining to user management has not been imported. 

Any thoughts on how to identify root cause? 

[Nic Brough -Adaptavist-]

When you move from Cloud to Server, the user IDs become disconnected from the Cloud user directory.

The export/import process is aware of this, and creates a system admin account for you to use to get into your server install and reconfigure it all.  Log in as sysadmin with a password of sysadmin too.

[Andy Heinzer]

Check your $JIRAHOME/log/atlassian-jira.log file during this import process.  I would expect that the logs will have an entry for the percentage of data imported from 1% through to 100%.  You should see all 100 entries in this log if the import was complete and successful.

However I have seen instances where the import failed, but do not provide any indication in the GUI.  As such if you can't login with the sysadmin/sysadmin credentials after a Cloud to Server migration, then check the logs for some errors, they would contain details of why the import failed. 

[Sravya Vuggina]

Hello @Nic Brough -Adaptavist-,

I did try with sysadmin/sysadmin credentials too. Unfortunately, it gives the same error or incorrect username and password.  

[Sravya Vuggina]

Hello @Andy Heinzer,

This time I increased JVM memory, created MSSQL database with collation and isolation as recommended in JIRA installation guidelines. Yet the import process failed at 69% (same as before) as here is what log file states. - The conversion of a datatime2 datatype to a datetime data type resulted in an out-of-range value. 

The schema of jiraissue table doesn't contain any fields with datatime2 database (atleast in 7.9.2). Wonder if that was how it was in prior versions. Is this a know issue? 

[Nic Brough -Adaptavist-]

If you're only getting a partial import, there's not a lot of point trying to log in - it won't have got far enough.

[Sravya Vuggina]

That makes sense. I believe the conversion issue is the roadblock for import. I've looked into project and few more tables, there is no data filled in. 

[Andy Heinzer]

Hi Sravya,

I have not been able to find any documented bug ticket or KB articles in Atlassian that reference that specific error message.  In our customer support portal, I did find a very small number of users that reported this problem with that error message.

In these, it appears that your data for some reason has that duedate value with a year of 0172.  In a datetime2 data type, that's fine.  However the datetime values have to be between 1753 and 9999.   But I have not been able to clarify why this might have been happening.  It's possible it was a still undocumented bug, but it might be something more environmentally specific here.

I don't know what value 0172 is supposed to represent for the year of a due date.  Is your data possibly using a non-Gregorian calendar?

In the other cases I found of this, it required the end user to open the backup.zip file and then manually edit the entities.xml file to replace these values in the backup file with valid datetime values.  Only then could you then save that file, add it back to the zip file and use that to complete the import.

[Sravya Vuggina]

Hello @Andy Heinzer,

Thanks for identifying the goofy duedate value. Regarding calendar, I believe not - as most of the projects in current JIRA cloud instance are operated here in USA. 

As recommended, I tried unzipping the file to alter entities.xml file. However, since the file is too big it isn't opening. Instead I have fixed the due date for this specific jira issue and taken backup again. While importing the latest backup file, I again ran into same issue. Not sure if the modifications made in JIRA got reflected to the database (was hoping it would).   

[Sravya Vuggina]

Finally I was able to restore!

Used 010Editor to open entities.xml file. Fixed duedate, and imported the data. It worked.. Thanks @Andy Heinzer! 
Logged in with sysadmin/sysadmin credentials.. Thanks @Nic Brough -Adaptavist-! 

[Andy Heinzer]

Glad to hear that helped!

[Mike Rathwell]

Way to go @Sravya Vuggina!!!

One thing I've also fallen into the habit of is slavishly documenting my steps I take to get something to work. I can't tell you how many times that:

• During a build out, the "next" step I take causes Something Bad to happen. With detailed documentation I can easily get back to where I was without having to figure out the previous steps again
• When I have to go back and do something again days, weeks, months, <insert time span here>, it is so much easier to have it written down how something that works fine can be changed, updated, recreated, etc.
• Allows others in the team to be able to either run my stuff in a pinch or hand it off to someone else so I can do something else fun.

Glad your odyssey is over.

mike 

[Vickey Palzor Lepcha]

The first thing I would do is - CHANGE MY DATABASE