Content of change history lost on Jira to Jira migration

[Thilo Brause]

We migrated a couple of projects from one Jira instance to another, using the default methodoloy of creating a backup and importing the projects one by one.

Somehow the content of changes have been lost in the process. The change history of an issue now looks like:

As you can see, all content about what is changed, has been lost during the migration.

Does anybody have an idea what could be the cause for that problem and how to fix it?

What is even more strange: Only one of six migrated projects is affected. But in this project all issues.

[Radek Dostál · Accepted solution]

If memory serves, this is the cause https://jira.atlassian.com/browse/JRASERVER-59681

If I am not mistaken, the problem is that one of those 6 projects is using either a single or multiple version-picker custom fields, and the other 5 are not.

There are a few comments in the above issue which you might want to check, you should be able to fix this either with SQL on the source instance, or by modifying the backup file (entities.xml in particular). I'd of course recommend the zip file because you're not risking breaking stuff, it's easier to just delete the project and re-import it again (or delete versions, components and issues if you are using the built-in project import, and not Project Configurator plugin).

Personally I've been modifying the entities.xml file in the backup file for imports to avoid this bug for years so I don't remember the steps by heart, I'd have to log on to the network and dig it up, I'll try tomorrow if I don't forget.

The last comment by Gawdi is pretty much what you need to do in the entities.xml file - have to change the oldvalue and newvalue values, so that the format is "right", but I must say the regex looks a little weird to me, I'm using a different one - you could try it though, maybe it's just a late hour for me and I have my regular expressions messed up. The essence is right though - it's those xhtml tags where ChangeItem is issue history, and those oldvalue newvalue values are bricking your change history. Basically during the import it fails to process the values and results in either none or partial change history for issues. When you "fix" the value format in the backup file, the issue history should import as expected.

I don't think I've ever come across any other problem with change history other than the version value format in the export file, so for me, this would be the first thing to check.

Again if I don't forget tomorrow I'll try to look up what regex my scripts are using, but in the meantime you might want to try what Gawdi posted on the bug, it's the right direction.

[Radek Dostál]

Okay so I've done some testing, first I created a single and multi select version fields, created 2 issues, flipped some history in, and then exported and reimported the project. I was able to reproduce the history being broken this way as I expected.

Then I "sanitized" the entities.xml file and did the import again, and this time, all history is correct.

This is the shell script I used:

#!/bin/bash

#remove empty brackets newvalue="[]" newvalue="", oldvalue="[]" oldvalue=""

#remove brackets from values, newvalue="[111]" newvalue="111", oldvalue="[123, 134]" oldvalue="123, 134"

#remove oldvalue="" and newvalue="", they should not be present in entities.xml, but could be created when we remove empty brackets in the first step


cat entities.xml \
| sed -E 's#(<ChangeItem .*fieldtype="custom" field=")(Single Version Picker Field|Multi Version Picker Field)(".*)(oldvalue=")\[\]"#\1\2\3\4"#' \
| sed -E 's#(<ChangeItem .*fieldtype="custom" field=")(Single Version Picker Field|Multi Version Picker Field)(".*)(newvalue=")\[\]"#\1\2\3\4"#' \
| sed -E 's#(<ChangeItem .*fieldtype="custom" field=")(Single Version Picker Field|Multi Version Picker Field)(".*)(oldvalue=")\[([0-9, ]*)\]"#\1\2\3\4\5"#' \
| sed -E 's#(<ChangeItem .*fieldtype="custom" field=")(Single Version Picker Field|Multi Version Picker Field)(".*)(newvalue=")\[([0-9, ]*)\]"#\1\2\3\4\5"#' \
| sed -E 's#(<ChangeItem .*fieldtype="custom" field=")(Single Version Picker Field|Multi Version Picker Field)(".*)(oldvalue=)""#\1\2\3\4#' \
| sed -E 's#(<ChangeItem .*fieldtype="custom" field=")(Single Version Picker Field|Multi Version Picker Field)(".*)(newvalue=)""#\1\2\3\4#' \
> entities.xml2

mv entities.xml2 entities.xml


# if you're not sure, you can get your version fields' names like so
# grep -E '<CustomField .*customfieldtypekey="com.atlassian.jira.plugin.system.customfieldtypes:(multiversion|version)"' entities.xml | grep -o 'name="[^"]*"' | sed 's#^name="##;s#"$##' > cfnames
# and then use that to construct the field="(name1|name2)"
# however, be aware that you may need to escape certain special characters to be sed-friendly, for example, 'Field Name (ver2)' needs to be escaped as 'Field Name \(ver2\)', and so forth

[Thilo Brause]

Dear Radek,

thank you for your effort and analysis. 

So far we assume it was a problem with Sprints as described in https://confluence.atlassian.com/jirakb/project-import-fails-with-numberformatexception-in-jira-server-816686500.html

After applying the fix described there the activity stream looks normal.

However there are some issues with versions in that project, so I will keep your hint in mend.

Best Regards,

   Thilo Brause