Speed up issue creation during big import, (maybe temporarily disable index for a project in Jira?)

[Jens Kisters __SeibertSolutions]

Hello,

we are currently importing a large amount of issues (700k - 800k) into a Jira 7.13.1 instance.

The target issue type has a minimal workflow but a couple of dozen custom fields among that some Stagil Asset Relation fields that create some issue Links when saved.

It currently takes about 5 seconds for a single issue creation,

we are looking to speed up the throughput, i am assuming indexing takes up a big part of the ressources consumed.

The instance has about 300 custom fields of which roughly 50% are global.

Is it possible to disable indexing and enable it, do a project index after the import has completed?

Removing the index post function from the create step of the workflow did not seem to have an effect.

We are creating the issues via the REST API by the way, a script reads them from the source database and keeps track which have already been imported.

i am aware that disabling indexing will make the imported issues unavailable in issue search.

kind regards

Jens

[Daniel Fietta]

Hi @Jens Kisters __SeibertSolutions,

Please check out our new version of STAGIL Assets which was released this week. We have significantly improved the performance of the REST API import when a STAGIL Assets Relation custom field is present on your screen.

Please try the new version and let us know if you are still experiencing performance issues.

Also, thank you for the hint about the field searchers. We will investigate dependencies regarding the performance.

Thank you for your investigations,
Daniel
STAGIL

[Jens Kisters __SeibertSolutions]

Hi Daniel, 

thanks for looking into this.

I have updated the the app and will see if there is a measurable speed up.

Cheers

Jens

[Jens Kisters __SeibertSolutions]

Hi @Daniel Fietta :

This chart shows the number of created issues per day.

The arrow shows the first day where the import has been running for 24 hours with the new Stagil Assets Version 4.4.22 installed.

Even though there is a slight rise it didnt exceed the level it already had with a new version a few weeks earlier.

kind regards

Jens

[STAGIL]

Hi Jens,

thanks for your post. Asset relations fields are more heavy than other custom fields since they manage Jira links. So during the creation not only the issues with their values are created but also links are created for every value in an asset relation field.

My team and I will investigate if the import is much slower than the import of an comparable amount of issue links.

Best regards
Björn from STAGIL (Product Owner of STAGIL Assets)

[Jens Kisters __SeibertSolutions]

Thanks a lot.

I am eager to hear the results, let me know if you need any details about our setup.

[Jens Kisters __SeibertSolutions]

Hi Björn AKA @STAGIL ,

i have made some more tests and found out that creating issues via the REST API takes far longer then creating an identical Issue via the Jira Clone function.

For this purpose i logged some timestamp in the create step of the workflow,

when cloning these were about 300ms apart,

when using the REST API they were about 1500ms apart.

Our Use Case contains 3 Stagil fields holding i issue each.

Cheers

Jens

[Jens Kisters __SeibertSolutions]

when working on this i have learned that when you log the time elapsed between the postfunction step "create the issue" and the last one,
that time is nowhere near the actual time it takes for the system to create the issue.

about 60% of the time passes before you can log your first timestamp.

So it was wrong to assume limiting custom field contexts does not speed up creation (original post edited)

I have now enhanced my logging (take the start timestamp via javascript on the client side) and will now see if there is a significant increase when deleting / adding contexts to custom fields.

In addition i have removed the searchers on most of the fields, it has shown to give a signifcant boost on creation performance for another issue type.

[Jens Kisters __SeibertSolutions]

@STAGIL GmbH @STAGIL : Any news on this one?

My team and I will investigate if the import is much slower than the import of an comparable amount of issue links.