Insight Import: Slow-down on processing step 6

[Thomas Scheibelhofer]

Hello,

currently we're experiencing a major slow-down of insight imports.

What took 7 minutes before, now takes over 4 hours and it's unclear what could cause this.

The import i'm doing in the screenshot is usually done within 2 - 3 minutes and is now at about 15 minutes.

I have a bit of a 'workaround' setup which let's all imports fail with "object_type_errors" on purpose, as it's not possible to filter for referenced items in the import.

So, my question is actually, has anyone else noticed similar behavior?

The system was updated to 5.4.6 on 17th or 18th, we started having the issues on Wednesday (21st).

Best regards

Thomas

[Alexander Sundström · Accepted solution]

Hi Thomas,

From what version did you update? 

Have you looked at the documentation at https://documentation.riada.se/insight/latest/system-requirements about the requirements and what can be done for performance tuning?

Also, it's a good idea to look in the atlassian-jira.log if it mentions JVM low on memory, as we have seen previously that this is usally the case behind slow imports.

Cheers
Alexander

[Thomas Scheibelhofer]

Hi Alexander,

thanks for your answer, i wanted to 'give the green light' just today.

We've had a major performance issues in one of the nodes (the scheduling node), which was restarted and now everything's back to normal.

Some processes have seemingly been taking up or freezing the CPUs. We'll now have to investigate whether Insight can be the cause of this or if we're having troubles on another front.

Regarding the system requirements, for now we should still be fine, at this point we have about 10.000 objects with a 3-node data center setup with ~ 10 GB JVM RAM and 12 cores per node.

Thank you and best regards

Thomas

[Alexander Sundström]

Thanks for the update Thomas, I'm glad to hear that the restart seems to have addressed the issue.

Let us know if you find any indications on Insight related to the performance.

Cheers!
Alexander