We are seeing very poor performance of our JIRA report macros. Specificaly, the JIRA Report blueprint-created pages. When we refresh these pages, it takes a very long time (30-60 seconds) to load. While loading, we observe 4 "JIRA macros executor" threads running. In fact, these threads have been "running" longer than they have been "waiting". Sometimes these pages simply timeout.
Our production JIRA server is not exceptionally busy AND the same filters that the Confluence report page uses return very quickly in JIRA.
Any ideas what could be causing such poor performance in the Confluence pages that access JIRA content?
Here is a thread dump:
2014-11-25 09:25:32 "Jira macros executor:thread-1" - Thread t@395 java.lang.Thread.State: RUNNABLE at java.math.BigInteger.mulAdd(BigInteger.java:1935) at java.math.BigInteger.montReduce(BigInteger.java:1880) at java.math.BigInteger.oddModPow(BigInteger.java:1850) at java.math.BigInteger.modPow(BigInteger.java:1599) at org.bouncycastle.crypto.engines.RSACoreEngine.processBlock(Unknown Source) at org.bouncycastle.crypto.engines.RSABlindedEngine.processBlock(Unknown Source) at org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.engineDoFinal(Unknown Source) at javax.crypto.Cipher.doFinal(Cipher.java:2087) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.generateV2EncryptedCertificate(BouncyCastleEncryptionProvider.java:332) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.createEncryptedCertificate(BouncyCastleEncryptionProvider.java:265) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:93) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.access$000(DefaultCurrentApplication.java:16) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:32) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:29) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3573) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) - locked <a89a00a> (a com.google.common.cache.LocalCache$StrongAccessWriteEntry) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3970) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4834) at com.google.common.cache.LocalCache$LocalManualCache.getUnchecked(LocalCache.java:4839) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:79) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:67) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:58) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.execute(TrustedRequest.java:38) at com.atlassian.confluence.extra.jira.DefaultJiraIssuesManager.retrieveXML(DefaultJiraIssuesManager.java:90) at com.atlassian.confluence.extra.jira.CacheJiraIssuesManager.retrieveXML(CacheJiraIssuesManager.java:41) at com.atlassian.confluence.extra.jira.DefaultJiraIssuesManager.retrieveXMLAsChannel(DefaultJiraIssuesManager.java:172) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.populateContextMapForStaticCountIssues(JiraIssuesMacro.java:907) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.createContextMapFromParams(JiraIssuesMacro.java:495) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.execute(JiraIssuesMacro.java:1195) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:78) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:19) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - locked <5189a5d5> (a java.util.concurrent.ThreadPoolExecutor$Worker) "Jira macros executor:thread-2" - Thread t@396 java.lang.Thread.State: RUNNABLE at java.math.BigInteger.mulAdd(BigInteger.java:1930) at java.math.BigInteger.squareToLen(BigInteger.java:1319) at java.math.BigInteger.oddModPow(BigInteger.java:1849) at java.math.BigInteger.modPow(BigInteger.java:1599) at org.bouncycastle.crypto.engines.RSACoreEngine.processBlock(Unknown Source) at org.bouncycastle.crypto.engines.RSABlindedEngine.processBlock(Unknown Source) at org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.engineDoFinal(Unknown Source) at javax.crypto.Cipher.doFinal(Cipher.java:2087) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.generateV2EncryptedCertificate(BouncyCastleEncryptionProvider.java:332) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.createEncryptedCertificate(BouncyCastleEncryptionProvider.java:265) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:93) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.access$000(DefaultCurrentApplication.java:16) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:32) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:29) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3573) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) - locked <14b76357> (a com.google.common.cache.LocalCache$StrongAccessWriteEntry) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3970) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4834) at com.google.common.cache.LocalCache$LocalManualCache.getUnchecked(LocalCache.java:4839) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:79) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:67) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:58) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.execute(TrustedRequest.java:38) at com.atlassian.confluence.extra.jira.DefaultJiraIssuesManager.retrieveXML(DefaultJiraIssuesManager.java:90) at com.atlassian.confluence.extra.jira.CacheJiraIssuesManager.retrieveXML(CacheJiraIssuesManager.java:41) at com.atlassian.confluence.extra.jira.DefaultJiraIssuesManager.retrieveXMLAsChannel(DefaultJiraIssuesManager.java:172) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.populateContextMapForStaticCountIssues(JiraIssuesMacro.java:907) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.createContextMapFromParams(JiraIssuesMacro.java:495) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.execute(JiraIssuesMacro.java:1195) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:78) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:19) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - locked <79868617> (a java.util.concurrent.ThreadPoolExecutor$Worker) "Jira macros executor:thread-3" - Thread t@397 java.lang.Thread.State: RUNNABLE at java.math.BigInteger.mulAdd(BigInteger.java:1930) at java.math.BigInteger.montReduce(BigInteger.java:1880) at java.math.BigInteger.oddModPow(BigInteger.java:1850) at java.math.BigInteger.modPow(BigInteger.java:1599) at org.bouncycastle.crypto.engines.RSACoreEngine.processBlock(Unknown Source) at org.bouncycastle.crypto.engines.RSABlindedEngine.processBlock(Unknown Source) at org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.engineDoFinal(Unknown Source) at javax.crypto.Cipher.doFinal(Cipher.java:2087) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.generateV2EncryptedCertificate(BouncyCastleEncryptionProvider.java:332) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.createEncryptedCertificate(BouncyCastleEncryptionProvider.java:265) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:93) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.access$000(DefaultCurrentApplication.java:16) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:32) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:29) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3573) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) - locked <24bb575c> (a com.google.common.cache.LocalCache$StrongAccessWriteEntry) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3970) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4834) at com.google.common.cache.LocalCache$LocalManualCache.getUnchecked(LocalCache.java:4839) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:79) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:67) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:58) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.execute(TrustedRequest.java:38) at com.atlassian.confluence.plugins.jiracharts.Base64JiraChartImageService.getBase64JiraChartImageModel(Base64JiraChartImageService.java:39) at com.atlassian.confluence.plugins.jiracharts.render.JiraImageChart.getImageSourceModel(JiraImageChart.java:53) at com.atlassian.confluence.plugins.jiracharts.render.PieChart.setupContext(PieChart.java:61) at com.atlassian.confluence.plugins.jiracharts.JiraChartMacro.execute(JiraChartMacro.java:93) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:78) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:19) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - locked <5535955e> (a java.util.concurrent.ThreadPoolExecutor$Worker) "Jira macros executor:thread-4" - Thread t@420 java.lang.Thread.State: RUNNABLE at java.math.BigInteger.mulAdd(BigInteger.java:1930) at java.math.BigInteger.squareToLen(BigInteger.java:1319) at java.math.BigInteger.oddModPow(BigInteger.java:1849) at java.math.BigInteger.modPow(BigInteger.java:1599) at org.bouncycastle.crypto.engines.RSACoreEngine.processBlock(Unknown Source) at org.bouncycastle.crypto.engines.RSABlindedEngine.processBlock(Unknown Source) at org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.engineDoFinal(Unknown Source) at javax.crypto.Cipher.doFinal(Cipher.java:2087) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.generateV2EncryptedCertificate(BouncyCastleEncryptionProvider.java:332) at com.atlassian.security.auth.trustedapps.BouncyCastleEncryptionProvider.createEncryptedCertificate(BouncyCastleEncryptionProvider.java:265) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:93) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.access$000(DefaultCurrentApplication.java:16) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:32) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication$1.load(DefaultCurrentApplication.java:29) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3573) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) - locked <51e4faa7> (a com.google.common.cache.LocalCache$StrongAccessWriteEntry) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3970) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4834) at com.google.common.cache.LocalCache$LocalManualCache.getUnchecked(LocalCache.java:4839) at com.atlassian.security.auth.trustedapps.DefaultCurrentApplication.encode(DefaultCurrentApplication.java:79) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:67) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.signRequest(TrustedRequest.java:58) at com.atlassian.applinks.core.auth.trusted.TrustedRequest.execute(TrustedRequest.java:38) at com.atlassian.confluence.extra.jira.DefaultJiraIssuesManager.retrieveXML(DefaultJiraIssuesManager.java:90) at com.atlassian.confluence.extra.jira.CacheJiraIssuesManager.retrieveXML(CacheJiraIssuesManager.java:41) at com.atlassian.confluence.extra.jira.DefaultJiraIssuesManager.retrieveXMLAsChannel(DefaultJiraIssuesManager.java:172) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.populateContextMapForStaticCountIssues(JiraIssuesMacro.java:907) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.createContextMapFromParams(JiraIssuesMacro.java:495) at com.atlassian.confluence.extra.jira.JiraIssuesMacro.execute(JiraIssuesMacro.java:1195) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:78) at com.atlassian.confluence.extra.jira.executor.StreamableMacroFutureTask.call(StreamableMacroFutureTask.java:19) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Locked ownable synchronizers: - locked <64a41f41> (a java.util.concurrent.ThreadPoolExecutor$Worker)
Notice exactly the same when many JIRA issue links are on a confluence page, e.g. by JIRA issues created via JIRA issues macro
We scheduled a restart of JIRA and cleaned out all the clean-able caches. We again do not have this problem. Will keep an eye on it and update here if it recurs.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
JIRA 6.3.8 Confluence 5.6.3
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
May be related to "https://ecosystem.atlassian.net/browse/TRUST-44";
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Online forums and learning are now in one easy-to-use experience.
By continuing, you accept the updated Community Terms of Use and acknowledge the Privacy Policy. Your public name, photo, and achievements may be publicly visible and available in search engines.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.