Forums

Articles
Create
cancel
Showing results for 
Search instead for 
Did you mean: 

URGENT!!! Server down

Nawaaz SOHAWON
Contributor
September 26, 2018

Kindly note that the JIRA service desk is down and I am prompted with the below error log:

 

HTTP Status 500 – Internal Server Error


Type Exception Report

Message TenantContextCommand short-circuited and fallback disabled.

Description The server encountered an unexpected condition that prevented it from fulfilling the request.

Exception

com.netflix.hystrix.exception.HystrixRuntimeException: TenantContextCommand short-circuited and fallback disabled.
	com.netflix.hystrix.AbstractCommand.handleFallbackDisabledByEmittingError(AbstractCommand.java:1052)
	com.netflix.hystrix.AbstractCommand.getFallbackOrThrowException(AbstractCommand.java:878)
	com.netflix.hystrix.AbstractCommand.handleShortCircuitViaFallback(AbstractCommand.java:982)
	com.netflix.hystrix.AbstractCommand.applyHystrixSemantics(AbstractCommand.java:557)
	com.netflix.hystrix.AbstractCommand.access$200(AbstractCommand.java:60)
	com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:419)
	com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:413)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
	rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:51)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
	rx.Observable.subscribe(Observable.java:10247)
	rx.Observable.subscribe(Observable.java:10214)
	rx.internal.operators.BlockingOperatorToFuture.toFuture(BlockingOperatorToFuture.java:51)
	rx.observables.BlockingObservable.toFuture(BlockingObservable.java:411)
	com.netflix.hystrix.HystrixCommand.queue(HystrixCommand.java:378)
	com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:344)
	com.atlassian.tenant.service.TenantContextLoader.fetch(TenantContextLoader.java:42)
	com.atlassian.tenant.impl.AbstractServiceTenantContextProvider.fetchTenantContext(AbstractServiceTenantContextProvider.java:58)
	com.atlassian.tenant.impl.AbstractTenantContextProvider.setTenantId(AbstractTenantContextProvider.java:132)
	com.atlassian.jira.tenant.JiraTenantContextProviderImpl.setTenantId(JiraTenantContextProviderImpl.java:78)
	com.atlassian.jira.web.filters.WorkContextInitializationFilter.setTenantId(WorkContextInitializationFilter.java:158)
	com.atlassian.jira.web.filters.WorkContextInitializationFilter.doFilter(WorkContextInitializationFilter.java:72)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.jira.security.ratelimit.AbstractJiraRateLimitingFilter.lambda$doFilterWithRateLimiting$0(AbstractJiraRateLimitingFilter.java:93)
	com.atlassian.ratelimit.core.impl.AbstractRateLimiter.runWithRateLimiting(AbstractRateLimiter.java:60)
	com.atlassian.jira.security.ratelimit.AbstractJiraRateLimitingFilter.doFilterWithRateLimiting(AbstractJiraRateLimitingFilter.java:86)
	com.atlassian.jira.security.ratelimit.AbstractJiraRateLimitingFilter.doFilter(AbstractJiraRateLimitingFilter.java:72)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.jira.util.zipkin.ZipkinTracingFilter.lambda$doFilter$1(ZipkinTracingFilter.java:85)
	com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:112)
	com.atlassian.jira.util.zipkin.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:73)
	com.atlassian.jira.web.filters.SecurityHttpHeaderRequestFilter.doFilter(SecurityHttpHeaderRequestFilter.java:41)
	com.atlassian.jira.internalstatus.RequestStatusFilter.doFilterOnce(RequestStatusFilter.java:54)
	com.atlassian.jira.web.filters.FilterOnceAbstractHttpFilter.doFilter(FilterOnceAbstractHttpFilter.java:25)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

Root Cause

java.lang.RuntimeException: Hystrix circuit short-circuited and is OPEN
	com.netflix.hystrix.AbstractCommand.handleShortCircuitViaFallback(AbstractCommand.java:979)
	com.netflix.hystrix.AbstractCommand.applyHystrixSemantics(AbstractCommand.java:557)
	com.netflix.hystrix.AbstractCommand.access$200(AbstractCommand.java:60)
	com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:419)
	com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:413)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
	rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
	rx.Observable.unsafeSubscribe(Observable.java:10151)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:51)
	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
	rx.Observable.subscribe(Observable.java:10247)
	rx.Observable.subscribe(Observable.java:10214)
	rx.internal.operators.BlockingOperatorToFuture.toFuture(BlockingOperatorToFuture.java:51)
	rx.observables.BlockingObservable.toFuture(BlockingObservable.java:411)
	com.netflix.hystrix.HystrixCommand.queue(HystrixCommand.java:378)
	com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:344)
	com.atlassian.tenant.service.TenantContextLoader.fetch(TenantContextLoader.java:42)
	com.atlassian.tenant.impl.AbstractServiceTenantContextProvider.fetchTenantContext(AbstractServiceTenantContextProvider.java:58)
	com.atlassian.tenant.impl.AbstractTenantContextProvider.setTenantId(AbstractTenantContextProvider.java:132)
	com.atlassian.jira.tenant.JiraTenantContextProviderImpl.setTenantId(JiraTenantContextProviderImpl.java:78)
	com.atlassian.jira.web.filters.WorkContextInitializationFilter.setTenantId(WorkContextInitializationFilter.java:158)
	com.atlassian.jira.web.filters.WorkContextInitializationFilter.doFilter(WorkContextInitializationFilter.java:72)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.jira.security.ratelimit.AbstractJiraRateLimitingFilter.lambda$doFilterWithRateLimiting$0(AbstractJiraRateLimitingFilter.java:93)
	com.atlassian.ratelimit.core.impl.AbstractRateLimiter.runWithRateLimiting(AbstractRateLimiter.java:60)
	com.atlassian.jira.security.ratelimit.AbstractJiraRateLimitingFilter.doFilterWithRateLimiting(AbstractJiraRateLimitingFilter.java:86)
	com.atlassian.jira.security.ratelimit.AbstractJiraRateLimitingFilter.doFilter(AbstractJiraRateLimitingFilter.java:72)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)
	com.atlassian.jira.util.zipkin.ZipkinTracingFilter.lambda$doFilter$1(ZipkinTracingFilter.java:85)
	com.github.kristofa.brave.servlet.BraveServletFilter.doFilter(BraveServletFilter.java:112)
	com.atlassian.jira.util.zipkin.ZipkinTracingFilter.doFilter(ZipkinTracingFilter.java:73)
	com.atlassian.jira.web.filters.SecurityHttpHeaderRequestFilter.doFilter(SecurityHttpHeaderRequestFilter.java:41)
	com.atlassian.jira.internalstatus.RequestStatusFilter.doFilterOnce(RequestStatusFilter.java:54)
	com.atlassian.jira.web.filters.FilterOnceAbstractHttpFilter.doFilter(FilterOnceAbstractHttpFilter.java:25)
	com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:32)

Note The full stack trace of the root cause is available in the server logs.


Apache Tomcat/9.0.10

 

Regards,

 

Nawaaz

1 answer

1 accepted

0 votes
Answer accepted
Suren Raj
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
September 26, 2018

Hi Nawaaz,

We're experiencing an incident that is affecting a group of our Cloud customers. This issue is high on our radar, and our engineering team is working actively to resolve the issue.

We sincerely apologize for the inconvenience caused due to this issue.

Please access any of our status pages below and click on *Subscribe* to receive updates about our progress on this incident:

Best Regards,
Suren Raj

Jack Brickey
Community Champion
September 26, 2018

But @Nawaaz SOHAWON indicates he is on Server.

Suggest an answer

Log in or Sign up to answer