Getting this error SpringMVC dispatcher [springMvc] could not be started
We have bitbucket hosted on our aws lightsail instance.
The instance has 8Gb ram, 2vCPUs, 160GB SSD.
We recently had to reboot our server as it was hanging. Jira sever is also installed on the same instance.
After the instance rebooted, JIRA worked fine but Bitbucket gives us Fatal error as you can see in the image attached below.
Restarted the server at around 9am
I have also attached the log files below
Help is much appreciated
Thank You
*****************************************************************************
Log files :
2019-01-14 08:22:28,135 INFO [pool-875-thread-1] c.a.b.s.s.t.DefaultElasticsearchConnectionTester Testing connection with Elasticsearch failed due to exception java.net.ConnectException: Connection refused: no further information
2019-01-14 08:22:28,136 ERROR [Caesium-1-1] c.a.b.i.s.c.s.p.AutomaticAuthenticationProvisioner Skipping automatic auth configuration: Elasticsearch instance is not available for connection.
2019-01-14 08:22:28,136 INFO [Caesium-1-1] c.a.b.i.s.i.jobs.StartupChecksJob Running startup jobs for search
2019-01-14 08:22:29,138 ERROR [Caesium-1-1] c.a.b.i.s.i.IndexingSynchronizationService Unable to establish a connection to Elasticsearch during index synchronisation.
2019-01-14 08:22:29,139 WARN [Caesium-1-1] c.a.b.i.s.c.cluster.ClusterJobRunner Job execution failed. Rescheduling. Job key: com.atlassian.bitbucket.internal.search.indexing.jobs.StartupChecksJob Job id: a80eba36-0d6c-4385-969a-edfdc7cf0186
2019-01-14 08:32:29,140 INFO [Caesium-1-1] c.a.b.i.s.i.jobs.StartupChecksJob Attempting provisioning in startup job
2019-01-14 08:32:30,145 INFO [pool-876-thread-1] c.a.b.s.s.t.DefaultElasticsearchConnectionTester Testing connection with Elasticsearch failed due to exception java.net.ConnectException: Connection refused: no further information
2019-01-14 08:32:30,147 ERROR [Caesium-1-1] c.a.b.i.s.c.s.p.AutomaticAuthenticationProvisioner Skipping automatic auth configuration: Elasticsearch instance is not available for connection.
2019-01-14 08:32:30,147 INFO [Caesium-1-1] c.a.b.i.s.i.jobs.StartupChecksJob Running startup jobs for search
2019-01-14 08:32:31,151 ERROR [Caesium-1-1] c.a.b.i.s.i.IndexingSynchronizationService Unable to establish a connection to Elasticsearch during index synchronisation.
2019-01-14 08:32:31,151 WARN [Caesium-1-1] c.a.b.i.s.c.cluster.ClusterJobRunner Job execution failed. Rescheduling. Job key: com.atlassian.bitbucket.internal.search.indexing.jobs.StartupChecksJob Job id: 35bea30b-8fe8-42ea-b29e-ddd88019a923
2019-01-14 08:42:31,152 INFO [Caesium-1-3] c.a.b.i.s.i.jobs.StartupChecksJob Attempting provisioning in startup job
2019-01-14 08:42:32,157 INFO [pool-877-thread-1] c.a.b.s.s.t.DefaultElasticsearchConnectionTester Testing connection with Elasticsearch failed due to exception java.net.ConnectException: Connection refused: no further information
2019-01-14 08:42:32,158 ERROR [Caesium-1-3] c.a.b.i.s.c.s.p.AutomaticAuthenticationProvisioner Skipping automatic auth configuration: Elasticsearch instance is not available for connection.
2019-01-14 08:42:32,158 INFO [Caesium-1-3] c.a.b.i.s.i.jobs.StartupChecksJob Running startup jobs for search
2019-01-14 08:42:33,161 ERROR [Caesium-1-3] c.a.b.i.s.i.IndexingSynchronizationService Unable to establish a connection to Elasticsearch during index synchronisation.
2019-01-14 08:42:33,161 WARN [Caesium-1-3] c.a.b.i.s.c.cluster.ClusterJobRunner Job execution failed. Rescheduling. Job key: com.atlassian.bitbucket.internal.search.indexing.jobs.StartupChecksJob Job id: 449605c6-0f8f-4a73-8490-0137e93cd882
2019-01-14 08:52:33,163 INFO [Caesium-1-4] c.a.b.i.s.i.jobs.StartupChecksJob Attempting provisioning in startup job
2019-01-14 08:52:34,170 INFO [pool-878-thread-1] c.a.b.s.s.t.DefaultElasticsearchConnectionTester Testing connection with Elasticsearch failed due to exception java.net.ConnectException: Connection refused: no further information
2019-01-14 08:52:34,171 ERROR [Caesium-1-4] c.a.b.i.s.c.s.p.AutomaticAuthenticationProvisioner Skipping automatic auth configuration: Elasticsearch instance is not available for connection.
2019-01-14 08:52:34,171 INFO [Caesium-1-4] c.a.b.i.s.i.jobs.StartupChecksJob Running startup jobs for search
2019-01-14 08:52:35,174 ERROR [Caesium-1-4] c.a.b.i.s.i.IndexingSynchronizationService Unable to establish a connection to Elasticsearch during index synchronisation.
2019-01-14 08:52:35,175 WARN [Caesium-1-4] c.a.b.i.s.c.cluster.ClusterJobRunner Job execution failed. Rescheduling. Job key: com.atlassian.bitbucket.internal.search.indexing.jobs.StartupChecksJob Job id: 408a011f-6e8f-420e-9b21-a463be68a8ee
2019-01-14 09:02:35,176 INFO [Caesium-1-3] c.a.b.i.s.i.jobs.StartupChecksJob Attempting provisioning in startup job
2019-01-14 09:02:36,181 INFO [pool-879-thread-1] c.a.b.s.s.t.DefaultElasticsearchConnectionTester Testing connection with Elasticsearch failed due to exception java.net.ConnectException: Connection refused: no further information
2019-01-14 09:02:36,183 ERROR [Caesium-1-3] c.a.b.i.s.c.s.p.AutomaticAuthenticationProvisioner Skipping automatic auth configuration: Elasticsearch instance is not available for connection.
2019-01-14 09:02:36,183 INFO [Caesium-1-3] c.a.b.i.s.i.jobs.StartupChecksJob Running startup jobs for search
2019-01-14 09:02:37,186 ERROR [Caesium-1-3] c.a.b.i.s.i.IndexingSynchronizationService Unable to establish a connection to Elasticsearch during index synchronisation.
2019-01-14 09:02:37,187 WARN [Caesium-1-3] c.a.b.i.s.c.cluster.ClusterJobRunner Job execution failed. Rescheduling. Job key: com.atlassian.bitbucket.internal.search.indexing.jobs.StartupChecksJob Job id: 89a73226-930d-4473-b7a5-facee922944e
2019-01-14 09:09:44,426 INFO [main] o.s.boot.SpringApplication
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.5.15.RELEASE)
2019-01-14 09:09:45,210 INFO [main] c.a.b.i.b.BitbucketServerApplication Starting BitbucketServerApplication on QLIK-SENSE with PID 2996 (C:\Atlassian\Bitbucket\5.15.1\app\WEB-INF\classes started by atlbitbucket in C:\Atlassian\Bitbucket\5.15.1\app)
2019-01-14 09:09:45,210 INFO [main] c.a.b.i.b.BitbucketServerApplication No active profile set, falling back to default profiles: default
2019-01-14 09:09:48,506 INFO [main] c.a.b.i.boot.log.BuildInfoLogger Starting Bitbucket 5.15.1 (93098cb built on Wed Nov 14 08:02:41 IST 2018)
2019-01-14 09:09:48,506 INFO [main] c.a.b.i.boot.log.BuildInfoLogger JVM: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_172-b11
2019-01-14 09:09:53,253 INFO [main] c.a.b.i.b.BitbucketServerApplication Started BitbucketServerApplication in 10.673 seconds (JVM running for 18.913)
2019-01-14 09:10:01,017 INFO [spring-startup] c.a.s.internal.home.HomeLockAcquirer Successfully acquired lock on home directory C:\Atlassian\ApplicationData\Bitbucket
2019-01-14 09:10:03,083 INFO [Thread-5] c.a.b.i.boot.log.BuildInfoLogger Bitbucket 5.15.1 has shut down
2019-01-14 09:10:04,536 ERROR [spring-startup] com.zaxxer.hikari.pool.HikariPool bitbucket - Exception during pool initialization.
org.h2.jdbc.JdbcSQLException: Error opening database: "Sleep interrupted" [8000-176]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:344)
at org.h2.message.DbException.get(DbException.java:167)
at org.h2.store.FileLock.getExceptionFatal(FileLock.java:457)
at org.h2.store.FileLock.sleep(FileLock.java:452)
at org.h2.store.FileLock.lockFile(FileLock.java:354)
at org.h2.store.FileLock.lock(FileLock.java:136)
at org.h2.engine.Database.open(Database.java:622)
at org.h2.engine.Database.openDatabase(Database.java:260)
at org.h2.engine.Database.<init>(Database.java:254)
at org.h2.engine.Engine.openSession(Engine.java:57)
at org.h2.engine.Engine.openSession(Engine.java:164)
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:142)
at org.h2.engine.Engine.createSession(Engine.java:125)
at org.h2.engine.Engine.createSession(Engine.java:27)
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:331)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:107)
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:91)
at org.h2.Driver.connect(Driver.java:74)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:117)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:123)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:375)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:204)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:445)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:516)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:116)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:72)
at com.atlassian.stash.internal.hikari.ExtendedHikariDataSource.<init>(ExtendedHikariDataSource.java:49)
at com.atlassian.stash.internal.hikari.HikariDataSourceFactory.create(HikariDataSourceFactory.java:70)
at org.springframework.context.support.PostProcessorRegistrationDelegate.registerBeanPostProcessors(PostProcessorRegistrationDelegate.java:225)
at org.springframework.context.support.AbstractApplicationContext.registerBeanPostProcessors(AbstractApplicationContext.java:703)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:528)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at java.lang.Thread.run(Thread.java:748)
... 92 frames trimmed
Caused by: java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at org.h2.store.FileLock.sleep(FileLock.java:450)
... 30 common frames omitted
2019-01-14 09:10:58,978 INFO [main] o.s.boot.SpringApplication
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v1.5.15.RELEASE)
2019-01-14 09:11:00,105 INFO [main] c.a.b.i.b.BitbucketServerApplication Starting BitbucketServerApplication on QLIK-SENSE with PID 3032 (C:\Atlassian\Bitbucket\5.15.1\app\WEB-INF\classes started by atlbitbucket in C:\Atlassian\Bitbucket\5.15.1\app)
2019-01-14 09:11:00,106 INFO [main] c.a.b.i.b.BitbucketServerApplication No active profile set, falling back to default profiles: default
2019-01-14 09:11:05,267 INFO [main] c.a.b.i.boot.log.BuildInfoLogger Starting Bitbucket 5.15.1 (93098cb built on Wed Nov 14 08:02:41 IST 2018)
2019-01-14 09:11:05,267 INFO [main] c.a.b.i.boot.log.BuildInfoLogger JVM: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_172-b11
2019-01-14 09:11:17,014 INFO [main] c.a.b.i.b.BitbucketServerApplication Started BitbucketServerApplication in 22.502 seconds (JVM running for 32.316)
2019-01-14 09:11:24,370 INFO [spring-startup] c.a.s.internal.home.HomeLockAcquirer Successfully acquired lock on home directory C:\Atlassian\ApplicationData\Bitbucket
2019-01-14 09:11:38,280 WARN [http-nio-7990-exec-10] c.a.j.s.w.c.s.JohnsonHttpRequestHandlerServlet HttpRequestHandlerServlet [scmServlet] cannot be initialised to service an incoming request; redirecting to /unavailable
2019-01-14 09:14:17,480 WARN [http-nio-7990-exec-9] c.a.j.s.w.c.s.JohnsonHttpRequestHandlerServlet HttpRequestHandlerServlet [streamsEmptyActivityServlet] cannot be initialised to service an incoming request; redirecting to /unavailable
2019-01-14 09:14:47,173 WARN [http-nio-7990-exec-8] c.a.j.s.w.c.s.JohnsonHttpRequestHandlerServlet HttpRequestHandlerServlet [streamsEmptyActivityServlet] cannot be initialised to service an incoming request; redirecting to /unavailable
2019-01-14 09:15:38,171 WARN [http-nio-7990-exec-8] c.a.j.s.w.c.s.JohnsonHttpRequestHandlerServlet HttpRequestHandlerServlet [streamsEmptyActivityServlet] cannot be initialised to service an incoming request; redirecting to /unavailable
2019-01-14 09:27:37,688 WARN [http-nio-7990-exec-1] c.a.j.s.w.c.s.JohnsonHttpRequestHandlerServlet HttpRequestHandlerServlet [streamsEmptyActivityServlet] cannot be initialised to service an incoming request; redirecting to /unavailable
2019-01-14 09:34:36,623 WARN [http-nio-7990-exec-8] c.a.j.s.w.c.s.JohnsonHttpRequestHandlerServlet HttpRequestHandlerServlet [scmServlet] cannot be initialised to service an incoming request; redirecting to /unavailable
2019-01-14 09:41:37,077 WARN [spring-startup] o.s.w.c.s.XmlWebApplicationContext Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'crowdAliasDao': Unsatisfied dependency expressed through method 'setSessionFactory' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sharedHomeLockAcquirer' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'licenseHelper' while setting constructor argument; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'licenseHelper' defined in URL [jar:file:/C:/Atlassian/Bitbucket/5.15.1/app/WEB-INF/lib/bitbucket-service-impl-5.15.1.jar!/com/atlassian/stash/internal/license/DefaultLicenseHelper.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'bootstrapLiquibase' defined in class path resource [stash-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by QLIK-SENSE (172.26.11.214) since 1/14/19 9:10 AM
2019-01-14 09:41:37,245 INFO [spring-startup] c.a.s.internal.home.HomeLockAcquirer Releasing lock on C:\Atlassian\ApplicationData\Bitbucket
2019-01-14 09:41:37,252 ERROR [spring-startup] c.a.j.s.w.s.JohnsonDispatcherServlet SpringMVC dispatcher [springMvc] could not be started
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'crowdAliasDao': Unsatisfied dependency expressed through method 'setSessionFactory' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sharedHomeLockAcquirer' defined in class path resource [stash-context.xml]: Cannot resolve reference to bean 'licenseHelper' while setting constructor argument; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'licenseHelper' defined in URL [jar:file:/C:/Atlassian/Bitbucket/5.15.1/app/WEB-INF/lib/bitbucket-service-impl-5.15.1.jar!/com/atlassian/stash/internal/license/DefaultLicenseHelper.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'bootstrapLiquibase' defined in class path resource [stash-context.xml]: Invocation of init method failed; nested exception is liquibase.exception.LockException: Could not acquire change log lock. Currently locked by QLIK-SENSE (172.26.11.214) since 1/14/19 9:10 AM
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:667)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
at org.springframework.context.supp.a.j.s.w.f.BypassableDelegatingFilterProxy Bypassing [authenticationAttributeFilter]; no Spring WebApplicationContext is available
