Elastic search indexing fails with: Failed to recreate index

[steven qent]

Bitbucket fails to create index on elastic search. We're using elastic search 7.1.1.

We're aware that atlassian does not recommend elastic search above 6.5.3, but we have a hard limit on our elastic search version of 7.0. 

We see that Bitbucket starts and then fails with

2019-06-29 06:14:06,908 ERROR [Caesium-1-2] c.a.b.i.s.i.IndexingSynchronizationService Failed to create indexes: ''[illegal_argument_exception, illegal_argument_exception, illegal_argument_exception, il$
2019-06-29 06:14:06,908 ERROR [Caesium-1-2] c.a.b.i.s.i.IndexingSynchronizationService Unable to synchronize the mapping in Elasticsearch

com.atlassian.bitbucket.internal.search.indexing.exceptions.IndexException: Failed to recreate index: [illegal_argument_exception, illegal_argument_exception, illegal_argument_exception, illegal_argument$
at com.atlassian.bitbucket.internal.search.indexing.upgrade.UpgradeTaskV3Point0.execute(UpgradeTaskV3Point0.java:52)
at com.atlassian.bitbucket.internal.search.indexing.upgrade.DefaultUpgradeService.upgrade(DefaultUpgradeService.java:45)
at com.atlassian.bitbucket.internal.search.indexing.IndexingSynchronizationService.synchronizeMapping(IndexingSynchronizationService.java:112)
at com.atlassian.bitbucket.internal.search.indexing.IndexingSynchronizationService.synchronizeStores(IndexingSynchronizationService.java:84)
at com.atlassian.bitbucket.internal.search.indexing.jobs.StartupChecksJob.run(StartupChecksJob.java:80)
at com.atlassian.bitbucket.internal.search.common.cluster.ClusterJobRunner.runJob(ClusterJobRunner.java:81)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
..

..

We're using Bitbucket version : 6.3.0

[Craig Castle-Mead]

Hi @Chris lerner 

6.9 doesn’t show as a supported ES version yet either. I’ve lodged a premier support ticket to see when newer ES version support will land. Will update when I hear back.

CCM

[Chris lerner]

Was there any resolution to this issue? Having the exact same error show up for Bitbucket 6.9.1 and ElasticSearch 7.5.1

[steven qent]

Thank you Craig. I'm certain that between ES6 and ES7 the index isn't compatible. Which is why we're trying to recreate the entire index. We have no permission related issues. 

[Craig Castle-Mead]

Hey @steven qent 

There are quite a few documented breaking changes In Elasticsearch 7 (https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#breaking_70_indices_changes) that may be to blame (assuming you’ve validated all your permissions are correct). 

If anyone else has managed to get BB working with ES7, would be interested to know. 

CCM