Elasticsearch errors after installing with standard Windows installer

[Geoff Ekar]

After running the standard windows installing and using it to point the home directory to a DFS network share via a symbolic link, elastic search throws errors of the type listed below. Are these referring to java permissions as configured in java security policy files, or disk/share permissions? If its referring to java permissions, can you provide guidence on the permissions that need granting for Elasticsearch? 

[2018-01-22T12:23:24,136][INFO ][o.e.n.Node ] [bitbucket_bundled] initializing ...
[2018-01-22T12:23:24,215][ERROR][o.e.b.Bootstrap ] Exception
java.security.AccessControlException: access denied ("java.io.FilePermission" "\\rti.local\apps\Bitbucket\shared\search\data\nodes\0" "write")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_102]
at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) ~[?:1.8.0_102]
at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799) ~[?:?]
at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491) ~[?:?]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_102]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_102]
at java.nio.file.Files.createDirectories(Files.java:727) ~[?:1.8.0_102]
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:92) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:226) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:264) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:244) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.5.3.jar:5.5.3]
[2018-01-22T12:23:24,230][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [bitbucket_bundled] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.security.AccessControlException: access denied ("java.io.FilePermission" "\\rti.local\apps\Bitbucket\shared\search\data\nodes\0" "write")
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.5.3.jar:5.5.3]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "\\rti.local\apps\Bitbucket\shared\search\data\nodes\0" "write")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_102]
at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) ~[?:1.8.0_102]
at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799) ~[?:?]
at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491) ~[?:?]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_102]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_102]
at java.nio.file.Files.createDirectories(Files.java:727) ~[?:1.8.0_102]
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:92) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:226) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:264) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:244) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.5.3.jar:5.5.3]
... 6 more
[2018-01-22T12:48:33,332][INFO ][o.e.n.Node ] [bitbucket_bundled] initializing ...
[2018-01-22T12:48:33,457][ERROR][o.e.b.Bootstrap ] Exception
java.security.AccessControlException: access denied ("java.io.FilePermission" "\\rti.local\apps\Bitbucket\shared\search\data\nodes\0" "write")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_102]
at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) ~[?:1.8.0_102]
at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799) ~[?:?]
at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491) ~[?:?]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_102]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_102]
at java.nio.file.Files.createDirectories(Files.java:727) ~[?:1.8.0_102]
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:92) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:226) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:264) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:244) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.main(Command.java:88) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.5.3.jar:5.5.3]
[2018-01-22T12:48:33,472][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [bitbucket_bundled] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.security.AccessControlException: access denied ("java.io.FilePermission" "\\rti.local\apps\Bitbucket\shared\search\data\nodes\0" "write")
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.5.3.jar:5.5.3]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "\\rti.local\apps\Bitbucket\shared\search\data\nodes\0" "write")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_102]
at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_102]
at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) ~[?:1.8.0_102]
at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799) ~[?:?]
at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491) ~[?:?]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_102]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_102]
at java.nio.file.Files.createDirectories(Files.java:727) ~[?:1.8.0_102]
at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:92) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-6.6.0.jar:6.6.0 5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30 07:29:46]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:226) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:264) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.node.Node.<init>(Node.java:244) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) ~[elasticsearch-5.5.3.jar:5.5.3]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.5.3.jar:5.5.3]
... 6 more

[ericblackburn]

Just deployed Bitbucket Server 6.5.1 with a sym link setup.  To leverage a network path, that is linked via a symbolic link, the sym link needs to be on the <Bitucket Home>/shared folder.  Seems that Bitbucket Data Center, natively supports the single network share design, is coded to support this, https://confluence.atlassian.com/bitbucketserver/install-bitbucket-data-center-872139817.html#InstallBitbucketDataCenter-4.Provisionyoursharedfilesystem.  Bitbucket Server is leveraging the same code set, so it works best by obeying Data Center's design contract.  A sym link on any other folder caused issues for us.

On a Windows system, we created the sym link via

mklink /d "<drive letter>:\Atlassian\ApplicationData\Bitbucket\shared"  "\\<network share path>\shared"

One more addition is needed to get Elasticsearch working.  Elasticsearch dislikes the sym link jump and will complain about permission issues.  To fix this, we updated the java.policy file at JAVA_HOME/jre/lib/security/java.policy.  Note, this JAVA_HOME is pointing to a JDK install.

The following needs to be appended to the grant{<end of this section>}; section. The following is a Windows path example.  The "-" is a recursive flag notation for the java.policy.

//Custom Policy for Bitbucket in order to support permissions needed for it to interact with the symbolic link to theshare
permission java.io.FilePermission "<Drive Letter>:\\Atlassian\\ApplicationData\\Bitbucket\\shared\\-", "read,write,delete,execute";
permission java.io.FilePermission "\\\\<network share path>\\shared\\-", "read,write,delete,execute";

Note, I don't know if the delete/execute permissions are needed.  I ran out of time on the project, and made the assumption that they would be.

[Ana Retamal]

Hi Geoff!

Looks like you might not have permission to create a file on that network share. Can you try and create a file in that directory from within Windows Explorer or the command line?

For more info regarding the Elasticsearch, you can read Install and configure a remote Elasticsearch instance.

You can also find some useful information in this thread.

Hope that helps!

Ana

[Geoff Ekar]

Thanks for the response. It didn't turn out to be file permissions. It turned out to be Java permissions. Permissions needed to be granted in the java security policy file. It seems that whatever process creates the java permissions during the installer doesn't do well with symbolic links and network shares. 

Its working now after amending the java policy to include the network location. 

[Ana Retamal]

Hi Geoff, thanks for letting us know, glad you figured it out!

Let us know if you need anything else.

Cheers :)

Ana

[Joe Bednarz]

I seem to be running into the same issue, and was wondering how you updated your policy file...

Thanks!