bitbucket service is starting and crashing immediately

[Dspiliopoulos]

hello,

We recently upgraded our bitbucket server to v5.9.1. Today the installed bitbucket service stopped initializing.

It looked like a java upgrade may have caused the inconsistency. We managed to solve the 

error pointing us to a non-existent jvm.dll path 

The service then would just start and stop. Could you please assist on giving us a lead?

We have considered the following and have not helped or not applied to us.

1. java home environment variable

2. spacing in the environment variable

3. jvm mismatch although there clearly was one on first observation but later corrected

4. we even deleted the existing service and recreated a new one

from the logs we only see the following

StartServiceCtrlDispatcher for 'AtlassianBitbucket' failed
[2018-04-19 09:53:23] [error] [ 320] The service process could not connect to the service controller.
[2018-04-19 09:53:23] [error] [ 320] Commons Daemon procrun failed with exit value: 4 (Failed to run service)
[2018-04-19 09:53:23] [error] [ 320] The service process could not connect to the service controller.
[2018-04-19 09:53:52] [info] [ 564] Commons Daemon procrun (1.0.15.0 64-bit) started

Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-04-19 12:02:38] [info] [ 5332] Running 'BITBUCKET' Service...
[2018-04-19 12:02:38] [info] [ 5864] Starting service...
[2018-04-19 12:02:39] [info] [ 5864] Service started in 1167 ms.
[2018-04-19 12:02:55] [info] [ 5332] Run service finished.
[2018-04-19 12:02:55] [info] [ 5332] Commons Daemon procrun finished

and from window event viewer

The Atlassian Bitbucket BITBUCKET service terminated with the following service-specific error:
Incorrect function.

bitbucket is running on a windows 2012 r2 box (64-bit)

we have installed a 64 bit java

java version "1.8.0_171"
Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)

[Erik Hobel]

Hello,

I had the same problem with my Stash installation. I solved it within the windows registry. I updated the JavaHome and the Jvm Attribute in the service definition

HKEY_LOCAL_MACHINE/SOFTWARE/Apache Software Foundation/Procrun 2.0/AtlassianStash/Parameters/Java

[Sooraj_Ravindran]

Hi Erik,

Thanks.

But I couldn't find any similar entry in my registry for bitbucket.

[Sooraj_Ravindran]

Thanks a lot. it was in different location. This has resolved my issue. Much appreciated.

[Christian Glockner]

Hi Dimitrios,

Are you able to start Bitbucket Server directly, i.e. not as a service? You can try running <BitbucketInstallationDir>\bin\start-bitbucket.bat from a command window. This will help you understand if there's something wrong with the service or Bitbucket Server itself.

Cheers,

Christian

Premier Support Engineer

Atlassian

[Dspiliopoulos]

Morning Christian,

Thanks for your feedback mate. Bitbucket server is actually running through the batch file. It was the only way to get the app running. 

We deleted the existing service from the upgrade , which was working fine a week now. We reconfigured a new service again yesterday with all necessary permissions to the directories but the behavior is the same. The unfortunate is that the errors I provided give us no lead. 

Any ideas? 

[Dspiliopoulos]

Hello Christian,

Hope you are well. Do you have any other ideas? Bitbucket has been working without problems but initialized through the batch. We restarted the host to see if the system anticipated any changes, but the Bitbucket service option still behaves like before. It starts and then stops.

The errors are the ones i listed above. 

from the logs we only see the following

StartServiceCtrlDispatcher for 'AtlassianBitbucket' failed
[2018-04-19 09:53:23] [error] [ 320] The service process could not connect to the service controller.
[2018-04-19 09:53:23] [error] [ 320] Commons Daemon procrun failed with exit value: 4 (Failed to run service)
[2018-04-19 09:53:23] [error] [ 320] The service process could not connect to the service controller.
[2018-04-19 09:53:52] [info] [ 564] Commons Daemon procrun (1.0.15.0 64-bit) started

Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-04-19 12:02:38] [info] [ 5332] Running 'BITBUCKET' Service...
[2018-04-19 12:02:38] [info] [ 5864] Starting service...
[2018-04-19 12:02:39] [info] [ 5864] Service started in 1167 ms.
[2018-04-19 12:02:55] [info] [ 5332] Run service finished.
[2018-04-19 12:02:55] [info] [ 5332] Commons Daemon procrun finished

and from window event viewer

The Atlassian Bitbucket BITBUCKET service terminated with the following service-specific error:
Incorrect function.

[CrownConsult]

Hi,

I have similar issue. Did you find a solution?

[HH Global Czechia, s.r.o.]

Hello,

Unfortunately none of the provided help back then gave us a solid lead. However we did manage to solve it by investigating the bitbucket-stderr logs. (see logs folder within the Atlassian Bitbucket Home folder.  (C:\Atlassian\ApplicationData\Bitbucket\logs)

Our problem turned out to be a permissions issue:

Caused by: java.io.FileNotFoundException: C:\Atlassian\ApplicationData\Bitbucket\shared\bitbucket.properties (Access is denied)

What you could check, is that the account you have assigned to impersonate your service account for the bitbucket service, has access to the file specified above.

We were expecting this to have propagated through when the permissions were set on the folder, but it seems this file did not inherit the permission.

Let me know if that worked for you

Thanks

[CrownConsult]

Thanks for the feedback.

I am issue is like below.

It gives an error that.\bin>start-bitbucket.bat
JRE_HOME "C:\Program Files\Java\jre1.8.0_171" contains spaces
Change to a location without spaces if this causes problems

But the applications start this manual way.

But if we go to windows service it doesnt.

Attached screen shot for reference.Any help would be highly appreciated.

Java_home is set

[CrownConsult]

error Log.

[2018-11-02 01:58:30] [info] [ 3904] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-11-02 01:58:31] [info] [ 3904] Running 'AtlassianBitbucket' Service...
[2018-11-02 01:58:31] [info] [ 3100] Starting service...
[2018-11-02 01:58:31] [error] [ 3100] Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll
[2018-11-02 01:58:31] [error] [ 3100] The system cannot find the path specified.
[2018-11-02 01:58:31] [error] [ 3100] ServiceStart returned 1
[2018-11-02 01:58:31] [error] [ 3100] The system cannot find the path specified.
[2018-11-02 01:58:58] [info] [ 3904] Run service finished.
[2018-11-02 01:58:58] [info] [ 3904] Commons Daemon procrun finished

[HH Global Czechia, s.r.o.]

Hello,

After reviewing your logs, the one mistake i see that could be causing your service stoppage is the spacing in the java home path.

Please make sure that the environmental variable is defined as follows:

C:\Progra~1\Java\jdk1.8.0_51

Also, make sure that you are targeting the correct Java home folder. What tends to happen is that Java auto updates and then points to a new directory. Make sure your jdk is pointing to 1.80.51

Let me know how that goes

[CrownConsult]

I have done changes as advised and its still the same.

Attached few screen shots and logs.

This error shows that its pointing to wrong directory.Where to change that?

"Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll"

[2018-11-07 09:34:38] [info]  [ 9968] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-11-07 09:34:38] [info]  [ 9968] Running 'AtlassianBitbucket' Service...
[2018-11-07 09:34:38] [info]  [12688] Starting service...
[2018-11-07 09:34:38] [error] [12688] Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll
[2018-11-07 09:34:38] [error] [12688] The system cannot find the path specified.
[2018-11-07 09:34:38] [error] [12688] ServiceStart returned 1
[2018-11-07 09:34:38] [error] [12688] The system cannot find the path specified.
[2018-11-07 09:34:38] [info]  [ 9968] Run service finished.
[2018-11-07 09:34:38] [info]  [ 9968] Commons Daemon procrun finished
[2018-11-07 09:35:41] [info]  [ 4076] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-11-07 09:35:42] [info]  [ 4076] Running 'AtlassianBitbucket' Service...
[2018-11-07 09:35:42] [info]  [ 1552] Starting service...
[2018-11-07 09:35:42] [error] [ 1552] Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll
[2018-11-07 09:35:42] [error] [ 1552] The system cannot find the path specified.
[2018-11-07 09:35:42] [error] [ 1552] ServiceStart returned 1
[2018-11-07 09:35:42] [error] [ 1552] The system cannot find the path specified.
[2018-11-07 09:36:00] [info]  [ 4076] Run service finished.
[2018-11-07 09:36:00] [info]  [ 4076] Commons Daemon procrun finished
[2018-11-07 09:38:34] [info]  [ 7680] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-11-07 09:38:35] [info]  [ 7680] Running 'AtlassianBitbucket' Service...
[2018-11-07 09:38:35] [info]  [ 7720] Starting service...
[2018-11-07 09:38:35] [error] [ 7720] Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll
[2018-11-07 09:38:35] [error] [ 7720] The system cannot find the path specified.
[2018-11-07 09:38:35] [error] [ 7720] ServiceStart returned 1
[2018-11-07 09:38:35] [error] [ 7720] The system cannot find the path specified.
[2018-11-07 09:38:35] [info]  [ 7680] Run service finished.
[2018-11-07 09:38:35] [info]  [ 7680] Commons Daemon procrun finished
[2018-11-07 09:39:57] [info]  [ 7852] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-11-07 09:39:57] [info]  [ 7852] Running 'AtlassianBitbucket' Service...
[2018-11-07 09:39:57] [info]  [ 2400] Starting service...
[2018-11-07 09:39:57] [error] [ 2400] Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll
[2018-11-07 09:39:57] [error] [ 2400] The system cannot find the path specified.
[2018-11-07 09:39:57] [error] [ 2400] ServiceStart returned 1
[2018-11-07 09:39:57] [error] [ 2400] The system cannot find the path specified.
[2018-11-07 09:39:57] [info]  [ 7852] Run service finished.
[2018-11-07 09:39:57] [info]  [ 7852] Commons Daemon procrun finished
[2018-11-07 09:40:26] [info]  [12256] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-11-07 09:40:26] [info]  [12256] Running 'AtlassianBitbucket' Service...
[2018-11-07 09:40:26] [info]  [ 7592] Starting service...
[2018-11-07 09:40:26] [error] [ 7592] Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll
[2018-11-07 09:40:26] [error] [ 7592] The system cannot find the path specified.
[2018-11-07 09:40:26] [error] [ 7592] ServiceStart returned 1
[2018-11-07 09:40:26] [error] [ 7592] The system cannot find the path specified.
[2018-11-07 09:40:26] [info]  [12256] Run service finished.
[2018-11-07 09:40:26] [info]  [12256] Commons Daemon procrun finished
[2018-11-07 09:44:18] [info]  [ 9928] Commons Daemon procrun (1.0.15.0 64-bit) started
[2018-11-07 09:44:18] [info]  [ 9928] Running 'AtlassianBitbucket' Service...
[2018-11-07 09:44:18] [info]  [12140] Starting service...
[2018-11-07 09:44:18] [error] [12140] Failed creating java c:\program files\java\jre1.8.0_151\bin\server\jvm.dll
[2018-11-07 09:44:18] [error] [12140] The system cannot find the path specified.
[2018-11-07 09:44:18] [error] [12140] ServiceStart returned 1
[2018-11-07 09:44:18] [error] [12140] The system cannot find the path specified.
[2018-11-07 09:44:18] [info]  [ 9928] Run service finished.
[2018-11-07 09:44:18] [info]  [ 9928] Commons Daemon procrun finished

[HH Global Czechia, s.r.o.]

Hello,

Have you changed it in the systems configured environment variables as advised?

You will find environment variables by right clicking computer, properties, advanced settings. You should have an environment variable listed there for your bitbucket installation i.e.

C:\Progra~1\Java\jdk1.8.0_51 or the latest Java update present on your system.

If the environment variable does not exist, you need to configure one.

Let me know the outcome.

The logs you provided all state a path cannot be found. So either your environment variable is not defined or it has that added space we discussed which needs to be replaced with the

~ or the variable is missing all together or pointing to your previous JRE install

[CrownConsult]

Hello Frantisek,

Good day.

I have tried all possible solutions.if I change the Varialble it doesn't start manually also.

So kind of stuck now and not sure what has been to done.