Add 2nd Node to Confluence Datacenter cluster?

[Robert Taylor]

I just setup the confluence datacenter on one server, its working fine. Now I need to add 2nd node to cluster. I have looked at all the steps on your website and it makes no sense to me. Do I need to install the application on the 2nd node 1st and then copy the install and local home directory from 1st node afterwards or do I just copy the folders from 1st node and some how start up the applications on 2nd node. Your instructions seems to miss a lot of steps, sorry if I should be aware, this is my 1st time working confluence.   This is all running on windows 2016 servers.  Also where are the windows instructions to install Synchrony in datacenter cluster?  I will install syncrony on same nodes as confluence.  I opened a ticket with attlasin, but i'm not sure support actually exist...


here are the steps that make no sense.
Add more Confluence nodes
4. Copy Confluence to second node
To copy Confluence to the second node:



Shut down Confluence on node 1
Shut down your application server on node 2, or stop it automatically loading web applications


Copy the installation directory from node 1 to node 2
Copy the local home directory from node 1 to node 2
If the file path of the local home directory is not the same on nodes 1 and 2 you'll need to update the <installation directory>/confluence/WEB-INF/classes/confluence-init.properties file on node 2 to point to the correct location.
Copying the local home directory ensures the Confluence search index, the database and cluster configuration, and any other settings are copied to node 2.

[AnnWorley]

Hi Robert,

I noticed you have a support request open where some detailed steps are being discussed for your Data Center evaluation. I added myself as a watcher on the ticket so we can circle back to this thread to share the resolution with the Community.

Thanks,

Ann

[Robert Taylor]

Hello Ann,

i'm getting a bit frustrated here.  So, i copied over the program folder and data folder to 2nd node.  I started the confluence batch file on 1st node, website is up and running.  I now started the confluence batch file one 2nd node, but it not loading and stopped at certain point.  I have copied install info.  I see errors about no matching network interface.  The only thing I can think of causing issue, is i'm using a domain based service account on 2nd node, simply because the SHARED directory is on network file server and the local confluence server system account can't write to it.  THis is taking me too much time to simply add a 2nd node.   Any ideas based on logs below?

thanks

03-Aug-2018 09:50:07.755 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server} Setting property 'debug' to '0' did not find a matching property.
03-Aug-2018 09:50:07.848 WARNING [main] org.apache.catalina.startup.SetAllPropertiesRule.begin [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'debug' to '0' did not find a matching property.
03-Aug-2018 09:50:07.973 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine} Setting property 'debug' to '0' did not find a matching property.
03-Aug-2018 09:50:07.973 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'debug' to '0' did not find a matching property.
03-Aug-2018 09:50:08.020 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property.
03-Aug-2018 09:50:08.051 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property.
03-Aug-2018 09:50:08.442 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8090"]
03-Aug-2018 09:50:08.520 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
03-Aug-2018 09:50:08.536 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 891 ms
03-Aug-2018 09:50:08.551 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Tomcat-Standalone]
03-Aug-2018 09:50:08.551 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/9.0.8
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
03-Aug-2018 09:50:12.442 INFO [localhost-startStop-2] org.apache.catalina.core.ApplicationContext.log Spring WebApplicationInitializers detected on classpath: [com.atlassian.synchrony.proxy.SynchronyDispatcherServletInitializer@219186b8]
2018-08-03 09:50:12,989 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] info Starting Confluence 6.10.1 [build 7702 based on commit hash c32f84c806260a834c2e109e904a2d8ee4f8e175] - synchrony version 2.1.0-master-ed94c233
03-Aug-2018 09:50:13.145 INFO [localhost-startStop-2] org.apache.catalina.core.ApplicationContext.log Initializing Spring FrameworkServlet 'dispatcher'
03-Aug-2018 09:50:13.145 INFO [localhost-startStop-2] org.springframework.web.servlet.DispatcherServlet.initServletBean FrameworkServlet 'dispatcher': initialization started
03-Aug-2018 09:50:13.161 INFO [localhost-startStop-2] org.springframework.web.context.support.AnnotationConfigWebApplicationContext.prepareRefresh Refreshing WebApplicationContext for namespace 'dispatcher-servlet': startup date [Fri Aug 03 09:50:13 EDT 2018]; root of context hierarchy
03-Aug-2018 09:50:13.333 INFO [localhost-startStop-2] org.springframework.web.context.support.AnnotationConfigWebApplicationContext.loadBeanDefinitions Registering annotated classes: [class com.atlassian.synchrony.proxy.websocket.WebSocketConfig,class com.atlassian.synchrony.proxy.web.SynchronyWebMvcConfig]
03-Aug-2018 09:50:14.396 INFO [localhost-startStop-2] org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler.initialize Initializing ExecutorService 'defaultSockJsTaskScheduler'
03-Aug-2018 09:50:14.505 INFO [localhost-startStop-2] org.springframework.web.socket.server.support.WebSocketHandlerMapping.registerHandler Mapped URL path [/v1/bayeux-sync1] onto handler of type [class org.springframework.web.socket.server.support.WebSocketHttpRequestHandler]
03-Aug-2018 09:50:14.880 INFO [localhost-startStop-2] org.springframework.web.servlet.handler.SimpleUrlHandlerMapping.registerHandler Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.DefaultServletHttpRequestHandler]
03-Aug-2018 09:50:15.630 INFO [localhost-startStop-2] org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.initControllerAdviceCache Looking for @ControllerAdvice: WebApplicationContext for namespace 'dispatcher-servlet': startup date [Fri Aug 03 09:50:13 EDT 2018]; root of context hierarchy
03-Aug-2018 09:50:15.880 INFO [localhost-startStop-2] org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping.register Mapped "{[/reload],methods=[PUT]}" onto public org.springframework.http.ResponseEntity com.atlassian.synchrony.proxy.web.SynchronyProxyRestController.reloadConfiguration(com.atlassian.synchrony.proxy.web.SynchronyProxyConfigPayload)
03-Aug-2018 09:50:15.880 INFO [localhost-startStop-2] org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping.register Mapped "{[/healthcheck]}" onto public com.atlassian.synchrony.proxy.web.HealthCheckResult com.atlassian.synchrony.proxy.web.SynchronyProxyRestController.getSynchronyProxyInfo()
03-Aug-2018 09:50:16.021 INFO [localhost-startStop-2] org.springframework.context.support.DefaultLifecycleProcessor.start Starting beans in phase 2147483647
03-Aug-2018 09:50:16.083 INFO [localhost-startStop-2] org.springframework.web.servlet.DispatcherServlet.initServletBean FrameworkServlet 'dispatcher': initialization completed in 2938 ms
2018-08-03 09:50:20,333 INFO [localhost-startStop-1] [atlassian.confluence.cluster.DefaultClusterConfigurationHelper] lambda$populateExistingClusterSetupConfig$2 Populating setup configuration if running with Cluster mode...
2018-08-03 09:50:20,786 INFO [localhost-startStop-1] [confluence.cluster.hazelcast.HazelcastClusterManager] configure Configuring Hazelcast with instanceName [confluence], join configuration TCP/IP member addresses: 172.18.25.169|172.18.25.103, network interfaces [] and local port 5801
2018-08-03 09:50:20,786 INFO [localhost-startStop-1] [confluence.cluster.hazelcast.HazelcastClusterManager] startCluster Starting the cluster.
2018-08-03 09:50:20,943 ERROR [localhost-startStop-1] [com.hazelcast.instance.AddressPicker] log [LOCAL] [dev] [3.8.6] Hazelcast CANNOT start on this node. No matching network interface found.
Interface matching must be either disabled or updated in the hazelcast.xml config file.
2018-08-03 09:50:20,958 ERROR [localhost-startStop-1] [com.hazelcast.instance.AddressPicker] log [LOCAL] [dev] [3.8.6] Hazelcast CANNOT start on this node. No matching network interface found.
Interface matching must be either disabled or updated in the hazelcast.xml config file.
java.lang.RuntimeException: Hazelcast CANNOT start on this node. No matching network interface found.
Interface matching must be either disabled or updated in the hazelcast.xml config file.
at com.hazelcast.instance.DefaultAddressPicker.pickInterfaceAddress(DefaultAddressPicker.java:199)
at com.hazelcast.instance.DefaultAddressPicker.pickAddress(DefaultAddressPicker.java:168)
at com.hazelcast.instance.DefaultAddressPicker.getPublicAddressByPortSearch(DefaultAddressPicker.java:108)
at com.hazelcast.instance.DefaultAddressPicker.pickAddress(DefaultAddressPicker.java:78)
at com.hazelcast.instance.Node.<init>(Node.java:183)
at com.hazelcast.instance.HazelcastInstanceImpl.createNode(HazelcastInstanceImpl.java:159)
at com.hazelcast.instance.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:127)
at com.hazelcast.instance.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:218)
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:176)
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:126)
at com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:58)
at com.atlassian.confluence.cluster.hazelcast.HazelcastClusterManager.startCluster(HazelcastClusterManager.java:317)
at com.atlassian.confluence.cluster.hazelcast.HazelcastClusterManager.reconfigure(HazelcastClusterManager.java:289)
at com.atlassian.confluence.cluster.DefaultClusterConfigurationHelper.bootstrapCluster(DefaultClusterConfigurationHelper.java:398)
at com.atlassian.confluence.setup.DefaultBootstrapManager.afterConfigurationLoaded(DefaultBootstrapManager.java:847)
at com.atlassian.config.bootstrap.DefaultAtlassianBootstrapManager.init(DefaultAtlassianBootstrapManager.java:75)
at com.atlassian.confluence.setup.DefaultBootstrapManager.init(DefaultBootstrapManager.java:181)
at com.atlassian.config.util.BootstrapUtils.init(BootstrapUtils.java:36)
at com.atlassian.confluence.setup.ConfluenceConfigurationListener.initialiseBootstrapContext(ConfluenceConfigurationListener.java:133)
at com.atlassian.confluence.setup.ConfluenceConfigurationListener.contextInitialized(ConfluenceConfigurationListener.java:64)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4627)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5091)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1427)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1417)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2018-08-03 09:50:20,990 ERROR [localhost-startStop-1] [atlassian.confluence.setup.ConfluenceConfigurationListener] error An error was encountered while bootstrapping Confluence (see below):
Hazelcast CANNOT start on this node. No matching network interface found.
Interface matching must be either disabled or updated in the hazelcast.xml config file.
java.lang.RuntimeException: Hazelcast CANNOT start on this node. No matching network interface found.
Interface matching must be either disabled or updated in the hazelcast.xml config file.
at com.hazelcast.instance.DefaultAddressPicker.pickInterfaceAddress(DefaultAddressPicker.java:199)
at com.hazelcast.instance.DefaultAddressPicker.pickAddress(DefaultAddressPicker.java:168)
at com.hazelcast.instance.DefaultAddressPicker.getPublicAddressByPortSearch(DefaultAddressPicker.java:108)
at com.hazelcast.instance.DefaultAddressPicker.pickAddress(DefaultAddressPicker.java:78)
at com.hazelcast.instance.Node.<init>(Node.java:183)
at com.hazelcast.instance.HazelcastInstanceImpl.createNode(HazelcastInstanceImpl.java:159)
at com.hazelcast.instance.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:127)
at com.hazelcast.instance.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:218)
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:176)
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:126)
at com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:58)
at com.atlassian.confluence.cluster.hazelcast.HazelcastClusterManager.startCluster(HazelcastClusterManager.java:317)
at com.atlassian.confluence.cluster.hazelcast.HazelcastClusterManager.reconfigure(HazelcastClusterManager.java:289)
at com.atlassian.confluence.cluster.DefaultClusterConfigurationHelper.bootstrapCluster(DefaultClusterConfigurationHelper.java:398)
at com.atlassian.confluence.setup.DefaultBootstrapManager.afterConfigurationLoaded(DefaultBootstrapManager.java:847)
at com.atlassian.config.bootstrap.DefaultAtlassianBootstrapManager.init(DefaultAtlassianBootstrapManager.java:75)
at com.atlassian.confluence.setup.DefaultBootstrapManager.init(DefaultBootstrapManager.java:181)
at com.atlassian.config.util.BootstrapUtils.init(BootstrapUtils.java:36)
at com.atlassian.confluence.setup.ConfluenceConfigurationListener.initialiseBootstrapContext(ConfluenceConfigurationListener.java:133)
at com.atlassian.confluence.setup.ConfluenceConfigurationListener.contextInitialized(ConfluenceConfigurationListener.java:64)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4627)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5091)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1427)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1417)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2018-08-03 09:50:21,021 WARN [localhost-startStop-1] [atlassian.confluence.plugin.PluginFrameworkContextListener] contextInitialized Not starting full plugin system due to upgrade
2018-08-03 09:50:22,271 INFO [localhost-startStop-1] [com.atlassian.confluence.lifecycle] init Confluence is ready to serve
03-Aug-2018 09:50:22.318 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8090"]
03-Aug-2018 09:50:22.412 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 13878 ms

[AnnWorley]

Hi Robert,

I understand you are frustrated with the time it's taking to add a second node to your Data Center instance. I checked out your support case and it looks like the new information posted here was not shared there so I had a look. 

Researching the error you posted, I found this knowledge base article: Hazelcast CANNOT start on this node. No matching network interface found.

I look forward to hearing whether the article helps in your case.

Please attach the latest logs to your support ticket so the support engineers can investigate more thoroughly.

Thanks,

Ann