Details
Blocker Description
I wish I could provide more information. This has happened once on a bare metal centos cluster while running vanilla continuous ingest of svn#1470047. There was nothing reported in the logs when one of the tservers just died after the system had been up for ~1 day. The out and err files were sparse, and the master only reported that it had lost connection with the tserver at the point when the tserver just stopped logging (it was overnight, so this was not witnessed until morning).
It recently happened again on a standalone instance on ec2 running ubuntu and svn#1477382. The instance had been running for ~7 hours. This time the gc, master, and tserver died. The gc died first, and then 2m:48s later the master died. 200ms later the tserver died. Again, there was no output in any of the out or err files for the processes. The logs also have no errors or warnings in them, just abrupt stops. The processes came up fine once restarted.