[CASSANDRA-16027] bin/cassandra may fail to execute in certain environments due to unhandled output pipe

• Type: Bug
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Component/s: Packaging
• Labels: lhf
• Bug Category: Packaging - All
• Severity: Low
• Complexity: Low Hanging Fruit
• Discovered By: User Report
• Platform: All
• Assignee: Unassigned
• Reporter: Christopher Bradford
• Created: 05/Aug/20 03:09
• Updated: 21/Mar/21 23:04

Description

While developing Docker images for C* 3.11.6 I noticed bin/cassandra failed to start when executed with nohup in a container based Red Hat Universal Base Image on Kubernetes. Curiously this issue could not be reproduced when simply running the container on Docker. Instead it had to be scheduled within a k8s pod.

 

Running an strace indicates the following:

[pid 3666026] write(2, "OpenJDK 64-Bit Server VM warning: ", 34) = -1 EPIPE (Broken pipe) 

 

The issue occurs before the system.log file is created. After further digging in to the command being run it was determined the bin/cassandra file was not redirecting all output pipes. In this particular environment this leads to the process going defunct and no Cassandra running.

 

The 4 lines starting with

exec $NUMACTL 

are not handling one of the outputs (stderr or stdout). As a workaround suffixing each line with

> /var/log/cassandra/stdout.log 2> /var/log/cassandra/stderr.log

resolves the issue and Cassandra starts without any issues.

 

Rather than bake this into our containers the fix should make its way upstream.