[SOLR-17216] Solr Operator probe logging should not leak basic auth credentials

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Authentication
• Labels: kubernetes security
• Assignee: Jason Gerlowski
• Reporter: Jason Gerlowski
• Created: 27/Mar/24 13:54
• Updated: 28/Apr/24 19:29
• Resolved: 18/Apr/24 15:12

Description

Credit for this report goes to "Flip Hess" from the mailing list

Solr Operator 0.8.0 uses "command-based" liveness and readiness probes in some circumstances. When auth is enabled and required on the probe endpoints, these command-based probes rely on the special JAVA_TOOL_OPTIONS env-var to tunnel the requisite settings and authentication credentials down through the bin/solr api invocation. In these cases the command run by the probe looks like:

JAVA_TOOL_OPTIONS="-Dbasicauth=$(cat /etc/secrets/sc-solrbackup-solrcloud-basic-auth/username):$(cat /etc/secrets/sc-solrbackup-solrcloud-basic-auth/password) -Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory" solr api -get "http://${SOLR_HOST}:8983/solr/admin/info/system"

When the JVM starts up it logs the value of JAVA_TOOL_OPTIONS (which contains the basic auth creds) to stdout. Usually this stdout is swallowed, but if the probe is unsuccessful the stdout ends up in a Kubernetes "event" (visible often in kubectl describe pod <podName>).

We should:
1. Change how these command-probes work, so that the stdout never includes auth credentials.
2. Create a CVE for the vulnerability.
3. Release the operator fix in a new release.