[SOLR-15630] MDC "prefix labels" should be handled by Layout - not hardcoded in solr code

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.0
• Assignee: Chris M. Hostetter
• Reporter: Chris M. Hostetter
• Created: 14/Sep/21 17:37
• Updated: 12/May/22 00:26
• Resolved: 16/Sep/21 22:45

Description

Prior to Solr 9, Solr code included hardcoded, single character, prefixes on all MDC values passed to the logging framework.

Starting with Solr 9, these hardcoded prefixes have been removed, and the Logging "appenders" are expected to provide any labeling/prefixing of what the values are. The default Solr log4j configuration has been updated to include the same single character prefixes (that were previously hardcoded).

Users who have a custom log4j configuration that wish to ensure Solr 9.x logs are consistently formatted after upgrading will need to make similar changes to their logging configuration files, by using %notEmpty macro patterns that include the prefixes, and wrapping those around the existing %X MDC patterns:

8.x

[%X{collection} %X{shard} %X{replica} %X{core}]

9.x

[%notEmpty{c:%X{collection}}%notEmpty{ s:%X{shard}}%notEmpty{ r:%X{replica}}%notEmpty{ x:%X{core}}]

---

Original Description

Currently, when Solr puts values in the MDC for logging, it always prepends them with a "CHAR" + "COLON" prefix that uniquely identifies what the value is – even though the MDC is a map and every value has a "key" associated with it that identifies what it is, ex...

  public static void setCollection(String collection) {
    if (collection != null) {
      MDC.put(COLLECTION_PROP, "c:" + collection);
    } else {
      MDC.remove(COLLECTION_PROP);
    }
  }

This choice evidently traces back to when MDC values where initially added SOLR-6673/SOLR-7590 (when Solr was still using slf4j->JUL IIUC) because that was best (only?) way folks found to ensure that log messages that did not have values for specific MDC keys weren't cluttered with valueless prefixes  even when a log message didn't correspond to a specific collection (or replica, or shard, etc...)

But with modern log4j, we can use use things like the %notEmpty syntax to prevent excessive verbosity when MDC values aren't set – allowing us to move the prefixes out of the Solr code and into Solr default log4j configuration – so that users who customize their log4j configuration with things like %mdc (or JSON logging) can get the "bare" values w/o hardcoded prefixes. (while keeping the exact same default log format ... or even improving it by removing some of the wasted whitespace)