Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-5969

Enable distributed tracing of requests

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Enable users to add diagnostic information to requests and trace them in the logs across servers.

      We have some metadata – e.g. a request UUID – that we log to every log line using Log4J's MDC. The UUID logging allows us to connect any log lines we have for a given request across servers. Sort of like Twitter's Zipkin.

      Currently we're using EmbeddedSolrServer without sharding, so adding the UUID is fairly simple, since everything is in one process and one thread. But, we're testing a sharded HTTP implementation and running into some difficulties getting this data passed around in a way that lets us trace all log lines generated by a request to its UUID.

      The first thing I tried was to add the UUID by adding it to the SolrParams. This achieves the goal of getting those values logged on the shards if a request is successful, but we miss having those values in the MDC if there are other log lines before the final log line. E.g. an Exception in a custom component.

      My current thought is that sending HTTP headers with diagnostic information would be very useful. Those could be placed in the MDC even before handing off to work to SolrDispatchFilter, so that any Solr problem will have the proper logging.

      I.e. every additional header added to a Solr request gets a "Solr-" prefix. On the server, we look for those headers and add them to the SLF4J MDC.

        Attachments

        1. SOLR-5969.diff
          17 kB
          Gregg Donovan
        2. SOLR-5969.diff
          18 kB
          Gregg Donovan

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                greggny3 Gregg Donovan
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: