Accumulo
  1. Accumulo
  2. ACCUMULO-1197

Pass Accumulo trace functionality through the DFSClient

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.0
    • Component/s: trace
    • Labels:

      Description

      The tracing functionality used by Accumulo provides nested regions of timing information for regular operations occurring inside of Accumulo, e.g. scans and compactions. There is basic functionality to view this information through the Accumulo monitor. This can be thought of as a distributed timing infrastructure for Accumulo which uses Accumulo to store its data.

      Currently, this tracing doesn't fall through into HDFS. It would be awesome to actually introspect through the DFSClient, all the way down to the datanode writing to local disk. A large portion of the task would be investigating ways to inject the ability for calling applications (Accumulo, in this case) to provide the trace client through the Hadoop datanode code and record the necessary timings

      Skills required would be a good understanding of Java. Some basic knowledge about Apache Hadoop would also be helpful.

        Issue Links

          Activity

          Hide
          Eric Newton added a comment -

          It would also be good if we merged any additional tracing effort with that done for HBase.

          Show
          Eric Newton added a comment - It would also be good if we merged any additional tracing effort with that done for HBase.
          Hide
          Jared Winick added a comment -

          In a informal exchange with Todd Lipcon he says they would like to get Dapper-like tracing into HDFS for the next major release as he thinks it is a critical feature for low-latency workloads. For this to all work though I guess everyone's take on "Dapper-like" needs to be similar.

          Show
          Jared Winick added a comment - In a informal exchange with Todd Lipcon he says they would like to get Dapper-like tracing into HDFS for the next major release as he thinks it is a critical feature for low-latency workloads. For this to all work though I guess everyone's take on "Dapper-like" needs to be similar.
          Hide
          Billie Rinaldi added a comment -

          Works with Hadoop 2.6.0 and greater.

          Show
          Billie Rinaldi added a comment - Works with Hadoop 2.6.0 and greater.

            People

            • Assignee:
              Billie Rinaldi
              Reporter:
              Josh Elser
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development