Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4486

Add log category for long-running DFSClient notices

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      There are a number of features in the DFS client which are transparent but can make a fairly big difference for performance – two in particular are short circuit reads and native checksumming. Because we don't want log spew for clients like "hadoop fs -cat" we currently log only at DEBUG level when these features are disabled. This makes it difficult to troubleshoot/verify for long-running perf-sensitive clients like HBase.

      One simple solution is to add a new log category - eg o.a.h.h.DFSClient.PerformanceAdvisory - which long-running clients could enable at DEBUG level without getting the full debug spew.

        Attachments

        1. HDFS-4486-20140820.patch
          3 kB
          Zhe Zhang
        2. HDFS-4486-20140821.patch
          10 kB
          Zhe Zhang
        3. hdfs-4486-20140821-2.patch
          10 kB
          Zhe Zhang
        4. HDFS-4486-20140822.patch
          10 kB
          Zhe Zhang
        5. HDFS-4486-20140822-2.patch
          11 kB
          Zhe Zhang

          Activity

            People

            • Assignee:
              zhz Zhe Zhang
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: