Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4486

Add log category for long-running DFSClient notices

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 2.6.0
    • None
    • None

    Description

      There are a number of features in the DFS client which are transparent but can make a fairly big difference for performance – two in particular are short circuit reads and native checksumming. Because we don't want log spew for clients like "hadoop fs -cat" we currently log only at DEBUG level when these features are disabled. This makes it difficult to troubleshoot/verify for long-running perf-sensitive clients like HBase.

      One simple solution is to add a new log category - eg o.a.h.h.DFSClient.PerformanceAdvisory - which long-running clients could enable at DEBUG level without getting the full debug spew.

      Attachments

        1. HDFS-4486-20140820.patch
          3 kB
          Zhe Zhang
        2. HDFS-4486-20140821.patch
          10 kB
          Zhe Zhang
        3. hdfs-4486-20140821-2.patch
          10 kB
          Zhe Zhang
        4. HDFS-4486-20140822.patch
          10 kB
          Zhe Zhang
        5. HDFS-4486-20140822-2.patch
          11 kB
          Zhe Zhang

        Activity

          People

            zhz Zhe Zhang
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: