Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4714

Log short messages in Namenode RPC server for exceptions meant for clients

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.7, 2.1.0-beta, 3.0.0-alpha1
    • 2.1.0-beta, 0.23.8
    • namenode
    • None
    • Reviewed

    Description

      Namenode can slow down significantly if a rogue client/job issues massive number of requests that will fail. E.g. permission denied, quota overage, etc. The major contributing factor in slow down is the long namenode log message, which includes full stack trace.

      Previously similar issues involving safe mode and standby node have been addressed and we can extend it to suppress logging stack traces for configured list of exception classes.

      Attachments

        1. hdfs-4714.patch.txt
          3 kB
          Kihwal Lee
        2. hdfs-4714.branch-2.patch.txt
          3 kB
          Kihwal Lee
        3. hdfs-4714.branch-0.23.patch.txt
          3 kB
          Kihwal Lee

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-8711 provide an option for IPC server users to avoid printing stack information for certain exceptions

          • Major - Major loss of function.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-3817 avoid printing stack information for SafeModeException

          • Major - Major loss of function.
          • Closed

          Activity

            People

              kihwal Kihwal Lee
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: