Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11461

DataNode Disk Outlier Detection

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha4
    • hdfs
    • None
    • Reviewed

    Description

      Similar to how DataNodes collect peer performance statistics, we can collect disk performance statistics per datanode and detect outliers among them, if any.

      Attachments

        1. image-2023-07-28-20-26-04-186.png
          312 kB
          Yuanbo Liu
        2. HDFS-11461-branch-2.000.patch
          36 kB
          Hanisha Koneru
        3. HDFS-11461.002.patch
          36 kB
          Hanisha Koneru
        4. HDFS-11461.001.patch
          34 kB
          Hanisha Koneru
        5. HDFS-11461.000.patch
          34 kB
          Hanisha Koneru

        Issue Links

          breaks

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-14235 Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon

          • Major - Major loss of function.
          • Resolved
          supercedes

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-11475 Unit Test for DataNode Disk Outlier Detection

          • Major - Major loss of function.
          • Open

          Activity

            People

              hanishakoneru Hanisha Koneru
              hanishakoneru Hanisha Koneru
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: