Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11847

Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha1
    • 3.1.0, 3.0.1
    • hdfs
    • None
    • Reviewed

    Description

      HDFS-10480 adds listOpenFiles option is to dfsadmin command to list all the open files in the system.

      Additionally, it would be very useful to only list open files that are blocking the DataNode decommissioning. With thousand+ node clusters, where there might be machines added and removed regularly for maintenance, any option to monitor and debug decommissioning status is very helpful. Proposal here is to add suboptions to listOpenFiles for the above case.

      Attachments

        1. HDFS-11847.01.patch
          42 kB
          Manoj Govindassamy
        2. HDFS-11847.02.patch
          56 kB
          Manoj Govindassamy
        3. HDFS-11847.03.patch
          60 kB
          Manoj Govindassamy
        4. HDFS-11847.04.patch
          60 kB
          Manoj Govindassamy
        5. HDFS-11847.05.patch
          61 kB
          Manoj Govindassamy

        Issue Links

          depends upon

          New Feature - A new feature of the product, which has yet to be developed. HDFS-10480 Add an admin command to list currently open files

          • Major - Major loss of function.
          • Resolved
          is a parent of

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-14507 Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-16059 dfsadmin -listOpenFiles -blockingDecommission can miss some files

          • Major - Major loss of function.
          • Open
          is required by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-12969 DfsAdmin listOpenFiles should report files by type

          • Major - Major loss of function.
          • Patch Available
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-4449 When a decommission is awaiting closure of live blocks, show the block IDs on the NameNode's UI report

          • Major - Major loss of function.
          • Open

          Activity

            People

              manojg Manoj Govindassamy
              manojg Manoj Govindassamy
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: