Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10930

Refactor: Wrap Datanode IO related operations

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha2
    • datanode
    • None
    • Reviewed

    Description

      Datanode IO (Disk/Network) related operations and instrumentations are currently spilled in many classes such as DataNode.java, BlockReceiver.java, BlockSender.java, FsDatasetImpl.java, FsVolumeImpl.java, DirectoryScanner.java, BlockScanner.java, FsDatasetAsyncDiskService.java, LocalReplica.java, LocalReplicaPipeline.java, Storage.java, etc.

      This ticket is opened to consolidate IO related operations for easy instrumentation, metrics collection, logging and trouble shooting.

      Attachments

        1. HDFS-10930.01.patch
          48 kB
          Xiaoyu Yao
        2. HDFS-10930.02.patch
          55 kB
          Xiaoyu Yao
        3. HDFS-10930.03.patch
          53 kB
          Xiaoyu Yao
        4. HDFS-10930.04.patch
          61 kB
          Xiaoyu Yao
        5. HDFS-10930.05.patch
          62 kB
          Xiaoyu Yao
        6. HDFS-10930.06.patch
          63 kB
          Xiaoyu Yao
        7. HDFS-10930.07.patch
          62 kB
          Xiaoyu Yao
        8. HDFS-10930.barnch-2.00.patch
          57 kB
          Xiaoyu Yao
        9. HDFS-10930-branch-2.00.patch
          57 kB
          Arpit Agarwal
        10. HDFS-10930-branch-2.001.patch
          57 kB
          Xiaoyu Yao
        11. HDFS-10930.08.patch
          62 kB
          Xiaoyu Yao
        12. HDFS-10930.09.patch
          62 kB
          Xiaoyu Yao
        13. HDFS-10930.10.patch
          65 kB
          Xiaoyu Yao
        14. HDFS-10930.11.patch
          65 kB
          Xiaoyu Yao
        15. HDFS-10930-branch-2.002.patch
          58 kB
          Xiaoyu Yao

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-11205 Fix findbugs issue with BlockPoolSlice#validateIntegrityAndSetLength

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-11253 FileInputStream leak on failure path in BlockSender

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-10958 Add instrumentation hooks around Datanode disk IO

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #170

          Activity

            People

              xyao Xiaoyu Yao
              xyao Xiaoyu Yao
              Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: