Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8031 Follow-on work for erasure coding phase I (striping layout)
  3. HDFS-9694

Make existing DFSClient#getFileChecksum() work for striped blocks

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Makes the getFileChecksum API works with striped layout EC files. Checksum computation done by block level in the distributed fashion. The current API does not support to compare the checksum generated with normal file and the checksum generated for the same file but in striped layout.
      Show
      Makes the getFileChecksum API works with striped layout EC files. Checksum computation done by block level in the distributed fashion. The current API does not support to compare the checksum generated with normal file and the checksum generated for the same file but in striped layout.

      Description

      This is a sub-task of HDFS-8430 and will get the existing API FileSystem#getFileChecksum(path) work for striped files. It will also refactor existing codes and layout basic work for subsequent tasks like support of the new API proposed there.

        Attachments

        1. HDFS-9694-v1.patch
          60 kB
          Kai Zheng
        2. HDFS-9694-v2.patch
          100 kB
          Kai Zheng
        3. HDFS-9694-v3.patch
          50 kB
          Kai Zheng
        4. HDFS-9694-v4.patch
          52 kB
          Kai Zheng
        5. HDFS-9694-v5.patch
          52 kB
          Kai Zheng
        6. HDFS-9694-v6.patch
          52 kB
          Kai Zheng
        7. HDFS-9694-v7.patch
          53 kB
          Kai Zheng
        8. HDFS-9694-v8.patch
          53 kB
          Kai Zheng
        9. HDFS-9694-v9.patch
          53 kB
          Kai Zheng

          Issue Links

          Is contained by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8430 Erasure coding: compute file checksum for striped files (stripe by stripe)

          • Major - Major loss of function.
          • In Progress
          is depended upon by

          Sub-task - The sub-task of the issue HDFS-9833 Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-12872 EC Checksum broken when BlockAccessToken is enabled

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

            Activity

              People

              • Assignee:
                drankye Kai Zheng
                Reporter:
                drankye Kai Zheng
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: