Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14147

Backport of HDFS-13056 to the 2.9 branch: "Expose file-level composite CRCs in HDFS"

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.9.0, 2.9.1, 2.9.2
    • Fix Version/s: None
    • Component/s: datanode, distcp, hdfs
    • Labels:
      None

      Description

      HDFS-13056, Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts, is a significant feature for storage agnostic CRC comparisons between HDFS and cloud object stores such as S3 and GCS. With the extensively installed base of Hadoop 2, it should make a lot of sense to have the feature in Hadoop 2.

      The plan is to start with the backporting to 2.9, followed by 2.8 and 2.7 in that order.

        Attachments

        1. HDFS-14147.pdf
          118 kB
          Yan
        2. HDFS-14147-branch-2.9-001.patch
          145 kB
          Yan
        3. HDFS-14147-branch-2.9-001.patch
          145 kB
          Yan

          Issue Links

            Activity

              People

              • Assignee:
                yzhou2001 Yan
                Reporter:
                yzhou2001 Yan
              • Votes:
                3 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated: