Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1
    • Component/s: hdfs-client
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      The default checksum algorithm used on HDFS is now CRC32C. Data from previous versions of Hadoop can still be read backwards-compatibly.

      Description

      Once the other subtasks/parts of HDFS-2080 are complete, CRC32C will be a much more efficient checksum algorithm than CRC32. Hence we should change the default checksum to CRC32C.

      However, in order to continue to support append against blocks created with the old checksum, we will need to implement some kind of handshaking in the write pipeline.

        Attachments

        1. hdfs-2130.txt
          33 kB
          Todd Lipcon
        2. hdfs-2130.txt
          32 kB
          Todd Lipcon
        3. hdfs-2130.txt
          32 kB
          Todd Lipcon

          Issue Links

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: