Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.23.1
    • hdfs-client
    • None
    • Reviewed
    • The default checksum algorithm used on HDFS is now CRC32C. Data from previous versions of Hadoop can still be read backwards-compatibly.

    Description

      Once the other subtasks/parts of HDFS-2080 are complete, CRC32C will be a much more efficient checksum algorithm than CRC32. Hence we should change the default checksum to CRC32C.

      However, in order to continue to support append against blocks created with the old checksum, we will need to implement some kind of handshaking in the write pipeline.

      Attachments

        1. hdfs-2130.txt
          32 kB
          Todd Lipcon
        2. hdfs-2130.txt
          32 kB
          Todd Lipcon
        3. hdfs-2130.txt
          33 kB
          Todd Lipcon

        Issue Links

          Activity

            People

              tlipcon Todd Lipcon
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: