Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8323

Low hanging checksum improvements

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Performance
    • Labels:
      None

      Description

      Over at Hadoop land, Todd Lipcon had done some improvements for checksums, a native implementation for CRC32C (HADOOP-7445) and bulk verify of checksums (HADOOP-7444).

      In HBase, we can do

      • Also develop a bulk verify API. Regardless of hbase.hstore.bytes.per.checksum we always want to verify of the whole checksum for the hfile block.
      • Enable NativeCrc32 to be used as a checksum algo. It is not clear how much gain we can expect over pure java CRC32.

      Though, longer term we should focus on convincing hdfs guys for inline checksums (HDFS-2699)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                enis Enis Soztutar
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: