Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9601

Support native CRC on byte arrays

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 3.0.0-alpha1
    • None
    • performance, util
    • Support NativeCrc32 verification for byte[] array backed buffers

    Description

      When we first implemented the Native CRC code, we only did so for direct byte buffers, because these correspond directly to native heap memory and thus make it easy to access via JNI. We'd generally assumed that accessing byte[] arrays from JNI was not efficient enough, but now that I know more about JNI I don't think that's true – we just need to make sure that the critical sections where we lock the buffers are short.

      Attachments

        1. HADOOP-9601-bench.patch
          2 kB
          Gopal Vijayaraghavan
        2. HADOOP-9601-rebase+benchmark.patch
          13 kB
          Gopal Vijayaraghavan
        3. HADOOP-9601-trunk-rebase.patch
          13 kB
          Gopal Vijayaraghavan
        4. HADOOP-9601-trunk-rebase-2.patch
          10 kB
          Gopal Vijayaraghavan
        5. HADOOP-9601-WIP-01.patch
          6 kB
          Gopal Vijayaraghavan
        6. HADOOP-9601-WIP-02.patch
          8 kB
          Gopal Vijayaraghavan

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10838 Byte array native checksumming

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9934 the generic (non x86) libhadoop CRC code makes unaligned loads and stores

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-8323 Low hanging checksum improvements

          • Major - Major loss of function.
          • Closed
          is superceded by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-10838 Byte array native checksumming

          • Major - Major loss of function.
          • Closed

          Activity

            People

              gopalv Gopal Vijayaraghavan
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: