Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11985 Improve Solaris support in Hadoop
  3. HADOOP-12012

Investigate JNI for improving byte array comparison performance

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: benchmarks, io, performance
    • Labels:
      None
    • Environment:

      All

      Description

      HADOOP-7761 added functionality to compare byte arrays by treating them as arrays of 64-bit longs for performance. However HADOOP-11466 reverted this change for the SPARC architecture as it causes misaligned traps which causes performance to be worse rather than better.

      Most platforms have a highly-optimised memcmp() libc function that uses processor-specific functionality to perform byte array comparison as quickly as is possible for the platform.

      We have done some preliminary benchmarking on Solaris that suggests that, for reasonably-sized byte arrays, JNI code using memcmp() outperforms both of the current Java byte-size and long-size implementations on both SPARC and x86 . We are confirming the results and will repeat the same benchmark on Linux and report the results here for discussion.

        Attachments

          Activity

            People

            • Assignee:
              alanburlison Alan Burlison
              Reporter:
              alanburlison Alan Burlison
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: