Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11985 Improve Solaris support in Hadoop
  3. HADOOP-12012

Investigate JNI for improving byte array comparison performance

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: benchmarks, io, performance
    • Labels:
      None
    • Environment:

      All

      Description

      HADOOP-7761 added functionality to compare byte arrays by treating them as arrays of 64-bit longs for performance. However HADOOP-11466 reverted this change for the SPARC architecture as it causes misaligned traps which causes performance to be worse rather than better.

      Most platforms have a highly-optimised memcmp() libc function that uses processor-specific functionality to perform byte array comparison as quickly as is possible for the platform.

      We have done some preliminary benchmarking on Solaris that suggests that, for reasonably-sized byte arrays, JNI code using memcmp() outperforms both of the current Java byte-size and long-size implementations on both SPARC and x86 . We are confirming the results and will repeat the same benchmark on Linux and report the results here for discussion.

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              alanburlison Alan Burlison Assign to me
              Reporter:
              alanburlison Alan Burlison

              Dates

              • Created:
                Updated:

                Issue deployment