Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7121

Use murmur hash to distribute HiveKey

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • Query Processor
    • None

    Description

      The current hashCode implementation produces poor parallelism when dealing with single integers or doubles.

      And for partitioned inserts into a 1 bucket table, there is a significant hotspot on Reducer #31.

      Removing the magic number 31 and using a more normal hash algorithm would help fix these hotspots.

      Attachments

        1. HIVE-7121.WIP.patch
          13 kB
          Gopal Vijayaraghavan
        2. HIVE-7121.3.patch
          13 kB
          Gunther Hagleitner
        3. HIVE-7121.2.patch
          13 kB
          Gopal Vijayaraghavan
        4. HIVE-7121.1.patch
          14 kB
          Gopal Vijayaraghavan

        Issue Links

          Activity

            People

              gopalv Gopal Vijayaraghavan
              gopalv Gopal Vijayaraghavan
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: