Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18910

Migrate to Murmur hash for shuffle and bucketing

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0, 3.0.0
    • 3.0.0
    • None
    • None

    Description

      Hive uses JAVA hash which is not as good as murmur for better distribution and efficiency in bucketing a table.

      Migrate to murmur hash but still keep backward compatibility for existing users so that they dont have to reload the existing tables.

      Attachments

        1. HIVE-18910.1.patch
          55 kB
          Deepak Jaiswal
        2. HIVE-18910.10.patch
          1.81 MB
          Deepak Jaiswal
        3. HIVE-18910.11.patch
          1.81 MB
          Deepak Jaiswal
        4. HIVE-18910.12.patch
          1.81 MB
          Deepak Jaiswal
        5. HIVE-18910.13.patch
          1.81 MB
          Deepak Jaiswal
        6. HIVE-18910.14.patch
          1.80 MB
          Deepak Jaiswal
        7. HIVE-18910.15.patch
          1.86 MB
          Deepak Jaiswal
        8. HIVE-18910.16.patch
          1.91 MB
          Deepak Jaiswal
        9. HIVE-18910.17.patch
          1.91 MB
          Deepak Jaiswal
        10. HIVE-18910.18.patch
          3.26 MB
          Deepak Jaiswal
        11. HIVE-18910.19.patch
          3.26 MB
          Deepak Jaiswal
        12. HIVE-18910.2.patch
          62 kB
          Deepak Jaiswal
        13. HIVE-18910.20.patch
          4.12 MB
          Deepak Jaiswal
        14. HIVE-18910.21.patch
          4.11 MB
          Deepak Jaiswal
        15. HIVE-18910.22.patch
          4.12 MB
          Deepak Jaiswal
        16. HIVE-18910.23.patch
          4.13 MB
          Deepak Jaiswal
        17. HIVE-18910.24.patch
          4.16 MB
          Deepak Jaiswal
        18. HIVE-18910.25.patch
          4.16 MB
          Deepak Jaiswal
        19. HIVE-18910.26.patch
          4.16 MB
          Deepak Jaiswal
        20. HIVE-18910.27.patch
          4.16 MB
          Deepak Jaiswal
        21. HIVE-18910.28.patch
          4.18 MB
          Deepak Jaiswal
        22. HIVE-18910.29.patch
          4.18 MB
          Deepak Jaiswal
        23. HIVE-18910.3.patch
          298 kB
          Deepak Jaiswal
        24. HIVE-18910.30.patch
          4.18 MB
          Deepak Jaiswal
        25. HIVE-18910.31.patch
          4.18 MB
          Deepak Jaiswal
        26. HIVE-18910.32.patch
          4.19 MB
          Deepak Jaiswal
        27. HIVE-18910.33.patch
          4.19 MB
          Deepak Jaiswal
        28. HIVE-18910.34.patch
          4.32 MB
          Deepak Jaiswal
        29. HIVE-18910.35.patch
          4.32 MB
          Deepak Jaiswal
        30. HIVE-18910.36.patch
          4.33 MB
          Deepak Jaiswal
        31. HIVE-18910.36.patch
          4.33 MB
          Deepak Jaiswal
        32. HIVE-18910.37.patch
          4.33 MB
          Deepak Jaiswal
        33. HIVE-18910.38.patch
          4.33 MB
          Deepak Jaiswal
        34. HIVE-18910.39.patch
          4.33 MB
          Deepak Jaiswal
        35. HIVE-18910.4.patch
          352 kB
          Deepak Jaiswal
        36. HIVE-18910.40.patch
          4.33 MB
          Deepak Jaiswal
        37. HIVE-18910.41.patch
          4.33 MB
          Deepak Jaiswal
        38. HIVE-18910.42.patch
          4.29 MB
          Deepak Jaiswal
        39. HIVE-18910.43.patch
          4.29 MB
          Deepak Jaiswal
        40. HIVE-18910.44.patch
          4.28 MB
          Deepak Jaiswal
        41. HIVE-18910.45.patch
          4.29 MB
          Deepak Jaiswal
        42. HIVE-18910.46.patch
          4.26 MB
          Deepak Jaiswal
        43. HIVE-18910.47.patch
          4.25 MB
          Deepak Jaiswal
        44. HIVE-18910.5.patch
          359 kB
          Deepak Jaiswal
        45. HIVE-18910.6.patch
          359 kB
          Deepak Jaiswal
        46. HIVE-18910.7.patch
          373 kB
          Deepak Jaiswal
        47. HIVE-18910.8.patch
          1.74 MB
          Deepak Jaiswal
        48. HIVE-18910.9.patch
          1.80 MB
          Deepak Jaiswal

        Issue Links

          relates to

          Sub-task - The sub-task of the issue SPARK-17495 Hive hash implementation

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-19386 Move TABLE_BUCKETING_VERSION to hive_metastore.thrift

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-19454 Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError

          • Major - Major loss of function.
          • Closed
          links to

          Web Link RB

          Activity

            People

              djaiswal Deepak Jaiswal
              djaiswal Deepak Jaiswal
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: