Hive
  1. Hive
  2. HIVE-6429

MapJoinKey has large memory overhead in typical cases

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      The only thing that MJK really needs it hashCode and equals (well, and construction), so there's no need to have array of writables in there. Assuming all the keys for a table have the same structure, for the common case where keys are primitive types, we can store something like a byte array combination of keys to reduce the memory usage. Will probably speed up compares too.

      1. HIVE-6429.WIP.patch
        49 kB
        Sergey Shelukhin
      2. HIVE-6429.patch
        49 kB
        Sergey Shelukhin
      3. HIVE-6429.01.patch
        50 kB
        Sergey Shelukhin
      4. HIVE-6429.02.patch
        50 kB
        Sergey Shelukhin
      5. HIVE-6429.03.patch
        51 kB
        Sergey Shelukhin
      6. HIVE-6429.04.patch
        54 kB
        Sergey Shelukhin
      7. HIVE-6429.05.patch
        75 kB
        Sergey Shelukhin
      8. HIVE-6429.06.patch
        76 kB
        Sergey Shelukhin
      9. HIVE-6429.07.patch
        76 kB
        Sergey Shelukhin
      10. HIVE-6429.08.patch
        78 kB
        Sergey Shelukhin
      11. HIVE-6429.09.patch
        59 kB
        Sergey Shelukhin
      12. HIVE-6429.10.patch
        56 kB
        Sergey Shelukhin

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Sergey Shelukhin
              Reporter:
              Sergey Shelukhin
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development