Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-4838

Refactor MapJoin HashMap code to improve testability and readability

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      MapJoin is an essential component for high performance joins in Hive and the current code has done great service for many years. However, the code is showing it's age and currently suffers from the following issues:

      • Uses static state via the MapJoinMetaData class to pass serialization metadata to the Key, Row classes.
      • The api of a logical "Table Container" is not defined and therefore it's unclear what apis HashMapWrapper
        needs to publicize. Additionally HashMapWrapper has many used public methods.
      • HashMapWrapper contains logic to serialize, test memory bounds, and implement the table container. Ideally these logical units could be seperated
      • HashTableSinkObjectCtx has unused fields and unused methods
      • CommonJoinOperator and children use ArrayList on left hand side when only List is required
      • There are unused classes MRU, DCLLItemm and classes which duplicate functionality MapJoinSingleKey and MapJoinDoubleKeys

        Attachments

        1. HIVE-4838.patch
          136 kB
          Brock Noland
        2. HIVE-4838.patch
          136 kB
          Brock Noland
        3. HIVE-4838.patch
          135 kB
          Brock Noland
        4. HIVE-4838.patch
          135 kB
          Brock Noland
        5. HIVE-4838.patch
          137 kB
          Brock Noland
        6. HIVE-4838.patch
          137 kB
          Brock Noland
        7. HIVE-4838.patch
          147 kB
          Brock Noland

          Issue Links

            Activity

              People

              • Assignee:
                brocknoland Brock Noland
                Reporter:
                brocknoland Brock Noland
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: