Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8607 Java 8 support for Hive
  3. HIVE-9181

Fix SkewJoinOptimizer related Java 8 ordering differences

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: Tests
    • Labels:
      None

      Description

      (1) SkewJoinOptimizer adds items to ParseContext.topOps is different order in Java 8 vs Java 7. This leads to differences in EXPLAIN output related to MapWork.

      (2) Dbproperties order for a "desc database" query is non-deterministic at the HiveMetastore end between Java 8 and Java 7 (because Thrift maps are un-ordered).

      create database test_db with dbproperties ('key1' = 'value1', 'key2' = 'value2');
      desc database extended test_db;
      

      At client side, we serialize the dbproperties map returned from HMS as string and write it to a temp file. The FetchTask then reads the temp file and returns the string back to the user. In test environment, for consistent ordering, we should sort the map before it is serialized to the temp file.

        Attachments

        1. HIVE-9181.patch
          44 kB
          Mohit Sabharwal
        2. HIVE-9181.patch
          44 kB
          Brock Noland
        3. HIVE-9181.1.patch
          44 kB
          Mohit Sabharwal

          Issue Links

            Activity

              People

              • Assignee:
                mohitsabharwal Mohit Sabharwal
                Reporter:
                mohitsabharwal Mohit Sabharwal
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: