Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: Testing Infrastructure
    • Labels:
      None

      Description

      Hash function differences between Java 7 and Java 8 lead to result order differences. While we have been able to fix a good number by converting hash maps to insert order hash maps, there are several cases where doing so is either not possible (because changes originate in external APIs) or change leads to even more out file differences.

      For example:

      (1) In TestCliDriver.testCliDriver_varchar_udf1, for the following query:

      select
        str_to_map('a:1,b:2,c:3',',',':'),
        str_to_map(cast('a:1,b:2,c:3' as varchar(20)),',',':')
      from varchar_udf_1 limit 1;”)
      

      the StandardMapObjectInspector used in LazySimpleSerDe to serialize the final output uses a HashMap. Changing it to LinkedHashMap will lead to several other q-test output differences.

      (2) In TestCliDriver.testCliDriver_parquet_map_null, data with map column is read from an Avro table into a Parquet table. Avro API, specifically GenericData.Record uses HashMap and returns data in different order.

      This patch adds supports to specify a hint called JAVA_VERSION_SPECIFIC_OUTPUT which may be added to a q-test, only if different outputs are expected for Java versions.

      For example:
      Under Java 7, test output file has ".java1.7.out" extension.
      Under Java 8, test output file has ".java1.8.out" extension.

      If hint is not added, we continue to generate a single ".out" file for the test.

        Attachments

        1. HIVE-9109.1.patch
          51 kB
          Mohit Sabharwal
        2. HIVE-9109.patch
          22 kB
          Mohit Sabharwal

          Issue Links

            Activity

              People

              • Assignee:
                mohitsabharwal Mohit Sabharwal
                Reporter:
                mohitsabharwal Mohit Sabharwal
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: