Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11097

HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.13.0, 0.14.0, 0.13.1, 1.0.0, 1.2.0
    • Fix Version/s: 1.3.0, 2.1.0
    • Component/s: File Formats
    • Labels:
      None
    • Environment:

      Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1

      Description

      Say we have a sql as

      create table if not exists test_orc_src (a int, b int, c int) stored as orc;
      create table if not exists test_orc_src2 (a int, b int, d int) stored as orc;
      insert overwrite table test_orc_src select 1,2,3 from src limit 1;
      insert overwrite table test_orc_src2 select 1,2,4 from src limit 1;
      set hive.auto.convert.join = false;
      set hive.execution.engine=mr;
      select
        tb.c
      from test.test_orc_src tb
      join (select * from test.test_orc_src2) tm
      on tb.a = tm.a
      where tb.b = 2
      

      The correct result is 3 but it produced no result.

      I find that in HiveInputFormat.pushProjectionsAndFilters

      match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key);
      

      It uses startsWith to combine aliases with path, so tm will match two alias in this case.

        Attachments

        1. HIVE-11097.1.patch
          1 kB
          Wan Chang
        2. HIVE-11097.2.patch
          4 kB
          Wan Chang
        3. HIVE-11097.3.patch
          6 kB
          Wan Chang
        4. HIVE-11097.4.patch
          7 kB
          Wan Chang
        5. HIVE-11097.5.patch
          7 kB
          Prasanth Jayachandran

          Issue Links

            Activity

              People

              • Assignee:
                wanchang Wan Chang
                Reporter:
                wanchang Wan Chang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: