Hive
  1. Hive
  2. HIVE-76

Column number mismatch between query and destination tables when alias.* expressions are present in the select list of a join

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.3.0
    • Component/s: Query Processor
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Column number mismatch between query and destination tables when alias.* expressions are present in the select list of a join. The reason is due to a bug in how the row resolver is constructed in SemanticAnalyzer.

      1. patch-76_1.txt
        51 kB
        Ashish Thusoo
      2. patch-76.txt
        38 kB
        Ashish Thusoo

        Activity

        Hide
        Ashish Thusoo added a comment -

        The following test case leads to the error:

        CREATE TABLE dest1(key1 INT, value1 STRING, key2 INT, value2 STRING) STORED AS TEXTFILE;

        FROM src src1 JOIN src src2 ON (src1.key = src2.key)
        INSERT OVERWRITE TABLE dest1 SELECT src1., src2.;

        where src has two columns (key INT, value STRING)

        In this situation we should not be getting a column mismatch error as the destination has 4 columns and SELECT src1., src2. also should
        have 4 columns. However, due to a bug in SemanticAnalyzer we add all the columns in the select list to the row resolver as many times as an
        expression of the form alias.* appears.

        As a result it is impossible to insert into a destination table. A workaround is to enumerate all the columns and not use alias.* in the select list
        with joins when inserting the results into a table.

        Show
        Ashish Thusoo added a comment - The following test case leads to the error: CREATE TABLE dest1(key1 INT, value1 STRING, key2 INT, value2 STRING) STORED AS TEXTFILE; FROM src src1 JOIN src src2 ON (src1.key = src2.key) INSERT OVERWRITE TABLE dest1 SELECT src1. , src2. ; where src has two columns (key INT, value STRING) In this situation we should not be getting a column mismatch error as the destination has 4 columns and SELECT src1. , src2. also should have 4 columns. However, due to a bug in SemanticAnalyzer we add all the columns in the select list to the row resolver as many times as an expression of the form alias.* appears. As a result it is impossible to insert into a destination table. A workaround is to enumerate all the columns and not use alias.* in the select list with joins when inserting the results into a table.
        Hide
        Ashish Thusoo added a comment -

        Fix for the bug and test case.

        Show
        Ashish Thusoo added a comment - Fix for the bug and test case.
        Hide
        Ashish Thusoo added a comment -

        submitted patch.

        Show
        Ashish Thusoo added a comment - submitted patch.
        Hide
        Namit Jain added a comment -

        can you add a select also in join10.q.
        I mean select without the explain also, like all other tests

        Show
        Namit Jain added a comment - can you add a select also in join10.q. I mean select without the explain also, like all other tests
        Hide
        Ashish Thusoo added a comment -

        New patch which also fixes join10.q

        Note that join10.q was already broken before.

        Show
        Ashish Thusoo added a comment - New patch which also fixes join10.q Note that join10.q was already broken before.
        Hide
        Namit Jain added a comment -

        +1

        Show
        Namit Jain added a comment - +1
        Hide
        dhruba borthakur added a comment -

        I just committed this. Thanks Ashish!

        Show
        dhruba borthakur added a comment - I just committed this. Thanks Ashish!

          People

          • Assignee:
            Ashish Thusoo
            Reporter:
            Ashish Thusoo
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development