Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.1
    • Fix Version/s: 0.14.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      need parser to recognize constructs such as :

      INSERT INTO Cust (Customer_Number, Balance, Address)
      VALUES (101, 50.00, '123 Main Street'), (102, 75.00, '123 Pine Ave');
      
      DELETE FROM Cust WHERE Balance > 5.0
      
      UPDATE Cust
      SET column1=value1,column2=value2,...
      WHERE some_column=some_value
      

      also useful

      select a,b from values((1,2),(3,4)) as FOO(a,b)
      

      This makes writing tests easier.

      Some references:
      http://dev.mysql.com/doc/refman/5.6/en/insert.html
      http://msdn.microsoft.com/en-us/library/dd776382.aspx
      http://www.postgresql.org/docs/9.1/static/sql-values.html

      1. HIVE-7646.3.patch
        22 kB
        Eugene Koifman
      2. HIVE-7646.2.patch
        51 kB
        Eugene Koifman
      3. HIVE-7646.1.patch
        16 kB
        Eugene Koifman
      4. HIVE-7646.patch
        8 kB
        Eugene Koifman

        Issue Links

          Activity

          Hide
          Eugene Koifman added a comment -

          changes to parse DELETE.

          Show
          Eugene Koifman added a comment - changes to parse DELETE.
          Hide
          Eugene Koifman added a comment -

          HIVE-7646.patch adds grammar for UPDATE/DELETE and some basic tests

          Show
          Eugene Koifman added a comment - HIVE-7646 .patch adds grammar for UPDATE/DELETE and some basic tests
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12661076/HIVE-7646.patch

          ERROR: -1 due to 2 failed/errored test(s), 5895 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/266/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/266/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-266/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12661076

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12661076/HIVE-7646.patch ERROR: -1 due to 2 failed/errored test(s), 5895 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/266/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/266/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-266/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12661076
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12662702/HIVE-7646.1.patch

          ERROR: -1 due to 25 failed/errored test(s), 5831 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_11
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_15
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_16
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_17
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_18
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_25
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_3
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_6
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/399/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/399/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-399/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 25 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12662702

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12662702/HIVE-7646.1.patch ERROR: -1 due to 25 failed/errored test(s), 5831 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_16 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_18 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9 Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/399/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/399/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-399/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 25 tests failed This message is automatically generated. ATTACHMENT ID: 12662702
          Hide
          Eugene Koifman added a comment -

          HIVE-7646.2.patch - make sure 'values' is still a valid identifier, even though it's a keyword (following previous practice to maintain backwards compat). Fix multirow values clause grammar wrt parenthesis.

          Show
          Eugene Koifman added a comment - HIVE-7646 .2.patch - make sure 'values' is still a valid identifier, even though it's a keyword (following previous practice to maintain backwards compat). Fix multirow values clause grammar wrt parenthesis.
          Hide
          Eugene Koifman added a comment -

          patch 2 accidentally include 'extra' stuff

          Show
          Eugene Koifman added a comment - patch 2 accidentally include 'extra' stuff
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12662934/HIVE-7646.3.patch

          ERROR: -1 due to 1 failed/errored test(s), 6020 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/410/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/410/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-410/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12662934

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12662934/HIVE-7646.3.patch ERROR: -1 due to 1 failed/errored test(s), 6020 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_8 Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/410/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/410/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-410/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12662934
          Hide
          Eugene Koifman added a comment -

          TestCliDriver.testCliDriver_auto_sortmerge_join_8 failed with
          < FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
          < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask

          which can't be caused by parser changes, I don't think.

          In any case it passes locally with the same patch so it's not related to the patch.

          Show
          Eugene Koifman added a comment - TestCliDriver.testCliDriver_auto_sortmerge_join_8 failed with < FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask < ATTEMPT: Execute BackupTask: org.apache.hadoop.hive.ql.exec.mr.MapRedTask which can't be caused by parser changes, I don't think. In any case it passes locally with the same patch so it's not related to the patch.
          Hide
          Alan Gates added a comment -

          +1

          This isn't a big deal and I don't think we should hold the patch for it, but I have a question. If I read the spec correctly a query like:
          select * from (values((1, 2, 3),(4, 5, 6));
          should be legal. At FromClauseParser.g, line 290 you are requiring a tableNameColList as part of the virtualTableSource. This means the user always has to do a table definition after the values clause, so the above would become:
          select * from (values((1, 2, 3),(4, 5, 6)) as foo(a int, b int, c int);
          This makes sense since the more common case is probably:
          select a, count(b) from (values((1, 2, 3),(4, 5, 6)) as foo(a int, b int, c int) group by a where c > 4;
          or something, in which case the table definition is required. But the main question is am I misreading the spec or are we just adding the requirement for the table definition in all cases? I think it's ok if we're adding it, as this is primarily for our own testing purposes.

          Show
          Alan Gates added a comment - +1 This isn't a big deal and I don't think we should hold the patch for it, but I have a question. If I read the spec correctly a query like: select * from (values((1, 2, 3),(4, 5, 6)); should be legal. At FromClauseParser.g, line 290 you are requiring a tableNameColList as part of the virtualTableSource. This means the user always has to do a table definition after the values clause, so the above would become: select * from (values((1, 2, 3),(4, 5, 6)) as foo(a int, b int, c int); This makes sense since the more common case is probably: select a, count(b) from (values((1, 2, 3),(4, 5, 6)) as foo(a int, b int, c int) group by a where c > 4; or something, in which case the table definition is required. But the main question is am I misreading the spec or are we just adding the requirement for the table definition in all cases? I think it's ok if we're adding it, as this is primarily for our own testing purposes.
          Hide
          Alan Gates added a comment -

          Patch checked in. Thanks Eugene.

          Show
          Alan Gates added a comment - Patch checked in. Thanks Eugene.
          Hide
          Lefty Leverenz added a comment -

          Doc note: This doesn't need a TODOC14 label because HIVE-7802 covers the documentation.

          Show
          Lefty Leverenz added a comment - Doc note: This doesn't need a TODOC14 label because HIVE-7802 covers the documentation.
          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.

            People

            • Assignee:
              Eugene Koifman
              Reporter:
              Eugene Koifman
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development