Hive
  1. Hive
  2. HIVE-1731

Improve miscellaneous error messages

    Details

    • Hadoop Flags:
      Reviewed

      Description

      This is a place for accumulating error message improvements so that we can update a bunch in batch.

      1. HIVE-1731.8-0.7.patch
        75 kB
        Syed S. Albiz
      2. HIVE-1731.7-0.7.patch
        76 kB
        Syed S. Albiz
      3. HIVE-1731.7.patch
        78 kB
        Syed S. Albiz
      4. HIVE-1731.6.patch
        59 kB
        Syed S. Albiz
      5. HIVE-1731.5.patch
        63 kB
        Syed S. Albiz
      6. HIVE-1731.4.patch
        64 kB
        Syed S. Albiz
      7. HIVE-1731.3.patch
        63 kB
        Syed S. Albiz
      8. HIVE-1731.2.patch
        45 kB
        Syed S. Albiz
      9. HIVE-1731.1.patch
        16 kB
        Syed S. Albiz
      10. 0002-update-test-cases-with-error-string-results.patch
        46 kB
        Syed S. Albiz
      11. 0001-Fixed-parse-error-line-number-issue.patch
        17 kB
        Syed S. Albiz
      12. 0001-Fixed-parse-error-line-number-issue.patch
        16 kB
        Syed S. Albiz

        Issue Links

          Activity

          Hide
          Adam Kramer added a comment -

          Syed, can you also link the other task to finish the other messages here? I'll start posting my new complaints about error messages there now since this one's done.

          Here's one:

          hive> select count(1), b.age from t1 a join t2 b ON a.key = b.key and a.ds='2011-06-10' AND b.ds='2011-06-10';
          FAILED: Error in semantic analysis: Line 1:17 Expression not in GROUP BY key 'age'

          ...this should say, 'b.age' and not just 'age.' Why? Because it's different from this:

          hive> select count(1), b.age from t1 a join t2 b ON a.key = b.key and a.ds='2011-06-10' AND b.ds='2011-06-10' GROUP BY age;
          FAILED: Error in semantic analysis: Line 1:17 Expression not in GROUP BY key 'age'

          ...which is many times more maddening than the first error message.

          Show
          Adam Kramer added a comment - Syed, can you also link the other task to finish the other messages here? I'll start posting my new complaints about error messages there now since this one's done. Here's one: hive> select count(1), b.age from t1 a join t2 b ON a.key = b.key and a.ds='2011-06-10' AND b.ds='2011-06-10'; FAILED: Error in semantic analysis: Line 1:17 Expression not in GROUP BY key 'age' ...this should say, 'b.age' and not just 'age.' Why? Because it's different from this: hive> select count(1), b.age from t1 a join t2 b ON a.key = b.key and a.ds='2011-06-10' AND b.ds='2011-06-10' GROUP BY age; FAILED: Error in semantic analysis: Line 1:17 Expression not in GROUP BY key 'age' ...which is many times more maddening than the first error message.
          Hide
          John Sichi added a comment -

          Committed to 0.7 branch. Syed, can you open followups for the remaining items? Feel free to chunk them into one or more issues depending on what makes sense, and copy the original comments from here (and link to this issue as related).

          Show
          John Sichi added a comment - Committed to 0.7 branch. Syed, can you open followups for the remaining items? Feel free to chunk them into one or more issues depending on what makes sense, and copy the original comments from here (and link to this issue as related).
          Hide
          Syed S. Albiz added a comment -

          "Error while creating HAR" false error slipped into previous patch for backport.

          Show
          Syed S. Albiz added a comment - "Error while creating HAR" false error slipped into previous patch for backport.
          Hide
          Syed S. Albiz added a comment -

          Backport updates to TestParseNegative and TestJdbcDriver testcases to 0.7 branch

          Show
          Syed S. Albiz added a comment - Backport updates to TestParseNegative and TestJdbcDriver testcases to 0.7 branch
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.20 #715 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.20/715/)
          HIVE-1731. Improve miscellaneous error messages
          (Syed Albiz via jvs)

          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.20 #715 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.20/715/ ) HIVE-1731 . Improve miscellaneous error messages (Syed Albiz via jvs)
          Hide
          John Sichi added a comment -

          I've committed to trunk. Syed, can you give me a backport HIVE-1731.7-0.7.patch?

          Show
          John Sichi added a comment - I've committed to trunk. Syed, can you give me a backport HIVE-1731 .7-0.7.patch?
          Hide
          Syed S. Albiz added a comment -

          Added testcase updates for jdbc and TestParseNegative.

          Show
          Syed S. Albiz added a comment - Added testcase updates for jdbc and TestParseNegative.
          Hide
          John Sichi added a comment -

          I got one failure in TestJdbcDriver, and 29 in TestParseNegative. Syed, you can reproduce these with

          ant -Dtestcase=TestJdbcDriver test
          ant -Dtestcase=TestParseNegative test

          Probably just need to update the expected results.

          Show
          John Sichi added a comment - I got one failure in TestJdbcDriver, and 29 in TestParseNegative. Syed, you can reproduce these with ant -Dtestcase=TestJdbcDriver test ant -Dtestcase=TestParseNegative test Probably just need to update the expected results.
          Hide
          Syed S. Albiz added a comment -

          Backport patch onto branch-0.7

          Show
          Syed S. Albiz added a comment - Backport patch onto branch-0.7
          Hide
          Carl Steinbach added a comment -

          @Syed: This is a great contribution, and it would be nice to include this in the 0.7.1 release. Do you think you have time to backport this to branch-0.7?

          Show
          Carl Steinbach added a comment - @Syed: This is a great contribution, and it would be nice to include this in the 0.7.1 release. Do you think you have time to backport this to branch-0.7?
          Hide
          John Sichi added a comment -

          +1. Will commit when tests pass. Need to open followups for the unaddressed items.

          Show
          John Sichi added a comment - +1. Will commit when tests pass. Need to open followups for the unaddressed items.
          Hide
          Syed S. Albiz added a comment -

          Regenerated overwrites to get rid of incorrect 'Error while creating HAR' overwrite.

          Show
          Syed S. Albiz added a comment - Regenerated overwrites to get rid of incorrect 'Error while creating HAR' overwrite.
          Hide
          Syed S. Albiz added a comment -

          Add a clientnegative testcase to check explicitly for the CharSetLiteral error message

          Show
          Syed S. Albiz added a comment - Add a clientnegative testcase to check explicitly for the CharSetLiteral error message
          Hide
          John Sichi added a comment -

          One review comment added in

          https://reviews.apache.org/r/699/

          Also, can you add a test case for the case in Thomas Lento's comment from 27/Apr/11? Or is there already one there and I missed it?

          Show
          John Sichi added a comment - One review comment added in https://reviews.apache.org/r/699/ Also, can you add a test case for the case in Thomas Lento's comment from 27/Apr/11? Or is there already one there and I missed it?
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/699/
          -----------------------------------------------------------

          Review request for hive.

          Summary
          -------

          Review by jvs.

          This addresses bug HIVE-1731.
          https://issues.apache.org/jira/browse/HIVE-1731

          Diffs


          ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java a68a5c3
          ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java d738c48
          ql/src/test/results/clientnegative/alter_view_failure6.q.out b9447fa
          ql/src/test/results/clientnegative/ambiguous_col.q.out 9838e27
          ql/src/test/results/clientnegative/archive1.q.out 0927686
          ql/src/test/results/clientnegative/bad_sample_clause.q.out 9ddb18f
          ql/src/test/results/clientnegative/clusterbydistributeby.q.out 9151ff2
          ql/src/test/results/clientnegative/clusterbyorderby.q.out 5a50355
          ql/src/test/results/clientnegative/clusterbysortby.q.out d79c319
          ql/src/test/results/clientnegative/clustern3.q.out 37c103f
          ql/src/test/results/clientnegative/clustern4.q.out fcfa605
          ql/src/test/results/clientnegative/column_rename3.q.out bdca52e
          ql/src/test/results/clientnegative/create_view_failure3.q.out 7f3a710
          ql/src/test/results/clientnegative/drop_function_failure.q.out 0629c37
          ql/src/test/results/clientnegative/drop_index_failure.q.out c91afac
          ql/src/test/results/clientnegative/drop_partition_failure.q.out 97b4839
          ql/src/test/results/clientnegative/dyn_part2.q.out 636096d
          ql/src/test/results/clientnegative/exim_00_unsupported_schema.q.out 744f518
          ql/src/test/results/clientnegative/fileformat_void_input.q.out ce62d79
          ql/src/test/results/clientnegative/groupby2_map_skew_multi_distinct.q.out 494db44
          ql/src/test/results/clientnegative/groupby2_multi_distinct.q.out e0a1dee
          ql/src/test/results/clientnegative/groupby3_map_skew_multi_distinct.q.out 5702d4f
          ql/src/test/results/clientnegative/groupby3_multi_distinct.q.out 5702d4f
          ql/src/test/results/clientnegative/groupby_key.q.out 8c30cc4
          ql/src/test/results/clientnegative/input1.q.out f4e830e
          ql/src/test/results/clientnegative/input2.q.out a8ba9c5
          ql/src/test/results/clientnegative/input_part0_neg.q.out 566b5bf
          ql/src/test/results/clientnegative/invalid_create_tbl2.q.out ae81881
          ql/src/test/results/clientnegative/invalid_select_expression.q.out 69ec9ef
          ql/src/test/results/clientnegative/invalid_tbl_name.q.out aff52fa
          ql/src/test/results/clientnegative/invalidate_view1.q.out be64a4d
          ql/src/test/results/clientnegative/join2.q.out 43565ee
          ql/src/test/results/clientnegative/joinneg.q.out b1d22f8
          ql/src/test/results/clientnegative/lateral_view_join.q.out b5a8c3b
          ql/src/test/results/clientnegative/load_part_nospec.q.out 6886e98
          ql/src/test/results/clientnegative/load_wrong_noof_part.q.out 7b083e3
          ql/src/test/results/clientnegative/nopart_insert.q.out 46e724f
          ql/src/test/results/clientnegative/nopart_load.q.out 84868e4
          ql/src/test/results/clientnegative/notable_alias3.q.out 15181a0
          ql/src/test/results/clientnegative/orderbysortby.q.out 28c3dd6
          ql/src/test/results/clientnegative/regex_col_1.q.out 224c271
          ql/src/test/results/clientnegative/regex_col_2.q.out 5449148
          ql/src/test/results/clientnegative/regex_col_groupby.q.out db7c4a8
          ql/src/test/results/clientnegative/sample.q.out 1363980
          ql/src/test/results/clientnegative/select_udtf_alias.q.out f468719
          ql/src/test/results/clientnegative/semijoin1.q.out 83235d0
          ql/src/test/results/clientnegative/semijoin2.q.out b846e8a
          ql/src/test/results/clientnegative/semijoin3.q.out 2481fc6
          ql/src/test/results/clientnegative/semijoin4.q.out 2c2a5df
          ql/src/test/results/clientnegative/show_tables_bad1.q.out 7a81c06
          ql/src/test/results/clientnegative/show_tables_bad2.q.out 7a81c06
          ql/src/test/results/clientnegative/smb_bucketmapjoin.q.out 5f0e654
          ql/src/test/results/clientnegative/strict_orderby.q.out fe8fd62
          ql/src/test/results/clientnegative/strict_pruning.q.out 3f6ce49
          ql/src/test/results/clientnegative/subq_insert.q.out da8d18a
          ql/src/test/results/clientnegative/udf_array_contains_wrong1.q.out a9fc662
          ql/src/test/results/clientnegative/udf_array_contains_wrong2.q.out 25459e1
          ql/src/test/results/clientnegative/udf_case_type_wrong.q.out 69ea973
          ql/src/test/results/clientnegative/udf_case_type_wrong2.q.out 356f91d
          ql/src/test/results/clientnegative/udf_case_type_wrong3.q.out fd13bda
          ql/src/test/results/clientnegative/udf_coalesce.q.out 44b5de9
          ql/src/test/results/clientnegative/udf_elt_wrong_args_len.q.out ef4e67f
          ql/src/test/results/clientnegative/udf_elt_wrong_type.q.out 1ca50f7
          ql/src/test/results/clientnegative/udf_field_wrong_args_len.q.out 42b24eb
          ql/src/test/results/clientnegative/udf_field_wrong_type.q.out 2c6a48e
          ql/src/test/results/clientnegative/udf_if_not_bool.q.out d7ae204
          ql/src/test/results/clientnegative/udf_if_wrong_args_len.q.out 8862493
          ql/src/test/results/clientnegative/udf_in.q.out 9dd7ee5
          ql/src/test/results/clientnegative/udf_instr_wrong_args_len.q.out 5465ebf
          ql/src/test/results/clientnegative/udf_instr_wrong_type.q.out c4854a6
          ql/src/test/results/clientnegative/udf_locate_wrong_args_len.q.out 6b5ad6f
          ql/src/test/results/clientnegative/udf_locate_wrong_type.q.out 3506901
          ql/src/test/results/clientnegative/udf_size_wrong_args_len.q.out edb9662
          ql/src/test/results/clientnegative/udf_size_wrong_type.q.out ea41485
          ql/src/test/results/clientnegative/udf_when_type_wrong.q.out ced2310
          ql/src/test/results/clientnegative/udf_when_type_wrong2.q.out 0911660
          ql/src/test/results/clientnegative/udf_when_type_wrong3.q.out 4836e89
          ql/src/test/results/clientnegative/union.q.out 2f670f9

          Diff: https://reviews.apache.org/r/699/diff

          Testing
          -------

          Thanks,

          John

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/699/ ----------------------------------------------------------- Review request for hive. Summary ------- Review by jvs. This addresses bug HIVE-1731 . https://issues.apache.org/jira/browse/HIVE-1731 Diffs ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java a68a5c3 ql/src/java/org/apache/hadoop/hive/ql/parse/ParseDriver.java d738c48 ql/src/test/results/clientnegative/alter_view_failure6.q.out b9447fa ql/src/test/results/clientnegative/ambiguous_col.q.out 9838e27 ql/src/test/results/clientnegative/archive1.q.out 0927686 ql/src/test/results/clientnegative/bad_sample_clause.q.out 9ddb18f ql/src/test/results/clientnegative/clusterbydistributeby.q.out 9151ff2 ql/src/test/results/clientnegative/clusterbyorderby.q.out 5a50355 ql/src/test/results/clientnegative/clusterbysortby.q.out d79c319 ql/src/test/results/clientnegative/clustern3.q.out 37c103f ql/src/test/results/clientnegative/clustern4.q.out fcfa605 ql/src/test/results/clientnegative/column_rename3.q.out bdca52e ql/src/test/results/clientnegative/create_view_failure3.q.out 7f3a710 ql/src/test/results/clientnegative/drop_function_failure.q.out 0629c37 ql/src/test/results/clientnegative/drop_index_failure.q.out c91afac ql/src/test/results/clientnegative/drop_partition_failure.q.out 97b4839 ql/src/test/results/clientnegative/dyn_part2.q.out 636096d ql/src/test/results/clientnegative/exim_00_unsupported_schema.q.out 744f518 ql/src/test/results/clientnegative/fileformat_void_input.q.out ce62d79 ql/src/test/results/clientnegative/groupby2_map_skew_multi_distinct.q.out 494db44 ql/src/test/results/clientnegative/groupby2_multi_distinct.q.out e0a1dee ql/src/test/results/clientnegative/groupby3_map_skew_multi_distinct.q.out 5702d4f ql/src/test/results/clientnegative/groupby3_multi_distinct.q.out 5702d4f ql/src/test/results/clientnegative/groupby_key.q.out 8c30cc4 ql/src/test/results/clientnegative/input1.q.out f4e830e ql/src/test/results/clientnegative/input2.q.out a8ba9c5 ql/src/test/results/clientnegative/input_part0_neg.q.out 566b5bf ql/src/test/results/clientnegative/invalid_create_tbl2.q.out ae81881 ql/src/test/results/clientnegative/invalid_select_expression.q.out 69ec9ef ql/src/test/results/clientnegative/invalid_tbl_name.q.out aff52fa ql/src/test/results/clientnegative/invalidate_view1.q.out be64a4d ql/src/test/results/clientnegative/join2.q.out 43565ee ql/src/test/results/clientnegative/joinneg.q.out b1d22f8 ql/src/test/results/clientnegative/lateral_view_join.q.out b5a8c3b ql/src/test/results/clientnegative/load_part_nospec.q.out 6886e98 ql/src/test/results/clientnegative/load_wrong_noof_part.q.out 7b083e3 ql/src/test/results/clientnegative/nopart_insert.q.out 46e724f ql/src/test/results/clientnegative/nopart_load.q.out 84868e4 ql/src/test/results/clientnegative/notable_alias3.q.out 15181a0 ql/src/test/results/clientnegative/orderbysortby.q.out 28c3dd6 ql/src/test/results/clientnegative/regex_col_1.q.out 224c271 ql/src/test/results/clientnegative/regex_col_2.q.out 5449148 ql/src/test/results/clientnegative/regex_col_groupby.q.out db7c4a8 ql/src/test/results/clientnegative/sample.q.out 1363980 ql/src/test/results/clientnegative/select_udtf_alias.q.out f468719 ql/src/test/results/clientnegative/semijoin1.q.out 83235d0 ql/src/test/results/clientnegative/semijoin2.q.out b846e8a ql/src/test/results/clientnegative/semijoin3.q.out 2481fc6 ql/src/test/results/clientnegative/semijoin4.q.out 2c2a5df ql/src/test/results/clientnegative/show_tables_bad1.q.out 7a81c06 ql/src/test/results/clientnegative/show_tables_bad2.q.out 7a81c06 ql/src/test/results/clientnegative/smb_bucketmapjoin.q.out 5f0e654 ql/src/test/results/clientnegative/strict_orderby.q.out fe8fd62 ql/src/test/results/clientnegative/strict_pruning.q.out 3f6ce49 ql/src/test/results/clientnegative/subq_insert.q.out da8d18a ql/src/test/results/clientnegative/udf_array_contains_wrong1.q.out a9fc662 ql/src/test/results/clientnegative/udf_array_contains_wrong2.q.out 25459e1 ql/src/test/results/clientnegative/udf_case_type_wrong.q.out 69ea973 ql/src/test/results/clientnegative/udf_case_type_wrong2.q.out 356f91d ql/src/test/results/clientnegative/udf_case_type_wrong3.q.out fd13bda ql/src/test/results/clientnegative/udf_coalesce.q.out 44b5de9 ql/src/test/results/clientnegative/udf_elt_wrong_args_len.q.out ef4e67f ql/src/test/results/clientnegative/udf_elt_wrong_type.q.out 1ca50f7 ql/src/test/results/clientnegative/udf_field_wrong_args_len.q.out 42b24eb ql/src/test/results/clientnegative/udf_field_wrong_type.q.out 2c6a48e ql/src/test/results/clientnegative/udf_if_not_bool.q.out d7ae204 ql/src/test/results/clientnegative/udf_if_wrong_args_len.q.out 8862493 ql/src/test/results/clientnegative/udf_in.q.out 9dd7ee5 ql/src/test/results/clientnegative/udf_instr_wrong_args_len.q.out 5465ebf ql/src/test/results/clientnegative/udf_instr_wrong_type.q.out c4854a6 ql/src/test/results/clientnegative/udf_locate_wrong_args_len.q.out 6b5ad6f ql/src/test/results/clientnegative/udf_locate_wrong_type.q.out 3506901 ql/src/test/results/clientnegative/udf_size_wrong_args_len.q.out edb9662 ql/src/test/results/clientnegative/udf_size_wrong_type.q.out ea41485 ql/src/test/results/clientnegative/udf_when_type_wrong.q.out ced2310 ql/src/test/results/clientnegative/udf_when_type_wrong2.q.out 0911660 ql/src/test/results/clientnegative/udf_when_type_wrong3.q.out 4836e89 ql/src/test/results/clientnegative/union.q.out 2f670f9 Diff: https://reviews.apache.org/r/699/diff Testing ------- Thanks, John
          Hide
          Syed S. Albiz added a comment -

          combined patches for easier reviewing

          Show
          Syed S. Albiz added a comment - combined patches for easier reviewing
          Hide
          Syed S. Albiz added a comment -

          didn't realize that git was adding a prefix, removed that and amended the patch names.

          Show
          Syed S. Albiz added a comment - didn't realize that git was adding a prefix, removed that and amended the patch names.
          Hide
          John Sichi added a comment -

          Syed, can you do the following:

          1) Create a combined patch with the code changes plus the error messages; name it HIVE-1731.1.patch (our naming convention). In a fresh checkout of Hive source, in the directory corresponding to trunk, make sure that the command "patch -p0 < HIVE-1731.1.patch" applies cleanly (I'm still getting prompts when I try it with your latest).

          2) Upload it to review board per the instructions in HowToContribute.

          3) Click Submit Patch again.

          4) Print out this JIRA and check off the comments for the ones which are addressed by your patch; let me take a look at that while reviewing, and then we can create followups for the remaining ones.

          Thanks!

          Show
          John Sichi added a comment - Syed, can you do the following: 1) Create a combined patch with the code changes plus the error messages; name it HIVE-1731 .1.patch (our naming convention). In a fresh checkout of Hive source, in the directory corresponding to trunk, make sure that the command "patch -p0 < HIVE-1731 .1.patch" applies cleanly (I'm still getting prompts when I try it with your latest). 2) Upload it to review board per the instructions in HowToContribute. 3) Click Submit Patch again. 4) Print out this JIRA and check off the comments for the ones which are addressed by your patch; let me take a look at that while reviewing, and then we can create followups for the remaining ones. Thanks!
          Hide
          Syed S. Albiz added a comment -

          updated clientnegative testcases using -Doverwite=true on test run

          Show
          Syed S. Albiz added a comment - updated clientnegative testcases using -Doverwite=true on test run
          Hide
          Syed S. Albiz added a comment -

          Fixed some typos and removed the git diffstat

          Show
          Syed S. Albiz added a comment - Fixed some typos and removed the git diffstat
          Hide
          Syed S. Albiz added a comment -

          Patch for parser error issues and touching up of error message strings. Errors generated during semantic analysis and execution touch different bits of code, and hence it makes sense to separate those patches as they address different issues.

          Show
          Syed S. Albiz added a comment - Patch for parser error issues and touching up of error message strings. Errors generated during semantic analysis and execution touch different bits of code, and hence it makes sense to separate those patches as they address different issues.
          Hide
          Syed S. Albiz added a comment -

          Will take a look

          Show
          Syed S. Albiz added a comment - Will take a look
          Hide
          Thomas Lento added a comment -

          I ran this:

          [Wed Apr 27 14:58:42 tlento@dev1158] hive -e "select _c17, count(1) from tmp_tl_foo group by _c17"

          I got this:

          FAILED: Parse Error: line 1:11 mismatched input ',' expecting CharSetLiteral in character string literal

          The problem, of course, is that I needed to run this:

          [Wed Apr 27 14:58:59 tlento@dev1158] hive -e "select \`_c17\`, count(1) from tmp_tl_foo group by \`_c17\`"

          Show
          Thomas Lento added a comment - I ran this: [Wed Apr 27 14:58:42 tlento@dev1158] hive -e "select _c17, count(1) from tmp_tl_foo group by _c17" I got this: FAILED: Parse Error: line 1:11 mismatched input ',' expecting CharSetLiteral in character string literal The problem, of course, is that I needed to run this: [Wed Apr 27 14:58:59 tlento@dev1158] hive -e "select \`_c17\`, count(1) from tmp_tl_foo group by \`_c17\`"
          Hide
          Adam Kramer added a comment -

          SELECT a.id, VAR(a.cnt) FROM mytable a
          FAILED: Error in semantic analysis: line 1:94 Expression Not In Group By Key a

          ...what this error message should say is "Function VAR is either undefined or it is not an aggregation function."
          ...and even if this were a GOOD message, it means "expression not in group by key a.cnt"...just saying a is useless.

          Show
          Adam Kramer added a comment - SELECT a.id, VAR(a.cnt) FROM mytable a FAILED: Error in semantic analysis: line 1:94 Expression Not In Group By Key a ...what this error message should say is "Function VAR is either undefined or it is not an aggregation function." ...and even if this were a GOOD message, it means "expression not in group by key a.cnt"...just saying a is useless.
          Hide
          Adam Kramer added a comment -

          hive> alter table my_table create partition(ds='2011-08-01');
          FAILED: Parse Error: line 1:12 cannot recognize input 'my_table' in alter table statement

          ...what it should say is, "I don't know what create partition means." If I change the word "create" to "add," this works.

          Show
          Adam Kramer added a comment - hive> alter table my_table create partition(ds='2011-08-01'); FAILED: Parse Error: line 1:12 cannot recognize input 'my_table' in alter table statement ...what it should say is, "I don't know what create partition means." If I change the word "create" to "add," this works.
          Hide
          Adam Kramer added a comment -

          FAILED - LOCKS ON THE UNDERLYING OBJECTS CANNOT BE ACQUIRED. RETRY AFTER SOME TIME

          should be

          FAILED - Somebody is currently running a query that will replace this table or partition. Retry after some time.

          or

          FAILED - Somebody is currently reading from a table or partition you are attempting to overwrite. Retry after some time.

          ...depending on whether the user is trying to read or write, respectively.

          Show
          Adam Kramer added a comment - FAILED - LOCKS ON THE UNDERLYING OBJECTS CANNOT BE ACQUIRED. RETRY AFTER SOME TIME should be FAILED - Somebody is currently running a query that will replace this table or partition. Retry after some time. or FAILED - Somebody is currently reading from a table or partition you are attempting to overwrite. Retry after some time. ...depending on whether the user is trying to read or write, respectively.
          Hide
          John Sichi added a comment -

          @Adam: I was referring to the kwRole spew.

          Critical mass: maybe we can bootcamp it

          Show
          John Sichi added a comment - @Adam: I was referring to the kwRole spew. Critical mass: maybe we can bootcamp it
          Hide
          Adam Kramer added a comment -

          @John: Which of my messages are you talking about? Also, what's critical mass on this jira?

          Show
          Adam Kramer added a comment - @John: Which of my messages are you talking about? Also, what's critical mass on this jira?
          Hide
          John Sichi added a comment -

          @Adam: Hmmm, this seems to be some fallout from the ANTLR hack we used in HIVE-78 to avoid making ROLE into a keyword.

          Show
          John Sichi added a comment - @Adam: Hmmm, this seems to be some fallout from the ANTLR hack we used in HIVE-78 to avoid making ROLE into a keyword.
          Hide
          Adam Kramer added a comment -

          hive> show partitios table_name_here ;
          FAILED: Parse Error: line 1:5 rule kwRole failed predicate:

          {input.LT(1).getText().equalsIgnoreCase("role")}

          ? in show role grants

          ...this is a typo. I meant "partitions." This error message did not help me understand that at all and provides misleading information.

          Should be

          FAILED: Parse Error: line 1:5 predicate "partitios" not understood; failed rule kwRole...

          Show
          Adam Kramer added a comment - hive> show partitios table_name_here ; FAILED: Parse Error: line 1:5 rule kwRole failed predicate: {input.LT(1).getText().equalsIgnoreCase("role")} ? in show role grants ...this is a typo. I meant "partitions." This error message did not help me understand that at all and provides misleading information. Should be FAILED: Parse Error: line 1:5 predicate "partitios" not understood; failed rule kwRole...
          Hide
          Adam Kramer added a comment -

          FAILED: Error in semantic analysis: AS clause has an invalid number of aliases

          ...this should provide the line number and column number on which the invalid AS clause begins and ends. Subqueries mean there could be more than one.

          Show
          Adam Kramer added a comment - FAILED: Error in semantic analysis: AS clause has an invalid number of aliases ...this should provide the line number and column number on which the invalid AS clause begins and ends. Subqueries mean there could be more than one.
          Hide
          Adam Kramer added a comment -

          hive> describe table_that_does_not_exist;
          FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

          ...this error message should say something like "Table does not exist."

          Show
          Adam Kramer added a comment - hive> describe table_that_does_not_exist; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask ...this error message should say something like "Table does not exist."
          Hide
          Adam Kramer added a comment -

          FAILED: Parse Error: line 0:-1 mismatched input '<EOF>' expecting ) in subquery source

          No error that refers to lines 0:-1 is ever useful. Here is an example query:

          INSERT OVERWRITE TABLE my_table
          SELECT TRANSFORM(b.user1, b.user2, b.cnt)
          USING '

          {tr}

          '
          AS c1,c2,c3,c4
          FROM (
          SELECT b.user1, b.user2, b.cnt FROM (
          SELECT user1, user2, COUNT(1) AS cnt
          FROM sourcetable
          WHERE ds > '2010-12-01' AND ds <= '2010-12-07'
          GROUP BY user1, user2
          DISTRIBUTE BY user1 SORT BY user1, cnt
          ) b;

          ...the problem here is that the inner query is not indented and lacks a ). This error message should report the error as being at the ;, which is to say 12:4. Since the message knows an rparen is missing, it should also provide the index of the lparen. It should read like this:

          FAILED: Parse Error: line 12:4 mismatched input '<EOF>' expecting ) in subquery source to close ( at line 5:6.

          Show
          Adam Kramer added a comment - FAILED: Parse Error: line 0:-1 mismatched input '<EOF>' expecting ) in subquery source No error that refers to lines 0:-1 is ever useful. Here is an example query: INSERT OVERWRITE TABLE my_table SELECT TRANSFORM(b.user1, b.user2, b.cnt) USING ' {tr} ' AS c1,c2,c3,c4 FROM ( SELECT b.user1, b.user2, b.cnt FROM ( SELECT user1, user2, COUNT(1) AS cnt FROM sourcetable WHERE ds > '2010-12-01' AND ds <= '2010-12-07' GROUP BY user1, user2 DISTRIBUTE BY user1 SORT BY user1, cnt ) b; ...the problem here is that the inner query is not indented and lacks a ). This error message should report the error as being at the ;, which is to say 12:4. Since the message knows an rparen is missing, it should also provide the index of the lparen. It should read like this: FAILED: Parse Error: line 12:4 mismatched input '<EOF>' expecting ) in subquery source to close ( at line 5:6.
          Hide
          Thomas Lento added a comment -

          FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

          The actual error was something to do with my regexp_extract syntax:

          index out of bounds error, no group 1

          I didn't specify a group in the regexp, and when I made the adjustment everything worked as expected. It seems like this should be caught earlier, or at least the error trace for the task with the most failures (or a random task with max_failures if there's more than one) should be dumped to the CLI.

          This one might belong in a different task, since it's a much harder problem to deal with the error traces for arbitrary functions (or, worse, custom scripts). However, in general "return code 2" is not useful. It only tells me I need to do a bunch of work to find out what's broken, and the only reason I know that is because I've been using Hive for a long time.

          I think in general "Return code 2" should indicate a problem with the query execution and include a pointer to the jobtracker logs, or (ideally) a pointer to the log output for the task(s) with the most failures (maybe the top 10 if there are a lot of tasks with max_failures).

          Show
          Thomas Lento added a comment - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask The actual error was something to do with my regexp_extract syntax: index out of bounds error, no group 1 I didn't specify a group in the regexp, and when I made the adjustment everything worked as expected. It seems like this should be caught earlier, or at least the error trace for the task with the most failures (or a random task with max_failures if there's more than one) should be dumped to the CLI. This one might belong in a different task, since it's a much harder problem to deal with the error traces for arbitrary functions (or, worse, custom scripts). However, in general "return code 2" is not useful. It only tells me I need to do a bunch of work to find out what's broken, and the only reason I know that is because I've been using Hive for a long time. I think in general "Return code 2" should indicate a problem with the query execution and include a pointer to the jobtracker logs, or (ideally) a pointer to the log output for the task(s) with the most failures (maybe the top 10 if there are a lot of tasks with max_failures).
          Show
          Adam Kramer added a comment - https://issues.apache.org/jira/browse/HIVE-1839 also.
          Hide
          Adam Kramer added a comment -

          From a UNION ALL query:

          FAILED: Error in semantic analysis: Schema of both sides of union should match: destinationid:_col1 _col2

          ...this should 1) provide a line number where the error is, 2) say how the schemata mismatch, and 3) use actual column names. destinationid is an actual column name, but I have no idea what _col1 and _col2 refer to.

          When I have 10 UNION ALLs on top of each other, this error message is very aggravating.

          Show
          Adam Kramer added a comment - From a UNION ALL query: FAILED: Error in semantic analysis: Schema of both sides of union should match: destinationid:_col1 _col2 ...this should 1) provide a line number where the error is, 2) say how the schemata mismatch, and 3) use actual column names. destinationid is an actual column name, but I have no idea what _col1 and _col2 refer to. When I have 10 UNION ALLs on top of each other, this error message is very aggravating.
          Hide
          Andrew Ryan added a comment -

          I am not a Hive developer, only a Hive user, and not a power user either.

          I took a look at what I think are the error messages:
          http://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java

          and here are my suggestions:
          1) Always have error messages start with a capital letter. Some do, some don't.
          2) Always make SQL keywords ALL CAPS in error messages. Not only is it helpful, but there are really potentially ambiguous cases. For example:
          "Create table as select command" could have very different interpretation than "CREATE TABLE AS SELECT command" in some contexts.
          3) Be consistent about case, and use capitalization to denote proper nouns or the beginning of sentences only. For example:
          NO_PARTITION_PREDICATE("No Partition Predicate Found")
          could probably be:
          NO_PARTITION_PREDICATE("No partition predicate found"),
          and both styles are mixed in the error messages, which is worse. At least if they were all one way or all the other way it would be easier to figure out.

          Show
          Andrew Ryan added a comment - I am not a Hive developer, only a Hive user, and not a power user either. I took a look at what I think are the error messages: http://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ErrorMsg.java and here are my suggestions: 1) Always have error messages start with a capital letter. Some do, some don't. 2) Always make SQL keywords ALL CAPS in error messages. Not only is it helpful, but there are really potentially ambiguous cases. For example: "Create table as select command" could have very different interpretation than "CREATE TABLE AS SELECT command" in some contexts. 3) Be consistent about case, and use capitalization to denote proper nouns or the beginning of sentences only. For example: NO_PARTITION_PREDICATE("No Partition Predicate Found") could probably be: NO_PARTITION_PREDICATE("No partition predicate found"), and both styles are mixed in the error messages, which is worse. At least if they were all one way or all the other way it would be easier to figure out.
          Hide
          John Sichi added a comment -

          From Andrew Ryan:

          Instead of

          FAILED: Error in semantic analysis: line 1:333 In strict mode, limit must be
          specified if ORDER BY is present

          make it

          FAILED: Error in semantic analysis: line 1:333 In strict mode, if ORDER BY
          is specified, LIMIT must also be specified

          Show
          John Sichi added a comment - From Andrew Ryan: Instead of FAILED: Error in semantic analysis: line 1:333 In strict mode, limit must be specified if ORDER BY is present make it FAILED: Error in semantic analysis: line 1:333 In strict mode, if ORDER BY is specified, LIMIT must also be specified

            People

            • Assignee:
              Syed S. Albiz
              Reporter:
              John Sichi
            • Votes:
              2 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development