Hive
  1. Hive
  2. HIVE-1977

DESCRIBE TABLE syntax doesn't support specifying a database qualified table name

    Details

    • Hadoop Flags:
      Reviewed

      Description

      The syntax for DESCRIBE is broken. It should be:

      DESCRIBE [EXTENDED] [database DOT]table [column]
      

      but is actually

      DESCRIBE [EXTENDED] table[DOT col_name]
      

      Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html

      1. HIVE-1977.6.patch.txt
        30 kB
        Zhenxiao Luo
      2. HIVE-1977.5.patch.txt
        51 kB
        Zhenxiao Luo
      3. HIVE-1977.4.patch.txt
        50 kB
        Zhenxiao Luo
      4. HIVE-1977.3.patch.txt
        51 kB
        Zhenxiao Luo
      5. HIVE-1977.2.patch.txt
        51 kB
        Zhenxiao Luo
      6. HIVE-1977.1.patch.txt
        53 kB
        Zhenxiao Luo

        Issue Links

          Activity

          Hide
          Carl Steinbach added a comment -

          I think we can fix this by introducing a new configuration property that controls whether DESCRIBE uses the standard syntax or the Hive syntax.

          Show
          Carl Steinbach added a comment - I think we can fix this by introducing a new configuration property that controls whether DESCRIBE uses the standard syntax or the Hive syntax.
          Hide
          Zhenxiao Luo added a comment -

          currently the following is working:

          1. $describe table;
          2. $describe table.column;

          while, these are not working:

          3. $describe column;
          4. $describe database.table;
          5. $describe database.table.column;

          always errors about the starting element is not a valid Table.

          Hive should support #3 and #5, since these are standard syntax, by updating Hive.g and related parser/lexer.

          For #4, a new configuration property may be needed to distinguish #2 with #4.
          Or, shall we always try matching both #2 and #4 when doing parsing? And error out if and only if neither matches? In this way, no new configuration property needed.

          This is also related to HIVE-2228.

          Show
          Zhenxiao Luo added a comment - currently the following is working: 1. $describe table; 2. $describe table.column; while, these are not working: 3. $describe column; 4. $describe database.table; 5. $describe database.table.column; always errors about the starting element is not a valid Table. Hive should support #3 and #5, since these are standard syntax, by updating Hive.g and related parser/lexer. For #4, a new configuration property may be needed to distinguish #2 with #4. Or, shall we always try matching both #2 and #4 when doing parsing? And error out if and only if neither matches? In this way, no new configuration property needed. This is also related to HIVE-2228 .
          Hide
          Carl Steinbach added a comment -

          3. $describe column;

          This doesn't make sense since tables in the same schema are allowed to have columns with the same name. The correct syntax for describing a column is:

          DESCRIBE table column

          or

          DESCRIBE db.table column

          Show
          Carl Steinbach added a comment - 3. $describe column; This doesn't make sense since tables in the same schema are allowed to have columns with the same name. The correct syntax for describing a column is: DESCRIBE table column or DESCRIBE db.table column
          Hide
          Zhenxiao Luo added a comment -

          @Carl: Get it. Should follow the correct syntax.

          In summary, should support the following:

          1. $describe table;
          2. $describe database.table;
          3. $describe table column;
          4. $describe database.table column;
          5. $describe table.column;
          6. $describe database.table.column;

          Currently,
          #1 is working
          #2 is error out for database not a valid table name
          #3 and #4 are having exceptions for invalid syntax
          #5 is working

          How about #6, are we going to support it?
          currently, #6 also error out for database not a valid table name

          And, to distinguish #2 and #5, a new configuration property is needed, or we try both, and error out only if neither matches?

          Show
          Zhenxiao Luo added a comment - @Carl: Get it. Should follow the correct syntax. In summary, should support the following: 1. $describe table; 2. $describe database.table; 3. $describe table column; 4. $describe database.table column; 5. $describe table.column; 6. $describe database.table.column; Currently, #1 is working #2 is error out for database not a valid table name #3 and #4 are having exceptions for invalid syntax #5 is working How about #6, are we going to support it? currently, #6 also error out for database not a valid table name And, to distinguish #2 and #5, a new configuration property is needed, or we try both, and error out only if neither matches?
          Hide
          Zhenxiao Luo added a comment -

          Review Request submitted at:
          https://reviews.facebook.net/D5763

          Show
          Zhenxiao Luo added a comment - Review Request submitted at: https://reviews.facebook.net/D5763
          Hide
          Namit Jain added a comment -

          Why do we need a new conf. for this ?
          I like your idea: we try both, and error out only if neither matches?

          Show
          Namit Jain added a comment - Why do we need a new conf. for this ? I like your idea: we try both, and error out only if neither matches?
          Hide
          Namit Jain added a comment -

          It is making the code complicated.
          Is the idea behind introducing a new conf. to handle the case if both table.column and database.table are valid ?
          Seems very unlikely, cant we create a hierarchy : database.table preceeds table.column.
          If the user really wants table.column in that case, they can always issue database.table.column which does not have any ambiguities.

          Show
          Namit Jain added a comment - It is making the code complicated. Is the idea behind introducing a new conf. to handle the case if both table.column and database.table are valid ? Seems very unlikely, cant we create a hierarchy : database.table preceeds table.column. If the user really wants table.column in that case, they can always issue database.table.column which does not have any ambiguities.
          Hide
          Zhenxiao Luo added a comment -

          @Namit: Thanks for your comments. I updated the patch, did the following:

          1. Instead of adding a new conf, try database.table first, if not valid(via tableValidCheck and databaseValidCheck), try table.column.
          2. get rid of isStandardSyntax, re-work the code

          Review request submitted at:
          https://reviews.facebook.net/D5763

          Show
          Zhenxiao Luo added a comment - @Namit: Thanks for your comments. I updated the patch, did the following: 1. Instead of adding a new conf, try database.table first, if not valid(via tableValidCheck and databaseValidCheck), try table.column. 2. get rid of isStandardSyntax, re-work the code Review request submitted at: https://reviews.facebook.net/D5763
          Hide
          Zhenxiao Luo added a comment -

          Get bug fixed in the previous patch.

          Get tests passed now. Review Request submitted at:
          https://reviews.facebook.net/D5763

          Show
          Zhenxiao Luo added a comment - Get bug fixed in the previous patch. Get tests passed now. Review Request submitted at: https://reviews.facebook.net/D5763
          Hide
          Namit Jain added a comment -

          minor comments

          Show
          Namit Jain added a comment - minor comments
          Hide
          Zhenxiao Luo added a comment -

          Comments addressed and review request submitted at:
          https://reviews.facebook.net/D5763

          Show
          Zhenxiao Luo added a comment - Comments addressed and review request submitted at: https://reviews.facebook.net/D5763
          Hide
          Zhenxiao Luo added a comment -

          testcase failure fixed in last patch. Also fix a comment error.

          New review request submitted at:
          https://reviews.facebook.net/D5763

          Show
          Zhenxiao Luo added a comment - testcase failure fixed in last patch. Also fix a comment error. New review request submitted at: https://reviews.facebook.net/D5763
          Hide
          Carl Steinbach added a comment -

          @Zhenxiao: Please see my comments on phabricator. Thanks.

          Show
          Carl Steinbach added a comment - @Zhenxiao: Please see my comments on phabricator. Thanks.
          Hide
          Zhenxiao Luo added a comment -

          @Carl: Thanks a lot for the comments. I updated my patch and submitted review request at:
          https://reviews.facebook.net/D5763

          Show
          Zhenxiao Luo added a comment - @Carl: Thanks a lot for the comments. I updated my patch and submitted review request at: https://reviews.facebook.net/D5763
          Hide
          Carl Steinbach added a comment -

          +1. Will commit if tests pass.

          Show
          Carl Steinbach added a comment - +1. Will commit if tests pass.
          Hide
          Carl Steinbach added a comment -

          @Zhenxiao: There are several test failures in TestParse. Can you please take a look? Thanks.

          Show
          Carl Steinbach added a comment - @Zhenxiao: There are several test failures in TestParse. Can you please take a look? Thanks.
          Hide
          Carl Steinbach added a comment -

          Here's the list of failures I see:

          org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1
          org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2
          org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3
          org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4
          org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5
          org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6

          Show
          Carl Steinbach added a comment - Here's the list of failures I see: org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6
          Show
          Gang Tim Liu added a comment - Fix is on the way: https://issues.apache.org/jira/browse/HIVE-3674 https://issues.apache.org/jira/browse/HIVE-3657
          Hide
          Carl Steinbach added a comment -

          Committed to trunk. Thanks Zhenxiao!

          Show
          Carl Steinbach added a comment - Committed to trunk. Thanks Zhenxiao!
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1777 (See https://builds.apache.org/job/Hive-trunk-h0.21/1777/)
          HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database qualified table name (Zhenxiao Luo via cws) (Revision 1406338)

          Result = FAILURE
          cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406338
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java
          • /hive/trunk/ql/src/test/queries/clientnegative/desc_failure3.q
          • /hive/trunk/ql/src/test/queries/clientpositive/describe_syntax.q
          • /hive/trunk/ql/src/test/results/clientnegative/desc_failure3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/describe_syntax.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1777 (See https://builds.apache.org/job/Hive-trunk-h0.21/1777/ ) HIVE-1977 . DESCRIBE TABLE syntax doesn't support specifying a database qualified table name (Zhenxiao Luo via cws) (Revision 1406338) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406338 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java /hive/trunk/ql/src/test/queries/clientnegative/desc_failure3.q /hive/trunk/ql/src/test/queries/clientpositive/describe_syntax.q /hive/trunk/ql/src/test/results/clientnegative/desc_failure3.q.out /hive/trunk/ql/src/test/results/clientpositive/describe_syntax.q.out
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
          HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database qualified table name (Zhenxiao Luo via cws) (Revision 1406338)

          Result = ABORTED
          cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406338
          Files :

          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
          • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java
          • /hive/trunk/ql/src/test/queries/clientnegative/desc_failure3.q
          • /hive/trunk/ql/src/test/queries/clientpositive/describe_syntax.q
          • /hive/trunk/ql/src/test/results/clientnegative/desc_failure3.q.out
          • /hive/trunk/ql/src/test/results/clientpositive/describe_syntax.q.out
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-1977 . DESCRIBE TABLE syntax doesn't support specifying a database qualified table name (Zhenxiao Luo via cws) (Revision 1406338) Result = ABORTED cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406338 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java /hive/trunk/ql/src/test/queries/clientnegative/desc_failure3.q /hive/trunk/ql/src/test/queries/clientpositive/describe_syntax.q /hive/trunk/ql/src/test/results/clientnegative/desc_failure3.q.out /hive/trunk/ql/src/test/results/clientpositive/describe_syntax.q.out
          Hide
          Ashutosh Chauhan added a comment -

          This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

          Show
          Ashutosh Chauhan added a comment - This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

            People

            • Assignee:
              Zhenxiao Luo
              Reporter:
              Carl Steinbach
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development