Hive
  1. Hive
  2. HIVE-4573

Support alternate table types for HiveServer2

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.12.0
    • Component/s: HiveServer2, JDBC
    • Labels:
      None
    • Release Note:
      Hide
      Adds new config parameter that needs to be documented:

      <property>
        <name>hive.server2.table.type.mapping</name>
        <value>HIVE</value>
        <description>
         This setting reflects how HiveServer will report the table types for JDBC and other
         client implementations that retrieves the available tables and supported table types
           HIVE : Exposes the hive's native table tyes like MANAGED_TABLE, EXTERNAL_TABLE, VIRTUAL_VIEW
           CLASSIC : More generic types like TABLE and VIEW
        </description>
      </property>
      Show
      Adds new config parameter that needs to be documented: <property>   <name>hive.server2.table.type.mapping</name>   <value>HIVE</value>   <description>    This setting reflects how HiveServer will report the table types for JDBC and other    client implementations that retrieves the available tables and supported table types      HIVE : Exposes the hive's native table tyes like MANAGED_TABLE, EXTERNAL_TABLE, VIRTUAL_VIEW      CLASSIC : More generic types like TABLE and VIEW   </description> </property>

      Description

      The getTables jdbc function no longer returns information when using normal JDBC table types like TABLE or VIEW. You must now use a more specific type such as MANAGED_TABLE or VIRTUAL_VIEW. An example application that will fail to return results against 0.10 is below, works without issue in 0.9. In my 0.10 test I used HS2.

      
      import java.sql.SQLException;
      import java.sql.Connection;
      import java.sql.ResultSet;
      import java.sql.Statement;
      import java.sql.DriverManager;
      import org.apache.hive.jdbc.HiveDriver;
      import java.sql.DatabaseMetaData;
      
      public class TestGet {
        private static String driverName = "org.apache.hive.jdbc.HiveDriver";
      
        /**
       * @param args
       * @throws SQLException
         */
        public static void main(String[] args) throws SQLException {
            try {
            Class.forName(driverName);
          } catch (ClassNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            System.exit(1);
          }
          Connection con = DriverManager.getConnection("jdbc:hive2://hostname:10000/default");
          DatabaseMetaData dbmd = con.getMetaData();
          String[] types = {"TABLE"};
          ResultSet rs = dbmd.getTables(null, null, "%", types);
            while (rs.next()) {
              System.out.println(rs.getString("TABLE_NAME"));
            }
          }
        }
      }
      
      1. HIVE-4573.2.patch
        18 kB
        Prasad Mujumdar
      2. HIVE-4573.1.patch
        17 kB
        Prasad Mujumdar

        Issue Links

          Activity

          Hide
          Lefty Leverenz added a comment -

          Doc note: This adds configuration parameter hive.server2.table.type.mapping to HiveConf.java, and HIVE-4998 changes its default to CLASSIC.

          Show
          Lefty Leverenz added a comment - Doc note: This adds configuration parameter hive.server2.table.type.mapping to HiveConf.java, and HIVE-4998 changes its default to CLASSIC. Configuration Properties – hive.server2.table.type.mapping
          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Sergey Shelukhin added a comment -

          ok, let's resurrect if it happens again... might be ivy cache, jenkins hit it too

          Show
          Sergey Shelukhin added a comment - ok, let's resurrect if it happens again... might be ivy cache, jenkins hit it too
          Hide
          Gunther Hagleitner added a comment -

          Alright. I have tried reproducing this several ways - no avail. I've checked the dependency chain. JDBC does depend on hive-common (albeit through a long transitive chain). I can't figure out why this is happening to you. I will monitor the situation but for now - very-clean is your very-slow friend.

          Show
          Gunther Hagleitner added a comment - Alright. I have tried reproducing this several ways - no avail. I've checked the dependency chain. JDBC does depend on hive-common (albeit through a long transitive chain). I can't figure out why this is happening to you. I will monitor the situation but for now - very-clean is your very-slow friend.
          Hide
          Gunther Hagleitner added a comment -

          I prefer fix over revert. Looking into this right now, but haven't been able to reproduce so far. I've heard from others that a "ant very-clean" will fix the issue if you see the same stuff. Hoping you can use this as a temporary workaround.

          Show
          Gunther Hagleitner added a comment - I prefer fix over revert. Looking into this right now, but haven't been able to reproduce so far. I've heard from others that a "ant very-clean" will fix the issue if you see the same stuff. Hoping you can use this as a temporary workaround.
          Hide
          Sergey Shelukhin added a comment -

          Do you guys want to fix or revert? For now I solved the problem by commenting it out. It may be because jdbc doesn't list common as dependency, but I didn't investigate a lot

          Show
          Sergey Shelukhin added a comment - Do you guys want to fix or revert? For now I solved the problem by commenting it out. It may be because jdbc doesn't list common as dependency, but I didn't investigate a lot
          Hide
          Sergey Shelukhin added a comment -

          This appears to break test-compile:

            [javac] Compiling 2 source files to /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/build/jdbc/test/classes
              [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:677: cannot find symbol
              [javac] symbol  : variable HIVE_SERVER2_TABLE_TYPE_MAPPING
              [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars
              [javac]     stmt.execute("set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname +
              [javac]                                            ^
              [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:684: cannot find symbol
              [javac] symbol  : variable HIVE_SERVER2_TABLE_TYPE_MAPPING
              [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars
              [javac]     stmt.execute("set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname +
              [javac]                                            ^
              [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:776: cannot find symbol
              [javac] symbol  : variable HIVE_SERVER2_TABLE_TYPE_MAPPING
              [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars
              [javac]     stmt.execute("set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname +
              [javac]                                            ^
              [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:784: cannot find symbol
              [javac] symbol  : variable HIVE_SERVER2_TABLE_TYPE_MAPPING
              [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars
              [javac]     stmt.execute("set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname +
              [javac]                                            ^
              [javac] Note: /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java uses unchecked or unsafe operations.
              [javac] Note: Recompile with -Xlint:unchecked for details.
              [javac] 4 errors
          
          Show
          Sergey Shelukhin added a comment - This appears to break test-compile: [javac] Compiling 2 source files to /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/build/jdbc/test/classes [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:677: cannot find symbol [javac] symbol : variable HIVE_SERVER2_TABLE_TYPE_MAPPING [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars [javac] stmt.execute( "set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname + [javac] ^ [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:684: cannot find symbol [javac] symbol : variable HIVE_SERVER2_TABLE_TYPE_MAPPING [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars [javac] stmt.execute( "set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname + [javac] ^ [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:776: cannot find symbol [javac] symbol : variable HIVE_SERVER2_TABLE_TYPE_MAPPING [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars [javac] stmt.execute( "set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname + [javac] ^ [javac] /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java:784: cannot find symbol [javac] symbol : variable HIVE_SERVER2_TABLE_TYPE_MAPPING [javac] location: class org.apache.hadoop.hive.conf.HiveConf.ConfVars [javac] stmt.execute( "set " + HiveConf.ConfVars.HIVE_SERVER2_TABLE_TYPE_MAPPING.varname + [javac] ^ [javac] Note: /x1/jenkins/jenkins-slave/workspace/Hive-trunk-hadoop2/hive/jdbc/src/test/org/apache/hadoop/hive/jdbc/TestJdbcDriver.java uses unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [javac] 4 errors
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hive-trunk-h0.21 #2248 (See https://builds.apache.org/job/Hive-trunk-h0.21/2248/)
          HIVE-4573: Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hive-trunk-h0.21 #2248 (See https://builds.apache.org/job/Hive-trunk-h0.21/2248/ ) HIVE-4573 : Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop2-ptest #47 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/47/)
          HIVE-4573: Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #47 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/47/ ) HIVE-4573 : Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hive-trunk-hadoop1-ptest #118 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/118/)
          HIVE-4573: Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hive-trunk-hadoop1-ptest #118 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/118/ ) HIVE-4573 : Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Hide
          Edward Capriolo added a comment -

          Im all about classic. Down with coke 2.

          Show
          Edward Capriolo added a comment - Im all about classic. Down with coke 2.
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hive-trunk-hadoop2 #330 (See https://builds.apache.org/job/Hive-trunk-hadoop2/330/)
          HIVE-4573: Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823)

          • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
          • /hive/trunk/conf/hive-default.xml.template
          • /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java
          • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #330 (See https://builds.apache.org/job/Hive-trunk-hadoop2/330/ ) HIVE-4573 : Support alternate table types for HiveServer2 (Prasad Mujumdar via Gunther Hagleitner) (gunther: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1510823 ) /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/jdbc/src/test/org/apache/hive/jdbc/TestJdbcDriver2.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/ClassicTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTableTypesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/GetTablesOperation.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/HiveTableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMapping.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/TableTypeMappingFactory.java
          Hide
          Gunther Hagleitner added a comment -

          Committed to trunk. Thanks Prasad!

          Show
          Gunther Hagleitner added a comment - Committed to trunk. Thanks Prasad!
          Hide
          Gunther Hagleitner added a comment -

          Edward Capriolo, Thejas M Nair, Prasad Mujumdar - If I understand this correctly this patch is needed as is and I am planning to commit in a few hours. The discussion on default, deprecation etc is happening on HIVE-4998. Speak up in the next couple of hours if you disagree.

          Show
          Gunther Hagleitner added a comment - Edward Capriolo , Thejas M Nair , Prasad Mujumdar - If I understand this correctly this patch is needed as is and I am planning to commit in a few hours. The discussion on default, deprecation etc is happening on HIVE-4998 . Speak up in the next couple of hours if you disagree.
          Hide
          Prasad Mujumdar added a comment -

          Thejas M Nair Thanks for setting up separate ticket for the default behavior. It's a good idea to decouple that discussion from the base implementation.

          Show
          Prasad Mujumdar added a comment - Thejas M Nair Thanks for setting up separate ticket for the default behavior. It's a good idea to decouple that discussion from the base implementation.
          Hide
          Thejas M Nair added a comment -

          Created HIVE-4998 to discuss the default behavior.

          Show
          Thejas M Nair added a comment - Created HIVE-4998 to discuss the default behavior.
          Hide
          Edward Capriolo added a comment -

          External tables are exceedingly rare and commonly misunderstood. I do not understand why a driver would care if a table was external or managed, that is just an implementation detail.

          Show
          Edward Capriolo added a comment - External tables are exceedingly rare and commonly misunderstood. I do not understand why a driver would care if a table was external or managed, that is just an implementation detail.
          Hide
          Thejas M Nair added a comment -

          Although it seems that eventually (.13?) you would want the default to be CLASSIC

          If we don't set the default to CLASSIC sooner, it would never happen. As time goes by, more applications would start relying this behavior.
          As N Campbell points out, the CLASSIC behavior is documented to be the 'normal' behavior. While we should aim for backward compatibility, I am not sure if that applies to bugs as well.

          The managed vs external table information can certainly be very useful. It would be good to get that without changing the server configuration. Should we rely on something like 'describe table extended' for that ?

          While I don't agree on the default, I don't think perfect should get in way of good. This improves things by making the classic behavior possible. We can discuss the default in a separate jira.
          +1 for the patch.

          Show
          Thejas M Nair added a comment - Although it seems that eventually (.13?) you would want the default to be CLASSIC If we don't set the default to CLASSIC sooner, it would never happen. As time goes by, more applications would start relying this behavior. As N Campbell points out, the CLASSIC behavior is documented to be the 'normal' behavior. While we should aim for backward compatibility, I am not sure if that applies to bugs as well. The managed vs external table information can certainly be very useful. It would be good to get that without changing the server configuration. Should we rely on something like 'describe table extended' for that ? While I don't agree on the default, I don't think perfect should get in way of good. This improves things by making the classic behavior possible. We can discuss the default in a separate jira. +1 for the patch.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12595799/HIVE-4573.2.patch

          ERROR: -1 due to 2 failed/errored test(s), 2762 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/300/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/300/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12595799/HIVE-4573.2.patch ERROR: -1 due to 2 failed/errored test(s), 2762 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/300/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/300/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 2 tests failed This message is automatically generated.
          Hide
          Gunther Hagleitner added a comment -

          Thanks! LGTM: +1

          Show
          Gunther Hagleitner added a comment - Thanks! LGTM: +1
          Hide
          Prasad Mujumdar added a comment -

          Updated the patch per review comments.

          Show
          Prasad Mujumdar added a comment - Updated the patch per review comments.
          Hide
          Gunther Hagleitner added a comment -

          I see. That makes sense. Although it seems that eventually (.13?) you would want the default to be CLASSIC (since it's easier for hive specific tools to set it to HIVE before they do their thing than for generic JDBC tools to set it to CLASSIC).

          Show
          Gunther Hagleitner added a comment - I see. That makes sense. Although it seems that eventually (.13?) you would want the default to be CLASSIC (since it's easier for hive specific tools to set it to HIVE before they do their thing than for generic JDBC tools to set it to CLASSIC).
          Hide
          Prasad Mujumdar added a comment -

          Gunther Hagleitner Thanks for taking a look.
          IMO, both use cases are valid. For applications or admin/monitoring tools developed specifically for hive, it would be useful to keep the hive specific details. Also we don't would break backward compatibility for existing apps that are expecting the hive way. That's my rationale for the fix.
          If that sounds reasonable, I will go ahead and make the necessary doc changes.

          Show
          Prasad Mujumdar added a comment - Gunther Hagleitner Thanks for taking a look. IMO, both use cases are valid. For applications or admin/monitoring tools developed specifically for hive, it would be useful to keep the hive specific details. Also we don't would break backward compatibility for existing apps that are expecting the hive way. That's my rationale for the fix. If that sounds reasonable, I will go ahead and make the necessary doc changes.
          Hide
          Gunther Hagleitner added a comment -

          Comments on RB.

          Show
          Gunther Hagleitner added a comment - Comments on RB.
          Hide
          Gunther Hagleitner added a comment -

          Seems like a lot of code for something that I expect everyone will want to have as "CLASSIC" anyways. It's almost more of a bug to me that we leak out these hive-only concepts.

          Questions: Can we get away with just classic? Can we at least mark HIVE as deprecated if we need both? Are we planning to always have both?

          If we go with the current code can you fill out the release notes and document the new config variable?

          Show
          Gunther Hagleitner added a comment - Seems like a lot of code for something that I expect everyone will want to have as "CLASSIC" anyways. It's almost more of a bug to me that we leak out these hive-only concepts. Questions: Can we get away with just classic? Can we at least mark HIVE as deprecated if we need both? Are we planning to always have both? If we go with the current code can you fill out the release notes and document the new config variable?
          Hide
          Prasad Mujumdar added a comment -
          Show
          Prasad Mujumdar added a comment - Review request on https://reviews.apache.org/r/13262/
          Hide
          N Campbell added a comment -

          while it is true that getTableTypes lists the types which exist, the Hive JDBC documentation continues to reference you to the standard JDBC pages where those types have existed since early days of JDBC.

          where was the change in types relative to prior releases etc documented re the change and purpose?

          as in ODBC, calling getTableTypes might help an application determine what types it 'expects' are shown to be supported or 'not'. However, for other types it becomes a crap shoot as to what a dynamic SQL application would presume to do re types listed that it is not familiar with.

          https://hive.apache.org/docs/r0.10.0/api/org/apache/hadoop/hive/jdbc/HiveDatabaseMetaData.html
          and follow the links from getTables().

          http://docs.oracle.com/javase/6/docs/api/java/sql/DatabaseMetaData.html?is-external=true#getTables(java.lang.String, java.lang.String, java.lang.String, java.lang.String[])

          Show
          N Campbell added a comment - while it is true that getTableTypes lists the types which exist, the Hive JDBC documentation continues to reference you to the standard JDBC pages where those types have existed since early days of JDBC. where was the change in types relative to prior releases etc documented re the change and purpose? as in ODBC, calling getTableTypes might help an application determine what types it 'expects' are shown to be supported or 'not'. However, for other types it becomes a crap shoot as to what a dynamic SQL application would presume to do re types listed that it is not familiar with. https://hive.apache.org/docs/r0.10.0/api/org/apache/hadoop/hive/jdbc/HiveDatabaseMetaData.html and follow the links from getTables(). http://docs.oracle.com/javase/6/docs/api/java/sql/DatabaseMetaData.html?is-external=true#getTables(java.lang.String , java.lang.String, java.lang.String, java.lang.String[])
          Hide
          Carl Steinbach added a comment - - edited

          The getTables jdbc function no longer returns information when using normal JDBC table types like TABLE or VIEW.

          It's not good that the behavior has changed between versions, nor that we're pretty clearly violating the principle of least astonishment for no good reason. At the same time it's unfair to say that the driver doesn't use "normal JDBC table types" because JDBC doesn't define any "normal table types". This also helps to explain why DatabaseMetaData.getTableTypes() exists.

          Show
          Carl Steinbach added a comment - - edited The getTables jdbc function no longer returns information when using normal JDBC table types like TABLE or VIEW. It's not good that the behavior has changed between versions, nor that we're pretty clearly violating the principle of least astonishment for no good reason. At the same time it's unfair to say that the driver doesn't use "normal JDBC table types" because JDBC doesn't define any "normal table types". This also helps to explain why DatabaseMetaData.getTableTypes() exists.

            People

            • Assignee:
              Prasad Mujumdar
              Reporter:
              Johndee Burks
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development