Hive
  1. Hive
  2. HIVE-1428

ALTER TABLE ADD PARTITION fails with a remote Thrift metastore

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.6.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD PARTITION commands will fail with an error similar to the following:

      [pradeepk@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE mytable add partition(datestamp = '20091101', srcid = '10',action) location '/user/pradeepk/mytable/20091101/10';"
      10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
      Hive history file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
      FAILED: Error in metadata: org.apache.thrift.TApplicationException: get_partition failed: unknown result
      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
      [pradeepk@chargesize:~/dev/howl]

      This is due to a check that tries to retrieve the partition to see if it exists. If it does not, an attempt is made to pass a null value from the metastore. Since thrift does not support null return values, an exception is thrown.

      1. HIVE-1428.patch
        23 kB
        Pradeep Kamath
      2. HIVE-1428-0.6.0-patch.txt
        32 kB
        Carl Steinbach
      3. HIVE-1428-2.patch
        37 kB
        Pradeep Kamath
      4. HIVE-1428-3.patch
        29 kB
        Pradeep Kamath
      5. TestHiveMetaStoreRemote.java
        13 kB
        Pradeep Kamath

        Activity

        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        John Sichi made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Status Reopened [ 4 ] Patch Available [ 10002 ]
        Carl Steinbach made changes -
        Fix Version/s 0.6.0 [ 12314524 ]
        Fix Version/s 0.7.0 [ 12315150 ]
        Affects Version/s 0.5.0 [ 12314156 ]
        Affects Version/s 0.6.0 [ 12314524 ]
        Affects Version/s 0.7.0 [ 12315150 ]
        Carl Steinbach made changes -
        Attachment HIVE-1428-0.6.0-patch.txt [ 12451981 ]
        Carl Steinbach made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        John Sichi made changes -
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        John Sichi made changes -
        Summary ALTER TABLE ADD PARTITION fails with a remote Thirft metastore ALTER TABLE ADD PARTITION fails with a remote Thrift metastore
        Fix Version/s 0.7.0 [ 12315150 ]
        Pradeep Kamath made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Pradeep Kamath made changes -
        Attachment HIVE-1428-3.patch [ 12449276 ]
        Paul Yang made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        John Sichi made changes -
        Assignee Pradeep Kamath [ pradeepkth ]
        Pradeep Kamath made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Pradeep Kamath made changes -
        Attachment HIVE-1428-2.patch [ 12448984 ]
        Pradeep Kamath made changes -
        Attachment TestHiveMetaStoreRemote.java [ 12448325 ]
        Paul Yang made changes -
        Description If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD PARTITION commands will fail with an error similar to the following:

        {code}
        [pradeepk@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE mytable add partition(datestamp = '20091101', srcid = '10',action) location '/user/pradeepk/mytable/20091101/10';"
        10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
        Hive history file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
        FAILED: Error in metadata: org.apache.thrift.TApplicationException: get_partition failed: unknown result
        FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
        [pradeepk@chargesize:~/dev/howl]
        {code}

        This is due to a check that tries to retrieve the partition to see if it exists. If it does not, an attempt is made to pass a null value from the metastore. Since thrift does not support null return values, an exception is thrown.
        If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD PARTITION commands will fail with an error similar to the following:

        [pradeepk@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE mytable add partition(datestamp = '20091101', srcid = '10',action) location '/user/pradeepk/mytable/20091101/10';"
        10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
        Hive history file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
        FAILED: Error in metadata: org.apache.thrift.TApplicationException: get_partition failed: unknown result
        FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
        [pradeepk@chargesize:~/dev/howl]

        This is due to a check that tries to retrieve the partition to see if it exists. If it does not, an attempt is made to pass a null value from the metastore. Since thrift does not support null return values, an exception is thrown.
        Pradeep Kamath made changes -
        Attachment HIVE-1428.patch [ 12448018 ]
        Paul Yang made changes -
        Field Original Value New Value
        Description If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD PARTITION commands will fail with an error similar to the following:

        {code}
        [pradeepk@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE mytable add partition(datestamp = '20091101', srcid = '10',action) location '/user/pradeepk/mytable/20091101/10';"
        10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
        Hive history file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
        FAILED: Error in metadata: org.apache.thrift.TApplicationException: get_partition failed: unknown result
        FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
        [pradeepk@chargesize:~/dev/howl]
        {code}

        This is due to a check that tries to retrieve the partition to check if it exists. If it does not, an attempt is made to pass a null partition value from the metastore. Since thrift does not support null return values, an exception is thrown when the CLI is configured to use a remote metastore.
        If the hive cli is configured to use a remote metastore, ALTER TABLE ... ADD PARTITION commands will fail with an error similar to the following:

        {code}
        [pradeepk@chargesize:~/dev/howl]hive --auxpath ult-serde.jar -e "ALTER TABLE mytable add partition(datestamp = '20091101', srcid = '10',action) location '/user/pradeepk/mytable/20091101/10';"
        10/06/16 17:08:59 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
        Hive history file=/tmp/pradeepk/hive_job_log_pradeepk_201006161709_1934304805.txt
        FAILED: Error in metadata: org.apache.thrift.TApplicationException: get_partition failed: unknown result
        FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
        [pradeepk@chargesize:~/dev/howl]
        {code}

        This is due to a check that tries to retrieve the partition to see if it exists. If it does not, an attempt is made to pass a null value from the metastore. Since thrift does not support null return values, an exception is thrown.
        Paul Yang created issue -

          People

          • Assignee:
            Pradeep Kamath
            Reporter:
            Paul Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development