Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8627

Compute stats on a table from impala caused the table to be corrupted

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.0, 0.13.1
    • Fix Version/s: 1.1.0
    • Component/s: Metastore
    • Labels:
      None

      Description

      Use impala 2.0 to connect to hive-0.13 Metastore.
      From impala, run the following queries:

      create table voter1(voter_id int,name string,age tinyint, registration
      string,contributions decimal(5,2),voterzone smallint,create_time timestamp) row
      format delimited fields terminated by '\t';
      load data inpath '/tmp/votertab' into table voter1;
      

      After this, can successfully select from table voter 1.

      Execute the following from impala shell:

      > compute stats voter1;
      

      After this, got the following error selecting from table voter1:

      > select * from voter1 limit 5;
      Query: select * from voter1 limit 5
      ERROR: AnalysisException: Failed to load metadata for table: default.voter1
      CAUSED BY: TableLoadingException: Failed to load metadata for table: voter1
      CAUSED BY: TTransportException: java.net.SocketException: Broken pipe
      CAUSED BY: SocketException: Broken pipe
      

      Below is the exception found in Hive log:

      org.apache.thrift.protocol.TProtocolException: Cannot write a TUnion with no set value!
      at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.java:240)
      at org.apache.thrift.TUnion$TUnionStandardScheme.write(TUnion.java:213)
      at org.apache.thrift.TUnion.write(TUnion.java:152)
      at org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj$ColumnStatisticsObjStandardScheme.write(ColumnStatisticsObj.java:550)
      at org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj$ColumnStatisticsObjStandardScheme.write(ColumnStatisticsObj.java:488)
      at org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj.write(ColumnStatisticsObj.java:414)
      at org.apache.hadoop.hive.metastore.api.TableStatsResult$TableStatsResultStandardScheme.write(TableStatsResult.java:388)
      at org.apache.hadoop.hive.metastore.api.TableStatsResult$TableStatsResultStandardScheme.write(TableStatsResult.java:338)
      at org.apache.hadoop.hive.metastore.api.TableStatsResult.write(TableStatsResult.java:288)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result$get_table_statistics_req_resultStandardScheme.write(ThriftHiveMetastore.java)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result$get_table_statistics_req_resultStandardScheme.write(ThriftHiveMetastore.java)
      at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_statistics_req_result.write(ThriftHiveMetastore.java)
      at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:53)
      at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
      at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)
      at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      

        Attachments

        1. HIVE-8627.patch
          1 kB
          Na Yang
        2. HIVE-8627.2.patch
          0.9 kB
          Na Yang

          Issue Links

            Activity

              People

              • Assignee:
                nyang Na Yang
                Reporter:
                nyang Na Yang
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: