[HIVE-6390] Read timeout on metastore client leads to out of sequence responses

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Component/s: Metastore
• Assignee: Unassigned
• Reporter: Amareshwari Sriramadasu
• Created: 07/Feb/14 05:45
• Updated: 07/Feb/14 06:09
• Resolved: 07/Feb/14 06:09

Description

When client application gets Read timeout on hive metastore client, the subsequent calls to metastore fail with out of sequence response. Then, the only way to get out of this is to restart the client application.

here are the exceptions:

2014-02-04 08:42:04,132 ERROR hive.log (MetaStoreUtils) - Got exception: org.apache.thrift.transport.TTransportException java.net.SocketTimeoutException: Read timed out
org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
        at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
        at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
        at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:912)

And subsequent calls to metastore get the following:

2014-02-04 08:43:14,273 ERROR hive.log (MetaStoreUtils) - Got exception: org.apache.thrift.TApplicationException get_tables failed: out of sequence response
org.apache.thrift.TApplicationException: get_tables failed: out of sequence response
        at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:76)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_tables(ThriftHiveMetastore.java:887)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_tables(ThriftHiveMetastore.java:873)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:912)