Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.14.0
-
None
-
Vmware Pseudo cluster
HDP 2.2 Ambari vanilla install
Centos 6.5
-
Hive Hadoop Tez Thrift HiveServer2
Description
Everything runs but dies after a few days with the Java heap space memory error - and with no activity on cluster. It failed most resently after 5 days.
I tried to fix it after noticing the SLF4J library duplication, wondering if conflicting libraries would be causing the error, but still fails.
Nagios reports HiveServer2 "Flapping"
HiveServer2.log output:
22488-2015-03-07 22:19:08,359 INFO [HiveServer2-Handler-Pool: Thread-13139]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(232)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V6
22489-2015-03-07 22:19:19,056 ERROR [HiveServer2-Handler-Pool: Thread-13139]: thrift.ProcessFunction (ProcessFunction.java:process(41)) - Internal error processing OpenSession
22490:java.lang.OutOfMemoryError: Java heap space
22491-2015-03-07 22:19:22,515 INFO [Thread-6]: server.HiveServer2 (HiveServer2.java:stop(299)) - Shutting down HiveServer2
22492-2015-03-07 22:19:22,516 INFO [Thread-6]: thrift.ThriftCLIService (ThriftCLIService.java:stop(137)) - Thrift server has stopped
22493-2015-03-07 22:19:22,516 INFO [Thread-6]: service.AbstractService (AbstractService.java:stop(125)) - Service:ThriftBinaryCLIService is stopped.
22494-2015-03-07 22:19:27,078 INFO [Thread-6]: service.AbstractService (AbstractService.java:stop(125)) - Service:OperationManager is stopped.
22495-2015-03-07 22:19:27,078 INFO [Thread-6]: service.AbstractService (AbstractService.java:stop(125)) - Service:SessionManager is stopped.
22496:2015-03-07 22:19:36,096 WARN [Thread-0]: util.ShutdownHookManager (ShutdownHookManager.java:run(56)) - ShutdownHook 'ClientFinalizer' failed, java.lang.OutOfMemoryError: Java heap space
22497:java.lang.OutOfMemoryError: Java heap space