Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
0.11.0
-
None
-
HADOOP ENVIRONMENT IS CDH5+CDH5-HIVE-0.11+PRESTO-0.56
Description
Hi friends:
Now I can't open the page https://groups.google.com/forum/#!forum/presto-users ,so show my question here.
I have started hiveserver and started presto-server on a machine with commands below:
hive --service hiveserver -p 9083
./launcher run
When I use the presto-client-cli command ./presto --server localhost:9083 --catalog hive --schema default ,the console shows presto:default>,input the command as show tables the console prints Error running command: java.nio.channels.ClosedChannelException,
and the hiveserver console print as below:
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "pool-1-thread-1" java.lang.OutOfMemoryError: Java heap space
at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:353)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
my configuration file below:
node.properties
node.environment=production
node.id=cc4a1bbf-5b98-4935-9fde-2cf1c98e8774
node.data-dir=/home/hadoop/cloudera-5.0.0/presto-0.56/presto/data
config.properties
coordinator=true
datasources=jmx
http-server.http.port=8080
presto-metastore.db.type=h2
presto-metastore.db.filename=/home/hadoop/cloudera-5.0.0/presto-0.56/presto/db/MetaStore
task.max-memory=1GB
discovery-server.enabled=true
discovery.uri=http://slave4:8080
jvm.config
-server
-Xmx16G
-XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent
-XX:+CMSClassUnloadingEnabled
-XX:+AggressiveOpts
-XX:+HeapDumpOnOutOfMemoryError
-XX:OnOutOfMemoryError=kill -9 %p
-XX:PermSize=150M
-XX:MaxPermSize=150M
-XX:ReservedCodeCacheSize=150M
-Xbootclasspath/p:/home/hadoop/cloudera-5.0.0/presto-0.56/presto-server-0.56/lib/floatingdecimal-0.1.jar
log.properties
com.facebook.presto=DEBUG
catalog/hive.properties
connector.name=hive-cdh4
hive.metastore.uri=thrift://slave4:9083
HADOOP ENVIRONMENT IS CDH5+CDH5-HIVE-0.11+PRESTO-0.56
Last I had increased the Java heap size for the Hive metastore,but it still given me the same error informations ,please help me to check if that is a bug of CDH5.Now I have no idea,god !
please help me ,thanks.
**********************************************************************************************************************************************************************
========================================================================================================
**********************************************************************************************************************************************************************
Add some informations as below:
Help,help,help!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I have test prest-server-0.55 and 0.56 and 0.57 on CDH4 +hive-0.10 or hive-0.11,but it still shown error informations above.
ON coordinator machine the directory etc and configuration files as below:
=================coordinator================================
----------------config.properties:
coordinator=true
datasources=jmx
http-server.http.port=8080
presto-metastore.db.type=h2
presto-metastore.db.filename=/home/hadoop/cloudera-5.0.0/presto-0.55/presto/db/MetaStore
task.max-memory=1GB
discovery-server.enabled=true
discovery.uri=http://name:8080
--------------jvm.config:
-server
-Xmx4G
-XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent
-XX:+CMSClassUnloadingEnabled
-XX:+AggressiveOpts
-XX:+HeapDumpOnOutOfMemoryError
-XX:OnOutOfMemoryError=kill -9 %p
-XX:PermSize=150M
-XX:MaxPermSize=150M
-XX:ReservedCodeCacheSize=150M
-Xbootclasspath/p:/home/hadoop/cloudera-5.0.0/presto-0.55/presto-server-0.55/lib/floatingdecimal-0.1.jar
-----------------log.properties:
com.facebook.presto=DEBUG
-----------------node.properties:
node.environment=production
#node.id=0699bf8f-ac4e-48f4-92a9-28c0d8862923
node.id=name
node.data-dir=/home/hadoop/cloudera-5.0.0/presto-0.55/presto/data
---------------hive.properties:
connector.name=hive-cdh4
hive.metastore.uri=thrift://name:10000
---------------jmx.properties:
connector.name=jmx
=================coordinator================================
=================woker====================================
----------------config.properties:
coordinator=false
datasources=jmx,hive
http-server.http.port=8080
presto-metastore.db.type=h2
presto-metastore.db.filename=/home/hadoop/cloudera-5.0.0/presto-0.55/presto/db/MetaStore
task.max-memory=1GB
discovery-server.enabled=true
discovery.uri=http://name:8080
--------------jvm.config:
-server
-Xmx4G
-XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent
-XX:+CMSClassUnloadingEnabled
-XX:+AggressiveOpts
-XX:+HeapDumpOnOutOfMemoryError
-XX:OnOutOfMemoryError=kill -9 %p
-XX:PermSize=150M
-XX:MaxPermSize=150M
-XX:ReservedCodeCacheSize=150M
-Xbootclasspath/p:/home/hadoop/cloudera-5.0.0/presto-0.55/presto-server-0.55/lib/floatingdecimal-0.1.jar
-----------------log.properties:
com.facebook.presto=DEBUG
-----------------node.properties:
node.environment=production
#node.id=773a6b4e-9fd0-4342-8d96-59d1f58ac7cd
node.id=data1
node.data-dir=/home/hadoop/cloudera-5.0.0/presto-0.55/presto/data
---------------hive.properties:
connector.name=hive-cdh4
hive.metastore.uri=thrift://name:10000
---------------jmx.properties:
connector.name=jmx
=================woker====================================
=================discovery-server===========================
--------------config.properties:
http-server.http.port=8411
discovery.uri=http://name:8080
--------------node.properties:
node.environment=production
#node.id=b45edeee-4870-420f-919b-9f8487b9750e
#node.id=0699bf8f-ac4e-48f4-92a9-28c0d8862923
node.id=name
node.data-dir=/home/hadoop/cloudera-5.0.0/presto-discovery-data
----------------jvm.config
-server
-Xmx1G
-XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent
-XX:+AggressiveOpts
-XX:+HeapDumpOnOutOfMemoryError
-XX:OnOutOfMemoryError=kill -9 %p
=================discovery-server===========================
I had started hiveserver and coordinator service and discovery-server on name machine,worker on data1 machine, all that ware Normal.When I started to use the command which was as below:
./presto --server name:10000 --catalog hive --schema default
the console of hiveserver that shown :
Exception in thread "pool-1-thread-1" java.lang.OutOfMemoryError: Java heap space
at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:353)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
MeanWhile,I had increased the java heap size of hive to 2g or 4g,however it still throwed exceptions as above.
Please give me some ideas because of you are Professional.
Thanks.