Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.3.0
-
None
-
None
-
None
Description
if i run "select * from table limit 10" in the hive, it can return successful. but when i was run "select max() from table" ,it is hang.
the output massage :-----------------------------------
Logging initialized using configuration in file:/app/apache-hive-2.3.0-bin/conf/hive-log4j2.properties Async: true
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Cannot run job locally: Input Size (= 562011785) is larger than hive.exec.mode.local.auto.inputbytes.max (= 256000000)。
The hive.log :---------------------------------------------------------------------
========"new HiveConf()"========
2018-09-27T08:55:46,192 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: lockmgr.DbLockManager (DbLockManager.java:lock(104)) - Response to queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 LockResponse(lockid:19039, state:ACQUIRED)
2018-09-27T08:55:46,193 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: lockmgr.DbTxnManager (DbTxnManager.java:startHeartbeat(507)) - Started heartbeat with delay/interval = 150000/150000 MILLISECONDS for query: root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1
2018-09-27T08:55:46,193 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: ql.Driver (Driver.java:execute(1734)) - Executing command(queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1): select count
from tick2
2018-09-27T08:55:46,195 WARN [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: ql.Driver (Driver.java:logMrWarning(2094)) - Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
2018-09-27T08:55:46,195 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: ql.Driver (SessionState.java:printInfo(1111)) - WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
2018-09-27T08:55:46,197 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: ql.Driver (SessionState.java:printInfo(1111)) - Query ID = root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1
2018-09-27T08:55:46,197 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: ql.Driver (SessionState.java:printInfo(1111)) - Total jobs = 1
2018-09-27T08:55:46,209 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: ql.Driver (SessionState.java:printInfo(1111)) - Launching Job 1 out of 1
2018-09-27T08:55:46,211 INFO [5cf99469-4f8e-49b1-9108-b94b3928a2df main]: ql.Driver (Driver.java:launchTask(2174)) - Starting task [Stage-1:MAPRED] in parallel
2018-09-27T08:55:46,212 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - Number of reduce tasks determined at compile time: 1
2018-09-27T08:55:46,212 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - In order to change the average load for a reducer (in bytes):
2018-09-27T08:55:46,212 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - set hive.exec.reducers.bytes.per.reducer=<number>
2018-09-27T08:55:46,212 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - In order to limit the maximum number of reducers:
2018-09-27T08:55:46,212 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - set hive.exec.reducers.max=<number>
2018-09-27T08:55:46,212 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - In order to set a constant number of reducers:
2018-09-27T08:55:46,212 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - set mapreduce.job.reduces=<number>
2018-09-27T08:55:46,225 INFO [Thread-6]: exec.Task (SessionState.java:printInfo(1111)) - Cannot run job locally: Input Size (= 562011785) is larger than hive.exec.mode.local.auto.inputbytes.max (= 256000000)
2018-09-27T08:55:46,231 INFO [Thread-6]: ql.Context (Context.java:getMRScratchDir(454)) - New scratch dir is hdfs://cluster/app/apache-hive-2.3.0-bin/tmp/hive-root/root/5cf99469-4f8e-49b1-9108-b94b3928a2df/hive_2018-09-27_08-55-41_865_3811548964141108768-2
2018-09-27T08:55:46,237 INFO [Thread-6]: mr.ExecDriver (ExecDriver.java:execute(287)) - Using org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
2018-09-27T08:55:46,241 INFO [Thread-6]: exec.Utilities (Utilities.java:getInputPaths(3023)) - Processing alias tick2
2018-09-27T08:55:46,241 INFO [Thread-6]: exec.Utilities (Utilities.java:getInputPaths(3052)) - Adding 1 inputs; the first input is hdfs://cluster/usr/hive/warehouse/tick2
2018-09-27T08:55:46,242 INFO [Thread-6]: exec.Utilities (Utilities.java:isEmptyPath(2332)) - Content Summary hdfs://cluster/usr/hive/warehouse/tick2length: 562011785 num files: 1 num directories: 1
2018-09-27T08:55:46,242 INFO [Thread-6]: ql.Context (Context.java:getMRScratchDir(454)) - New scratch dir is hdfs://cluster/app/apache-hive-2.3.0-bin/tmp/hive-root/root/5cf99469-4f8e-49b1-9108-b94b3928a2df/hive_2018-09-27_08-55-41_865_3811548964141108768-2
2018-09-27T08:55:46,331 INFO [Thread-6]: exec.SerializationUtilities (SerializationUtilities.java:serializePlan(450)) - Serializing MapWork using kryo
2018-09-27T08:55:46,471 INFO [Thread-6]: Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1173)) - mapred.submit.replication is deprecated. Instead, use mapreduce.client.submit.file.replication
2018-09-27T08:55:46,476 INFO [Thread-6]: exec.Utilities (Utilities.java:setBaseWork(616)) - Serialized plan (via FILE) - name: null size: 6.48KB
2018-09-27T08:55:46,479 INFO [Thread-6]: exec.SerializationUtilities (SerializationUtilities.java:serializePlan(450)) - Serializing ReduceWork using kryo
2018-09-27T08:55:46,499 INFO [Thread-6]: exec.Utilities (Utilities.java:setBaseWork(616)) - Serialized plan (via FILE) - name: null size: 5.67KB
2018-09-27T08:55:46,736 INFO [Thread-6]: exec.Utilities (Utilities.java:getBaseWork(407)) - PLAN PATH = hdfs://cluster/app/apache-hive-2.3.0-bin/tmp/hive-root/root/5cf99469-4f8e-49b1-9108-b94b3928a2df/hive_2018-09-27_08-55-41_865_3811548964141108768-2/-mr-10005/0be9de35-7cf8-4896-8496-964bbeebc9c8/map.xml
2018-09-27T08:55:46,736 INFO [Thread-6]: exec.Utilities (Utilities.java:getBaseWork(407)) - PLAN PATH = hdfs://cluster/app/apache-hive-2.3.0-bin/tmp/hive-root/root/5cf99469-4f8e-49b1-9108-b94b3928a2df/hive_2018-09-27_08-55-41_865_3811548964141108768-2/-mr-10005/0be9de35-7cf8-4896-8496-964bbeebc9c8/reduce.xml
2018-09-27T08:58:16,195 INFO [Heartbeater-0]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:00:46,194 INFO [Heartbeater-0]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:03:16,194 INFO [Heartbeater-1]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:05:46,193 INFO [Heartbeater-0]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:08:16,194 INFO [Heartbeater-2]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:10:46,198 INFO [Heartbeater-2]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:13:16,193 INFO [Heartbeater-2]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:15:46,194 INFO [Heartbeater-2]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:18:16,193 INFO [Heartbeater-2]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0
2018-09-27T09:20:46,194 INFO [Heartbeater-2]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(455)) - Sending heartbeat for txnid:0 and lockid:19039 queryId=root_20180927085541_71b0e652-ad29-4a2f-ac53-1e9a4f34cec1 txnid:0