Total jobs = 4 Stage-2 is selected by condition resolver. Launching Job 1 out of 4 Number of reduce tasks not specified. Estimated from input data size: 2 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer= In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapreduce.job.reduces= Starting Job = job_1456406330006_0120, Tracking URL = http://namenode1:8088/proxy/application_1456406330006_0120/ Kill Command = /home/bigdata/software/hadoop/bin/hadoop job -kill job_1456406330006_0120 Hadoop job information for Stage-2: number of mappers: 3; number of reducers: 2 2016-03-08 11:12:39,692 Stage-2 map = 0%, reduce = 0% 2016-03-08 11:13:04,381 Stage-2 map = 22%, reduce = 0%, Cumulative CPU 52.96 sec 2016-03-08 11:13:06,906 Stage-2 map = 24%, reduce = 0%, Cumulative CPU 62.13 sec 2016-03-08 11:13:10,074 Stage-2 map = 57%, reduce = 0%, Cumulative CPU 71.26 sec 2016-03-08 11:13:13,220 Stage-2 map = 61%, reduce = 0%, Cumulative CPU 78.35 sec 2016-03-08 11:13:16,396 Stage-2 map = 66%, reduce = 0%, Cumulative CPU 84.46 sec 2016-03-08 11:13:17,435 Stage-2 map = 67%, reduce = 0%, Cumulative CPU 85.33 sec 2016-03-08 11:13:19,512 Stage-2 map = 82%, reduce = 0%, Cumulative CPU 89.22 sec 2016-03-08 11:13:30,982 Stage-2 map = 89%, reduce = 0%, Cumulative CPU 103.13 sec 2016-03-08 11:13:34,108 Stage-2 map = 91%, reduce = 0%, Cumulative CPU 106.31 sec 2016-03-08 11:13:37,220 Stage-2 map = 94%, reduce = 0%, Cumulative CPU 109.35 sec 2016-03-08 11:13:40,353 Stage-2 map = 96%, reduce = 0%, Cumulative CPU 112.43 sec 2016-03-08 11:13:43,481 Stage-2 map = 99%, reduce = 0%, Cumulative CPU 115.43 sec 2016-03-08 11:13:45,564 Stage-2 map = 100%, reduce = 0%, Cumulative CPU 117.08 sec 2016-03-08 11:13:59,483 Stage-2 map = 100%, reduce = 67%, Cumulative CPU 128.42 sec 2016-03-08 11:14:02,652 Stage-2 map = 100%, reduce = 68%, Cumulative CPU 134.4 sec 2016-03-08 11:14:05,810 Stage-2 map = 100%, reduce = 69%, Cumulative CPU 140.35 sec 2016-03-08 11:14:12,148 Stage-2 map = 100%, reduce = 70%, Cumulative CPU 152.21 sec 2016-03-08 11:14:15,305 Stage-2 map = 100%, reduce = 71%, Cumulative CPU 158.1 sec 2016-03-08 11:14:18,462 Stage-2 map = 100%, reduce = 72%, Cumulative CPU 164.05 sec 2016-03-08 11:14:21,609 Stage-2 map = 100%, reduce = 73%, Cumulative CPU 170.06 sec 2016-03-08 11:14:24,788 Stage-2 map = 100%, reduce = 74%, Cumulative CPU 175.87 sec 2016-03-08 11:14:27,930 Stage-2 map = 100%, reduce = 75%, Cumulative CPU 181.83 sec 2016-03-08 11:14:30,018 Stage-2 map = 100%, reduce = 87%, Cumulative CPU 187.28 sec 2016-03-08 11:14:33,127 Stage-2 map = 100%, reduce = 100%, Cumulative CPU 189.84 sec MapReduce Total cumulative CPU time: 3 minutes 9 seconds 840 msec Ended Job = job_1456406330006_0120 Stage-10 is filtered out by condition resolver. Stage-11 is filtered out by condition resolver. Stage-1 is selected by condition resolver. Launching Job 2 out of 4 Number of reduce tasks not specified. Estimated from input data size: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer= In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapreduce.job.reduces= Starting Job = job_1456406330006_0121, Tracking URL = http://namenode1:8088/proxy/application_1456406330006_0121/ Kill Command = /home/bigdata/software/hadoop/bin/hadoop job -kill job_1456406330006_0121 Hadoop job information for Stage-1: number of mappers: 3; number of reducers: 1 2016-03-08 11:14:46,010 Stage-1 map = 0%, reduce = 0% 2016-03-08 11:15:17,284 Stage-1 map = 22%, reduce = 0%, Cumulative CPU 73.52 sec 2016-03-08 11:15:19,372 Stage-1 map = 36%, reduce = 0%, Cumulative CPU 79.55 sec 2016-03-08 11:15:20,418 Stage-1 map = 38%, reduce = 0%, Cumulative CPU 82.83 sec 2016-03-08 11:15:22,514 Stage-1 map = 60%, reduce = 0%, Cumulative CPU 88.86 sec 2016-03-08 11:15:23,561 Stage-1 map = 65%, reduce = 0%, Cumulative CPU 91.91 sec 2016-03-08 11:15:25,652 Stage-1 map = 70%, reduce = 0%, Cumulative CPU 100.69 sec 2016-03-08 11:15:28,790 Stage-1 map = 73%, reduce = 0%, Cumulative CPU 106.75 sec 2016-03-08 11:15:31,918 Stage-1 map = 78%, reduce = 0%, Cumulative CPU 112.85 sec 2016-03-08 11:15:32,955 Stage-1 map = 81%, reduce = 0%, Cumulative CPU 114.28 sec 2016-03-08 11:15:35,039 Stage-1 map = 89%, reduce = 0%, Cumulative CPU 117.28 sec 2016-03-08 11:15:38,150 Stage-1 map = 90%, reduce = 0%, Cumulative CPU 120.49 sec 2016-03-08 11:15:41,258 Stage-1 map = 92%, reduce = 0%, Cumulative CPU 123.52 sec 2016-03-08 11:15:43,324 Stage-1 map = 95%, reduce = 0%, Cumulative CPU 126.55 sec 2016-03-08 11:15:46,426 Stage-1 map = 99%, reduce = 0%, Cumulative CPU 129.6 sec 2016-03-08 11:15:48,493 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 131.03 sec 2016-03-08 11:16:00,943 Stage-1 map = 100%, reduce = 67%, Cumulative CPU 139.38 sec 2016-03-08 11:16:32,038 Stage-1 map = 100%, reduce = 68%, Cumulative CPU 170.06 sec 2016-03-08 11:17:08,228 Stage-1 map = 100%, reduce = 69%, Cumulative CPU 206.62 sec 2016-03-08 11:17:26,806 Stage-1 map = 100%, reduce = 70%, Cumulative CPU 224.93 sec 2016-03-08 11:17:44,321 Stage-1 map = 100%, reduce = 71%, Cumulative CPU 243.26 sec 2016-03-08 11:17:59,795 Stage-1 map = 100%, reduce = 72%, Cumulative CPU 258.61 sec 2016-03-08 11:18:15,236 Stage-1 map = 100%, reduce = 73%, Cumulative CPU 273.85 sec 2016-03-08 11:18:31,803 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 290.39 sec MapReduce Total cumulative CPU time: 4 minutes 50 seconds 390 msec Ended Job = job_1456406330006_0121 MapReduce Jobs Launched: Job 0: Map: 3 Reduce: 2 Cumulative CPU: 189.84 sec HDFS Read: 395481001 HDFS Write: 46672295 SUCCESS Job 1: Map: 3 Reduce: 1 Cumulative CPU: 290.39 sec HDFS Read: 258625897 HDFS Write: 34464801 SUCCESS Total MapReduce CPU Time Spent: 8 minutes 0 seconds 230 msec OK Time taken: 368.879 seconds, Fetched: 174099803 row(s)