Total jobs = 8 Stage-2 is selected by condition resolver. Stage-18 is filtered out by condition resolver. Stage-19 is filtered out by condition resolver. Stage-1 is selected by condition resolver. Launching Job 1 out of 8 Launching Job 2 out of 8 Number of reduce tasks not specified. Estimated from input data size: 2 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer= In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapreduce.job.reduces= Number of reduce tasks not specified. Estimated from input data size: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer= In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapreduce.job.reduces= Starting Job = job_1456406330006_0117, Tracking URL = http://namenode1:8088/proxy/application_1456406330006_0117/ Kill Command = /home/bigdata/software/hadoop/bin/hadoop job -kill job_1456406330006_0117 Starting Job = job_1456406330006_0118, Tracking URL = http://namenode1:8088/proxy/application_1456406330006_0118/ Kill Command = /home/bigdata/software/hadoop/bin/hadoop job -kill job_1456406330006_0118 Hadoop job information for Stage-2: number of mappers: 3; number of reducers: 2 2016-03-08 10:59:13,468 Stage-2 map = 0%, reduce = 0% Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2016-03-08 10:59:15,208 Stage-1 map = 0%, reduce = 0% 2016-03-08 10:59:24,900 Stage-1 map = 50%, reduce = 0%, Cumulative CPU 1.18 sec 2016-03-08 10:59:33,746 Stage-2 map = 22%, reduce = 0%, Cumulative CPU 38.73 sec 2016-03-08 10:59:36,891 Stage-2 map = 23%, reduce = 0%, Cumulative CPU 47.46 sec 2016-03-08 10:59:40,041 Stage-2 map = 32%, reduce = 0%, Cumulative CPU 55.67 sec 2016-03-08 10:59:41,078 Stage-2 map = 33%, reduce = 0%, Cumulative CPU 59.05 sec 2016-03-08 10:59:41,800 Stage-1 map = 71%, reduce = 0%, Cumulative CPU 22.31 sec 2016-03-08 10:59:47,332 Stage-2 map = 56%, reduce = 0%, Cumulative CPU 71.82 sec 2016-03-08 10:59:49,440 Stage-2 map = 71%, reduce = 0%, Cumulative CPU 75.79 sec 2016-03-08 10:59:50,480 Stage-2 map = 73%, reduce = 0%, Cumulative CPU 78.89 sec 2016-03-08 10:59:53,628 Stage-2 map = 77%, reduce = 0%, Cumulative CPU 85.14 sec 2016-03-08 10:59:56,747 Stage-2 map = 82%, reduce = 0%, Cumulative CPU 92.18 sec 2016-03-08 10:59:57,441 Stage-1 map = 83%, reduce = 0%, Cumulative CPU 36.69 sec 2016-03-08 11:00:02,671 Stage-1 map = 86%, reduce = 0%, Cumulative CPU 42.47 sec 2016-03-08 11:00:04,046 Stage-2 map = 89%, reduce = 0%, Cumulative CPU 102.42 sec 2016-03-08 11:00:05,802 Stage-1 map = 91%, reduce = 0%, Cumulative CPU 45.04 sec 2016-03-08 11:00:07,180 Stage-2 map = 91%, reduce = 0%, Cumulative CPU 105.65 sec 2016-03-08 11:00:08,937 Stage-1 map = 97%, reduce = 0%, Cumulative CPU 47.9 sec 2016-03-08 11:00:10,294 Stage-2 map = 93%, reduce = 0%, Cumulative CPU 108.66 sec 2016-03-08 11:00:11,034 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 49.35 sec 2016-03-08 11:00:13,390 Stage-2 map = 96%, reduce = 0%, Cumulative CPU 111.75 sec 2016-03-08 11:00:16,489 Stage-2 map = 98%, reduce = 0%, Cumulative CPU 114.8 sec 2016-03-08 11:00:19,588 Stage-2 map = 100%, reduce = 0%, Cumulative CPU 117.25 sec 2016-03-08 11:00:23,525 Stage-1 map = 100%, reduce = 68%, Cumulative CPU 56.19 sec 2016-03-08 11:00:26,744 Stage-1 map = 100%, reduce = 69%, Cumulative CPU 59.25 sec 2016-03-08 11:00:29,948 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 62.52 sec MapReduce Total cumulative CPU time: 1 minutes 2 seconds 520 msec Ended Job = job_1456406330006_0118 2016-03-08 11:00:33,440 Stage-2 map = 100%, reduce = 67%, Cumulative CPU 128.54 sec 2016-03-08 11:00:36,601 Stage-2 map = 100%, reduce = 68%, Cumulative CPU 134.52 sec 2016-03-08 11:00:39,761 Stage-2 map = 100%, reduce = 69%, Cumulative CPU 140.18 sec 2016-03-08 11:00:46,089 Stage-2 map = 100%, reduce = 70%, Cumulative CPU 151.97 sec 2016-03-08 11:00:49,245 Stage-2 map = 100%, reduce = 71%, Cumulative CPU 157.93 sec 2016-03-08 11:00:52,402 Stage-2 map = 100%, reduce = 72%, Cumulative CPU 163.94 sec 2016-03-08 11:00:55,551 Stage-2 map = 100%, reduce = 73%, Cumulative CPU 169.81 sec 2016-03-08 11:00:58,695 Stage-2 map = 100%, reduce = 74%, Cumulative CPU 175.78 sec 2016-03-08 11:01:03,961 Stage-2 map = 100%, reduce = 75%, Cumulative CPU 184.59 sec 2016-03-08 11:01:06,056 Stage-2 map = 100%, reduce = 100%, Cumulative CPU 190.86 sec MapReduce Total cumulative CPU time: 3 minutes 10 seconds 860 msec Ended Job = job_1456406330006_0117 Stage-22 is filtered out by condition resolver. Stage-1 is selected by condition resolver. Launching Job 3 out of 8 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer= In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapreduce.job.reduces= Starting Job = job_1456406330006_0119, Tracking URL = http://namenode1:8088/proxy/application_1456406330006_0119/ Kill Command = /home/bigdata/software/hadoop/bin/hadoop job -kill job_1456406330006_0119 Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2016-03-08 11:01:20,939 Stage-1 map = 0%, reduce = 0% 2016-03-08 11:01:30,517 Stage-1 map = 50%, reduce = 0%, Cumulative CPU 1.08 sec 2016-03-08 11:01:47,229 Stage-1 map = 71%, reduce = 0%, Cumulative CPU 24.26 sec 2016-03-08 11:01:59,691 Stage-1 map = 83%, reduce = 0%, Cumulative CPU 38.07 sec 2016-03-08 11:02:02,795 Stage-1 map = 85%, reduce = 0%, Cumulative CPU 41.33 sec 2016-03-08 11:02:05,900 Stage-1 map = 90%, reduce = 0%, Cumulative CPU 44.37 sec 2016-03-08 11:02:08,993 Stage-1 map = 98%, reduce = 0%, Cumulative CPU 47.39 sec 2016-03-08 11:02:10,028 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 48.77 sec 2016-03-08 11:02:22,570 Stage-1 map = 100%, reduce = 68%, Cumulative CPU 56.05 sec 2016-03-08 11:02:25,669 Stage-1 map = 100%, reduce = 69%, Cumulative CPU 59.12 sec 2016-03-08 11:02:27,746 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 61.75 sec MapReduce Total cumulative CPU time: 1 minutes 1 seconds 750 msec Ended Job = job_1456406330006_0119 MapReduce Jobs Launched: Job 0: Map: 2 Reduce: 1 Cumulative CPU: 62.52 sec HDFS Read: 211953231 HDFS Write: 8 SUCCESS Job 1: Map: 3 Reduce: 2 Cumulative CPU: 190.86 sec HDFS Read: 395481001 HDFS Write: 46672295 SUCCESS Job 2: Map: 2 Reduce: 1 Cumulative CPU: 61.75 sec HDFS Read: 211953231 HDFS Write: 8 SUCCESS Total MapReduce CPU Time Spent: 5 minutes 15 seconds 130 msec OK Time taken: 217.251 seconds