Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetupMethods] 2014-03-06 18:59:44,781 WARN [pool-1-thread-1] conf.Configuration (Configuration.java:(611)) - DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively 2014-03-06 18:59:45,981 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.tasktracker.map.tasks.maximum is deprecated. Instead, use mapreduce.tasktracker.map.tasks.maximum 2014-03-06 18:59:45,981 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.tasktracker.reduce.tasks.maximum is deprecated. Instead, use mapreduce.tasktracker.reduce.tasks.maximum SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/Users/rkanter/.m2/repository/org/slf4j/slf4j-simple/1.7.5/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/Users/rkanter/.m2/repository/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory] 2014-03-06 18:59:46,374 WARN [pool-1-thread-1] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Formatting using clusterid: testClusterID 2014-03-06 18:59:46,683 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(676)) - fsLock is fair:true 2014-03-06 18:59:46,730 INFO [pool-1-thread-1] namenode.HostFileManager (HostFileManager.java:refresh(304)) - read includes: HostSet( ) 2014-03-06 18:59:46,730 INFO [pool-1-thread-1] namenode.HostFileManager (HostFileManager.java:refresh(311)) - read excludes: HostSet( ) 2014-03-06 18:59:46,731 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping 2014-03-06 18:59:46,732 INFO [pool-1-thread-1] blockmanagement.DatanodeManager (DatanodeManager.java:(229)) - dfs.block.invalidate.limit=1000 2014-03-06 18:59:46,732 INFO [pool-1-thread-1] blockmanagement.DatanodeManager (DatanodeManager.java:(235)) - dfs.namenode.datanode.registration.ip-hostname-check=true 2014-03-06 18:59:46,735 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map BlocksMap 2014-03-06 18:59:46,736 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:46,738 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 2.0% max memory 910.5 MB = 18.2 MB 2014-03-06 18:59:46,738 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^21 = 2097152 entries 2014-03-06 18:59:46,752 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:createBlockTokenSecretManager(348)) - dfs.block.access.token.enable=false 2014-03-06 18:59:46,752 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(333)) - defaultReplication = 2 2014-03-06 18:59:46,754 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(334)) - maxReplication = 512 2014-03-06 18:59:46,755 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(335)) - minReplication = 1 2014-03-06 18:59:46,755 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(336)) - maxReplicationStreams = 2 2014-03-06 18:59:46,755 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(337)) - shouldCheckForEnoughRacks = false 2014-03-06 18:59:46,755 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(338)) - replicationRecheckInterval = 3000 2014-03-06 18:59:46,755 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(339)) - encryptDataTransfer = false 2014-03-06 18:59:46,755 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(340)) - maxNumBlocksToLog = 1000 2014-03-06 18:59:46,763 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(693)) - fsOwner = rkanter (auth:SIMPLE) 2014-03-06 18:59:46,764 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(694)) - supergroup = supergroup 2014-03-06 18:59:46,764 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(695)) - isPermissionEnabled = true 2014-03-06 18:59:46,764 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(706)) - HA Enabled: false 2014-03-06 18:59:46,768 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(743)) - Append Enabled: true 2014-03-06 18:59:46,992 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map INodeMap 2014-03-06 18:59:46,992 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:46,993 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 1.0% max memory 910.5 MB = 9.1 MB 2014-03-06 18:59:46,993 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^20 = 1048576 entries 2014-03-06 18:59:46,994 INFO [pool-1-thread-1] namenode.NameNode (FSDirectory.java:(195)) - Caching file names occuring more than 10 times 2014-03-06 18:59:47,003 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map cachedBlocks 2014-03-06 18:59:47,003 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:47,004 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 0.25% max memory 910.5 MB = 2.3 MB 2014-03-06 18:59:47,004 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^18 = 262144 entries 2014-03-06 18:59:47,006 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(4668)) - dfs.namenode.safemode.threshold-pct = 0.9990000128746033 2014-03-06 18:59:47,006 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(4669)) - dfs.namenode.safemode.min.datanodes = 0 2014-03-06 18:59:47,006 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(4670)) - dfs.namenode.safemode.extension = 0 2014-03-06 18:59:47,008 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:initRetryCache(815)) - Retry cache on namenode is enabled 2014-03-06 18:59:47,008 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:initRetryCache(823)) - Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis 2014-03-06 18:59:47,012 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map Namenode Retry Cache 2014-03-06 18:59:47,012 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:47,012 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 0.029999999329447746% max memory 910.5 MB = 279.7 KB 2014-03-06 18:59:47,012 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^15 = 32768 entries 2014-03-06 18:59:47,016 INFO [pool-1-thread-1] namenode.AclConfigFlag (AclConfigFlag.java:(40)) - ACLs enabled? false 2014-03-06 18:59:47,097 INFO [pool-1-thread-1] namenode.FSImage (FSImage.java:format(148)) - Allocated new BlockPoolId: BP-1007909283-172.16.1.64-1394161187028 2014-03-06 18:59:47,109 INFO [pool-1-thread-1] common.Storage (NNStorage.java:format(550)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/name1 has been successfully formatted. 2014-03-06 18:59:47,110 INFO [pool-1-thread-1] common.Storage (NNStorage.java:format(550)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/name2 has been successfully formatted. 2014-03-06 18:59:47,242 INFO [pool-1-thread-1] namenode.NNStorageRetentionManager (NNStorageRetentionManager.java:getImageTxIdToRetain(201)) - Going to retain 1 images with txid >= 0 2014-03-06 18:59:47,244 INFO [pool-1-thread-1] namenode.NameNode (NameNode.java:createNameNode(1226)) - createNameNode [] 2014-03-06 18:59:47,287 WARN [pool-1-thread-1] impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2014-03-06 18:59:47,381 INFO [pool-1-thread-1] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(344)) - Scheduled snapshot period at 10 second(s). 2014-03-06 18:59:47,381 INFO [pool-1-thread-1] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(183)) - NameNode metrics system started 2014-03-06 18:59:47,425 INFO [pool-1-thread-1] hdfs.DFSUtil (DFSUtil.java:httpServerTemplateForNNAndJN(1655)) - Starting web server as: ${dfs.web.authentication.kerberos.principal} 2014-03-06 18:59:47,425 INFO [pool-1-thread-1] hdfs.DFSUtil (DFSUtil.java:httpServerTemplateForNNAndJN(1666)) - Starting Web-server for hdfs at: http://localhost:0 [pool-1-thread-1] INFO org.mortbay.log - Logging to org.slf4j.impl.SimpleLogger(org.mortbay.log) via org.mortbay.log.Slf4jLog 2014-03-06 18:59:47,498 INFO [pool-1-thread-1] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.namenode is not defined 2014-03-06 18:59:47,514 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addGlobalFilter(658)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2014-03-06 18:59:47,517 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(636)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs 2014-03-06 18:59:47,517 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2014-03-06 18:59:47,517 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2014-03-06 18:59:47,553 INFO [pool-1-thread-1] http.HttpServer2 (NameNodeHttpServer.java:initWebHdfs(86)) - Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter) 2014-03-06 18:59:47,555 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addJerseyResourcePackage(562)) - addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/* 2014-03-06 18:59:47,584 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:openListeners(846)) - Jetty bound to port 52680 [pool-1-thread-1] INFO org.mortbay.log - jetty-6.1.14 [pool-1-thread-1] INFO org.mortbay.log - Extract jar:file:/Users/rkanter/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.3.0-cdh5.1.0-SNAPSHOT/hadoop-hdfs-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar!/webapps/hdfs to /var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/Jetty_localhost_52680_hdfs____.i9xv8l/webapp [pool-1-thread-1] WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter - 'signature.secret' configuration not set, using a random value as secret [pool-1-thread-1] INFO org.mortbay.log - Started SelectChannelConnector@localhost:52680 2014-03-06 18:59:47,875 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(676)) - fsLock is fair:true 2014-03-06 18:59:47,875 INFO [pool-1-thread-1] namenode.HostFileManager (HostFileManager.java:refresh(304)) - read includes: HostSet( ) 2014-03-06 18:59:47,876 INFO [pool-1-thread-1] namenode.HostFileManager (HostFileManager.java:refresh(311)) - read excludes: HostSet( ) 2014-03-06 18:59:47,876 INFO [pool-1-thread-1] blockmanagement.DatanodeManager (DatanodeManager.java:(229)) - dfs.block.invalidate.limit=1000 2014-03-06 18:59:47,876 INFO [pool-1-thread-1] blockmanagement.DatanodeManager (DatanodeManager.java:(235)) - dfs.namenode.datanode.registration.ip-hostname-check=true 2014-03-06 18:59:47,876 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map BlocksMap 2014-03-06 18:59:47,876 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:47,877 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 2.0% max memory 910.5 MB = 18.2 MB 2014-03-06 18:59:47,877 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^21 = 2097152 entries 2014-03-06 18:59:47,882 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:createBlockTokenSecretManager(348)) - dfs.block.access.token.enable=false 2014-03-06 18:59:47,883 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(333)) - defaultReplication = 2 2014-03-06 18:59:47,883 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(334)) - maxReplication = 512 2014-03-06 18:59:47,883 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(335)) - minReplication = 1 2014-03-06 18:59:47,883 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(336)) - maxReplicationStreams = 2 2014-03-06 18:59:47,884 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(337)) - shouldCheckForEnoughRacks = false 2014-03-06 18:59:47,884 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(338)) - replicationRecheckInterval = 3000 2014-03-06 18:59:47,884 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(339)) - encryptDataTransfer = false 2014-03-06 18:59:47,884 INFO [pool-1-thread-1] blockmanagement.BlockManager (BlockManager.java:(340)) - maxNumBlocksToLog = 1000 2014-03-06 18:59:47,885 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(693)) - fsOwner = rkanter (auth:SIMPLE) 2014-03-06 18:59:47,885 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(694)) - supergroup = supergroup 2014-03-06 18:59:47,885 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(695)) - isPermissionEnabled = true 2014-03-06 18:59:47,886 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(706)) - HA Enabled: false 2014-03-06 18:59:47,887 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(743)) - Append Enabled: true 2014-03-06 18:59:47,887 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map INodeMap 2014-03-06 18:59:47,887 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:47,888 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 1.0% max memory 910.5 MB = 9.1 MB 2014-03-06 18:59:47,888 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^20 = 1048576 entries 2014-03-06 18:59:47,894 INFO [pool-1-thread-1] namenode.NameNode (FSDirectory.java:(195)) - Caching file names occuring more than 10 times 2014-03-06 18:59:47,895 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map cachedBlocks 2014-03-06 18:59:47,895 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:47,895 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 0.25% max memory 910.5 MB = 2.3 MB 2014-03-06 18:59:47,895 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^18 = 262144 entries 2014-03-06 18:59:47,896 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(4668)) - dfs.namenode.safemode.threshold-pct = 0.9990000128746033 2014-03-06 18:59:47,896 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(4669)) - dfs.namenode.safemode.min.datanodes = 0 2014-03-06 18:59:47,897 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:(4670)) - dfs.namenode.safemode.extension = 0 2014-03-06 18:59:47,897 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:initRetryCache(815)) - Retry cache on namenode is enabled 2014-03-06 18:59:47,897 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:initRetryCache(823)) - Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis 2014-03-06 18:59:47,897 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map Namenode Retry Cache 2014-03-06 18:59:47,897 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:47,898 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 0.029999999329447746% max memory 910.5 MB = 279.7 KB 2014-03-06 18:59:47,898 INFO [pool-1-thread-1] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^15 = 32768 entries 2014-03-06 18:59:47,898 INFO [pool-1-thread-1] namenode.AclConfigFlag (AclConfigFlag.java:(40)) - ACLs enabled? false 2014-03-06 18:59:47,900 INFO [pool-1-thread-1] common.Storage (Storage.java:tryLock(702)) - Lock on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/name1/in_use.lock acquired by nodename 21390@rkanter-mbp.local 2014-03-06 18:59:47,902 INFO [pool-1-thread-1] common.Storage (Storage.java:tryLock(702)) - Lock on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/name2/in_use.lock acquired by nodename 21390@rkanter-mbp.local 2014-03-06 18:59:47,905 INFO [pool-1-thread-1] namenode.FileJournalManager (FileJournalManager.java:recoverUnfinalizedSegments(322)) - Recovering unfinalized segments in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/name1/current 2014-03-06 18:59:47,905 INFO [pool-1-thread-1] namenode.FileJournalManager (FileJournalManager.java:recoverUnfinalizedSegments(322)) - Recovering unfinalized segments in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/name2/current 2014-03-06 18:59:47,906 INFO [pool-1-thread-1] namenode.FSImage (FSImage.java:loadFSImage(633)) - No edit log streams selected. 2014-03-06 18:59:47,945 INFO [pool-1-thread-1] namenode.FSImageFormatPBINode (FSImageFormatPBINode.java:loadINodeSection(173)) - Loading 1 INodes. 2014-03-06 18:59:47,956 INFO [pool-1-thread-1] namenode.FSImageFormatProtobuf (FSImageFormatProtobuf.java:load(170)) - Loaded FSImage in 0 seconds. 2014-03-06 18:59:47,957 INFO [pool-1-thread-1] namenode.FSImage (FSImage.java:loadFSImage(906)) - Loaded image for txid 0 from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/name1/current/fsimage_0000000000000000000 2014-03-06 18:59:47,961 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:loadFSImage(884)) - Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false) 2014-03-06 18:59:47,961 INFO [pool-1-thread-1] namenode.FSEditLog (FSEditLog.java:startLogSegment(1113)) - Starting log segment at 1 2014-03-06 18:59:48,038 INFO [pool-1-thread-1] namenode.NameCache (NameCache.java:initialized(143)) - initialized with 0 entries 0 lookups 2014-03-06 18:59:48,039 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:loadFromDisk(644)) - Finished loading FSImage in 141 msecs 2014-03-06 18:59:48,279 INFO [pool-1-thread-1] namenode.NameNode (NameNodeRpcServer.java:(283)) - RPC server is binding to localhost:0 2014-03-06 18:59:48,288 INFO [pool-1-thread-1] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 18:59:48,305 INFO [Socket Reader #1 for port 52681] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52681 2014-03-06 18:59:48,341 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:registerMBean(5587)) - Registered FSNamesystemState MBean 2014-03-06 18:59:48,370 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:getCompleteBlocksTotal(5244)) - Number of blocks under construction: 0 2014-03-06 18:59:48,371 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:getCompleteBlocksTotal(5244)) - Number of blocks under construction: 0 2014-03-06 18:59:48,371 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:initializeReplQueues(1059)) - initializing replication queues 2014-03-06 18:59:48,371 INFO [pool-1-thread-1] hdfs.StateChange (FSNamesystem.java:leave(4742)) - STATE* Leaving safe mode after 0 secs 2014-03-06 18:59:48,372 INFO [pool-1-thread-1] hdfs.StateChange (FSNamesystem.java:leave(4753)) - STATE* Network topology has 0 racks and 0 datanodes 2014-03-06 18:59:48,372 INFO [pool-1-thread-1] hdfs.StateChange (FSNamesystem.java:leave(4756)) - STATE* UnderReplicatedBlocks has 0 blocks 2014-03-06 18:59:48,375 INFO [Replication Queue Initializer] blockmanagement.BlockManager (BlockManager.java:processMisReplicatesAsync(2478)) - Total number of blocks = 0 2014-03-06 18:59:48,375 INFO [Replication Queue Initializer] blockmanagement.BlockManager (BlockManager.java:processMisReplicatesAsync(2479)) - Number of invalid blocks = 0 2014-03-06 18:59:48,375 INFO [Replication Queue Initializer] blockmanagement.BlockManager (BlockManager.java:processMisReplicatesAsync(2480)) - Number of under-replicated blocks = 0 2014-03-06 18:59:48,375 INFO [Replication Queue Initializer] blockmanagement.BlockManager (BlockManager.java:processMisReplicatesAsync(2481)) - Number of over-replicated blocks = 0 2014-03-06 18:59:48,375 INFO [Replication Queue Initializer] blockmanagement.BlockManager (BlockManager.java:processMisReplicatesAsync(2483)) - Number of blocks being written = 0 2014-03-06 18:59:48,375 INFO [Replication Queue Initializer] hdfs.StateChange (BlockManager.java:processMisReplicatesAsync(2484)) - STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 4 msec 2014-03-06 18:59:48,406 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 18:59:48,406 INFO [IPC Server listener on 52681] ipc.Server (Server.java:run(644)) - IPC Server listener on 52681: starting 2014-03-06 18:59:48,410 INFO [pool-1-thread-1] namenode.NameNode (NameNode.java:startCommonServices(538)) - NameNode RPC up at: localhost/127.0.0.1:52681 2014-03-06 18:59:48,410 INFO [pool-1-thread-1] namenode.FSNamesystem (FSNamesystem.java:startActiveServices(984)) - Starting services required for active state 2014-03-06 18:59:48,414 INFO [CacheReplicationMonitor(2067277312)] blockmanagement.CacheReplicationMonitor (CacheReplicationMonitor.java:run(159)) - Starting CacheReplicationMonitor with interval 300000 milliseconds 2014-03-06 18:59:48,414 INFO [CacheReplicationMonitor(2067277312)] blockmanagement.CacheReplicationMonitor (CacheReplicationMonitor.java:run(172)) - Rescanning because of pending operations 2014-03-06 18:59:48,415 INFO [CacheReplicationMonitor(2067277312)] blockmanagement.CacheReplicationMonitor (CacheReplicationMonitor.java:run(202)) - Scanned 0 directive(s) and 0 block(s) in 1 millisecond(s). 2014-03-06 18:59:48,820 INFO [pool-1-thread-1] hdfs.MiniDFSCluster (MiniDFSCluster.java:startDataNodes(1263)) - Starting DataNode 0 with dfs.datanode.data.dir: [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1,[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2 2014-03-06 18:59:48,939 INFO [pool-1-thread-1] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DataNode metrics system started (again) 2014-03-06 18:59:48,941 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:(277)) - Configured hostname is 127.0.0.1 2014-03-06 18:59:48,944 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:startDataNode(726)) - Starting DataNode with maxLockedMemory = 0 2014-03-06 18:59:48,958 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:initDataXceiver(526)) - Opened streaming server at /127.0.0.1:52682 2014-03-06 18:59:48,961 INFO [pool-1-thread-1] datanode.DataNode (DataXceiverServer.java:(73)) - Balancing bandwith is 1048576 bytes/s 2014-03-06 18:59:48,971 INFO [pool-1-thread-1] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.datanode is not defined 2014-03-06 18:59:48,971 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addGlobalFilter(658)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2014-03-06 18:59:48,972 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(636)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode 2014-03-06 18:59:48,973 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2014-03-06 18:59:48,973 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2014-03-06 18:59:48,978 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addJerseyResourcePackage(562)) - addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/* 2014-03-06 18:59:48,979 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:openListeners(846)) - Jetty bound to port 52683 [pool-1-thread-1] INFO org.mortbay.log - jetty-6.1.14 [pool-1-thread-1] INFO org.mortbay.log - Extract jar:file:/Users/rkanter/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.3.0-cdh5.1.0-SNAPSHOT/hadoop-hdfs-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar!/webapps/datanode to /var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/Jetty_localhost_52683_datanode____fr1g4b/webapp [pool-1-thread-1] INFO org.mortbay.log - Started SelectChannelConnector@localhost:52683 2014-03-06 18:59:49,107 INFO [pool-1-thread-1] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 18:59:49,108 INFO [Socket Reader #1 for port 52684] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52684 2014-03-06 18:59:49,116 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:initIpcServer(428)) - Opened IPC server at /127.0.0.1:52684 2014-03-06 18:59:49,125 INFO [pool-1-thread-1] datanode.DataNode (BlockPoolManager.java:refreshNamenodes(148)) - Refresh request received for nameservices: null 2014-03-06 18:59:49,127 INFO [pool-1-thread-1] datanode.DataNode (BlockPoolManager.java:doRefreshNamenodes(193)) - Starting BPOfferServices for nameservices: 2014-03-06 18:59:49,139 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:run(807)) - Block pool (Datanode Uuid unassigned) service to localhost/127.0.0.1:52681 starting to offer service 2014-03-06 18:59:49,141 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 18:59:49,142 INFO [IPC Server listener on 52684] ipc.Server (Server.java:run(644)) - IPC Server listener on 52684: starting 2014-03-06 18:59:49,144 INFO [pool-1-thread-1] hdfs.MiniDFSCluster (MiniDFSCluster.java:startDataNodes(1263)) - Starting DataNode 1 with dfs.datanode.data.dir: [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3,[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4 2014-03-06 18:59:49,198 INFO [pool-1-thread-1] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DataNode metrics system started (again) 2014-03-06 18:59:49,198 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:(277)) - Configured hostname is 127.0.0.1 2014-03-06 18:59:49,199 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:startDataNode(726)) - Starting DataNode with maxLockedMemory = 0 2014-03-06 18:59:49,199 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:initDataXceiver(526)) - Opened streaming server at /127.0.0.1:52685 2014-03-06 18:59:49,200 INFO [pool-1-thread-1] datanode.DataNode (DataXceiverServer.java:(73)) - Balancing bandwith is 1048576 bytes/s 2014-03-06 18:59:49,201 INFO [pool-1-thread-1] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.datanode is not defined 2014-03-06 18:59:49,202 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addGlobalFilter(658)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2014-03-06 18:59:49,203 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(636)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode 2014-03-06 18:59:49,203 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2014-03-06 18:59:49,203 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2014-03-06 18:59:49,204 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:addJerseyResourcePackage(562)) - addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.datanode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/* 2014-03-06 18:59:49,205 INFO [pool-1-thread-1] http.HttpServer2 (HttpServer2.java:openListeners(846)) - Jetty bound to port 52686 [pool-1-thread-1] INFO org.mortbay.log - jetty-6.1.14 [pool-1-thread-1] INFO org.mortbay.log - Extract jar:file:/Users/rkanter/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.3.0-cdh5.1.0-SNAPSHOT/hadoop-hdfs-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar!/webapps/datanode to /var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/Jetty_localhost_52686_datanode____16zihq/webapp [pool-1-thread-1] INFO org.mortbay.log - Started SelectChannelConnector@localhost:52686 2014-03-06 18:59:49,314 INFO [pool-1-thread-1] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 18:59:49,315 INFO [Socket Reader #1 for port 52688] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52688 2014-03-06 18:59:49,320 INFO [pool-1-thread-1] datanode.DataNode (DataNode.java:initIpcServer(428)) - Opened IPC server at /127.0.0.1:52688 2014-03-06 18:59:49,327 INFO [pool-1-thread-1] datanode.DataNode (BlockPoolManager.java:refreshNamenodes(148)) - Refresh request received for nameservices: null 2014-03-06 18:59:49,327 INFO [pool-1-thread-1] datanode.DataNode (BlockPoolManager.java:doRefreshNamenodes(193)) - Starting BPOfferServices for nameservices: 2014-03-06 18:59:49,329 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:run(807)) - Block pool (Datanode Uuid unassigned) service to localhost/127.0.0.1:52681 starting to offer service 2014-03-06 18:59:49,329 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 18:59:49,329 INFO [IPC Server listener on 52688] ipc.Server (Server.java:run(644)) - IPC Server listener on 52688: starting 2014-03-06 18:59:49,472 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(173)) - Data-node version: -55 and name-node layout version: -55 2014-03-06 18:59:49,472 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(173)) - Data-node version: -55 and name-node layout version: -55 2014-03-06 18:59:49,473 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:tryLock(702)) - Lock on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/in_use.lock acquired by nodename 21390@rkanter-mbp.local 2014-03-06 18:59:49,473 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:tryLock(702)) - Lock on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/in_use.lock acquired by nodename 21390@rkanter-mbp.local 2014-03-06 18:59:49,474 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(197)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1 is not formatted 2014-03-06 18:59:49,474 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(198)) - Formatting ... 2014-03-06 18:59:49,474 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(197)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3 is not formatted 2014-03-06 18:59:49,474 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(198)) - Formatting ... 2014-03-06 18:59:49,476 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:tryLock(702)) - Lock on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/in_use.lock acquired by nodename 21390@rkanter-mbp.local 2014-03-06 18:59:49,477 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:tryLock(702)) - Lock on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/in_use.lock acquired by nodename 21390@rkanter-mbp.local 2014-03-06 18:59:49,477 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(197)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2 is not formatted 2014-03-06 18:59:49,477 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(197)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4 is not formatted 2014-03-06 18:59:49,477 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(198)) - Formatting ... 2014-03-06 18:59:49,477 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (DataStorage.java:recoverTransitionRead(198)) - Formatting ... 2014-03-06 18:59:49,564 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(106)) - Analyzing storage directories for bpid BP-1007909283-172.16.1.64-1394161187028 2014-03-06 18:59:49,564 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(106)) - Analyzing storage directories for bpid BP-1007909283-172.16.1.64-1394161187028 2014-03-06 18:59:49,564 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:lock(666)) - Locking is disabled 2014-03-06 18:59:49,564 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:lock(666)) - Locking is disabled 2014-03-06 18:59:49,565 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(128)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028 is not formatted. 2014-03-06 18:59:49,565 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(128)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028 is not formatted. 2014-03-06 18:59:49,565 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(129)) - Formatting ... 2014-03-06 18:59:49,566 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:format(181)) - Formatting block pool BP-1007909283-172.16.1.64-1394161187028 directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current 2014-03-06 18:59:49,565 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(129)) - Formatting ... 2014-03-06 18:59:49,566 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:format(181)) - Formatting block pool BP-1007909283-172.16.1.64-1394161187028 directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current 2014-03-06 18:59:49,566 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:lock(666)) - Locking is disabled 2014-03-06 18:59:49,566 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (Storage.java:lock(666)) - Locking is disabled 2014-03-06 18:59:49,567 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(128)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028 is not formatted. 2014-03-06 18:59:49,567 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(128)) - Storage directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028 is not formatted. 2014-03-06 18:59:49,567 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(129)) - Formatting ... 2014-03-06 18:59:49,567 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:recoverTransitionRead(129)) - Formatting ... 2014-03-06 18:59:49,567 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:format(181)) - Formatting block pool BP-1007909283-172.16.1.64-1394161187028 directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current 2014-03-06 18:59:49,567 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:format(181)) - Formatting block pool BP-1007909283-172.16.1.64-1394161187028 directory /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current 2014-03-06 18:59:49,568 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:doTransition(252)) - Restored 0 block files from trash. 2014-03-06 18:59:49,568 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:doTransition(252)) - Restored 0 block files from trash. 2014-03-06 18:59:49,569 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:doTransition(252)) - Restored 0 block files from trash. 2014-03-06 18:59:49,569 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] common.Storage (BlockPoolSliceStorage.java:doTransition(252)) - Restored 0 block files from trash. 2014-03-06 18:59:49,570 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (DataNode.java:initStorage(926)) - Setting up storage: nsid=330784823;bpid=BP-1007909283-172.16.1.64-1394161187028;lv=-55;nsInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0;bpid=BP-1007909283-172.16.1.64-1394161187028;dnuuid=null 2014-03-06 18:59:49,570 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (DataNode.java:initStorage(926)) - Setting up storage: nsid=330784823;bpid=BP-1007909283-172.16.1.64-1394161187028;lv=-55;nsInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0;bpid=BP-1007909283-172.16.1.64-1394161187028;dnuuid=null 2014-03-06 18:59:49,571 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (DataNode.java:checkDatanodeUuid(766)) - Generated and persisted new Datanode UUID a305cec0-5002-4fa9-8de0-6cd548029493 2014-03-06 18:59:49,571 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (DataNode.java:checkDatanodeUuid(766)) - Generated and persisted new Datanode UUID baef3ef0-1673-49bb-ac59-3f31d58cd534 2014-03-06 18:59:49,588 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:(209)) - Added volume - /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current, StorageType: DISK 2014-03-06 18:59:49,588 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:(209)) - Added volume - /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current, StorageType: DISK 2014-03-06 18:59:49,588 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:(209)) - Added volume - /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current, StorageType: DISK 2014-03-06 18:59:49,589 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:(209)) - Added volume - /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current, StorageType: DISK 2014-03-06 18:59:49,601 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:registerMBean(1363)) - Registered FSDatasetState MBean 2014-03-06 18:59:49,601 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:registerMBean(1363)) - Registered FSDatasetState MBean 2014-03-06 18:59:49,608 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DirectoryScanner (DirectoryScanner.java:start(330)) - Periodic Directory Tree Verification scan starting at 1394179836607 with interval 21600000 2014-03-06 18:59:49,608 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DirectoryScanner (DirectoryScanner.java:start(330)) - Periodic Directory Tree Verification scan starting at 1394161738607 with interval 21600000 2014-03-06 18:59:49,609 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:addBlockPool(1732)) - Adding block pool BP-1007909283-172.16.1.64-1394161187028 2014-03-06 18:59:49,609 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsDatasetImpl.java:addBlockPool(1732)) - Adding block pool BP-1007909283-172.16.1.64-1394161187028 2014-03-06 18:59:49,610 INFO [Thread-91] impl.FsDatasetImpl (FsVolumeList.java:run(207)) - Scanning block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current... 2014-03-06 18:59:49,610 INFO [Thread-92] impl.FsDatasetImpl (FsVolumeList.java:run(207)) - Scanning block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current... 2014-03-06 18:59:49,611 INFO [Thread-93] impl.FsDatasetImpl (FsVolumeList.java:run(207)) - Scanning block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current... 2014-03-06 18:59:49,611 INFO [Thread-94] impl.FsDatasetImpl (FsVolumeList.java:run(207)) - Scanning block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current... 2014-03-06 18:59:49,650 INFO [Thread-92] impl.FsDatasetImpl (FsVolumeList.java:run(212)) - Time taken to scan block pool BP-1007909283-172.16.1.64-1394161187028 on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current: 40ms 2014-03-06 18:59:49,650 INFO [Thread-94] impl.FsDatasetImpl (FsVolumeList.java:run(212)) - Time taken to scan block pool BP-1007909283-172.16.1.64-1394161187028 on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current: 39ms 2014-03-06 18:59:49,650 INFO [Thread-93] impl.FsDatasetImpl (FsVolumeList.java:run(212)) - Time taken to scan block pool BP-1007909283-172.16.1.64-1394161187028 on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current: 39ms 2014-03-06 18:59:49,651 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsVolumeList.java:addBlockPool(236)) - Total time to scan all replicas for block pool BP-1007909283-172.16.1.64-1394161187028: 41ms 2014-03-06 18:59:49,650 INFO [Thread-91] impl.FsDatasetImpl (FsVolumeList.java:run(212)) - Time taken to scan block pool BP-1007909283-172.16.1.64-1394161187028 on /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current: 40ms 2014-03-06 18:59:49,651 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsVolumeList.java:addBlockPool(236)) - Total time to scan all replicas for block pool BP-1007909283-172.16.1.64-1394161187028: 42ms 2014-03-06 18:59:49,652 INFO [Thread-99] impl.FsDatasetImpl (FsVolumeList.java:run(108)) - Adding replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current... 2014-03-06 18:59:49,652 INFO [Thread-100] impl.FsDatasetImpl (FsVolumeList.java:run(108)) - Adding replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current... 2014-03-06 18:59:49,652 INFO [Thread-101] impl.FsDatasetImpl (FsVolumeList.java:run(108)) - Adding replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current... 2014-03-06 18:59:49,652 INFO [Thread-99] impl.FsDatasetImpl (FsVolumeList.java:run(113)) - Time to add replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current: 0ms 2014-03-06 18:59:49,652 INFO [Thread-101] impl.FsDatasetImpl (FsVolumeList.java:run(113)) - Time to add replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current: 0ms 2014-03-06 18:59:49,652 INFO [Thread-102] impl.FsDatasetImpl (FsVolumeList.java:run(108)) - Adding replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current... 2014-03-06 18:59:49,652 INFO [Thread-100] impl.FsDatasetImpl (FsVolumeList.java:run(113)) - Time to add replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current: 0ms 2014-03-06 18:59:49,653 INFO [Thread-102] impl.FsDatasetImpl (FsVolumeList.java:run(113)) - Time to add replicas to map for block pool BP-1007909283-172.16.1.64-1394161187028 on volume /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current: 0ms 2014-03-06 18:59:49,653 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsVolumeList.java:getAllVolumesMap(136)) - Total time to add all replicas to map: 1ms 2014-03-06 18:59:49,653 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] impl.FsDatasetImpl (FsVolumeList.java:getAllVolumesMap(136)) - Total time to add all replicas to map: 2ms 2014-03-06 18:59:49,656 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:register(767)) - Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid null) service to localhost/127.0.0.1:52681 beginning handshake with NN 2014-03-06 18:59:49,656 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:register(767)) - Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid null) service to localhost/127.0.0.1:52681 beginning handshake with NN 2014-03-06 18:59:49,669 INFO [IPC Server handler 3 on 52681] hdfs.StateChange (DatanodeManager.java:registerDatanode(787)) - BLOCK* registerDatanode: from DatanodeRegistration(127.0.0.1, datanodeUuid=a305cec0-5002-4fa9-8de0-6cd548029493, infoPort=52686, ipcPort=52688, storageInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0) storage a305cec0-5002-4fa9-8de0-6cd548029493 2014-03-06 18:59:49,672 INFO [IPC Server handler 3 on 52681] net.NetworkTopology (NetworkTopology.java:add(413)) - Adding a new node: /default-rack/127.0.0.1:52685 2014-03-06 18:59:49,673 INFO [IPC Server handler 4 on 52681] hdfs.StateChange (DatanodeManager.java:registerDatanode(787)) - BLOCK* registerDatanode: from DatanodeRegistration(127.0.0.1, datanodeUuid=baef3ef0-1673-49bb-ac59-3f31d58cd534, infoPort=52683, ipcPort=52684, storageInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0) storage baef3ef0-1673-49bb-ac59-3f31d58cd534 2014-03-06 18:59:49,673 INFO [IPC Server handler 4 on 52681] net.NetworkTopology (NetworkTopology.java:add(413)) - Adding a new node: /default-rack/127.0.0.1:52682 2014-03-06 18:59:49,676 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:register(780)) - Block pool Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid null) service to localhost/127.0.0.1:52681 successfully registered with NN 2014-03-06 18:59:49,676 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:register(780)) - Block pool Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid null) service to localhost/127.0.0.1:52681 successfully registered with NN 2014-03-06 18:59:49,677 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:offerService(639)) - For namenode localhost/127.0.0.1:52681 using DELETEREPORT_INTERVAL of 300000 msec BLOCKREPORT_INTERVAL of 21600000msec CACHEREPORT_INTERVAL of 10000msec Initial delay: 0msec; heartBeatInterval=3000 2014-03-06 18:59:49,677 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:offerService(639)) - For namenode localhost/127.0.0.1:52681 using DELETEREPORT_INTERVAL of 300000 msec BLOCKREPORT_INTERVAL of 21600000msec CACHEREPORT_INTERVAL of 10000msec Initial delay: 0msec; heartBeatInterval=3000 2014-03-06 18:59:49,714 INFO [IPC Server handler 1 on 52681] blockmanagement.DatanodeDescriptor (DatanodeDescriptor.java:updateStorage(670)) - Adding new storage ID DS-9d151ffe-72c8-4f4a-8368-39290a0a904c for DN 127.0.0.1:52682 2014-03-06 18:59:49,714 INFO [IPC Server handler 1 on 52681] blockmanagement.DatanodeDescriptor (DatanodeDescriptor.java:updateStorage(670)) - Adding new storage ID DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6 for DN 127.0.0.1:52682 2014-03-06 18:59:49,714 INFO [IPC Server handler 5 on 52681] blockmanagement.DatanodeDescriptor (DatanodeDescriptor.java:updateStorage(670)) - Adding new storage ID DS-fdea22e5-520f-41a1-ae24-5c89db493ed9 for DN 127.0.0.1:52685 2014-03-06 18:59:49,715 INFO [IPC Server handler 5 on 52681] blockmanagement.DatanodeDescriptor (DatanodeDescriptor.java:updateStorage(670)) - Adding new storage ID DS-2ebb2e2d-4942-4815-af28-c14d569871af for DN 127.0.0.1:52685 2014-03-06 18:59:49,734 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPOfferService.java:updateActorStatesFromHeartbeat(435)) - Namenode Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid baef3ef0-1673-49bb-ac59-3f31d58cd534) service to localhost/127.0.0.1:52681 trying to claim ACTIVE state with txid=1 2014-03-06 18:59:49,734 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPOfferService.java:updateActorStatesFromHeartbeat(435)) - Namenode Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid a305cec0-5002-4fa9-8de0-6cd548029493) service to localhost/127.0.0.1:52681 trying to claim ACTIVE state with txid=1 2014-03-06 18:59:49,734 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPOfferService.java:updateActorStatesFromHeartbeat(447)) - Acknowledging ACTIVE Namenode Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid a305cec0-5002-4fa9-8de0-6cd548029493) service to localhost/127.0.0.1:52681 2014-03-06 18:59:49,734 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPOfferService.java:updateActorStatesFromHeartbeat(447)) - Acknowledging ACTIVE Namenode Block pool BP-1007909283-172.16.1.64-1394161187028 (Datanode Uuid baef3ef0-1673-49bb-ac59-3f31d58cd534) service to localhost/127.0.0.1:52681 2014-03-06 18:59:49,748 INFO [IPC Server handler 8 on 52681] blockmanagement.BlockManager (BlockManager.java:processReport(1699)) - BLOCK* processReport: Received first block report from org.apache.hadoop.hdfs.server.protocol.DatanodeStorage@7d9c4388 after starting up or becoming active. Its block contents are no longer considered stale 2014-03-06 18:59:49,749 INFO [IPC Server handler 8 on 52681] BlockStateChange (BlockManager.java:processReport(1715)) - BLOCK* processReport: from storage DS-9d151ffe-72c8-4f4a-8368-39290a0a904c node DatanodeRegistration(127.0.0.1, datanodeUuid=baef3ef0-1673-49bb-ac59-3f31d58cd534, infoPort=52683, ipcPort=52684, storageInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0), blocks: 0, processing time: 3 msecs 2014-03-06 18:59:49,749 INFO [IPC Server handler 6 on 52681] blockmanagement.BlockManager (BlockManager.java:processReport(1699)) - BLOCK* processReport: Received first block report from org.apache.hadoop.hdfs.server.protocol.DatanodeStorage@cfefe1ed after starting up or becoming active. Its block contents are no longer considered stale 2014-03-06 18:59:49,750 INFO [IPC Server handler 6 on 52681] BlockStateChange (BlockManager.java:processReport(1715)) - BLOCK* processReport: from storage DS-2ebb2e2d-4942-4815-af28-c14d569871af node DatanodeRegistration(127.0.0.1, datanodeUuid=a305cec0-5002-4fa9-8de0-6cd548029493, infoPort=52686, ipcPort=52688, storageInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0), blocks: 0, processing time: 1 msecs 2014-03-06 18:59:49,750 INFO [IPC Server handler 8 on 52681] blockmanagement.BlockManager (BlockManager.java:processReport(1699)) - BLOCK* processReport: Received first block report from org.apache.hadoop.hdfs.server.protocol.DatanodeStorage@db5aa53b after starting up or becoming active. Its block contents are no longer considered stale 2014-03-06 18:59:49,750 INFO [IPC Server handler 8 on 52681] BlockStateChange (BlockManager.java:processReport(1715)) - BLOCK* processReport: from storage DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6 node DatanodeRegistration(127.0.0.1, datanodeUuid=baef3ef0-1673-49bb-ac59-3f31d58cd534, infoPort=52683, ipcPort=52684, storageInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0), blocks: 0, processing time: 0 msecs 2014-03-06 18:59:49,750 INFO [IPC Server handler 6 on 52681] blockmanagement.BlockManager (BlockManager.java:processReport(1699)) - BLOCK* processReport: Received first block report from org.apache.hadoop.hdfs.server.protocol.DatanodeStorage@220c9655 after starting up or becoming active. Its block contents are no longer considered stale 2014-03-06 18:59:49,750 INFO [IPC Server handler 6 on 52681] BlockStateChange (BlockManager.java:processReport(1715)) - BLOCK* processReport: from storage DS-fdea22e5-520f-41a1-ae24-5c89db493ed9 node DatanodeRegistration(127.0.0.1, datanodeUuid=a305cec0-5002-4fa9-8de0-6cd548029493, infoPort=52686, ipcPort=52688, storageInfo=lv=-55;cid=testClusterID;nsid=330784823;c=0), blocks: 0, processing time: 0 msecs 2014-03-06 18:59:49,771 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:blockReport(500)) - Sent 1 blockreports 0 blocks total. Took 2 msec to generate and 33 msecs for RPC and NN processing. Got back commands none 2014-03-06 18:59:49,771 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataNode (BPServiceActor.java:blockReport(500)) - Sent 1 blockreports 0 blocks total. Took 2 msec to generate and 34 msecs for RPC and NN processing. Got back commands none 2014-03-06 18:59:49,775 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map BlockMap 2014-03-06 18:59:49,775 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:49,775 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(354)) - Computing capacity for map BlockMap 2014-03-06 18:59:49,775 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(355)) - VM type = 64-bit 2014-03-06 18:59:49,776 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 0.5% max memory 910.5 MB = 4.6 MB 2014-03-06 18:59:49,776 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^19 = 524288 entries 2014-03-06 18:59:49,776 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(356)) - 0.5% max memory 910.5 MB = 4.6 MB 2014-03-06 18:59:49,803 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:(186)) - Periodic Block Verification Scanner initialized with interval 504 hours for block pool BP-1007909283-172.16.1.64-1394161187028 2014-03-06 18:59:49,803 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] util.GSet (LightWeightGSet.java:computeCapacity(361)) - capacity = 2^19 = 524288 entries 2014-03-06 18:59:49,806 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:(186)) - Periodic Block Verification Scanner initialized with interval 504 hours for block pool BP-1007909283-172.16.1.64-1394161187028 2014-03-06 18:59:49,814 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataBlockScanner (DataBlockScanner.java:addBlockPool(265)) - Added bpid=BP-1007909283-172.16.1.64-1394161187028 to blockPoolScannerMap, new size=1 2014-03-06 18:59:49,814 INFO [DataNode: [[[DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/, [DISK]file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/]] heartbeating to localhost/127.0.0.1:52681] datanode.DataBlockScanner (DataBlockScanner.java:addBlockPool(265)) - Added bpid=BP-1007909283-172.16.1.64-1394161187028 to blockPoolScannerMap, new size=1 2014-03-06 18:59:49,867 INFO [pool-1-thread-1] hdfs.MiniDFSCluster (MiniDFSCluster.java:waitActive(2018)) - Cluster is active 2014-03-06 18:59:49,896 INFO [IPC Server handler 9 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/rkanter/target/test-data dst=null perm=rkanter:supergroup:rwxr-xr-x 2014-03-06 18:59:49,900 INFO [IPC Server handler 0 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/rkanter/target/test-data/minicluster/mapred dst=null perm=rkanter:supergroup:rwxr-xr-x 2014-03-06 18:59:49,901 INFO [IPC Server handler 2 on 52681] namenode.EditLogFileOutputStream (EditLogFileOutputStream.java:flushAndSync(199)) - Nothing to flush 2014-03-06 18:59:49,902 INFO [IPC Server handler 2 on 52681] namenode.EditLogFileOutputStream (EditLogFileOutputStream.java:flushAndSync(199)) - Nothing to flush 2014-03-06 18:59:49,902 INFO [IPC Server handler 2 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user dst=null perm=rkanter:supergroup:rwxr-xr-x 2014-03-06 18:59:49,904 INFO [IPC Server handler 3 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp dst=null perm=rkanter:supergroup:rwxr-xr-x 2014-03-06 18:59:49,906 INFO [IPC Server handler 4 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/hadoop/mapred/system dst=null perm=rkanter:supergroup:rwxr-xr-x 2014-03-06 18:59:49,910 INFO [IPC Server handler 5 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/user/rkanter/target/test-data dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:49,913 INFO [IPC Server handler 1 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/user/rkanter/target/test-data/minicluster dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:49,916 INFO [IPC Server handler 8 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/user/rkanter/target/test-data/minicluster/mapred dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:49,917 INFO [IPC Server handler 6 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/user dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:49,919 INFO [IPC Server handler 7 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:49,922 INFO [IPC Server handler 9 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/hadoop/mapred/system dst=null perm=rkanter:supergroup:rwx------ 2014-03-06 18:59:49,950 INFO [IPC Server handler 0 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 2014-03-06 18:59:49,976 INFO [IPC Server handler 2 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=rkanter:supergroup:rw-r--r-- 2014-03-06 18:59:50,011 INFO [IPC Server handler 3 on 52681] hdfs.StateChange (FSNamesystem.java:saveAllocatedBlock(3026)) - BLOCK* allocateBlock: /user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 2014-03-06 18:59:50,063 INFO [DataXceiver for client DFSClient_NONMAPREDUCE_-2110652364_8 at /127.0.0.1:52690 [Receiving block BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001]] datanode.DataNode (DataXceiver.java:writeBlock(569)) - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001 src: /127.0.0.1:52690 dest: /127.0.0.1:52682 2014-03-06 18:59:50,076 INFO [DataXceiver for client DFSClient_NONMAPREDUCE_-2110652364_8 at /127.0.0.1:52691 [Receiving block BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001]] datanode.DataNode (DataXceiver.java:writeBlock(569)) - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001 src: /127.0.0.1:52691 dest: /127.0.0.1:52685 2014-03-06 18:59:50,139 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, type=LAST_IN_PIPELINE, downstreams=0:[]] DataNode.clienttrace (BlockReceiver.java:finalizeBlock(1215)) - src: /127.0.0.1:52691, dest: /127.0.0.1:52685, bytes: 485190, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, duration: 45221000 2014-03-06 18:59:50,140 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, type=LAST_IN_PIPELINE, downstreams=0:[]] datanode.DataNode (BlockReceiver.java:run(1196)) - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 2014-03-06 18:59:50,140 INFO [IPC Server handler 8 on 52681] BlockStateChange (BlockManager.java:logAddStoredBlock(2339)) - BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52685 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} size 0 2014-03-06 18:59:50,141 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, type=HAS_DOWNSTREAM_IN_PIPELINE] DataNode.clienttrace (BlockReceiver.java:finalizeBlock(1215)) - src: /127.0.0.1:52690, dest: /127.0.0.1:52682, bytes: 485190, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, duration: 47832000 2014-03-06 18:59:50,142 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, type=HAS_DOWNSTREAM_IN_PIPELINE] datanode.DataNode (BlockReceiver.java:run(1196)) - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 2014-03-06 18:59:50,148 INFO [IPC Server handler 6 on 52681] BlockStateChange (BlockManager.java:logAddStoredBlock(2339)) - BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52682 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} size 0 2014-03-06 18:59:50,150 INFO [IPC Server handler 7 on 52681] hdfs.StateChange (FSNamesystem.java:completeFile(2959)) - DIR* completeFile: /user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar is closed by DFSClient_NONMAPREDUCE_-2110652364_8 2014-03-06 18:59:50,155 INFO [IPC Server handler 9 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=rkanter:supergroup:rwxr--r-- 2014-03-06 18:59:50,164 INFO [IPC Server handler 0 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 2014-03-06 18:59:50,167 INFO [IPC Server handler 2 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=rkanter:supergroup:rw-r--r-- 2014-03-06 18:59:50,172 INFO [IPC Server handler 3 on 52681] hdfs.StateChange (FSNamesystem.java:saveAllocatedBlock(3026)) - BLOCK* allocateBlock: /user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741826_1002{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 2014-03-06 18:59:50,175 INFO [DataXceiver for client DFSClient_NONMAPREDUCE_-2110652364_8 at /127.0.0.1:52692 [Receiving block BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002]] datanode.DataNode (DataXceiver.java:writeBlock(569)) - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002 src: /127.0.0.1:52692 dest: /127.0.0.1:52682 2014-03-06 18:59:50,177 INFO [DataXceiver for client DFSClient_NONMAPREDUCE_-2110652364_8 at /127.0.0.1:52693 [Receiving block BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002]] datanode.DataNode (DataXceiver.java:writeBlock(569)) - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002 src: /127.0.0.1:52693 dest: /127.0.0.1:52685 2014-03-06 18:59:50,194 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, type=LAST_IN_PIPELINE, downstreams=0:[]] DataNode.clienttrace (BlockReceiver.java:finalizeBlock(1215)) - src: /127.0.0.1:52693, dest: /127.0.0.1:52685, bytes: 1470916, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, duration: 15007000 2014-03-06 18:59:50,194 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, type=LAST_IN_PIPELINE, downstreams=0:[]] datanode.DataNode (BlockReceiver.java:run(1196)) - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 2014-03-06 18:59:50,195 INFO [IPC Server handler 1 on 52681] BlockStateChange (BlockManager.java:logAddStoredBlock(2339)) - BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52685 is added to blk_1073741826_1002{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} size 0 2014-03-06 18:59:50,196 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, type=HAS_DOWNSTREAM_IN_PIPELINE] DataNode.clienttrace (BlockReceiver.java:finalizeBlock(1215)) - src: /127.0.0.1:52692, dest: /127.0.0.1:52682, bytes: 1470916, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, duration: 16549000 2014-03-06 18:59:50,197 INFO [PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, type=HAS_DOWNSTREAM_IN_PIPELINE] datanode.DataNode (BlockReceiver.java:run(1196)) - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 2014-03-06 18:59:50,198 INFO [IPC Server handler 8 on 52681] BlockStateChange (BlockManager.java:logAddStoredBlock(2339)) - BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52682 is added to blk_1073741826_1002{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} size 0 2014-03-06 18:59:50,206 INFO [IPC Server handler 6 on 52681] hdfs.StateChange (FSNamesystem.java:completeFile(2959)) - DIR* completeFile: /user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar is closed by DFSClient_NONMAPREDUCE_-2110652364_8 2014-03-06 18:59:50,209 INFO [IPC Server handler 7 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=rkanter:supergroup:rwxr--r-- 2014-03-06 18:59:50,269 INFO [IPC Server handler 9 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging dst=null perm=null 2014-03-06 18:59:50,270 INFO [pool-1-thread-1] v2.MiniMRYarnCluster (MiniMRYarnCluster.java:serviceInit(150)) - mkdir: hdfs://localhost:52681/tmp/hadoop-yarn/staging 2014-03-06 18:59:50,273 INFO [IPC Server handler 0 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:50,280 INFO [IPC Server handler 2 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/history/done dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:50,357 INFO [pool-1-thread-1] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2052)) - yarn-site.xml not found 2014-03-06 18:59:50,358 INFO [pool-1-thread-1] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2052)) - core-site.xml not found 2014-03-06 18:59:50,358 INFO [pool-1-thread-1] security.Groups (Groups.java:refresh(162)) - clearing userToGroupsMap cache 2014-03-06 18:59:50,361 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher 2014-03-06 18:59:50,392 INFO [pool-1-thread-1] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2014-03-06 18:59:50,397 INFO [pool-1-thread-1] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(75)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2014-03-06 18:59:50,401 INFO [pool-1-thread-1] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:rollMasterKey(107)) - Rolling master-key for amrm-tokens 2014-03-06 18:59:50,410 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2014-03-06 18:59:50,412 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2014-03-06 18:59:50,413 INFO [pool-1-thread-1] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(252)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler 2014-03-06 18:59:50,451 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2014-03-06 18:59:50,452 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2014-03-06 18:59:50,454 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2014-03-06 18:59:50,456 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2014-03-06 18:59:50,478 INFO [pool-1-thread-1] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - ResourceManager metrics system started (again) 2014-03-06 18:59:50,506 WARN [pool-1-thread-1] fair.AllocationFileLoaderService (AllocationFileLoaderService.java:getAllocationFile(166)) - fair-scheduler.xml not found on the classpath. 2014-03-06 18:59:50,563 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2014-03-06 18:59:50,568 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2014-03-06 18:59:50,571 INFO [pool-1-thread-1] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2014-03-06 18:59:50,572 INFO [pool-1-thread-1] util.HostsFileReader (HostsFileReader.java:refresh(129)) - Refreshing hosts (include/exclude) list 2014-03-06 18:59:50,574 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$2 2014-03-06 18:59:50,576 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0 2014-03-06 18:59:50,576 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1 2014-03-06 18:59:50,576 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2 2014-03-06 18:59:50,576 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3 2014-03-06 18:59:50,577 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-0_0 2014-03-06 18:59:50,577 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-0_1 2014-03-06 18:59:50,577 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-0_2 2014-03-06 18:59:50,577 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-0_3 2014-03-06 18:59:50,578 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(511)) - Starting NM: 0 2014-03-06 18:59:50,655 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2014-03-06 18:59:50,657 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2014-03-06 18:59:50,658 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2014-03-06 18:59:50,658 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2014-03-06 18:59:50,659 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2014-03-06 18:59:50,659 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2014-03-06 18:59:50,665 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2014-03-06 18:59:50,665 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2014-03-06 18:59:50,666 INFO [pool-1-thread-1] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - NodeManager metrics system started (again) 2014-03-06 18:59:50,673 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2014-03-06 18:59:50,673 INFO [pool-1-thread-1] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(196)) - per directory file limit = 8192 2014-03-06 18:59:50,961 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2014-03-06 18:59:50,984 WARN [pool-1-thread-1] containermanager.AuxServices (AuxServices.java:serviceInit(114)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config. 2014-03-06 18:59:50,984 INFO [pool-1-thread-1] containermanager.AuxServices (AuxServices.java:addService(67)) - Adding auxiliary service httpshuffle, "mapreduce_shuffle" 2014-03-06 18:59:51,021 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(98)) - Using ResourceCalculatorPlugin : null 2014-03-06 18:59:51,021 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(103)) - Using ResourceCalculatorProcessTree : null 2014-03-06 18:59:51,022 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(127)) - Physical memory check enabled: false 2014-03-06 18:59:51,022 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(128)) - Virtual memory check enabled: false 2014-03-06 18:59:51,022 INFO [pool-1-thread-1] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(163)) - Initialized nodemanager for null: physical-memory=4096 virtual-memory=8602 virtual-cores=8 2014-03-06 18:59:51,023 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0 2014-03-06 18:59:51,023 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1 2014-03-06 18:59:51,023 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2 2014-03-06 18:59:51,023 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created localDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3 2014-03-06 18:59:51,023 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-1_0 2014-03-06 18:59:51,023 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-1_1 2014-03-06 18:59:51,024 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-1_2 2014-03-06 18:59:51,024 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(529)) - Created logDir in /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-logDir-nm-1_3 2014-03-06 18:59:51,024 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(511)) - Starting NM: 1 2014-03-06 18:59:51,041 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ShortCircuitedNodeManager 2014-03-06 18:59:51,042 INFO [pool-1-thread-1] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - NodeManager metrics system started (again) 2014-03-06 18:59:51,045 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2014-03-06 18:59:51,045 INFO [pool-1-thread-1] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(196)) - per directory file limit = 8192 2014-03-06 18:59:51,335 INFO [pool-1-thread-1] event.AsyncDispatcher (AsyncDispatcher.java:register(196)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2014-03-06 18:59:51,337 WARN [pool-1-thread-1] containermanager.AuxServices (AuxServices.java:serviceInit(114)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config. 2014-03-06 18:59:51,337 INFO [pool-1-thread-1] containermanager.AuxServices (AuxServices.java:addService(67)) - Adding auxiliary service httpshuffle, "mapreduce_shuffle" 2014-03-06 18:59:51,341 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(98)) - Using ResourceCalculatorPlugin : null 2014-03-06 18:59:51,341 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(103)) - Using ResourceCalculatorProcessTree : null 2014-03-06 18:59:51,341 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(127)) - Physical memory check enabled: false 2014-03-06 18:59:51,342 INFO [pool-1-thread-1] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(128)) - Virtual memory check enabled: false 2014-03-06 18:59:51,342 INFO [pool-1-thread-1] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(163)) - Initialized nodemanager for null: physical-memory=4096 virtual-memory=8602 virtual-cores=8 2014-03-06 18:59:51,372 INFO [pool-1-thread-1] hs.JobHistory (JobHistory.java:serviceInit(83)) - JobHistory Init 2014-03-06 18:59:51,416 INFO [IPC Server handler 3 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done dst=null perm=null 2014-03-06 18:59:51,421 INFO [IPC Server handler 4 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 2014-03-06 18:59:51,423 INFO [IPC Server handler 5 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=rkanter:supergroup:rwxrwxrwx 2014-03-06 18:59:51,424 INFO [IPC Server handler 1 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 2014-03-06 18:59:51,424 INFO [pool-1-thread-1] hs.HistoryFileManager (HistoryFileManager.java:mkdir(645)) - Perms after creating 511, Expected: 1023 2014-03-06 18:59:51,424 INFO [pool-1-thread-1] hs.HistoryFileManager (HistoryFileManager.java:mkdir(648)) - Explicitly setting permissions to : 1023, rwxrwxrwt 2014-03-06 18:59:51,426 INFO [IPC Server handler 8 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=rkanter:supergroup:rwxrwxrwt 2014-03-06 18:59:54,615 INFO [Thread-84] datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(429)) - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001 2014-03-06 18:59:54,624 INFO [Thread-83] datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(429)) - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002 2014-03-06 18:59:54,829 INFO [Thread-83] datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(429)) - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001 2014-03-06 18:59:54,831 INFO [Thread-84] datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(429)) - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002 2014-03-06 19:00:01,430 INFO [pool-1-thread-1] hs.HistoryFileManager (HistoryFileManager.java:initExisting(664)) - Initializing Existing Jobs... 2014-03-06 19:00:01,436 INFO [IPC Server handler 6 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done dst=null perm=null 2014-03-06 19:00:01,441 INFO [pool-1-thread-1] hs.CachedHistoryStorage (CachedHistoryStorage.java:serviceInit(66)) - CachedHistoryStorage Init 2014-03-06 19:00:01,458 INFO [pool-1-thread-1] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:01,459 INFO [Socket Reader #1 for port 52695] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52695 2014-03-06 19:00:01,462 INFO [pool-1-thread-1] v2.MiniMRYarnCluster (MiniMRYarnCluster.java:serviceStart(210)) - Waiting for HistoryServer to start... 2014-03-06 19:00:01,462 INFO [Thread-182] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - JobHistoryServer metrics system started (again) 2014-03-06 19:00:01,463 INFO [Thread-182] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(223)) - Updating the current master key for generating delegation tokens 2014-03-06 19:00:01,465 INFO [Thread[Thread-183,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(536)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2014-03-06 19:00:01,465 INFO [Thread[Thread-183,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(223)) - Updating the current master key for generating delegation tokens 2014-03-06 19:00:01,473 INFO [Thread-182] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.jobhistory is not defined 2014-03-06 19:00:01,474 INFO [Thread-182] http.HttpServer2 (HttpServer2.java:addGlobalFilter(658)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2014-03-06 19:00:01,475 INFO [Thread-182] http.HttpServer2 (HttpServer2.java:addFilter(636)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context jobhistory 2014-03-06 19:00:01,475 INFO [Thread-182] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2014-03-06 19:00:01,475 INFO [Thread-182] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2014-03-06 19:00:01,477 INFO [Thread-182] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /jobhistory/* 2014-03-06 19:00:01,478 INFO [Thread-182] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /ws/* 2014-03-06 19:00:01,481 INFO [Thread-182] http.HttpServer2 (HttpServer2.java:openListeners(846)) - Jetty bound to port 52696 [Thread-182] INFO org.mortbay.log - jetty-6.1.14 [Thread-182] INFO org.mortbay.log - Extract jar:file:/Users/rkanter/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.3.0-cdh5.1.0-SNAPSHOT/hadoop-yarn-common-2.3.0-cdh5.1.0-SNAPSHOT.jar!/webapps/jobhistory to /var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/Jetty_rkanter.mbp_local_52696_jobhistory____lfqmfo/webapp [Thread-182] INFO org.mortbay.log - Started SelectChannelConnector@rkanter-mbp.local:52696 [Thread-182] INFO org.apache.hadoop.yarn.webapp.WebApps - Web app /jobhistory started at 52696 [Thread-182] INFO org.apache.hadoop.yarn.webapp.WebApps - Registered webapp guice modules 2014-03-06 19:00:01,942 INFO [Thread-182] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:01,943 INFO [Socket Reader #1 for port 52697] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52697 2014-03-06 19:00:01,947 INFO [Thread-182] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.mapreduce.v2.api.HSClientProtocolPB to the server 2014-03-06 19:00:01,947 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:01,947 INFO [IPC Server listener on 52697] ipc.Server (Server.java:run(644)) - IPC Server listener on 52697: starting 2014-03-06 19:00:01,949 INFO [Thread-182] hs.HistoryClientService (HistoryClientService.java:serviceStart(142)) - Instantiated MRClientService at 172.16.1.64/172.16.1.64:52697 2014-03-06 19:00:01,950 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:01,950 INFO [IPC Server listener on 52695] ipc.Server (Server.java:run(644)) - IPC Server listener on 52695: starting 2014-03-06 19:00:01,952 INFO [IPC Server handler 0 on 52681] FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7724)) - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/fail dst=null perm=null 2014-03-06 19:00:02,964 INFO [pool-1-thread-1] v2.MiniMRYarnCluster (MiniMRYarnCluster.java:serviceStart(227)) - MiniMRYARN ResourceManager address: rkanter-mbp.local:0 2014-03-06 19:00:02,964 INFO [pool-1-thread-1] v2.MiniMRYarnCluster (MiniMRYarnCluster.java:serviceStart(229)) - MiniMRYARN ResourceManager web address: rkanter-mbp.local:0 2014-03-06 19:00:02,964 INFO [pool-1-thread-1] v2.MiniMRYarnCluster (MiniMRYarnCluster.java:serviceStart(231)) - MiniMRYARN HistoryServer address: 172.16.1.64:52697 2014-03-06 19:00:02,965 INFO [pool-1-thread-1] v2.MiniMRYarnCluster (MiniMRYarnCluster.java:serviceStart(233)) - MiniMRYARN HistoryServer web address: rkanter-mbp.local:52696 2014-03-06 19:00:02,966 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(295)) - Waiting for RM to start... 2014-03-06 19:00:02,966 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(841)) - Transitioning to active state 2014-03-06 19:00:02,970 INFO [RM-0] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(103)) - Rolling master-key for container-tokens 2014-03-06 19:00:02,970 INFO [Timer-5] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:rollMasterKey(107)) - Rolling master-key for amrm-tokens 2014-03-06 19:00:02,974 INFO [RM-0] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(95)) - Rolling master-key for nm-tokens 2014-03-06 19:00:02,975 INFO [RM-0] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(223)) - Updating the current master key for generating delegation tokens 2014-03-06 19:00:02,975 INFO [RM-0] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(85)) - storing master key with keyID 1 2014-03-06 19:00:02,976 INFO [Thread[Thread-205,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(536)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2014-03-06 19:00:02,976 INFO [Thread[Thread-205,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(223)) - Updating the current master key for generating delegation tokens 2014-03-06 19:00:02,976 INFO [Thread[Thread-205,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(85)) - storing master key with keyID 2 2014-03-06 19:00:03,013 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:03,013 INFO [Socket Reader #1 for port 52698] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52698 2014-03-06 19:00:03,016 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2014-03-06 19:00:03,016 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:03,016 INFO [IPC Server listener on 52698] ipc.Server (Server.java:run(644)) - IPC Server listener on 52698: starting 2014-03-06 19:00:03,041 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:03,042 INFO [Socket Reader #1 for port 52699] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52699 2014-03-06 19:00:03,044 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2014-03-06 19:00:03,045 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:03,045 INFO [IPC Server listener on 52699] ipc.Server (Server.java:run(644)) - IPC Server listener on 52699: starting 2014-03-06 19:00:03,085 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:03,086 INFO [Socket Reader #1 for port 52700] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52700 2014-03-06 19:00:03,089 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2014-03-06 19:00:03,089 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:03,089 INFO [IPC Server listener on 52700] ipc.Server (Server.java:run(644)) - IPC Server listener on 52700: starting 2014-03-06 19:00:03,095 INFO [RM-0] resourcemanager.ResourceManager (ResourceManager.java:transitionToActive(844)) - Transitioned to active state 2014-03-06 19:00:03,099 INFO [RM-0] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.resourcemanager is not defined 2014-03-06 19:00:03,100 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addGlobalFilter(658)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2014-03-06 19:00:03,100 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(636)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2014-03-06 19:00:03,100 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2014-03-06 19:00:03,100 INFO [RM-0] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2014-03-06 19:00:03,101 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /cluster/* 2014-03-06 19:00:03,101 INFO [RM-0] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /ws/* 2014-03-06 19:00:03,102 INFO [RM-0] http.HttpServer2 (HttpServer2.java:openListeners(846)) - Jetty bound to port 52701 [RM-0] INFO org.mortbay.log - jetty-6.1.14 [RM-0] INFO org.mortbay.log - Extract jar:file:/Users/rkanter/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.3.0-cdh5.1.0-SNAPSHOT/hadoop-yarn-common-2.3.0-cdh5.1.0-SNAPSHOT.jar!/webapps/cluster to /var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/Jetty_rkanter.mbp_local_52701_cluster____.2w11yk/webapp Mar 06, 2014 7:00:03 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.mapreduce.v2.hs.webapp.HsWebServices as a root resource class Mar 06, 2014 7:00:03 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.mapreduce.v2.hs.webapp.JAXBContextResolver as a provider class Mar 06, 2014 7:00:03 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Mar 06, 2014 7:00:03 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Mar 06, 2014 7:00:03 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.mapreduce.v2.hs.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Mar 06, 2014 7:00:03 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Mar 06, 2014 7:00:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.mapreduce.v2.hs.webapp.HsWebServices to GuiceManagedComponentProvider with the scope "PerRequest" [RM-0] INFO org.mortbay.log - Started SelectChannelConnector@rkanter-mbp.local:52701 [RM-0] INFO org.apache.hadoop.yarn.webapp.WebApps - Web app /cluster started at 52701 Mar 06, 2014 7:00:04 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. [RM-0] INFO org.apache.hadoop.yarn.webapp.WebApps - Registered webapp guice modules 2014-03-06 19:00:04,311 INFO [RM-0] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:04,312 INFO [Socket Reader #1 for port 52702] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52702 2014-03-06 19:00:04,319 INFO [RM-0] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2014-03-06 19:00:04,320 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:04,320 INFO [IPC Server listener on 52702] ipc.Server (Server.java:run(644)) - IPC Server listener on 52702: starting 2014-03-06 19:00:04,467 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(307)) - MiniYARN ResourceManager address: 172.16.1.64:52700 2014-03-06 19:00:04,467 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:startResourceManager(309)) - MiniYARN ResourceManager web address: rkanter-mbp.local:52701 2014-03-06 19:00:04,467 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(426)) - MiniYARN ResourceManager address: 172.16.1.64:52700 2014-03-06 19:00:04,467 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(428)) - MiniYARN ResourceManager web address: rkanter-mbp.local:52701 2014-03-06 19:00:04,468 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(546)) - Waiting for NM 0 to start... 2014-03-06 19:00:04,483 INFO [Thread-378] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:04,483 INFO [Socket Reader #1 for port 52703] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52703 2014-03-06 19:00:04,486 INFO [Thread-378] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2014-03-06 19:00:04,486 INFO [Thread-378] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(283)) - Blocking new container-requests as container manager rpc server is still starting. 2014-03-06 19:00:04,487 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:04,487 INFO [IPC Server listener on 52703] ipc.Server (Server.java:run(644)) - IPC Server listener on 52703: starting 2014-03-06 19:00:04,495 INFO [Thread-378] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(189)) - Updating node address : 172.16.1.64:52703 2014-03-06 19:00:04,496 INFO [Thread-378] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(294)) - ContainerManager started at /172.16.1.64:52703 2014-03-06 19:00:04,503 INFO [Thread-378] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:04,504 INFO [Socket Reader #1 for port 52704] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52704 2014-03-06 19:00:04,510 INFO [Thread-378] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2014-03-06 19:00:04,510 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:04,510 INFO [IPC Server listener on 52704] ipc.Server (Server.java:run(644)) - IPC Server listener on 52704: starting 2014-03-06 19:00:04,512 INFO [Thread-378] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(263)) - Localizer started on port 52704 2014-03-06 19:00:04,529 INFO [Thread-378] mapred.IndexCache (IndexCache.java:(47)) - IndexCache created with max memory = 10485760 2014-03-06 19:00:04,549 INFO [Thread-378] mapred.ShuffleHandler (ShuffleHandler.java:serviceStart(326)) - httpshuffle listening on port 52705 2014-03-06 19:00:04,550 INFO [Thread-378] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 2014-03-06 19:00:04,550 INFO [Thread-378] webapp.WebServer (WebServer.java:serviceStart(60)) - Instantiating NMWebApp at rkanter-mbp.local:0 2014-03-06 19:00:04,551 INFO [Thread-378] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2014-03-06 19:00:04,551 INFO [Thread-378] http.HttpServer2 (HttpServer2.java:addGlobalFilter(658)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2014-03-06 19:00:04,552 INFO [Thread-378] http.HttpServer2 (HttpServer2.java:addFilter(636)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2014-03-06 19:00:04,552 INFO [Thread-378] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2014-03-06 19:00:04,552 INFO [Thread-378] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2014-03-06 19:00:04,552 INFO [Thread-378] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /node/* 2014-03-06 19:00:04,553 INFO [Thread-378] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /ws/* 2014-03-06 19:00:04,553 INFO [Thread-378] http.HttpServer2 (HttpServer2.java:openListeners(846)) - Jetty bound to port 52706 [Thread-378] INFO org.mortbay.log - jetty-6.1.14 [Thread-378] INFO org.mortbay.log - Extract jar:file:/Users/rkanter/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.3.0-cdh5.1.0-SNAPSHOT/hadoop-yarn-common-2.3.0-cdh5.1.0-SNAPSHOT.jar!/webapps/node to /var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/Jetty_rkanter.mbp_local_52706_node____r83lsj/webapp Mar 06, 2014 7:00:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class Mar 06, 2014 7:00:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class Mar 06, 2014 7:00:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Mar 06, 2014 7:00:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Mar 06, 2014 7:00:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Mar 06, 2014 7:00:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Mar 06, 2014 7:00:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" [Thread-378] INFO org.mortbay.log - Started SelectChannelConnector@rkanter-mbp.local:52706 [Thread-378] INFO org.apache.hadoop.yarn.webapp.WebApps - Web app /node started at 52706 Mar 06, 2014 7:00:05 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. [Thread-378] INFO org.apache.hadoop.yarn.webapp.WebApps - Registered webapp guice modules 2014-03-06 19:00:05,012 INFO [Thread-378] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(244)) - Registering with RM using finished containers :[] 2014-03-06 19:00:05,029 INFO [Thread-378] util.RackResolver (RackResolver.java:coreResolve(109)) - Resolved 172.16.1.64 to /default-rack 2014-03-06 19:00:05,036 INFO [Thread-378] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(304)) - NodeManager from node 172.16.1.64(cmPort: 52703 httpPort: 52706) registered with capability: , assigned nodeId 172.16.1.64:52703 2014-03-06 19:00:05,036 INFO [Thread-378] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(71)) - Rolling master-key for container-tokens, got key with id -66622771 2014-03-06 19:00:05,038 INFO [Thread-378] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(65)) - Rolling master-key for nm-tokens, got key with id :1122039547 2014-03-06 19:00:05,038 INFO [Thread-378] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(294)) - Registered with ResourceManager as 172.16.1.64:52703 with total resource of 2014-03-06 19:00:05,038 INFO [Thread-378] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(296)) - Notifying ContainerManager to unblock new container-requests 2014-03-06 19:00:05,041 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(384)) - 172.16.1.64:52703 Node Transitioned from NEW to RUNNING 2014-03-06 19:00:05,044 INFO [ResourceManager Event Processor] fair.FairScheduler (FairScheduler.java:addNode(818)) - Added node 172.16.1.64:52703 cluster capacity: 2014-03-06 19:00:05,469 INFO [pool-1-thread-1] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(546)) - Waiting for NM 1 to start... 2014-03-06 19:00:05,473 INFO [Thread-416] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:05,474 INFO [Socket Reader #1 for port 52707] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52707 2014-03-06 19:00:05,478 INFO [Thread-416] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2014-03-06 19:00:05,479 INFO [Thread-416] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(283)) - Blocking new container-requests as container manager rpc server is still starting. 2014-03-06 19:00:05,479 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:05,480 INFO [IPC Server listener on 52707] ipc.Server (Server.java:run(644)) - IPC Server listener on 52707: starting 2014-03-06 19:00:05,484 INFO [Thread-416] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(189)) - Updating node address : 172.16.1.64:52707 2014-03-06 19:00:05,484 INFO [Thread-416] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(294)) - ContainerManager started at /172.16.1.64:52707 2014-03-06 19:00:05,485 INFO [Thread-416] ipc.CallQueueManager (CallQueueManager.java:(49)) - Using callQueue class java.util.concurrent.LinkedBlockingQueue 2014-03-06 19:00:05,486 INFO [Socket Reader #1 for port 52708] ipc.Server (Server.java:run(575)) - Starting Socket Reader #1 for port 52708 2014-03-06 19:00:05,488 INFO [Thread-416] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2014-03-06 19:00:05,489 INFO [IPC Server Responder] ipc.Server (Server.java:run(797)) - IPC Server Responder: starting 2014-03-06 19:00:05,489 INFO [IPC Server listener on 52708] ipc.Server (Server.java:run(644)) - IPC Server listener on 52708: starting 2014-03-06 19:00:05,490 INFO [Thread-416] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(263)) - Localizer started on port 52708 2014-03-06 19:00:05,491 INFO [Thread-416] mapred.IndexCache (IndexCache.java:(47)) - IndexCache created with max memory = 10485760 2014-03-06 19:00:05,492 INFO [Thread-416] mapred.ShuffleHandler (ShuffleHandler.java:serviceStart(326)) - httpshuffle listening on port 52709 2014-03-06 19:00:05,493 INFO [Thread-416] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 2014-03-06 19:00:05,493 INFO [Thread-416] webapp.WebServer (WebServer.java:serviceStart(60)) - Instantiating NMWebApp at rkanter-mbp.local:0 2014-03-06 19:00:05,494 INFO [Thread-416] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(80)) - Http request log for http.requests.nodemanager is not defined 2014-03-06 19:00:05,494 INFO [Thread-416] http.HttpServer2 (HttpServer2.java:addGlobalFilter(658)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter) 2014-03-06 19:00:05,495 INFO [Thread-416] http.HttpServer2 (HttpServer2.java:addFilter(636)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2014-03-06 19:00:05,495 INFO [Thread-416] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2014-03-06 19:00:05,495 INFO [Thread-416] http.HttpServer2 (HttpServer2.java:addFilter(643)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2014-03-06 19:00:05,496 INFO [Thread-416] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /node/* 2014-03-06 19:00:05,496 INFO [Thread-416] http.HttpServer2 (HttpServer2.java:initializeWebServer(392)) - adding path spec: /ws/* 2014-03-06 19:00:05,496 INFO [Thread-416] http.HttpServer2 (HttpServer2.java:openListeners(846)) - Jetty bound to port 52710 [Thread-416] INFO org.mortbay.log - jetty-6.1.14 [Thread-416] INFO org.mortbay.log - Extract jar:file:/Users/rkanter/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.3.0-cdh5.1.0-SNAPSHOT/hadoop-yarn-common-2.3.0-cdh5.1.0-SNAPSHOT.jar!/webapps/node to /var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/Jetty_rkanter.mbp_local_52710_node____.9rke6s/webapp Mar 06, 2014 7:00:05 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class Mar 06, 2014 7:00:05 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class Mar 06, 2014 7:00:05 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class Mar 06, 2014 7:00:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Mar 06, 2014 7:00:05 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" Mar 06, 2014 7:00:05 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Mar 06, 2014 7:00:05 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton" [Thread-416] INFO org.mortbay.log - Started SelectChannelConnector@rkanter-mbp.local:52710 [Thread-416] INFO org.apache.hadoop.yarn.webapp.WebApps - Web app /node started at 52710 Mar 06, 2014 7:00:05 PM com.google.inject.servlet.GuiceFilter setPipeline WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. [Thread-416] INFO org.apache.hadoop.yarn.webapp.WebApps - Registered webapp guice modules 2014-03-06 19:00:05,841 INFO [Thread-416] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(244)) - Registering with RM using finished containers :[] 2014-03-06 19:00:05,841 INFO [Thread-416] util.RackResolver (RackResolver.java:coreResolve(109)) - Resolved 172.16.1.64 to /default-rack 2014-03-06 19:00:05,841 INFO [Thread-416] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(304)) - NodeManager from node 172.16.1.64(cmPort: 52707 httpPort: 52710) registered with capability: , assigned nodeId 172.16.1.64:52707 2014-03-06 19:00:05,841 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(384)) - 172.16.1.64:52707 Node Transitioned from NEW to RUNNING 2014-03-06 19:00:05,841 INFO [Thread-416] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(71)) - Rolling master-key for container-tokens, got key with id -66622771 2014-03-06 19:00:05,841 INFO [Thread-416] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(65)) - Rolling master-key for nm-tokens, got key with id :1122039547 2014-03-06 19:00:05,841 INFO [ResourceManager Event Processor] fair.FairScheduler (FairScheduler.java:addNode(818)) - Added node 172.16.1.64:52707 cluster capacity: 2014-03-06 19:00:05,842 INFO [Thread-416] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(294)) - Registered with ResourceManager as 172.16.1.64:52707 with total resource of 2014-03-06 19:00:05,842 INFO [Thread-416] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(296)) - Notifying ContainerManager to unblock new container-requests 2014-03-06 19:00:06,473 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2014-03-06 19:00:06,473 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - fs.default.name is deprecated. Instead, use fs.defaultFS 2014-03-06 19:00:06,498 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - dfs.permissions is deprecated. Instead, use dfs.permissions.enabled 2014-03-06 19:00:06,638 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 2014-03-06 19:00:06,638 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 2014-03-06 19:00:06,638 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 2014-03-06 19:00:06,638 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 2014-03-06 19:00:06,638 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 2014-03-06 19:00:06,638 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 2014-03-06 19:00:06,639 INFO [pool-1-thread-1] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 2014-03-06 19:00:06,776 INFO [Thread-455] metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(416)) - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2014-03-06 19:00:06,817 INFO [Thread-455] metastore.ObjectStore (ObjectStore.java:initialize(243)) - ObjectStore, initialize called 2014-03-06 19:00:07,049 INFO [Thread-455] DataNucleus.Persistence (Log4JLogger.java:info(77)) - Property datanucleus.cache.level2 unknown - will be ignored [Thread-455] WARN com.jolbox.bonecp.BoneCPConfig - Max Connections < 1. Setting to 20 2014-03-06 19:00:08,934 INFO [Thread-455] metastore.ObjectStore (ObjectStore.java:getPMF(312)) - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" [Thread-455] WARN com.jolbox.bonecp.BoneCPConfig - Max Connections < 1. Setting to 20 2014-03-06 19:00:09,003 INFO [Thread-455] metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:(101)) - MySQL check failed, assuming we are not on mysql: Lexical error at line 1, column 5. Encountered: "@" (64), after : "". 2014-03-06 19:00:10,104 INFO [Thread-455] metastore.ObjectStore (ObjectStore.java:setConf(226)) - Initialized ObjectStore 2014-03-06 19:00:10,249 WARN [Thread-455] metastore.ObjectStore (ObjectStore.java:checkSchema(5910)) - Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 0.12.0 2014-03-06 19:00:10,574 INFO [Thread-455] metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(4301)) - Starting DB backed MetaStore Server 2014-03-06 19:00:10,577 INFO [Thread-455] metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(4313)) - Started the new metaserver on port [30044]... 2014-03-06 19:00:10,577 INFO [Thread-455] metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(4315)) - Options.minWorkerThreads = 200 2014-03-06 19:00:10,577 INFO [Thread-455] metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(4317)) - Options.maxWorkerThreads = 100000 2014-03-06 19:00:10,577 INFO [Thread-455] metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(4319)) - TCP keepalive = true 2014-03-06 19:00:16,854 WARN [pool-1-thread-1] conf.HiveConf (HiveConf.java:initialize(1160)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore. 2014-03-06 19:00:16,908 INFO [pool-1-thread-1] hive.metastore (HiveMetaStoreClient.java:open(249)) - Trying to connect to metastore with URI thrift://localhost:30044 2014-03-06 19:00:16,927 INFO [pool-1-thread-1] hive.metastore (HiveMetaStoreClient.java:open(337)) - Connected to metastore. 2014-03-06 19:00:16,938 INFO [pool-31-thread-1] metastore.HiveMetaStore (HiveMetaStore.java:logInfo(459)) - 1: source:/127.0.0.1 get_databases: NonExistentDatabaseUsedForHealthCheck 2014-03-06 19:00:16,939 INFO [pool-31-thread-1] HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(242)) - ugi=rkanter ip=/127.0.0.1 cmd=source:/127.0.0.1 get_databases: NonExistentDatabaseUsedForHealthCheck 2014-03-06 19:00:16,942 INFO [pool-31-thread-1] metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(416)) - 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2014-03-06 19:00:16,943 INFO [pool-31-thread-1] metastore.ObjectStore (ObjectStore.java:initialize(243)) - ObjectStore, initialize called 2014-03-06 19:00:16,945 INFO [pool-31-thread-1] metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:(101)) - MySQL check failed, assuming we are not on mysql: Lexical error at line 1, column 5. Encountered: "@" (64), after : "". 2014-03-06 19:00:16,947 INFO [pool-31-thread-1] DataNucleus.Query (Log4JLogger.java:info(77)) - Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 2014-03-06 19:00:16,947 INFO [pool-31-thread-1] metastore.ObjectStore (ObjectStore.java:setConf(226)) - Initialized ObjectStore 2014-03-06 19:00:16,980 INFO [pool-1-thread-1] hadoop.TestMapReduceActionExecutor (XLog.java:log(539)) - Metastore server started at thrift://localhost:30044 19:00:16,996 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:00:16,996 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:00:16,996 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:00:16,996 INFO XLogService:539 - Log4j reload interval [disabled] 19:00:16,998 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:00:16,998 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetupMethods] 19:00:16,999 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetupMethods/conf] 19:00:16,999 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf file [oozie-site.xml] 19:00:17,006 INFO ConfigurationService:539 - USER[-] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:00:17,011 INFO deprecation:1009 - user.name is deprecated. Instead, use mapreduce.job.user.name 19:00:17,012 WARN Services:542 - USER[-] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:00:17,013 INFO Services:539 - USER[-] GROUP[-] Exiting null Entering NORMAL 19:00:17,013 INFO Services:539 - USER[-] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan8535750957788997179.dir] 19:00:17,014 INFO ConfigurationService:539 - USER[-] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:00:17,021 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:00:17,021 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:00:17,021 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:00:17,046 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] log4j:WARN No appenders could be found for logger (openjpa.Runtime). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. 19:00:19,116 INFO JPAService:539 - USER[-] GROUP[-] All entities initialized 19:00:19,117 INFO JPAService:539 - USER[-] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:00:19,121 INFO Services:539 - Initialized 19:00:19,121 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:00:19,121 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:00:19,268 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:00:19,269 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:00:19,269 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:00:19,269 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:00:19,269 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:00:19,269 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:00:19,269 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:00:19,269 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:00:19,270 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:00:19,270 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/097f9315-2975-4453-a711-fcf115f73d19] 19:00:19,356 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:00:19,356 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:00:19,356 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:00:19,356 INFO XLogService:539 - Log4j reload interval [disabled] 19:00:19,356 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:00:19,357 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetupMethods] 19:00:19,357 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetupMethods/conf] 19:00:19,357 INFO ConfigurationService:539 - USER[-] GROUP[-] Oozie conf file [oozie-site.xml] 19:00:19,363 INFO ConfigurationService:539 - USER[-] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:00:19,363 INFO ConfigurationService:539 - USER[-] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:00:19,363 INFO ConfigurationService:539 - USER[-] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:00:19,365 WARN Services:542 - USER[-] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:00:19,365 INFO Services:539 - USER[-] GROUP[-] Exiting null Entering NORMAL 19:00:19,365 INFO Services:539 - USER[-] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan7004856025324114619.dir] 19:00:19,406 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:00:19,406 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:00:19,406 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:00:19,406 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:00:19,407 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:00:19,407 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:00:19,407 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:00:19,407 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:00:19,407 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:00:19,407 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:00:19,407 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:00:19,408 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:00:19,409 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:00:19,410 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:00:19,410 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:00:19,410 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:00:19,410 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:00:19,410 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:00:19,410 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:00:19,410 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:00:19,417 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [kill] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [resume] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [suspend] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [bundle_kill] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [bundle_resume] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [bundle_suspend] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_kill] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_change] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_resume] 19:00:19,418 DEBUG CallableQueueService:545 - USER[-] GROUP[-] Adding interrupt type [coord_suspend] 19:00:19,423 WARN ConfigurationService:542 - USER[-] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:00:19,424 WARN ConfigurationService:542 - USER[-] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:00:19,445 WARN ConfigurationService:542 - USER[-] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:00:19,445 WARN AuthorizationService:542 - USER[-] GROUP[-] Oozie running with authorization disabled 19:00:19,445 INFO HadoopAccessorService:539 - USER[-] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:00:19,445 INFO HadoopAccessorService:539 - USER[-] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:00:19,445 INFO HadoopAccessorService:539 - USER[-] GROUP[-] Oozie Kerberos Authentication [disabled] 19:00:19,475 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:00:19,475 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:00:20,056 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] All entities initialized 19:00:20,056 INFO JPAService:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:00:20,090 INFO Services:539 - Initialized 19:00:20,090 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:00:20,090 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:00:20,125 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:00:20,130 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741827_1003{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:20,134 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003 src: /127.0.0.1:52713 dest: /127.0.0.1:52685 19:00:20,136 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003 src: /127.0.0.1:52714 dest: /127.0.0.1:52682 19:00:20,140 INFO clienttrace:1215 - src: /127.0.0.1:52714, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1757895107_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003, duration: 2940000 19:00:20,140 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,142 INFO clienttrace:1215 - src: /127.0.0.1:52713, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1757895107_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003, duration: 4156000 19:00:20,142 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,148 INFO StateChange:2959 - DIR* completeFile: /user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_1757895107_8 19:00:20,202 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,204 INFO deprecation:1009 - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 19:00:20,204 INFO deprecation:1009 - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 19:00:20,205 INFO ConfigurationService:539 - USER[-] GROUP[-] Programmatic configuration change, property[oozie.action.mapreduce.uber.jar.enable]=[true] 19:00:20,223 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,226 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml dst=null perm=null 19:00:20,246 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,254 INFO deprecation:1009 - mapred.job.name is deprecated. Instead, use mapreduce.job.name 19:00:20,264 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,267 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,317 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741828_1004{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:20,320 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741828_1004 src: /127.0.0.1:52716 dest: /127.0.0.1:52685 19:00:20,323 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741828_1004 src: /127.0.0.1:52717 dest: /127.0.0.1:52682 19:00:20,330 INFO clienttrace:1215 - src: /127.0.0.1:52717, dest: /127.0.0.1:52682, bytes: 75388, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741828_1004, duration: 5138000 19:00:20,330 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741828_1004, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,331 INFO clienttrace:1215 - src: /127.0.0.1:52716, dest: /127.0.0.1:52685, bytes: 75388, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741828_1004, duration: 6285000 19:00:20,332 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741828_1004, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,334 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,336 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,338 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,342 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741829_1005{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:20,345 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741829_1005 src: /127.0.0.1:52718 dest: /127.0.0.1:52685 19:00:20,347 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741829_1005 src: /127.0.0.1:52719 dest: /127.0.0.1:52682 19:00:20,350 INFO clienttrace:1215 - src: /127.0.0.1:52719, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741829_1005, duration: 2379000 19:00:20,351 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741829_1005, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,352 INFO clienttrace:1215 - src: /127.0.0.1:52718, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741829_1005, duration: 3553000 19:00:20,352 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741829_1005, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,356 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,357 INFO deprecation:1009 - mapred.job.queue.name is deprecated. Instead, use mapreduce.job.queuename 19:00:20,374 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,377 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml dst=null perm=null 19:00:20,393 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,398 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,402 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=null 19:00:20,403 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,475 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741830_1006{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:20,479 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741830_1006 src: /127.0.0.1:52720 dest: /127.0.0.1:52682 19:00:20,481 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741830_1006 src: /127.0.0.1:52721 dest: /127.0.0.1:52685 19:00:20,487 INFO clienttrace:1215 - src: /127.0.0.1:52721, dest: /127.0.0.1:52685, bytes: 75388, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741830_1006, duration: 3943000 19:00:20,487 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741830_1006, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,488 INFO clienttrace:1215 - src: /127.0.0.1:52720, dest: /127.0.0.1:52682, bytes: 75388, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741830_1006, duration: 5433000 19:00:20,489 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741830_1006, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,491 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,492 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,494 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=null 19:00:20,495 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,497 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741831_1007{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:20,500 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741831_1007 src: /127.0.0.1:52722 dest: /127.0.0.1:52685 19:00:20,501 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741831_1007 src: /127.0.0.1:52723 dest: /127.0.0.1:52682 19:00:20,505 INFO clienttrace:1215 - src: /127.0.0.1:52723, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741831_1007, duration: 2629000 19:00:20,506 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741831_1007, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,507 INFO clienttrace:1215 - src: /127.0.0.1:52722, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741831_1007, duration: 4043000 19:00:20,507 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741831_1007, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,509 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,530 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,533 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml dst=null perm=null 19:00:20,560 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,567 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,569 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=null 19:00:20,570 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,613 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741832_1008{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:20,615 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741832_1008 src: /127.0.0.1:52724 dest: /127.0.0.1:52685 19:00:20,617 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741832_1008 src: /127.0.0.1:52725 dest: /127.0.0.1:52682 19:00:20,623 INFO clienttrace:1215 - src: /127.0.0.1:52725, dest: /127.0.0.1:52682, bytes: 75431, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741832_1008, duration: 4646000 19:00:20,623 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741832_1008, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,624 INFO clienttrace:1215 - src: /127.0.0.1:52724, dest: /127.0.0.1:52685, bytes: 75431, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741832_1008, duration: 5939000 19:00:20,624 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741832_1008, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,627 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,628 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,631 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=null 19:00:20,632 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,635 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741833_1009{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:20,637 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741833_1009 src: /127.0.0.1:52726 dest: /127.0.0.1:52685 19:00:20,639 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741833_1009 src: /127.0.0.1:52727 dest: /127.0.0.1:52682 19:00:20,643 INFO clienttrace:1215 - src: /127.0.0.1:52727, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741833_1009, duration: 2751000 19:00:20,643 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741833_1009, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,646 INFO clienttrace:1215 - src: /127.0.0.1:52726, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741833_1009, duration: 5595000 19:00:20,646 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741833_1009, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,648 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,669 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,670 WARN MapReduceActionExecutor:542 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] The oozie.mapreduce.uber.jar property is only applicable for MapReduce (notstreaming nor pipes) workflows, ignoring 19:00:20,671 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml dst=null perm=null 19:00:20,689 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,696 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,699 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=null 19:00:20,700 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,715 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741834_1010{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:20,718 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741834_1010 src: /127.0.0.1:52728 dest: /127.0.0.1:52685 19:00:20,719 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741834_1010 src: /127.0.0.1:52729 dest: /127.0.0.1:52682 19:00:20,726 INFO clienttrace:1215 - src: /127.0.0.1:52729, dest: /127.0.0.1:52682, bytes: 75579, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741834_1010, duration: 4990000 19:00:20,726 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741834_1010, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,728 INFO clienttrace:1215 - src: /127.0.0.1:52728, dest: /127.0.0.1:52685, bytes: 75579, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741834_1010, duration: 6322000 19:00:20,728 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741834_1010, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,732 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,734 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,736 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=null 19:00:20,737 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,739 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741835_1011{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:20,741 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741835_1011 src: /127.0.0.1:52730 dest: /127.0.0.1:52685 19:00:20,743 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741835_1011 src: /127.0.0.1:52731 dest: /127.0.0.1:52682 19:00:20,746 INFO clienttrace:1215 - src: /127.0.0.1:52731, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741835_1011, duration: 1947000 19:00:20,747 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741835_1011, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,747 INFO clienttrace:1215 - src: /127.0.0.1:52730, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741835_1011, duration: 2857000 19:00:20,747 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741835_1011, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,750 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,801 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,804 WARN MapReduceActionExecutor:542 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] The oozie.mapreduce.uber.jar property is only applicable for MapReduce (notstreaming nor pipes) workflows, ignoring 19:00:20,805 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml dst=null perm=null 19:00:20,822 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,826 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,829 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=null 19:00:20,829 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,836 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741836_1012{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:20,839 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741836_1012 src: /127.0.0.1:52732 dest: /127.0.0.1:52685 19:00:20,840 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741836_1012 src: /127.0.0.1:52733 dest: /127.0.0.1:52682 19:00:20,843 INFO clienttrace:1215 - src: /127.0.0.1:52733, dest: /127.0.0.1:52682, bytes: 75347, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741836_1012, duration: 2451000 19:00:20,844 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741836_1012, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,844 INFO clienttrace:1215 - src: /127.0.0.1:52732, dest: /127.0.0.1:52685, bytes: 75347, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741836_1012, duration: 3150000 19:00:20,844 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741836_1012, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,846 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,847 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,849 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=null 19:00:20,850 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,852 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741837_1013{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:20,854 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741837_1013 src: /127.0.0.1:52734 dest: /127.0.0.1:52685 19:00:20,855 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741837_1013 src: /127.0.0.1:52735 dest: /127.0.0.1:52682 19:00:20,858 INFO clienttrace:1215 - src: /127.0.0.1:52735, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741837_1013, duration: 1622000 19:00:20,858 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741837_1013, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,859 INFO clienttrace:1215 - src: /127.0.0.1:52734, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741837_1013, duration: 2297000 19:00:20,859 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741837_1013, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,862 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,878 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,879 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/097f9315-2975-4453-a711-fcf115f73d19/app/workflow.xml dst=null perm=null 19:00:20,894 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,897 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,900 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=null 19:00:20,901 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,908 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741838_1014{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:20,910 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014 src: /127.0.0.1:52736 dest: /127.0.0.1:52682 19:00:20,911 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014 src: /127.0.0.1:52737 dest: /127.0.0.1:52685 19:00:20,915 INFO clienttrace:1215 - src: /127.0.0.1:52737, dest: /127.0.0.1:52685, bytes: 75248, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014, duration: 2206000 19:00:20,915 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,915 INFO clienttrace:1215 - src: /127.0.0.1:52736, dest: /127.0.0.1:52682, bytes: 75248, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014, duration: 2903000 19:00:20,916 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,917 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/action.xml is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,919 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input dst=null perm=test:supergroup:rwxrwxrwx 19:00:20,921 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=null 19:00:20,922 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:00:20,924 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741839_1015{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:20,926 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015 src: /127.0.0.1:52738 dest: /127.0.0.1:52685 19:00:20,927 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015 src: /127.0.0.1:52739 dest: /127.0.0.1:52682 19:00:20,930 INFO clienttrace:1215 - src: /127.0.0.1:52739, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015, duration: 1647000 19:00:20,930 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:20,931 INFO clienttrace:1215 - src: /127.0.0.1:52738, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1039094556_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015, duration: 2409000 19:00:20,931 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:20,933 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190019423-oozie-rkan-W/mr-action--map-reduce/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-1039094556_8 19:00:20,933 INFO ConfigurationService:539 - USER[-] GROUP[-] Programmatic configuration change, property[oozie.action.mapreduce.uber.jar.enable]=[false] 19:00:20,948 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,949 INFO ConfigurationService:539 - USER[-] GROUP[-] Programmatic configuration change, property[oozie.action.mapreduce.uber.jar.enable]=[false] 19:00:20,962 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,977 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:20,981 INFO Services:539 - Shutdown 19:00:21,388 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741828_1004, blk_1073741829_1005, blk_1073741830_1006, blk_1073741831_1007, blk_1073741832_1008, blk_1073741833_1009, blk_1073741834_1010, blk_1073741835_1011, blk_1073741836_1012, blk_1073741837_1013] 19:00:21,693 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:00:21,696 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741840_1016{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:21,699 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016 src: /127.0.0.1:52741 dest: /127.0.0.1:52685 19:00:21,700 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016 src: /127.0.0.1:52742 dest: /127.0.0.1:52682 19:00:21,703 INFO clienttrace:1215 - src: /127.0.0.1:52742, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2043022519_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016, duration: 1627000 19:00:21,703 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:21,703 INFO clienttrace:1215 - src: /127.0.0.1:52741, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2043022519_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016, duration: 2224000 19:00:21,704 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:21,706 INFO StateChange:2959 - DIR* completeFile: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input/data.txt is closed by DFSClient_NONMAPREDUCE_2043022519_8 19:00:21,717 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:00:21,718 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741841_1017{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:00:21,720 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017 src: /127.0.0.1:52743 dest: /127.0.0.1:52682 19:00:21,722 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017 src: /127.0.0.1:52744 dest: /127.0.0.1:52685 19:00:21,725 INFO clienttrace:1215 - src: /127.0.0.1:52744, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2043022519_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017, duration: 1852000 19:00:21,725 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:21,725 INFO clienttrace:1215 - src: /127.0.0.1:52743, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2043022519_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017, duration: 2386000 19:00:21,725 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:21,729 INFO StateChange:2959 - DIR* completeFile: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_2043022519_8 19:00:21,731 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:00:21,733 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741842_1018{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:00:21,735 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018 src: /127.0.0.1:52745 dest: /127.0.0.1:52682 19:00:21,736 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018 src: /127.0.0.1:52746 dest: /127.0.0.1:52685 19:00:21,741 INFO clienttrace:1215 - src: /127.0.0.1:52746, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2043022519_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018, duration: 3259000 19:00:21,741 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:21,742 INFO clienttrace:1215 - src: /127.0.0.1:52745, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2043022519_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018, duration: 4005000 19:00:21,742 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:21,744 INFO StateChange:2959 - DIR* completeFile: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_2043022519_8 19:00:21,748 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java dst=null perm=null 19:00:21,750 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:00:21,754 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java.tmp dst=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java perm=test:supergroup:rwxrwxrwx 19:00:21,757 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/workflow.xml dst=null perm=null 19:00:21,804 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/workflow.xml dst=null perm=null 19:00:21,837 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java dst=null perm=test:supergroup:rwxrwxrwx 19:00:21,839 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:21,847 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741843_1019{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:21,850 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019 src: /127.0.0.1:52748 dest: /127.0.0.1:52685 19:00:21,851 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019 src: /127.0.0.1:52749 dest: /127.0.0.1:52682 19:00:21,854 INFO clienttrace:1215 - src: /127.0.0.1:52749, dest: /127.0.0.1:52682, bytes: 77265, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019, duration: 1520000 19:00:21,854 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:21,854 INFO clienttrace:1215 - src: /127.0.0.1:52748, dest: /127.0.0.1:52685, bytes: 77265, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019, duration: 2257000 19:00:21,855 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:21,857 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action.xml is closed by DFSClient_NONMAPREDUCE_451532259_8 19:00:21,858 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input dst=null perm=test:supergroup:rwxrwxrwx 19:00:21,861 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:00:21,865 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741844_1020{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:21,867 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020 src: /127.0.0.1:52750 dest: /127.0.0.1:52685 19:00:21,868 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020 src: /127.0.0.1:52751 dest: /127.0.0.1:52682 19:00:21,872 INFO clienttrace:1215 - src: /127.0.0.1:52751, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020, duration: 3211000 19:00:21,873 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:21,873 INFO clienttrace:1215 - src: /127.0.0.1:52750, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020, duration: 3877000 19:00:21,874 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:21,875 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_451532259_8 19:00:21,992 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161221992, maxDate=1394766021992, sequenceNumber=1, masterKeyId=2 19:00:21,994 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 1 19:00:22,001 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/0000000-140306190021371-oozie-rkan-W@mr-action dst=null perm=null 19:00:22,003 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161222003, maxDate=1394766022003, sequenceNumber=2, masterKeyId=2 19:00:22,003 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 2 19:00:22,032 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:00:22,038 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:00:22,040 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=test:supergroup:rwx------ 19:00:22,050 INFO ClientRMService:229 - Allocated new applicationId: 1 19:00:22,060 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001 dst=null perm=null 19:00:22,063 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001 dst=null perm=test:supergroup:rwxrwxrwx 19:00:22,064 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001 dst=null perm=test:supergroup:rwx------ 19:00:22,067 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:22,068 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:22,069 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:22,071 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:22,072 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:00:22,073 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:00:22,074 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:00:22,075 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:22,076 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:22,078 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:22,079 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:22,080 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib dst=null perm=null 19:00:22,081 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app dst=null perm=null 19:00:22,082 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a dst=null perm=null 19:00:22,083 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:00:22,105 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:00:22,107 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:00:22,109 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:00:22,121 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:00:22,123 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=test:supergroup:rw-r--r-- 19:00:22,126 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split 19:00:22,127 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=null 19:00:22,132 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741845_1021{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:00:22,134 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021 src: /127.0.0.1:52753 dest: /127.0.0.1:52682 19:00:22,136 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021 src: /127.0.0.1:52754 dest: /127.0.0.1:52685 19:00:22,139 INFO clienttrace:1215 - src: /127.0.0.1:52754, dest: /127.0.0.1:52685, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021, duration: 1916000 19:00:22,139 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:22,140 INFO clienttrace:1215 - src: /127.0.0.1:52753, dest: /127.0.0.1:52682, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021, duration: 2684000 19:00:22,140 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:22,142 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split is closed by DFSClient_NONMAPREDUCE_451532259_8 19:00:22,144 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:00:22,146 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:00:22,149 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741846_1022{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:22,151 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022 src: /127.0.0.1:52755 dest: /127.0.0.1:52685 19:00:22,153 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022 src: /127.0.0.1:52756 dest: /127.0.0.1:52682 19:00:22,156 INFO clienttrace:1215 - src: /127.0.0.1:52756, dest: /127.0.0.1:52682, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022, duration: 1321000 19:00:22,156 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:22,157 INFO clienttrace:1215 - src: /127.0.0.1:52755, dest: /127.0.0.1:52685, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022, duration: 2168000 19:00:22,157 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:22,159 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_451532259_8 19:00:22,166 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:22,168 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:00:22,178 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741847_1023{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:22,180 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023 src: /127.0.0.1:52757 dest: /127.0.0.1:52685 19:00:22,182 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023 src: /127.0.0.1:52758 dest: /127.0.0.1:52682 19:00:22,185 INFO clienttrace:1215 - src: /127.0.0.1:52758, dest: /127.0.0.1:52682, bytes: 79743, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023, duration: 1947000 19:00:22,185 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:22,186 INFO clienttrace:1215 - src: /127.0.0.1:52757, dest: /127.0.0.1:52685, bytes: 79743, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023, duration: 2731000 19:00:22,186 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:22,188 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml is closed by DFSClient_NONMAPREDUCE_451532259_8 19:00:22,252 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001 dst=null perm=null 19:00:22,269 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:22,270 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:22,272 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=null 19:00:22,273 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=null 19:00:22,274 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=null 19:00:22,275 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=null 19:00:22,277 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:22,278 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:22,279 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:22,280 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:22,281 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:22,282 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:22,283 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:22,285 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:22,286 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:22,352 INFO ClientRMService:342 - Application with id 1 submitted by user test 19:00:22,353 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0001 19:00:22,354 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0001 19:00:22,354 INFO RMAppImpl:643 - application_1394161202967_0001 State change from NEW to NEW_SAVING 19:00:22,364 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0001 19:00:22,373 INFO RMAppImpl:643 - application_1394161202967_0001 State change from NEW_SAVING to SUBMITTED 19:00:22,377 INFO FairScheduler:638 - Accepted application application_1394161202967_0001 from user: test, in queue: default, currently num of applications: 1 19:00:22,382 INFO RMAppImpl:643 - application_1394161202967_0001 State change from SUBMITTED to ACCEPTED 19:00:22,383 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0001_000001 19:00:22,384 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from NEW to SUBMITTED 19:00:22,390 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0001_000001 to scheduler from user: test 19:00:22,392 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from SUBMITTED to SCHEDULED 19:00:22,644 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161222644, maxDate=1394766022644, sequenceNumber=3, masterKeyId=2 19:00:22,645 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 3 19:00:22,695 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741828_1004 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741828 for deletion 19:00:22,697 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741829_1005 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741829 for deletion 19:00:22,697 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741830_1006 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741830 for deletion 19:00:22,697 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741828_1004 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741828 19:00:22,697 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741831_1007 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741831 for deletion 19:00:22,697 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741829_1005 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741829 19:00:22,697 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741832_1008 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741832 for deletion 19:00:22,697 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741830_1006 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741830 19:00:22,698 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741831_1007 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741831 19:00:22,698 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741833_1009 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741833 for deletion 19:00:22,698 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741834_1010 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741834 for deletion 19:00:22,698 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741832_1008 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741832 19:00:22,698 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741835_1011 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741835 for deletion 19:00:22,698 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741833_1009 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741833 19:00:22,698 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741836_1012 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741836 for deletion 19:00:22,698 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741834_1010 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741834 19:00:22,698 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741837_1013 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741837 for deletion 19:00:22,698 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741835_1011 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741835 19:00:22,698 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741836_1012 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741836 19:00:22,699 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741837_1013 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741837 19:00:22,890 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000001 Container Transitioned from NEW to ALLOCATED 19:00:22,890 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000001 19:00:22,892 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0001_01_000001 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:00:22,896 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:00:22,896 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0001 AttemptId: appattempt_1394161202967_0001_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0001_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:00:22,897 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:00:22,907 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:00:22,910 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0001_000001 19:00:22,924 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0001_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0001_000001 19:00:22,924 INFO AMLauncher:190 - Command to launch container container_1394161202967_0001_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:00:22,966 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0001_01_000001 by user test 19:00:22,986 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0001 19:00:22,988 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000001 19:00:22,989 INFO Application:450 - Application application_1394161202967_0001 transitioned from NEW to INITING 19:00:22,989 INFO Application:296 - Adding container_1394161202967_0001_01_000001 to application application_1394161202967_0001 19:00:22,994 INFO Application:450 - Application application_1394161202967_0001 transitioned from INITING to RUNNING 19:00:22,997 INFO Container:888 - Container container_1394161202967_0001_01_000001 transitioned from NEW to LOCALIZING 19:00:22,997 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0001 19:00:22,999 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0001_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0001_000001 19:00:22,999 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from ALLOCATED to LAUNCHED 19:00:23,008 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:00:23,008 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split transitioned from INIT to DOWNLOADING 19:00:23,008 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml transitioned from INIT to DOWNLOADING 19:00:23,008 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:00:23,008 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar transitioned from INIT to DOWNLOADING 19:00:23,009 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar transitioned from INIT to DOWNLOADING 19:00:23,009 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0001_01_000001 19:00:23,011 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar, 1394161221728, FILE, null } 19:00:23,020 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar, 1394161190149, FILE, null } 19:00:23,020 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar, 1394161190198, FILE, null } 19:00:23,227 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0001_01_000001.tokens. Credentials list: 19:00:23,467 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:23,477 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:23,467 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:23,505 INFO DefaultContainerExecutor:469 - Initializing user test 19:00:23,506 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:23,506 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:23,506 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:23,557 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:00:23,557 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:00:23,557 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib dst=null perm=null 19:00:23,590 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app dst=null perm=null 19:00:23,590 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:00:23,590 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:00:23,615 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:00:23,615 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:00:23,615 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a dst=null perm=null 19:00:23,671 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:23,682 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:23,671 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:00:23,718 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:23,718 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:23,718 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:23,750 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:23,750 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:23,750 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:23,812 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:23,985 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:23,985 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:23,985 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:23,999 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:00:24,063 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0001_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000001.tokens 19:00:24,087 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0001 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0001 19:00:24,328 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52762, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017, duration: 3967000 19:00:24,340 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52763, bytes: 488982, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, duration: 16084000 19:00:24,466 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741828_1004, blk_1073741829_1005, blk_1073741830_1006, blk_1073741831_1007, blk_1073741832_1008, blk_1073741833_1009, blk_1073741834_1010, blk_1073741835_1011, blk_1073741836_1012, blk_1073741837_1013] 19:00:24,471 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/filecache/10/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:00:24,572 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52761, bytes: 1482408, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, duration: 246852000 19:00:24,630 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/filecache/11/MRAppJar.jar) transitioned from DOWNLOADING to LOCALIZED 19:00:24,731 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/filecache/12/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar) transitioned from DOWNLOADING to LOCALIZED 19:00:24,863 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021 19:00:24,863 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003 19:00:24,865 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017 19:00:24,865 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020 19:00:24,866 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741827_1003 19:00:24,866 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016 19:00:24,866 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018 19:00:24,867 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022 19:00:24,867 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023 19:00:24,868 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014 19:00:24,868 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741830_1006 19:00:24,868 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017 19:00:24,869 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015 19:00:24,870 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741828_1004 19:00:24,870 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741829_1005 19:00:24,871 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=null 19:00:24,871 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023 19:00:24,871 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021 19:00:24,872 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741832_1008 19:00:24,872 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741831_1007 19:00:24,873 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019 19:00:24,873 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=null 19:00:24,874 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=null 19:00:24,897 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52762, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_795437370_807, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022, duration: 356000 19:00:25,012 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0001/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:00:25,071 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741834_1010 19:00:25,077 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741839_1015 19:00:25,106 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016 19:00:25,106 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741835_1011 19:00:25,106 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741833_1009 19:00:25,107 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741842_1018 19:00:25,107 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022 19:00:25,108 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=null 19:00:25,109 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741836_1012 19:00:25,109 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020 19:00:25,110 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741837_1013 19:00:25,110 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=null 19:00:25,112 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=null 19:00:25,136 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52763, bytes: 176, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_795437370_807, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021, duration: 340000 19:00:25,258 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0001/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:00:25,270 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741838_1014 19:00:25,304 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019 19:00:25,369 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:25,380 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:25,381 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:25,405 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52761, bytes: 80367, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_795437370_807, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023, duration: 409000 19:00:25,483 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0001/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:00:25,493 INFO Container:888 - Container container_1394161202967_0001_01_000001 transitioned from LOCALIZING to LOCALIZED 19:00:25,626 INFO Container:888 - Container container_1394161202967_0001_01_000001 transitioned from LOCALIZED to RUNNING 19:00:25,633 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:25,752 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741828_1004 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741828 for deletion 19:00:25,753 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741829_1005 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741829 for deletion 19:00:25,778 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741828_1004 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741828 19:00:25,789 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741830_1006 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741830 for deletion 19:00:25,789 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741829_1005 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741829 19:00:25,789 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741831_1007 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741831 for deletion 19:00:25,790 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741830_1006 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741830 19:00:25,790 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741832_1008 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741832 for deletion 19:00:25,790 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741831_1007 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741831 19:00:25,790 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741833_1009 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741833 for deletion 19:00:25,791 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741832_1008 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741832 19:00:25,791 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741834_1010 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741834 for deletion 19:00:25,791 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741835_1011 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741835 for deletion 19:00:25,791 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741833_1009 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741833 19:00:25,791 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741836_1012 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741836 for deletion 19:00:25,791 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741835_1011 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741835 19:00:25,792 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741837_1013 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741837 for deletion 19:00:25,792 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741834_1010 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741834 19:00:25,792 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741837_1013 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741837 19:00:25,823 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741836_1012 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741836 19:00:26,034 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000001/default_container_executor.sh] 19:00:28,342 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:00:28,380 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/COMMIT_STARTED dst=null perm=null 19:00:28,381 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/COMMIT_SUCCESS dst=null perm=null 19:00:28,383 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/COMMIT_FAIL dst=null perm=null 19:00:28,445 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001 dst=null perm=null 19:00:28,446 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:00:28,448 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:00:28,453 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=test:supergroup:rwxrwx--- 19:00:28,456 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:00:28,933 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=null 19:00:28,942 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.splitmetainfo dst=null perm=null 19:00:29,010 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52767, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741846_1022, duration: 3551000 19:00:30,072 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0001_000001 19:00:30,073 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0001 APPATTEMPTID=appattempt_1394161202967_0001_000001 19:00:30,073 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from LAUNCHED to RUNNING 19:00:30,073 INFO RMAppImpl:643 - application_1394161202967_0001 State change from ACCEPTED to RUNNING 19:00:30,126 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:00:30,129 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=test:supergroup:rwxrwxrwx 19:00:30,190 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:30,399 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741848_1024{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:30,446 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024 src: /127.0.0.1:52772 dest: /127.0.0.1:52685 19:00:30,448 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024 src: /127.0.0.1:52773 dest: /127.0.0.1:52682 19:00:30,479 INFO clienttrace:1215 - src: /127.0.0.1:52773, dest: /127.0.0.1:52682, bytes: 92703, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024, duration: 29663000 19:00:30,480 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:30,481 INFO clienttrace:1215 - src: /127.0.0.1:52772, dest: /127.0.0.1:52685, bytes: 92703, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024, duration: 30742000 19:00:30,481 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:30,486 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1_conf.xml is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:30,511 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001.summary dst=null perm=test:supergroup:rw-rw-rw- 19:00:30,514 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741849_1025{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:30,516 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025 src: /127.0.0.1:52774 dest: /127.0.0.1:52685 19:00:30,518 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025 src: /127.0.0.1:52775 dest: /127.0.0.1:52682 19:00:30,521 INFO clienttrace:1215 - src: /127.0.0.1:52775, dest: /127.0.0.1:52682, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025, duration: 1622000 19:00:30,521 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:30,522 INFO clienttrace:1215 - src: /127.0.0.1:52774, dest: /127.0.0.1:52685, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025, duration: 2221000 19:00:30,522 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:30,524 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001.summary is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:31,176 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000002 Container Transitioned from NEW to ALLOCATED 19:00:31,176 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000002 19:00:31,176 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0001_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:00:31,467 INFO JobHistory:198 - History Cleaner started 19:00:31,470 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done dst=null perm=null 19:00:31,471 INFO JobHistory:204 - History Cleaner complete 19:00:32,152 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:00:32,209 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:32,210 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:32,222 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:32,224 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:32,225 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:32,234 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:32,236 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:32,237 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:32,238 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:32,240 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:32,241 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:32,306 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0001_01_000002 by user test 19:00:32,306 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0001 19:00:32,306 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000002 19:00:32,307 INFO Application:450 - Application application_1394161202967_0001 transitioned from NEW to INITING 19:00:32,307 INFO Application:296 - Adding container_1394161202967_0001_01_000002 to application application_1394161202967_0001 19:00:32,308 INFO Application:450 - Application application_1394161202967_0001 transitioned from INITING to RUNNING 19:00:32,308 INFO Container:888 - Container container_1394161202967_0001_01_000002 transitioned from NEW to LOCALIZING 19:00:32,308 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0001 19:00:32,308 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0001 19:00:32,308 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:00:32,309 INFO ShuffleHandler:263 - Added token for job_1394161202967_0001 19:00:32,309 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml transitioned from INIT to DOWNLOADING 19:00:32,310 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar transitioned from INIT to DOWNLOADING 19:00:32,310 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:00:32,310 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar transitioned from INIT to DOWNLOADING 19:00:32,310 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0001_01_000002 19:00:32,310 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar, 1394161190149, FILE, null } 19:00:32,313 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar, 1394161221728, FILE, null } 19:00:32,313 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar, 1394161190198, FILE, null } 19:00:32,508 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0001_01_000002.tokens. Credentials list: 19:00:32,633 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:32,633 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:32,635 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:32,668 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:32,668 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:32,668 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:32,710 INFO DefaultContainerExecutor:469 - Initializing user test 19:00:32,712 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:00:32,712 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib dst=null perm=null 19:00:32,712 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:00:32,788 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:00:32,788 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:00:32,788 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app dst=null perm=null 19:00:32,815 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:00:32,815 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a dst=null perm=null 19:00:32,815 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:00:32,901 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:00:32,901 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:32,902 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:32,934 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:32,934 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:32,934 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:32,959 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:32,959 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:32,959 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:33,023 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:33,023 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:33,023 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:33,103 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:33,183 INFO AppSchedulingInfo:166 - checking for deactivate... 19:00:33,184 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:00:33,300 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52778, bytes: 488982, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741825_1001, duration: 108469000 19:00:33,365 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52780, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741841_1017, duration: 1912000 19:00:33,652 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0001_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000002.tokens 19:00:33,656 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52779, bytes: 1482408, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741826_1002, duration: 437329000 19:00:33,657 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0001 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0001 19:00:33,657 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/filecache/10/MRAppJar.jar) transitioned from DOWNLOADING to LOCALIZED 19:00:33,657 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/filecache/11/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:00:33,940 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/filecache/12/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar) transitioned from DOWNLOADING to LOCALIZED 19:00:34,122 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:34,124 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:34,126 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml dst=null perm=null 19:00:34,152 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52778, bytes: 80367, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-779876508_946, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741847_1023, duration: 404000 19:00:34,317 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0001/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:00:34,324 INFO Container:888 - Container container_1394161202967_0001_01_000002 transitioned from LOCALIZING to LOCALIZED 19:00:34,411 INFO Container:888 - Container container_1394161202967_0001_01_000002 transitioned from LOCALIZED to RUNNING 19:00:34,411 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:34,635 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000002/default_container_executor.sh] 19:00:34,877 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025 19:00:34,878 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024 19:00:35,346 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741849_1025 19:00:35,347 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024 19:00:37,443 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job.split dst=null perm=null 19:00:37,529 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52786, bytes: 176, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741845_1021, duration: 476000 19:00:37,582 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=null 19:00:37,599 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0001_m_000000_0/part-00000 dst=null perm=test:supergroup:rw-rw-rw- 19:00:37,622 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/0000000-140306190021371-oozie-rkan-W@mr-action dst=null perm=null 19:00:37,628 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/0000000-140306190021371-oozie-rkan-W@mr-action dst=null perm=test:supergroup:rw-rw-rw- 19:00:37,640 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/0000000-140306190021371-oozie-rkan-W@mr-action. BP-1007909283-172.16.1.64-1394161187028 blk_1073741850_1026{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:37,686 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026 src: /127.0.0.1:52787 dest: /127.0.0.1:52682 19:00:37,688 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026 src: /127.0.0.1:52788 dest: /127.0.0.1:52685 19:00:37,713 INFO clienttrace:1215 - src: /127.0.0.1:52788, dest: /127.0.0.1:52685, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026, duration: 23834000 19:00:37,713 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:37,714 INFO clienttrace:1215 - src: /127.0.0.1:52787, dest: /127.0.0.1:52682, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026, duration: 24550000 19:00:37,714 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:37,718 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/0000000-140306190021371-oozie-rkan-W@mr-action is closed by DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1 19:00:37,725 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52789, bytes: 9, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741844_1020, duration: 427000 19:00:37,796 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:00:37,797 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:00:37,815 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52786, bytes: 77869, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741843_1019, duration: 1153000 19:00:38,488 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output dst=null perm=null 19:00:38,492 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:00:38,494 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:00:38,519 INFO ClientRMService:229 - Allocated new applicationId: 2 19:00:38,541 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002 dst=null perm=null 19:00:38,547 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002 dst=null perm=test:supergroup:rwxrwxrwx 19:00:38,558 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002 dst=null perm=test:supergroup:rwx------ 19:00:38,566 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:38,567 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:38,569 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:38,570 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:38,571 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:00:38,572 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:00:38,572 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:00:38,573 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:00:38,574 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:00:38,576 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:38,577 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:38,578 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib dst=null perm=null 19:00:38,579 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app dst=null perm=null 19:00:38,580 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a dst=null perm=null 19:00:38,581 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:00:38,595 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input dst=null perm=null 19:00:38,598 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input dst=null perm=null 19:00:38,601 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input dst=null perm=null 19:00:38,631 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:00:38,634 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=test:supergroup:rw-r--r-- 19:00:38,639 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split 19:00:38,641 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:38,649 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741851_1027{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:00:38,651 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027 src: /127.0.0.1:52791 dest: /127.0.0.1:52682 19:00:38,652 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027 src: /127.0.0.1:52792 dest: /127.0.0.1:52685 19:00:38,656 INFO clienttrace:1215 - src: /127.0.0.1:52792, dest: /127.0.0.1:52685, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027, duration: 2022000 19:00:38,656 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:38,656 INFO clienttrace:1215 - src: /127.0.0.1:52791, dest: /127.0.0.1:52682, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027, duration: 2332000 19:00:38,656 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:38,658 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split is closed by DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1 19:00:38,660 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:00:38,662 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:00:38,667 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741852_1028{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:00:38,670 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028 src: /127.0.0.1:52793 dest: /127.0.0.1:52682 19:00:38,671 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028 src: /127.0.0.1:52794 dest: /127.0.0.1:52685 19:00:38,673 INFO clienttrace:1215 - src: /127.0.0.1:52794, dest: /127.0.0.1:52685, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028, duration: 1448000 19:00:38,674 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:38,674 INFO clienttrace:1215 - src: /127.0.0.1:52793, dest: /127.0.0.1:52682, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028, duration: 2201000 19:00:38,675 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:38,677 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo is closed by DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1 19:00:38,690 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:38,692 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:00:38,726 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741853_1029{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:38,729 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029 src: /127.0.0.1:52795 dest: /127.0.0.1:52685 19:00:38,731 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029 src: /127.0.0.1:52796 dest: /127.0.0.1:52682 19:00:38,734 INFO clienttrace:1215 - src: /127.0.0.1:52796, dest: /127.0.0.1:52682, bytes: 77358, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029, duration: 1989000 19:00:38,734 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:38,735 INFO clienttrace:1215 - src: /127.0.0.1:52795, dest: /127.0.0.1:52685, bytes: 77358, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029, duration: 2650000 19:00:38,735 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:38,736 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml is closed by DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1 19:00:38,923 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002 dst=null perm=null 19:00:39,106 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:39,108 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:39,114 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:39,115 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:39,117 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=null 19:00:39,118 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=null 19:00:39,123 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:39,125 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:39,126 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:39,127 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:39,128 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:39,129 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:39,131 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:39,132 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:39,133 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:39,191 INFO ClientRMService:342 - Application with id 2 submitted by user test 19:00:39,191 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0002 19:00:39,191 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0002 19:00:39,192 INFO RMAppImpl:643 - application_1394161202967_0002 State change from NEW to NEW_SAVING 19:00:39,192 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0002 19:00:39,192 INFO RMAppImpl:643 - application_1394161202967_0002 State change from NEW_SAVING to SUBMITTED 19:00:39,192 INFO FairScheduler:638 - Accepted application application_1394161202967_0002 from user: test, in queue: default, currently num of applications: 2 19:00:39,193 INFO RMAppImpl:643 - application_1394161202967_0002 State change from SUBMITTED to ACCEPTED 19:00:39,193 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0002_000001 19:00:39,193 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from NEW to SUBMITTED 19:00:39,193 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0002_000001 to scheduler from user: test 19:00:39,194 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from SUBMITTED to SCHEDULED 19:00:39,197 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000001 Container Transitioned from NEW to ALLOCATED 19:00:39,197 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000001 19:00:39,197 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0002_01_000001 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:00:39,198 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:00:39,198 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0002 AttemptId: appattempt_1394161202967_0002_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0002_01_000001, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] 19:00:39,199 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:00:39,199 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:00:39,199 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0002_000001 19:00:39,201 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0002_01_000001, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0002_000001 19:00:39,201 INFO AMLauncher:190 - Command to launch container container_1394161202967_0002_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:00:39,209 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0002_01_000001 by user test 19:00:39,210 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0002 19:00:39,210 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000001 19:00:39,210 INFO Application:450 - Application application_1394161202967_0002 transitioned from NEW to INITING 19:00:39,210 INFO Application:296 - Adding container_1394161202967_0002_01_000001 to application application_1394161202967_0002 19:00:39,211 INFO Application:450 - Application application_1394161202967_0002 transitioned from INITING to RUNNING 19:00:39,211 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0002_01_000001, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0002_000001 19:00:39,211 INFO Container:888 - Container container_1394161202967_0002_01_000001 transitioned from NEW to LOCALIZING 19:00:39,211 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from ALLOCATED to LAUNCHED 19:00:39,211 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0002 19:00:39,211 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:00:39,211 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split transitioned from INIT to DOWNLOADING 19:00:39,212 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml transitioned from INIT to DOWNLOADING 19:00:39,212 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0002_01_000001 19:00:39,263 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0002_01_000001.tokens. Credentials list: 19:00:39,387 INFO DefaultContainerExecutor:469 - Initializing user test 19:00:39,519 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=test:supergroup:rw-rw-rw- 19:00:39,579 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq. BP-1007909283-172.16.1.64-1394161187028 blk_1073741854_1030{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:00:39,643 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030 src: /127.0.0.1:52799 dest: /127.0.0.1:52682 19:00:39,683 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030 src: /127.0.0.1:52800 dest: /127.0.0.1:52685 19:00:39,742 INFO clienttrace:1215 - src: /127.0.0.1:52800, dest: /127.0.0.1:52685, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, duration: 7518000 19:00:39,752 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:39,789 INFO clienttrace:1215 - src: /127.0.0.1:52799, dest: /127.0.0.1:52682, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, duration: 66883000 19:00:39,811 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:39,828 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq is closed by DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1 19:00:39,845 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0001_m_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0001_m_000000_0_1012047672_1 19:00:39,883 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0002_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000001.tokens 19:00:39,883 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0002 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0002 19:00:39,895 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029 19:00:39,905 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028 19:00:39,905 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027 19:00:39,906 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026 19:00:39,906 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0001_m_000000_0 dst=null perm=null 19:00:39,906 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030 19:00:39,928 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0001_m_000000_0 dst=null perm=null 19:00:39,953 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0001_m_000000 dst=null perm=null 19:00:40,008 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0001_m_000000_0 dst=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0001_m_000000 perm=test:supergroup:rwxrwxrwx 19:00:40,070 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0001_01_000002 19:00:40,071 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000002 19:00:40,071 INFO Container:888 - Container container_1394161202967_0001_01_000002 transitioned from RUNNING to KILLING 19:00:40,072 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0001_01_000002 19:00:40,078 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:00:40,127 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0001_01_000002 is : 143 19:00:40,225 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741855_1031{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:00:40,225 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:00:40,289 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:40,295 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031 src: /127.0.0.1:52802 dest: /127.0.0.1:52682 19:00:40,301 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=null 19:00:40,302 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031 src: /127.0.0.1:52803 dest: /127.0.0.1:52685 19:00:40,303 INFO Container:888 - Container container_1394161202967_0001_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:00:40,306 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000002 19:00:40,306 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=null 19:00:40,307 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000002 19:00:40,307 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000002 19:00:40,308 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000002 19:00:40,308 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000002 19:00:40,309 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=null 19:00:40,311 INFO Container:888 - Container container_1394161202967_0001_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:00:40,311 INFO Application:339 - Removing container_1394161202967_0001_01_000002 from application application_1394161202967_0001 19:00:40,311 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:40,311 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=null 19:00:40,311 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0001 19:00:40,312 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1.jhist for DFSClient_NONMAPREDUCE_2014257723_1 19:00:40,314 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:00:40,315 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:00:40,316 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0001_m_000000 dst=null perm=null 19:00:40,358 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/part-00000 dst=null perm=null 19:00:40,360 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028 19:00:40,360 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52805, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1322947898_1043, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028, duration: 309000 19:00:40,361 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029 19:00:40,396 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027 19:00:40,402 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0001_m_000000/part-00000 dst=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/part-00000 perm=test:supergroup:rw-rw-rw- 19:00:40,459 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741850_1026 19:00:40,493 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030 19:00:40,521 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_temporary dst=null perm=null 19:00:40,552 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0002/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:00:40,554 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:00:40,588 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:40,636 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:00:40,679 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:40,681 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:40,683 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:40,685 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:40,726 INFO clienttrace:1215 - src: /127.0.0.1:52803, dest: /127.0.0.1:52685, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031, duration: 422051000 19:00:40,726 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:40,726 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52806, bytes: 283, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1322947898_1043, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027, duration: 238000 19:00:40,727 INFO clienttrace:1215 - src: /127.0.0.1:52802, dest: /127.0.0.1:52682, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031, duration: 422814000 19:00:40,727 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:40,765 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1.jhist is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:40,817 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:00:40,872 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741856_1032{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:40,875 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0002/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:00:40,876 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032 src: /127.0.0.1:52807 dest: /127.0.0.1:52682 19:00:40,878 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032 src: /127.0.0.1:52808 dest: /127.0.0.1:52685 19:00:40,958 INFO clienttrace:1215 - src: /127.0.0.1:52808, dest: /127.0.0.1:52685, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032, duration: 21761000 19:00:40,963 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:40,992 INFO clienttrace:1215 - src: /127.0.0.1:52807, dest: /127.0.0.1:52682, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032, duration: 74322000 19:00:40,994 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:40,995 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:40,996 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary_tmp is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:40,996 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:40,998 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:41,007 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:00:41,039 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1.jhist dst=null perm=null 19:00:41,041 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist_tmp dst=null perm=null 19:00:41,041 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52806, bytes: 77966, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1322947898_1043, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029, duration: 421000 19:00:41,079 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1.jhist dst=null perm=null 19:00:41,079 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0001_01_000002 19:00:41,136 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist_tmp dst=null perm=null 19:00:41,170 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000002 Container Transitioned from RUNNING to COMPLETED 19:00:41,180 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0001_01_000002 in state: COMPLETED event:FINISHED 19:00:41,181 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000002 19:00:41,181 INFO FSSchedulerNode:162 - Released container container_1394161202967_0001_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:00:41,187 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0001_000001 released container container_1394161202967_0001_01_000002 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:00:41,188 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1.jhist dst=null perm=null 19:00:41,218 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0002/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:00:41,218 INFO Container:888 - Container container_1394161202967_0002_01_000001 transitioned from LOCALIZING to LOCALIZED 19:00:41,220 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:00:41,223 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52809, bytes: 20872, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031, duration: 369000 19:00:41,226 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741857_1033{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:41,229 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033 src: /127.0.0.1:52810 dest: /127.0.0.1:52685 19:00:41,232 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033 src: /127.0.0.1:52811 dest: /127.0.0.1:52682 19:00:41,237 INFO clienttrace:1215 - src: /127.0.0.1:52811, dest: /127.0.0.1:52682, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033, duration: 2001000 19:00:41,237 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:41,238 INFO clienttrace:1215 - src: /127.0.0.1:52810, dest: /127.0.0.1:52685, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033, duration: 3084000 19:00:41,238 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:41,240 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist_tmp is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:41,242 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:00:41,244 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1_conf.xml dst=null perm=null 19:00:41,277 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml_tmp dst=null perm=null 19:00:41,279 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1_conf.xml dst=null perm=null 19:00:41,312 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml_tmp dst=null perm=null 19:00:41,330 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001/job_1394161202967_0001_1_conf.xml dst=null perm=null 19:00:41,402 INFO Container:888 - Container container_1394161202967_0002_01_000001 transitioned from LOCALIZED to RUNNING 19:00:41,405 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:41,497 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:00:41,534 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52812, bytes: 93431, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741848_1024, duration: 547000 19:00:41,568 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741858_1034{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:00:41,634 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034 src: /127.0.0.1:52813 dest: /127.0.0.1:52685 19:00:41,667 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034 src: /127.0.0.1:52814 dest: /127.0.0.1:52682 19:00:41,726 INFO clienttrace:1215 - src: /127.0.0.1:52814, dest: /127.0.0.1:52682, bytes: 92703, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034, duration: 32759000 19:00:41,739 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:41,741 INFO clienttrace:1215 - src: /127.0.0.1:52813, dest: /127.0.0.1:52685, bytes: 92703, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2014257723_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034, duration: 71210000 19:00:41,741 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:41,772 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_2014257723_1 19:00:41,804 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:00:41,870 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary perm=test:supergroup:rwxrwx--- 19:00:41,934 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml perm=test:supergroup:rwxrwx--- 19:00:41,934 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000001/default_container_executor.sh] 19:00:41,976 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist perm=test:supergroup:rwxrwx--- 19:00:41,990 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0001_000001 with final state: FINISHING 19:00:41,991 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from RUNNING to FINAL_SAVING 19:00:41,991 INFO RMAppImpl:880 - Updating application application_1394161202967_0001 with final state: FINISHING 19:00:41,992 INFO RMAppImpl:643 - application_1394161202967_0001 State change from RUNNING to FINAL_SAVING 19:00:41,992 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0001 19:00:41,996 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from FINAL_SAVING to FINISHING 19:00:41,996 INFO RMAppImpl:643 - application_1394161202967_0001 State change from FINAL_SAVING to FINISHING 19:00:43,006 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0001 dst=null perm=null 19:00:43,218 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:00:43,240 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:00:43,241 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:00:43,245 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52805, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, duration: 303000 19:00:43,321 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161243321, maxDate=1394766043321, sequenceNumber=4, masterKeyId=2 19:00:43,321 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 4 19:00:43,347 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:00:43,350 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:00:43,361 INFO CompletedJob:95 - Loading job: job_1394161202967_0001 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist 19:00:43,361 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist] 19:00:43,362 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist dst=null perm=null 19:00:43,367 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist dst=null perm=null 19:00:43,374 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52806, bytes: 20872, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033, duration: 312000 19:00:43,823 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary dst=null perm=null 19:00:43,825 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52805, bytes: 420, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032, duration: 196000 19:00:43,825 INFO JobSummary:372 - jobId=job_1394161202967_0001,submitTime=1394161222308,launchTime=1394161230117,firstMapTaskLaunchTime=1394161232313,firstReduceTaskLaunchTime=0,finishTime=1394161240681,resourcesPerMap=1024,resourcesPerReduce=0,numMaps=1,numReduces=0,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=7,reduceSlotSeconds=0,jobName=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190021371-oozie-rkan-W 19:00:43,826 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary] 19:00:43,827 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001.summary dst=null perm=null 19:00:43,830 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:00:43,833 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=rkanter:supergroup:rwxrwx--- 19:00:43,834 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:00:43,834 INFO HistoryFileManager:969 - Perms after creating 504, Expected: 504 19:00:43,835 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist 19:00:43,842 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0001-1394161222308-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161240681-1-0-SUCCEEDED-root.test-1394161230117.jhist perm=test:supergroup:rwxrwx--- 19:00:43,843 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0001_conf.xml 19:00:43,845 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0001_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0001_conf.xml perm=test:supergroup:rwxrwx--- 19:00:43,882 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:00:43,883 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:00:43,885 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:00:43,888 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52805, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, duration: 256000 19:00:43,955 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161243955, maxDate=1394766043955, sequenceNumber=5, masterKeyId=2 19:00:43,955 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 5 19:00:44,240 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:00:44,276 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/COMMIT_STARTED dst=null perm=null 19:00:44,278 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/COMMIT_SUCCESS dst=null perm=null 19:00:44,279 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/COMMIT_FAIL dst=null perm=null 19:00:44,354 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002 dst=null perm=null 19:00:44,355 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:00:44,357 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:00:44,758 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=null 19:00:44,767 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.splitmetainfo dst=null perm=null 19:00:44,834 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52822, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741852_1028, duration: 304000 19:00:44,915 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033 19:00:44,915 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031 19:00:44,915 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032 19:00:44,917 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034 19:00:45,521 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741855_1031 19:00:45,524 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741858_1034 19:00:45,525 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741856_1032 19:00:45,525 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741857_1033 19:00:45,530 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741856_1032, blk_1073741845_1021, blk_1073741846_1022, blk_1073741847_1023, blk_1073741848_1024, blk_1073741849_1025, blk_1073741855_1031] 19:00:45,891 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0002_000001 19:00:45,892 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0002 APPATTEMPTID=appattempt_1394161202967_0002_000001 19:00:45,892 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from LAUNCHED to RUNNING 19:00:45,892 INFO RMAppImpl:643 - application_1394161202967_0002 State change from ACCEPTED to RUNNING 19:00:45,919 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:00:45,921 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1 dst=null perm=test:testg:rwxrwxrwx 19:00:45,981 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:00:46,202 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741859_1035{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:46,254 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035 src: /127.0.0.1:52828 dest: /127.0.0.1:52685 19:00:46,255 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035 src: /127.0.0.1:52829 dest: /127.0.0.1:52682 19:00:46,292 INFO clienttrace:1215 - src: /127.0.0.1:52829, dest: /127.0.0.1:52682, bytes: 91661, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035, duration: 35323000 19:00:46,292 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:46,293 INFO clienttrace:1215 - src: /127.0.0.1:52828, dest: /127.0.0.1:52685, bytes: 91661, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035, duration: 36145000 19:00:46,293 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:46,298 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1_conf.xml is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:46,324 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002.summary dst=null perm=test:supergroup:rw-rw-rw- 19:00:46,327 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741860_1036{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:46,329 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036 src: /127.0.0.1:52830 dest: /127.0.0.1:52682 19:00:46,331 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036 src: /127.0.0.1:52831 dest: /127.0.0.1:52685 19:00:46,335 INFO clienttrace:1215 - src: /127.0.0.1:52831, dest: /127.0.0.1:52685, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036, duration: 2212000 19:00:46,336 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:46,336 INFO clienttrace:1215 - src: /127.0.0.1:52830, dest: /127.0.0.1:52682, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036, duration: 3198000 19:00:46,337 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:46,339 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002.summary is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:46,688 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741856_1032 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741856 for deletion 19:00:46,688 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741845_1021 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741845 for deletion 19:00:46,688 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741846_1022 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741846 for deletion 19:00:46,689 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741856_1032 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741856 19:00:46,689 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741847_1023 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741847 for deletion 19:00:46,689 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741845_1021 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741845 19:00:46,689 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741848_1024 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741848 for deletion 19:00:46,689 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741846_1022 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741846 19:00:46,689 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741847_1023 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741847 19:00:46,689 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741849_1025 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741849 for deletion 19:00:46,690 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741848_1024 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741848 19:00:46,690 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741855_1031 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741855 for deletion 19:00:46,690 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741849_1025 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741849 19:00:46,691 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741855_1031 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741855 19:00:47,087 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000002 Container Transitioned from NEW to ALLOCATED 19:00:47,087 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000002 19:00:47,087 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0002_01_000002 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:00:47,114 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000003 Container Transitioned from NEW to ALLOCATED 19:00:47,114 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000003 19:00:47,114 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0002_01_000003 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:00:47,943 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:00:47,944 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:00:48,007 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:48,009 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:48,023 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:48,026 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:48,029 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:48,039 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:48,041 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:48,044 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:48,045 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:00:48,047 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:00:48,049 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/app/lib/test.jar dst=null perm=null 19:00:48,128 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0002_01_000003 by user test 19:00:48,128 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0002_01_000002 by user test 19:00:48,128 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000003 19:00:48,128 INFO Application:296 - Adding container_1394161202967_0002_01_000003 to application application_1394161202967_0002 19:00:48,128 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0002 19:00:48,129 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000002 19:00:48,132 INFO Container:888 - Container container_1394161202967_0002_01_000003 transitioned from NEW to LOCALIZING 19:00:48,132 INFO Application:450 - Application application_1394161202967_0002 transitioned from NEW to INITING 19:00:48,132 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0002 19:00:48,132 INFO Application:296 - Adding container_1394161202967_0002_01_000002 to application application_1394161202967_0002 19:00:48,133 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0002 19:00:48,133 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:00:48,133 INFO Application:450 - Application application_1394161202967_0002 transitioned from INITING to RUNNING 19:00:48,133 INFO ShuffleHandler:263 - Added token for job_1394161202967_0002 19:00:48,133 INFO Container:888 - Container container_1394161202967_0002_01_000002 transitioned from NEW to LOCALIZING 19:00:48,133 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0002 19:00:48,134 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0002 19:00:48,134 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:00:48,134 INFO Container:888 - Container container_1394161202967_0002_01_000003 transitioned from LOCALIZING to LOCALIZED 19:00:48,134 INFO ShuffleHandler:263 - Added token for job_1394161202967_0002 19:00:48,134 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml transitioned from INIT to DOWNLOADING 19:00:48,135 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0002_01_000002 19:00:48,184 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/nmPrivate/container_1394161202967_0002_01_000002.tokens. Credentials list: 19:00:48,254 INFO DefaultContainerExecutor:469 - Initializing user test 19:00:48,482 INFO Container:888 - Container container_1394161202967_0002_01_000003 transitioned from LOCALIZED to RUNNING 19:00:48,482 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:48,540 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741856_1032, blk_1073741845_1021, blk_1073741846_1022, blk_1073741847_1023, blk_1073741848_1024, blk_1073741849_1025, blk_1073741855_1031] 19:00:48,631 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/nmPrivate/container_1394161202967_0002_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000002.tokens 19:00:48,632 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0002 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0002 19:00:48,837 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000003/default_container_executor.sh] 19:00:48,944 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:48,945 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:48,947 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml dst=null perm=null 19:00:48,962 INFO AppSchedulingInfo:166 - checking for deactivate... 19:00:48,963 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52836, bytes: 77966, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_965317933_1194, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741853_1029, duration: 556000 19:00:49,042 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0002/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:00:49,042 INFO Container:888 - Container container_1394161202967_0002_01_000002 transitioned from LOCALIZING to LOCALIZED 19:00:49,065 INFO ContainerLaunch:322 - Container container_1394161202967_0001_01_000001 succeeded 19:00:49,065 INFO Container:888 - Container container_1394161202967_0001_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:00:49,066 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0001_01_000001 19:00:49,095 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:00:49,116 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:00:49,157 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000001 19:00:49,160 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000001 19:00:49,160 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000001 19:00:49,161 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0001/container_1394161202967_0001_01_000001 19:00:49,166 INFO Container:888 - Container container_1394161202967_0002_01_000002 transitioned from LOCALIZED to RUNNING 19:00:49,177 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000001 19:00:49,187 INFO Container:888 - Container container_1394161202967_0001_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 19:00:49,187 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:49,187 INFO Application:339 - Removing container_1394161202967_0001_01_000001 from application application_1394161202967_0001 19:00:49,187 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:49,187 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0001 19:00:49,416 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000002/default_container_executor.sh] 19:00:49,689 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741856_1032 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741856 for deletion 19:00:49,689 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741845_1021 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741845 for deletion 19:00:49,689 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741846_1022 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741846 for deletion 19:00:49,690 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741847_1023 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741847 for deletion 19:00:49,690 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741856_1032 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741856 19:00:49,690 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741848_1024 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741848 for deletion 19:00:49,690 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741849_1025 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741849 for deletion 19:00:49,690 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741855_1031 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741855 for deletion 19:00:49,690 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741846_1022 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741846 19:00:49,690 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741845_1021 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741845 19:00:49,691 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741848_1024 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741848 19:00:49,691 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741847_1023 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741847 19:00:49,691 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741849_1025 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741849 19:00:49,691 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741855_1031 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741855 19:00:49,923 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035 19:00:49,923 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036 19:00:50,094 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0001_01_000001 19:00:50,096 INFO RMContainerImpl:220 - container_1394161202967_0001_01_000001 Container Transitioned from RUNNING to COMPLETED 19:00:50,096 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0001_01_000001 in state: COMPLETED event:FINISHED 19:00:50,096 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0001_000001 19:00:50,096 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0001 CONTAINERID=container_1394161202967_0001_01_000001 19:00:50,096 INFO FSSchedulerNode:162 - Released container container_1394161202967_0001_01_000001 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:00:50,096 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0001_000001 released container container_1394161202967_0001_01_000001 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:00:50,098 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0001_000001 State change from FINISHING to FINISHED 19:00:50,100 INFO RMAppImpl:643 - application_1394161202967_0001 State change from FINISHING to FINISHED 19:00:50,101 INFO FairScheduler:720 - Application appattempt_1394161202967_0001_000001 is done. finalState=FINISHED 19:00:50,101 INFO AppSchedulingInfo:108 - Application application_1394161202967_0001 requests cleared 19:00:50,101 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0001_000001 19:00:50,102 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0001 19:00:50,105 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0001,name=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190021371-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0001/jobhistory/job/job_1394161202967_0001,appMasterHost=rkanter-mbp.local,startTime=1394161222352,finishTime=1394161241991,finalStatus=SUCCEEDED 19:00:50,111 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0001_01_000001 19:00:50,119 INFO Application:450 - Application application_1394161202967_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:00:50,120 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0001 19:00:50,121 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0001 19:00:50,121 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0001 19:00:50,121 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0001 19:00:50,121 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0001 19:00:50,123 INFO Application:450 - Application application_1394161202967_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:00:50,124 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0001, with delay of 10800 seconds 19:00:50,531 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035 19:00:50,532 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741860_1036 19:00:51,097 INFO Application:450 - Application application_1394161202967_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:00:51,098 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0001 19:00:51,098 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0001 19:00:51,098 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0001 19:00:51,098 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0001 19:00:51,098 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0001 19:00:51,099 INFO Application:450 - Application application_1394161202967_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:00:51,099 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0001, with delay of 10800 seconds 19:00:52,232 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:52,331 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52841, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0002_m_000001_0_-1344416268_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027, duration: 653000 19:00:52,384 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input/data.txt dst=null perm=null 19:00:52,387 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52841, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0002_m_000001_0_-1344416268_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016, duration: 471000 19:00:52,607 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_m_000001_0 dst=null perm=null 19:00:52,647 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0002_01_000003 19:00:52,647 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000003 19:00:52,647 INFO Container:888 - Container container_1394161202967_0002_01_000003 transitioned from RUNNING to KILLING 19:00:52,647 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0002_01_000003 19:00:52,673 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741861_1037{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:52,698 INFO FSEditLog:674 - Number of transactions: 288 Total time for transactions(ms): 151 Number of transactions batched in Syncs: 11 Number of syncs: 188 SyncTimes(ms): 190 56 19:00:52,700 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0002_01_000003 is : 143 19:00:52,705 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037 src: /127.0.0.1:52842 dest: /127.0.0.1:52685 19:00:52,707 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037 src: /127.0.0.1:52843 dest: /127.0.0.1:52682 19:00:52,712 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1.jhist for DFSClient_NONMAPREDUCE_2092908579_1 19:00:52,726 INFO Container:888 - Container container_1394161202967_0002_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:00:52,727 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000003 19:00:52,727 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000003 19:00:52,727 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000003 19:00:52,728 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000003 19:00:52,728 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000003 19:00:52,728 INFO Container:888 - Container container_1394161202967_0002_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:00:52,728 INFO Application:339 - Removing container_1394161202967_0002_01_000003 from application application_1394161202967_0002 19:00:52,728 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:52,728 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0002 19:00:52,771 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job.split dst=null perm=null 19:00:52,851 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52845, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0002_m_000000_0_1039293782_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741851_1027, duration: 228000 19:00:52,906 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/input/data.txt dst=null perm=null 19:00:52,984 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52845, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0002_m_000000_0_1039293782_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741840_1016, duration: 421000 19:00:53,004 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_m_000000_0 dst=null perm=null 19:00:53,017 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0002_01_000002 19:00:53,017 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000002 19:00:53,017 INFO Container:888 - Container container_1394161202967_0002_01_000002 transitioned from RUNNING to KILLING 19:00:53,017 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0002_01_000002 19:00:53,055 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0002_01_000002 is : 143 19:00:53,075 INFO Container:888 - Container container_1394161202967_0002_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:00:53,076 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000002 19:00:53,076 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000002 19:00:53,076 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000002 19:00:53,076 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000002 19:00:53,077 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000002 19:00:53,077 INFO Container:888 - Container container_1394161202967_0002_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:00:53,077 INFO Application:339 - Removing container_1394161202967_0002_01_000002 from application application_1394161202967_0002 19:00:53,077 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:53,077 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0002 19:00:53,649 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0002_01_000003 19:00:53,651 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000003 Container Transitioned from RUNNING to COMPLETED 19:00:53,652 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0002_01_000003 in state: COMPLETED event:FINISHED 19:00:53,652 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000003 19:00:53,652 INFO FSSchedulerNode:162 - Released container container_1394161202967_0002_01_000003 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:00:53,652 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0002_000001 released container container_1394161202967_0002_01_000003 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:00:54,019 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0002_01_000002 19:00:54,019 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000002 Container Transitioned from RUNNING to COMPLETED 19:00:54,020 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0002_01_000002 in state: COMPLETED event:FINISHED 19:00:54,020 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000002 19:00:54,020 INFO FSSchedulerNode:162 - Released container container_1394161202967_0002_01_000002 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:00:54,020 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0002_000001 released container container_1394161202967_0002_01_000002 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:00:54,020 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000004 Container Transitioned from NEW to ALLOCATED 19:00:54,020 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000004 19:00:54,021 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0002_01_000004 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:00:55,017 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 19:00:55,032 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0002_01_000004 by user test 19:00:55,032 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000004 19:00:55,032 INFO Application:296 - Adding container_1394161202967_0002_01_000004 to application application_1394161202967_0002 19:00:55,033 INFO Container:888 - Container container_1394161202967_0002_01_000004 transitioned from NEW to LOCALIZING 19:00:55,033 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0002 19:00:55,033 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0002 19:00:55,033 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:00:55,033 INFO ShuffleHandler:263 - Added token for job_1394161202967_0002 19:00:55,034 INFO Container:888 - Container container_1394161202967_0002_01_000004 transitioned from LOCALIZING to LOCALIZED 19:00:55,117 INFO Container:888 - Container container_1394161202967_0002_01_000004 transitioned from LOCALIZED to RUNNING 19:00:55,118 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:55,332 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000004/default_container_executor.sh] 19:00:56,021 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000004 Container Transitioned from ACQUIRED to RUNNING 19:00:56,025 INFO AppSchedulingInfo:166 - checking for deactivate... 19:00:58,654 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_r_000000_0/part-00000 dst=null perm=test:testg:rw-rw-rw- 19:00:58,694 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_r_000000_0/part-00000. BP-1007909283-172.16.1.64-1394161187028 blk_1073741862_1038{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:58,793 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038 src: /127.0.0.1:52850 dest: /127.0.0.1:52682 19:00:58,795 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038 src: /127.0.0.1:52851 dest: /127.0.0.1:52685 19:00:58,828 INFO clienttrace:1215 - src: /127.0.0.1:52851, dest: /127.0.0.1:52685, bytes: 16, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0002_r_000000_0_78613495_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038, duration: 32087000 19:00:58,829 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:58,829 INFO clienttrace:1215 - src: /127.0.0.1:52850, dest: /127.0.0.1:52682, bytes: 16, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0002_r_000000_0_78613495_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038, duration: 32925000 19:00:58,830 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:58,833 INFO StateChange:2959 - DIR* completeFile: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_r_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0002_r_000000_0_78613495_1 19:00:58,845 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_r_000000_0 dst=null perm=null 19:00:58,853 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_r_000000_0 dst=null perm=null 19:00:58,854 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/task_1394161202967_0002_r_000000 dst=null perm=null 19:00:58,858 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/_temporary/attempt_1394161202967_0002_r_000000_0 dst=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/task_1394161202967_0002_r_000000 perm=test:testg:rwxrwxrwx 19:00:58,872 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0002_01_000004 19:00:58,872 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000004 19:00:58,872 INFO Container:888 - Container container_1394161202967_0002_01_000004 transitioned from RUNNING to KILLING 19:00:58,872 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0002_01_000004 19:00:58,926 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0002_01_000004 is : 143 19:00:58,927 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:00:58,930 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:58,934 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1 dst=null perm=null 19:00:58,941 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output dst=null perm=null 19:00:58,942 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output dst=null perm=null 19:00:58,943 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/task_1394161202967_0002_r_000000 dst=null perm=null 19:00:58,944 INFO Container:888 - Container container_1394161202967_0002_01_000004 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:00:58,945 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/part-00000 dst=null perm=null 19:00:58,945 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000004 19:00:58,945 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000004 19:00:58,945 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000004 19:00:58,946 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000004 19:00:58,946 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000004 19:00:58,946 INFO Container:888 - Container container_1394161202967_0002_01_000004 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:00:58,946 INFO Application:339 - Removing container_1394161202967_0002_01_000004 from application application_1394161202967_0002 19:00:58,947 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:00:58,947 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0002 19:00:58,950 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary/1/task_1394161202967_0002_r_000000/part-00000 dst=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/part-00000 perm=test:testg:rw-rw-rw- 19:00:58,956 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_temporary dst=null perm=null 19:00:58,961 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_SUCCESS dst=null perm=test:testg:rw-rw-rw- 19:00:58,963 INFO StateChange:2959 - DIR* completeFile: /user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:58,965 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:00:58,967 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:58,990 INFO clienttrace:1215 - src: /127.0.0.1:52843, dest: /127.0.0.1:52682, bytes: 39301, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037, duration: 6281476000 19:00:58,991 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:58,992 INFO clienttrace:1215 - src: /127.0.0.1:52842, dest: /127.0.0.1:52685, bytes: 39301, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037, duration: 6283204000 19:00:58,992 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:58,994 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1.jhist is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:58,997 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:00:58,999 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741863_1039{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:59,002 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039 src: /127.0.0.1:52852 dest: /127.0.0.1:52682 19:00:59,004 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039 src: /127.0.0.1:52853 dest: /127.0.0.1:52685 19:00:59,007 INFO clienttrace:1215 - src: /127.0.0.1:52853, dest: /127.0.0.1:52685, bytes: 429, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039, duration: 1797000 19:00:59,007 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:59,008 INFO clienttrace:1215 - src: /127.0.0.1:52852, dest: /127.0.0.1:52682, bytes: 429, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039, duration: 2729000 19:00:59,008 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:59,011 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary_tmp is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:59,015 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:00:59,021 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1.jhist dst=null perm=null 19:00:59,022 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist_tmp dst=null perm=null 19:00:59,026 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1.jhist dst=null perm=null 19:00:59,027 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist_tmp dst=null perm=null 19:00:59,028 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1.jhist dst=null perm=null 19:00:59,031 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:00:59,034 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52854, bytes: 39609, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037, duration: 247000 19:00:59,036 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741864_1040{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:00:59,038 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040 src: /127.0.0.1:52855 dest: /127.0.0.1:52685 19:00:59,040 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040 src: /127.0.0.1:52856 dest: /127.0.0.1:52682 19:00:59,044 INFO clienttrace:1215 - src: /127.0.0.1:52856, dest: /127.0.0.1:52682, bytes: 39301, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040, duration: 2716000 19:00:59,044 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:59,045 INFO clienttrace:1215 - src: /127.0.0.1:52855, dest: /127.0.0.1:52685, bytes: 39301, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040, duration: 3162000 19:00:59,045 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:59,047 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist_tmp is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:59,049 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:00:59,050 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1_conf.xml dst=null perm=null 19:00:59,051 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml_tmp dst=null perm=null 19:00:59,052 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1_conf.xml dst=null perm=null 19:00:59,053 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml_tmp dst=null perm=null 19:00:59,054 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002/job_1394161202967_0002_1_conf.xml dst=null perm=null 19:00:59,057 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:00:59,059 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52854, bytes: 92381, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741859_1035, duration: 359000 19:00:59,061 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741865_1041{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:00:59,063 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041 src: /127.0.0.1:52857 dest: /127.0.0.1:52682 19:00:59,064 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041 src: /127.0.0.1:52858 dest: /127.0.0.1:52685 19:00:59,068 INFO clienttrace:1215 - src: /127.0.0.1:52858, dest: /127.0.0.1:52685, bytes: 91661, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041, duration: 2149000 19:00:59,068 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:00:59,069 INFO clienttrace:1215 - src: /127.0.0.1:52857, dest: /127.0.0.1:52682, bytes: 91661, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2092908579_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041, duration: 2676000 19:00:59,069 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:00:59,071 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_2092908579_1 19:00:59,073 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:00:59,075 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary perm=test:supergroup:rwxrwx--- 19:00:59,077 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml perm=test:supergroup:rwxrwx--- 19:00:59,080 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist perm=test:supergroup:rwxrwx--- 19:00:59,086 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0002_000001 with final state: FINISHING 19:00:59,087 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from RUNNING to FINAL_SAVING 19:00:59,087 INFO RMAppImpl:880 - Updating application application_1394161202967_0002 with final state: FINISHING 19:00:59,087 INFO RMAppImpl:643 - application_1394161202967_0002 State change from RUNNING to FINAL_SAVING 19:00:59,087 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0002 19:00:59,089 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from FINAL_SAVING to FINISHING 19:00:59,089 INFO RMAppImpl:643 - application_1394161202967_0002 State change from FINAL_SAVING to FINISHING 19:00:59,873 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0002_01_000004 19:00:59,874 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000004 Container Transitioned from RUNNING to COMPLETED 19:00:59,874 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0002_01_000004 in state: COMPLETED event:FINISHED 19:00:59,874 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000004 19:00:59,874 INFO FSSchedulerNode:162 - Released container container_1394161202967_0002_01_000004 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:00:59,874 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0002_000001 released container container_1394161202967_0002_01_000004 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:00:59,929 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040 19:00:59,929 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039 19:00:59,930 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038 19:00:59,931 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037 19:00:59,932 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041 19:01:00,095 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0002 dst=null perm=null 19:01:00,277 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161260277, maxDate=1394766060277, sequenceNumber=6, masterKeyId=2 19:01:00,277 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 6 19:01:00,283 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:01:00,285 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:01:00,287 INFO CompletedJob:95 - Loading job: job_1394161202967_0002 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist 19:01:00,287 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist] 19:01:00,288 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist dst=null perm=null 19:01:00,290 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist dst=null perm=null 19:01:00,293 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52863, bytes: 39609, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040, duration: 231000 19:01:00,352 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary dst=null perm=null 19:01:00,355 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52864, bytes: 433, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039, duration: 204000 19:01:00,356 INFO JobSummary:372 - jobId=job_1394161202967_0002,submitTime=1394161239191,launchTime=1394161245910,firstMapTaskLaunchTime=1394161248139,firstReduceTaskLaunchTime=1394161255033,finishTime=1394161258968,resourcesPerMap=1024,resourcesPerReduce=1024,numMaps=2,numReduces=1,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=9,reduceSlotSeconds=3,jobName=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190021371-oozie-rkan-W 19:01:00,356 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary] 19:01:00,357 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002.summary dst=null perm=null 19:01:00,359 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:01:00,360 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist 19:01:00,361 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0002-1394161239191-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161258968-2-1-SUCCEEDED-root.test-1394161245910.jhist perm=test:supergroup:rwxrwx--- 19:01:00,362 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0002_conf.xml 19:01:00,363 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0002_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0002_conf.xml perm=test:supergroup:rwxrwx--- 19:01:00,384 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:00,387 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:00,388 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:00,390 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52863, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, duration: 221000 19:01:00,409 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:00,410 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:00,412 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:00,415 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52864, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_451532259_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741854_1030, duration: 620000 19:01:00,483 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java dst=null perm=null 19:01:00,485 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190021371-oozie-rkan-W/map-reduce--java dst=null perm=null 19:01:00,522 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161260522, maxDate=1394766060522, sequenceNumber=7, masterKeyId=2 19:01:00,522 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 7 19:01:00,538 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741864_1040 19:01:00,539 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741861_1037 19:01:00,539 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741862_1038 19:01:00,541 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741865_1041 19:01:00,541 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741863_1039 19:01:00,542 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741843_1019, blk_1073741859_1035, blk_1073741844_1020, blk_1073741860_1036, blk_1073741861_1037, blk_1073741863_1039, blk_1073741850_1026, blk_1073741851_1027, blk_1073741852_1028, blk_1073741853_1029, blk_1073741854_1030] 19:01:01,314 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:01:01,317 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741866_1042{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:01,320 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042 src: /127.0.0.1:52868 dest: /127.0.0.1:52682 19:01:01,322 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042 src: /127.0.0.1:52869 dest: /127.0.0.1:52685 19:01:01,324 INFO clienttrace:1215 - src: /127.0.0.1:52869, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_640800170_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042, duration: 1326000 19:01:01,325 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:01,325 INFO clienttrace:1215 - src: /127.0.0.1:52868, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_640800170_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042, duration: 1880000 19:01:01,325 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:01,328 INFO StateChange:2959 - DIR* completeFile: /user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input/data.txt is closed by DFSClient_NONMAPREDUCE_640800170_8 19:01:01,338 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:01:01,340 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741867_1043{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:01,342 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043 src: /127.0.0.1:52870 dest: /127.0.0.1:52685 19:01:01,344 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043 src: /127.0.0.1:52871 dest: /127.0.0.1:52682 19:01:01,346 INFO clienttrace:1215 - src: /127.0.0.1:52871, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_640800170_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043, duration: 1283000 19:01:01,346 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:01,347 INFO clienttrace:1215 - src: /127.0.0.1:52870, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_640800170_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043, duration: 1882000 19:01:01,347 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:01,349 INFO StateChange:2959 - DIR* completeFile: /user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_640800170_8 19:01:01,352 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:01:01,354 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741868_1044{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:01:01,356 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044 src: /127.0.0.1:52872 dest: /127.0.0.1:52682 19:01:01,357 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044 src: /127.0.0.1:52873 dest: /127.0.0.1:52685 19:01:01,359 INFO clienttrace:1215 - src: /127.0.0.1:52873, dest: /127.0.0.1:52685, bytes: 1872, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_640800170_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044, duration: 1252000 19:01:01,359 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:01,360 INFO clienttrace:1215 - src: /127.0.0.1:52872, dest: /127.0.0.1:52682, bytes: 1872, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_640800170_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044, duration: 1899000 19:01:01,360 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:01,362 INFO StateChange:2959 - DIR* completeFile: /user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_640800170_8 19:01:01,366 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java dst=null perm=null 19:01:01,368 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:01:01,371 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java.tmp dst=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java perm=test:supergroup:rwxrwxrwx 19:01:01,372 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/workflow.xml dst=null perm=null 19:01:01,452 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/workflow.xml dst=null perm=null 19:01:01,489 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java dst=null perm=test:supergroup:rwxrwxrwx 19:01:01,492 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:01,502 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741869_1045{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:01,505 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045 src: /127.0.0.1:52875 dest: /127.0.0.1:52685 19:01:01,506 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045 src: /127.0.0.1:52876 dest: /127.0.0.1:52682 19:01:01,509 INFO clienttrace:1215 - src: /127.0.0.1:52876, dest: /127.0.0.1:52682, bytes: 77744, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045, duration: 1656000 19:01:01,509 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:01,509 INFO clienttrace:1215 - src: /127.0.0.1:52875, dest: /127.0.0.1:52685, bytes: 77744, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045, duration: 2255000 19:01:01,510 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:01,512 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action.xml is closed by DFSClient_NONMAPREDUCE_-341598792_8 19:01:01,513 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input dst=null perm=test:supergroup:rwxrwxrwx 19:01:01,515 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:01:01,517 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741870_1046{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:01,519 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046 src: /127.0.0.1:52877 dest: /127.0.0.1:52682 19:01:01,520 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046 src: /127.0.0.1:52878 dest: /127.0.0.1:52685 19:01:01,522 INFO clienttrace:1215 - src: /127.0.0.1:52878, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046, duration: 1295000 19:01:01,522 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:01,523 INFO clienttrace:1215 - src: /127.0.0.1:52877, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046, duration: 1735000 19:01:01,523 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:01,525 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-341598792_8 19:01:01,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741859_1035 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741859 for deletion 19:01:01,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741843_1019 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741843 for deletion 19:01:01,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741860_1036 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741860 for deletion 19:01:01,970 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741859_1035 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741859 19:01:01,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741844_1020 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741844 for deletion 19:01:01,970 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741860_1036 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741860 19:01:01,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741843_1019 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741843 19:01:01,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741861_1037 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741861 for deletion 19:01:01,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741844_1020 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741844 19:01:01,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741863_1039 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741863 for deletion 19:01:01,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741861_1037 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741861 19:01:01,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741850_1026 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741850 for deletion 19:01:01,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741863_1039 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741863 19:01:01,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741851_1027 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741851 for deletion 19:01:01,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741850_1026 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741850 19:01:01,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741852_1028 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741852 for deletion 19:01:01,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741851_1027 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741851 19:01:01,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741853_1029 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741853 for deletion 19:01:01,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741852_1028 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741852 19:01:01,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741854_1030 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741854 for deletion 19:01:01,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741853_1029 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741853 19:01:01,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741854_1030 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741854 19:01:01,982 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161261982, maxDate=1394766061982, sequenceNumber=8, masterKeyId=2 19:01:01,982 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 8 19:01:01,983 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/0000000-140306190100960-oozie-rkan-W@mr-action dst=null perm=null 19:01:01,984 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161261984, maxDate=1394766061984, sequenceNumber=9, masterKeyId=2 19:01:01,984 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 9 19:01:02,001 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:01:02,002 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:02,003 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:02,004 INFO ClientRMService:229 - Allocated new applicationId: 3 19:01:02,006 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003 dst=null perm=null 19:01:02,007 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003 dst=null perm=test:supergroup:rwxrwxrwx 19:01:02,009 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003 dst=null perm=test:supergroup:rwx------ 19:01:02,010 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:02,011 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:02,012 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:02,013 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:02,014 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:01:02,014 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:01:02,015 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:01:02,016 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:02,017 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:02,018 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:02,019 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:02,020 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib dst=null perm=null 19:01:02,020 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app dst=null perm=null 19:01:02,021 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c dst=null perm=null 19:01:02,022 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:02,024 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:01:02,025 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:01:02,026 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:01:02,029 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:01:02,031 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=test:supergroup:rw-r--r-- 19:01:02,032 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split 19:01:02,032 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=null 19:01:02,034 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741871_1047{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:02,036 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047 src: /127.0.0.1:52880 dest: /127.0.0.1:52685 19:01:02,037 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047 src: /127.0.0.1:52881 dest: /127.0.0.1:52682 19:01:02,040 INFO clienttrace:1215 - src: /127.0.0.1:52881, dest: /127.0.0.1:52682, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047, duration: 1630000 19:01:02,040 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:02,040 INFO clienttrace:1215 - src: /127.0.0.1:52880, dest: /127.0.0.1:52685, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047, duration: 2008000 19:01:02,041 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:02,043 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split is closed by DFSClient_NONMAPREDUCE_-341598792_8 19:01:02,045 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:01:02,047 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:01:02,048 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741872_1048{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:01:02,050 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048 src: /127.0.0.1:52882 dest: /127.0.0.1:52682 19:01:02,051 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048 src: /127.0.0.1:52883 dest: /127.0.0.1:52685 19:01:02,053 INFO clienttrace:1215 - src: /127.0.0.1:52883, dest: /127.0.0.1:52685, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048, duration: 1143000 19:01:02,054 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:02,054 INFO clienttrace:1215 - src: /127.0.0.1:52882, dest: /127.0.0.1:52682, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048, duration: 1726000 19:01:02,054 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:02,056 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_-341598792_8 19:01:02,058 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:02,060 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:01:02,066 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741873_1049{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:01:02,068 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049 src: /127.0.0.1:52884 dest: /127.0.0.1:52682 19:01:02,069 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049 src: /127.0.0.1:52885 dest: /127.0.0.1:52685 19:01:02,072 INFO clienttrace:1215 - src: /127.0.0.1:52885, dest: /127.0.0.1:52685, bytes: 79936, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049, duration: 1754000 19:01:02,072 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:02,073 INFO clienttrace:1215 - src: /127.0.0.1:52884, dest: /127.0.0.1:52682, bytes: 79936, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049, duration: 2242000 19:01:02,073 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:02,075 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml is closed by DFSClient_NONMAPREDUCE_-341598792_8 19:01:02,076 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003 dst=null perm=null 19:01:02,077 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=null 19:01:02,078 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=null 19:01:02,080 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=null 19:01:02,081 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=null 19:01:02,081 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=null 19:01:02,082 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=null 19:01:02,085 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:02,085 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:02,086 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:02,087 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:02,088 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:02,089 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:02,090 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:02,091 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:02,093 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:02,094 INFO ClientRMService:342 - Application with id 3 submitted by user test 19:01:02,094 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0003 19:01:02,094 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0003 19:01:02,094 INFO RMAppImpl:643 - application_1394161202967_0003 State change from NEW to NEW_SAVING 19:01:02,096 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0003 19:01:02,096 INFO RMAppImpl:643 - application_1394161202967_0003 State change from NEW_SAVING to SUBMITTED 19:01:02,096 INFO FairScheduler:638 - Accepted application application_1394161202967_0003 from user: test, in queue: default, currently num of applications: 2 19:01:02,096 INFO RMAppImpl:643 - application_1394161202967_0003 State change from SUBMITTED to ACCEPTED 19:01:02,098 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0003_000001 19:01:02,098 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from NEW to SUBMITTED 19:01:02,098 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0003_000001 to scheduler from user: test 19:01:02,099 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from SUBMITTED to SCHEDULED 19:01:02,134 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161262134, maxDate=1394766062134, sequenceNumber=10, masterKeyId=2 19:01:02,134 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 10 19:01:02,969 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000001 Container Transitioned from NEW to ALLOCATED 19:01:02,970 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000001 19:01:02,970 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0003_01_000001 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:01:02,971 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:01:02,971 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0003 AttemptId: appattempt_1394161202967_0003_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0003_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:01:02,971 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:01:02,972 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:01:02,972 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0003_000001 19:01:02,974 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0003_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0003_000001 19:01:02,974 INFO AMLauncher:190 - Command to launch container container_1394161202967_0003_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:01:02,982 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0003_01_000001 by user test 19:01:02,983 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0003 19:01:02,983 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000001 19:01:02,983 INFO Application:450 - Application application_1394161202967_0003 transitioned from NEW to INITING 19:01:02,983 INFO Application:296 - Adding container_1394161202967_0003_01_000001 to application application_1394161202967_0003 19:01:02,984 INFO Application:450 - Application application_1394161202967_0003 transitioned from INITING to RUNNING 19:01:02,984 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0003_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0003_000001 19:01:02,984 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from ALLOCATED to LAUNCHED 19:01:02,984 INFO Container:888 - Container container_1394161202967_0003_01_000001 transitioned from NEW to LOCALIZING 19:01:02,984 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0003 19:01:02,985 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:01:02,985 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split transitioned from INIT to DOWNLOADING 19:01:02,985 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml transitioned from INIT to DOWNLOADING 19:01:02,985 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:01:02,985 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0003_01_000001 19:01:02,986 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar, 1394161261349, FILE, null } 19:01:03,076 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0003_01_000001.tokens. Credentials list: 19:01:03,134 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:03,154 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:03,176 INFO DefaultContainerExecutor:469 - Initializing user test 19:01:03,178 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib dst=null perm=null 19:01:03,210 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app dst=null perm=null 19:01:03,251 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c dst=null perm=null 19:01:03,275 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:03,313 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:03,338 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:03,358 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:03,379 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:03,464 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52887, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043, duration: 5116000 19:01:03,549 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741843_1019, blk_1073741859_1035, blk_1073741844_1020, blk_1073741860_1036, blk_1073741861_1037, blk_1073741863_1039, blk_1073741850_1026, blk_1073741851_1027, blk_1073741852_1028, blk_1073741853_1029, blk_1073741854_1030] 19:01:03,603 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/filecache/13/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:01:03,621 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0003_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000001.tokens 19:01:03,621 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003 19:01:03,791 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=null 19:01:03,793 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=null 19:01:03,794 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=null 19:01:03,818 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52887, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1893801486_1423, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048, duration: 273000 19:01:03,880 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0003/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:01:03,933 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=null 19:01:03,935 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=null 19:01:03,937 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=null 19:01:03,951 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52890, bytes: 176, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1893801486_1423, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047, duration: 226000 19:01:03,996 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:01:04,051 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0003/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:01:04,112 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=null 19:01:04,114 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=null 19:01:04,116 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=null 19:01:04,128 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52887, bytes: 80564, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1893801486_1423, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049, duration: 431000 19:01:04,185 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:01:04,185 INFO Container:888 - Container container_1394161202967_0003_01_000001 transitioned from LOCALIZING to LOCALIZED 19:01:04,260 INFO Container:888 - Container container_1394161202967_0003_01_000001 transitioned from LOCALIZED to RUNNING 19:01:04,260 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:04,467 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000001/default_container_executor.sh] 19:01:04,937 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043 19:01:04,938 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048 19:01:04,939 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046 19:01:04,940 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045 19:01:04,940 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047 19:01:04,941 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049 19:01:04,942 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044 19:01:04,942 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042 19:01:04,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741859_1035 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741859 for deletion 19:01:04,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741843_1019 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741843 for deletion 19:01:04,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741860_1036 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741860 for deletion 19:01:04,970 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741859_1035 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741859 19:01:04,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741844_1020 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741844 for deletion 19:01:04,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741861_1037 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741861 for deletion 19:01:04,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741843_1019 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741843 19:01:04,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741863_1039 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741863 for deletion 19:01:04,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741860_1036 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741860 19:01:04,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741850_1026 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741850 for deletion 19:01:04,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741861_1037 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741861 19:01:04,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741851_1027 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741851 for deletion 19:01:04,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741844_1020 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741844 19:01:04,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741852_1028 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741852 for deletion 19:01:04,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741853_1029 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741853 for deletion 19:01:04,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741863_1039 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741863 19:01:04,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741854_1030 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741854 for deletion 19:01:04,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741850_1026 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741850 19:01:04,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741851_1027 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741851 19:01:04,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741852_1028 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741852 19:01:04,973 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741853_1029 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741853 19:01:04,973 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741854_1030 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741854 19:01:05,546 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741873_1049 19:01:05,547 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042 19:01:05,547 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047 19:01:05,547 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043 19:01:05,548 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048 19:01:05,549 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045 19:01:05,549 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046 19:01:05,550 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741868_1044 19:01:06,253 INFO ContainerLaunch:322 - Container container_1394161202967_0002_01_000001 succeeded 19:01:06,253 INFO Container:888 - Container container_1394161202967_0002_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:01:06,253 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0002_01_000001 19:01:06,270 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000001 19:01:06,270 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000001 19:01:06,270 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000001 19:01:06,271 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0002/container_1394161202967_0002_01_000001 19:01:06,271 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000001 19:01:06,271 INFO Container:888 - Container container_1394161202967_0002_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 19:01:06,271 INFO Application:339 - Removing container_1394161202967_0002_01_000001 from application application_1394161202967_0002 19:01:06,271 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:06,272 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0002 19:01:06,842 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:06,876 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/COMMIT_STARTED dst=null perm=null 19:01:06,877 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/COMMIT_SUCCESS dst=null perm=null 19:01:06,878 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/COMMIT_FAIL dst=null perm=null 19:01:06,943 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003 dst=null perm=null 19:01:06,945 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:01:06,946 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:01:06,982 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0002_01_000001 19:01:06,983 INFO RMContainerImpl:220 - container_1394161202967_0002_01_000001 Container Transitioned from RUNNING to COMPLETED 19:01:06,983 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0002_000001 19:01:06,983 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0002_01_000001 in state: COMPLETED event:FINISHED 19:01:06,983 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0002_000001 State change from FINISHING to FINISHED 19:01:06,983 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0002 CONTAINERID=container_1394161202967_0002_01_000001 19:01:06,984 INFO RMAppImpl:643 - application_1394161202967_0002 State change from FINISHING to FINISHED 19:01:06,984 INFO FSSchedulerNode:162 - Released container container_1394161202967_0002_01_000001 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:01:06,984 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0002_000001 released container container_1394161202967_0002_01_000001 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: FINISHED 19:01:06,984 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0002 19:01:06,984 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0002_000001 19:01:06,984 INFO FairScheduler:720 - Application appattempt_1394161202967_0002_000001 is done. finalState=FINISHED 19:01:06,984 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0002,name=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190021371-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0002/jobhistory/job/job_1394161202967_0002,appMasterHost=rkanter-mbp.local,startTime=1394161239191,finishTime=1394161259087,finalStatus=SUCCEEDED 19:01:06,984 INFO AppSchedulingInfo:108 - Application application_1394161202967_0002 requests cleared 19:01:06,991 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0002_01_000001 19:01:07,343 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=null 19:01:07,352 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.splitmetainfo dst=null perm=null 19:01:07,417 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52923, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741872_1048, duration: 216000 19:01:07,983 INFO Application:450 - Application application_1394161202967_0002 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:01:07,983 INFO Application:450 - Application application_1394161202967_0002 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:01:07,983 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0002 19:01:07,984 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0002 19:01:07,984 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0002 19:01:07,984 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0002 19:01:07,984 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0002 19:01:07,984 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0002 19:01:07,984 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0002 19:01:07,984 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0002 19:01:07,985 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0002 19:01:07,985 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0002 19:01:07,985 INFO Application:450 - Application application_1394161202967_0002 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:01:07,985 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0002, with delay of 10800 seconds 19:01:07,985 INFO Application:450 - Application application_1394161202967_0002 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:01:07,986 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0002, with delay of 10800 seconds 19:01:08,410 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0003_000001 19:01:08,411 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0003 APPATTEMPTID=appattempt_1394161202967_0003_000001 19:01:08,411 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from LAUNCHED to RUNNING 19:01:08,411 INFO RMAppImpl:643 - application_1394161202967_0003 State change from ACCEPTED to RUNNING 19:01:08,457 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:01:08,464 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=test:supergroup:rwxrwxrwx 19:01:08,533 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:08,746 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741874_1050{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:08,800 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050 src: /127.0.0.1:52929 dest: /127.0.0.1:52682 19:01:08,802 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050 src: /127.0.0.1:52930 dest: /127.0.0.1:52685 19:01:08,836 INFO clienttrace:1215 - src: /127.0.0.1:52930, dest: /127.0.0.1:52685, bytes: 92920, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050, duration: 33129000 19:01:08,837 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:08,837 INFO clienttrace:1215 - src: /127.0.0.1:52929, dest: /127.0.0.1:52682, bytes: 92920, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050, duration: 34240000 19:01:08,838 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:08,842 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1_conf.xml is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:08,866 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003.summary dst=null perm=test:supergroup:rw-rw-rw- 19:01:08,869 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741875_1051{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:01:08,872 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051 src: /127.0.0.1:52931 dest: /127.0.0.1:52682 19:01:08,873 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051 src: /127.0.0.1:52932 dest: /127.0.0.1:52685 19:01:08,876 INFO clienttrace:1215 - src: /127.0.0.1:52932, dest: /127.0.0.1:52685, bytes: 377, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051, duration: 1435000 19:01:08,876 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:08,877 INFO clienttrace:1215 - src: /127.0.0.1:52931, dest: /127.0.0.1:52682, bytes: 377, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051, duration: 2375000 19:01:08,877 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:08,879 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003.summary is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:09,946 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050 19:01:09,947 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051 19:01:09,985 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000002 Container Transitioned from NEW to ALLOCATED 19:01:09,985 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000002 19:01:09,985 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0003_01_000002 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:01:10,475 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:01:10,527 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=null 19:01:10,528 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.xml dst=null perm=null 19:01:10,546 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:10,547 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:10,548 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:10,553 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741875_1051 19:01:10,554 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050 19:01:10,557 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:10,558 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:10,560 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:10,561 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:10,562 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:10,563 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:10,619 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0003_01_000002 by user test 19:01:10,619 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000002 19:01:10,619 INFO Application:296 - Adding container_1394161202967_0003_01_000002 to application application_1394161202967_0003 19:01:10,620 INFO Container:888 - Container container_1394161202967_0003_01_000002 transitioned from NEW to LOCALIZING 19:01:10,620 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0003 19:01:10,620 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0003 19:01:10,620 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:01:10,620 INFO ShuffleHandler:263 - Added token for job_1394161202967_0003 19:01:10,621 INFO Container:888 - Container container_1394161202967_0003_01_000002 transitioned from LOCALIZING to LOCALIZED 19:01:10,690 INFO Container:888 - Container container_1394161202967_0003_01_000002 transitioned from LOCALIZED to RUNNING 19:01:10,690 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:10,898 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000002/default_container_executor.sh] 19:01:10,986 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:01:11,491 INFO AppSchedulingInfo:166 - checking for deactivate... 19:01:13,918 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job.split dst=null perm=null 19:01:14,005 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52937, bytes: 176, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741871_1047, duration: 284000 19:01:14,089 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=null 19:01:14,112 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0003_m_000000_0/part-00000 dst=null perm=test:supergroup:rw-rw-rw- 19:01:14,136 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/0000000-140306190100960-oozie-rkan-W@mr-action dst=null perm=null 19:01:14,140 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/0000000-140306190100960-oozie-rkan-W@mr-action dst=null perm=test:supergroup:rw-rw-rw- 19:01:14,151 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/0000000-140306190100960-oozie-rkan-W@mr-action. BP-1007909283-172.16.1.64-1394161187028 blk_1073741876_1052{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:14,196 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052 src: /127.0.0.1:52938 dest: /127.0.0.1:52685 19:01:14,198 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052 src: /127.0.0.1:52939 dest: /127.0.0.1:52682 19:01:14,220 INFO clienttrace:1215 - src: /127.0.0.1:52939, dest: /127.0.0.1:52682, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052, duration: 21274000 19:01:14,221 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:14,221 INFO clienttrace:1215 - src: /127.0.0.1:52938, dest: /127.0.0.1:52685, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052, duration: 22088000 19:01:14,221 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:14,225 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/0000000-140306190100960-oozie-rkan-W@mr-action is closed by DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1 19:01:14,232 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52937, bytes: 9, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741870_1046, duration: 501000 19:01:14,291 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:01:14,292 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:01:14,305 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52940, bytes: 78352, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741869_1045, duration: 335000 19:01:14,950 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052 19:01:14,952 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/output dst=null perm=null 19:01:14,958 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:14,959 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:14,988 INFO ClientRMService:229 - Allocated new applicationId: 4 19:01:15,005 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004 dst=null perm=null 19:01:15,023 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004 dst=null perm=test:supergroup:rwxrwxrwx 19:01:15,033 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004 dst=null perm=test:supergroup:rwx------ 19:01:15,041 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:15,043 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:15,044 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:15,045 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:15,046 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:01:15,047 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:01:15,047 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:01:15,048 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:15,049 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:15,051 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:15,052 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:15,053 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib dst=null perm=null 19:01:15,054 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app dst=null perm=null 19:01:15,055 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c dst=null perm=null 19:01:15,056 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:15,069 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input dst=null perm=null 19:01:15,074 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input dst=null perm=null 19:01:15,078 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input dst=null perm=null 19:01:15,105 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:01:15,108 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=test:supergroup:rw-r--r-- 19:01:15,112 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split 19:01:15,113 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:15,126 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741877_1053{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:15,129 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053 src: /127.0.0.1:52942 dest: /127.0.0.1:52685 19:01:15,130 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053 src: /127.0.0.1:52943 dest: /127.0.0.1:52682 19:01:15,133 INFO clienttrace:1215 - src: /127.0.0.1:52943, dest: /127.0.0.1:52682, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053, duration: 1354000 19:01:15,133 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:15,133 INFO clienttrace:1215 - src: /127.0.0.1:52942, dest: /127.0.0.1:52685, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053, duration: 2001000 19:01:15,134 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:15,135 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split is closed by DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1 19:01:15,137 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:01:15,139 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:01:15,144 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741878_1054{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:15,147 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054 src: /127.0.0.1:52944 dest: /127.0.0.1:52685 19:01:15,149 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054 src: /127.0.0.1:52945 dest: /127.0.0.1:52682 19:01:15,153 INFO clienttrace:1215 - src: /127.0.0.1:52945, dest: /127.0.0.1:52682, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054, duration: 1428000 19:01:15,153 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:15,153 INFO clienttrace:1215 - src: /127.0.0.1:52944, dest: /127.0.0.1:52685, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054, duration: 1785000 19:01:15,154 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:15,155 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo is closed by DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1 19:01:15,168 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:15,170 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:01:15,207 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741879_1055{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:01:15,209 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055 src: /127.0.0.1:52946 dest: /127.0.0.1:52682 19:01:15,211 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055 src: /127.0.0.1:52947 dest: /127.0.0.1:52685 19:01:15,214 INFO clienttrace:1215 - src: /127.0.0.1:52947, dest: /127.0.0.1:52685, bytes: 77832, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055, duration: 1955000 19:01:15,214 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:15,215 INFO clienttrace:1215 - src: /127.0.0.1:52946, dest: /127.0.0.1:52682, bytes: 77832, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055, duration: 2713000 19:01:15,215 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:15,217 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml is closed by DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1 19:01:15,400 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004 dst=null perm=null 19:01:15,557 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741876_1052 19:01:15,558 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053 19:01:15,559 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055 19:01:15,559 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054 19:01:15,588 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:15,590 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:15,595 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:15,596 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:15,597 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=null 19:01:15,598 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=null 19:01:15,602 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:15,603 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:15,604 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:15,605 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:15,606 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:15,607 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:15,608 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:15,610 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:15,611 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:15,666 INFO ClientRMService:342 - Application with id 4 submitted by user test 19:01:15,666 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0004 19:01:15,667 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0004 19:01:15,667 INFO RMAppImpl:643 - application_1394161202967_0004 State change from NEW to NEW_SAVING 19:01:15,667 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0004 19:01:15,667 INFO RMAppImpl:643 - application_1394161202967_0004 State change from NEW_SAVING to SUBMITTED 19:01:15,667 INFO FairScheduler:638 - Accepted application application_1394161202967_0004 from user: test, in queue: default, currently num of applications: 2 19:01:15,668 INFO RMAppImpl:643 - application_1394161202967_0004 State change from SUBMITTED to ACCEPTED 19:01:15,668 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0004_000001 19:01:15,668 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from NEW to SUBMITTED 19:01:15,668 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0004_000001 to scheduler from user: test 19:01:15,668 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from SUBMITTED to SCHEDULED 19:01:15,862 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=test:supergroup:rw-rw-rw- 19:01:15,893 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq. BP-1007909283-172.16.1.64-1394161187028 blk_1073741880_1056{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:15,897 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056 src: /127.0.0.1:52949 dest: /127.0.0.1:52685 19:01:15,898 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056 src: /127.0.0.1:52950 dest: /127.0.0.1:52682 19:01:15,900 INFO clienttrace:1215 - src: /127.0.0.1:52950, dest: /127.0.0.1:52682, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056, duration: 1256000 19:01:15,900 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:15,901 INFO clienttrace:1215 - src: /127.0.0.1:52949, dest: /127.0.0.1:52685, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056, duration: 1941000 19:01:15,901 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:15,903 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq is closed by DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1 19:01:15,910 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0003_m_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0003_m_000000_0_270692958_1 19:01:15,931 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0003_m_000000_0 dst=null perm=null 19:01:15,937 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0003_m_000000_0 dst=null perm=null 19:01:15,938 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0003_m_000000 dst=null perm=null 19:01:15,943 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0003_m_000000_0 dst=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0003_m_000000 perm=test:supergroup:rwxrwxrwx 19:01:15,965 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0003_01_000002 19:01:15,966 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000002 19:01:15,966 INFO Container:888 - Container container_1394161202967_0003_01_000002 transitioned from RUNNING to KILLING 19:01:15,966 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0003_01_000002 19:01:16,003 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0004 19:01:16,027 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0003_01_000002 is : 143 19:01:16,029 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741881_1057{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:16,029 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000001 Container Transitioned from NEW to RESERVED 19:01:16,029 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:01:16,029 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0004 reserved container container_1394161202967_0004_01_000001 on node host: 172.16.1.64:52707 #containers=2 available= used=, currently has 1 at priority 0; currentReservation 2048 19:01:16,029 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0004_01_000001 on node host: 172.16.1.64:52707 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@49728191 19:01:16,031 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000002 Container Transitioned from NEW to ALLOCATED 19:01:16,032 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000002 19:01:16,032 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:01:16,032 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057 src: /127.0.0.1:52951 dest: /127.0.0.1:52682 19:01:16,033 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:01:16,033 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:16,033 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0004 AttemptId: appattempt_1394161202967_0004_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0004_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] 19:01:16,033 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:01:16,034 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:01:16,034 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057 src: /127.0.0.1:52952 dest: /127.0.0.1:52685 19:01:16,034 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0004_000001 19:01:16,036 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0004_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0004_000001 19:01:16,036 INFO AMLauncher:190 - Command to launch container container_1394161202967_0004_01_000002 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:01:16,040 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1.jhist for DFSClient_NONMAPREDUCE_-992373219_1 19:01:16,040 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=null 19:01:16,043 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0004_01_000002 by user test 19:01:16,044 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0004 19:01:16,044 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000002 19:01:16,045 INFO Application:450 - Application application_1394161202967_0004 transitioned from NEW to INITING 19:01:16,045 INFO Application:296 - Adding container_1394161202967_0004_01_000002 to application application_1394161202967_0004 19:01:16,045 INFO Application:450 - Application application_1394161202967_0004 transitioned from INITING to RUNNING 19:01:16,045 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0004_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0004_000001 19:01:16,046 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from ALLOCATED to LAUNCHED 19:01:16,046 INFO Container:888 - Container container_1394161202967_0004_01_000002 transitioned from NEW to LOCALIZING 19:01:16,046 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0004 19:01:16,046 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:01:16,046 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split transitioned from INIT to DOWNLOADING 19:01:16,046 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml transitioned from INIT to DOWNLOADING 19:01:16,046 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:01:16,047 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0004_01_000002 19:01:16,047 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar, 1394161261349, FILE, null } 19:01:16,048 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:01:16,050 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:01:16,110 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0003_m_000000 dst=null perm=null 19:01:16,162 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/part-00000 dst=null perm=null 19:01:16,191 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/nmPrivate/container_1394161202967_0004_01_000002.tokens. Credentials list: 19:01:16,242 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0003_m_000000/part-00000 dst=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/part-00000 perm=test:supergroup:rw-rw-rw- 19:01:16,253 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:16,272 INFO Container:888 - Container container_1394161202967_0003_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:01:16,274 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000002 19:01:16,274 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000002 19:01:16,274 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000002 19:01:16,274 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_temporary dst=null perm=null 19:01:16,275 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:16,275 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000002 19:01:16,314 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000002 19:01:16,318 INFO Container:888 - Container container_1394161202967_0003_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:01:16,318 INFO Application:339 - Removing container_1394161202967_0003_01_000002 from application application_1394161202967_0003 19:01:16,318 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:16,319 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0003 19:01:16,326 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib dst=null perm=null 19:01:16,326 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:01:16,343 INFO DefaultContainerExecutor:469 - Initializing user test 19:01:16,345 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app dst=null perm=null 19:01:16,349 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:16,372 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c dst=null perm=null 19:01:16,402 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:01:16,404 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:16,434 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:16,462 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:16,495 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:16,526 INFO clienttrace:1215 - src: /127.0.0.1:52952, dest: /127.0.0.1:52685, bytes: 20702, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057, duration: 461358000 19:01:16,528 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:16,527 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:16,556 INFO clienttrace:1215 - src: /127.0.0.1:52951, dest: /127.0.0.1:52682, bytes: 20702, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057, duration: 493260000 19:01:16,556 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:16,579 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:16,583 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1.jhist is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:16,640 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:01:16,691 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52955, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741867_1043, duration: 24140000 19:01:16,692 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741882_1058{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:16,753 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058 src: /127.0.0.1:52956 dest: /127.0.0.1:52685 19:01:16,828 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058 src: /127.0.0.1:52957 dest: /127.0.0.1:52682 19:01:16,881 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/filecache/13/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:01:16,904 INFO clienttrace:1215 - src: /127.0.0.1:52957, dest: /127.0.0.1:52682, bytes: 421, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058, duration: 13710000 19:01:16,906 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:16,907 INFO clienttrace:1215 - src: /127.0.0.1:52956, dest: /127.0.0.1:52685, bytes: 421, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058, duration: 31172000 19:01:16,907 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:16,917 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary_tmp is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:16,942 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/nmPrivate/container_1394161202967_0004_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000002.tokens 19:01:16,943 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0004 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0004 19:01:16,943 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:01:16,973 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0003_01_000002 19:01:16,976 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1.jhist dst=null perm=null 19:01:16,977 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000002 Container Transitioned from RUNNING to COMPLETED 19:01:16,978 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0003_01_000002 in state: COMPLETED event:FINISHED 19:01:16,978 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000002 19:01:16,978 INFO FSSchedulerNode:162 - Released container container_1394161202967_0003_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:01:16,978 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0003_000001 released container container_1394161202967_0003_01_000002 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:01:16,984 INFO FairScheduler:1042 - Releasing reservation that cannot be satisfied for application appattempt_1394161202967_0004_000001 on node host: 172.16.1.64:52707 #containers=1 available= used= 19:01:17,012 INFO FSSchedulerApp:123 - Application application_1394161202967_0004 unreserved on node host: 172.16.1.64:52707 #containers=1 available= used=, currently has 0 at priority 0; currentReservation 19:01:17,013 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist_tmp dst=null perm=null 19:01:17,016 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:01:17,049 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1.jhist dst=null perm=null 19:01:17,080 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist_tmp dst=null perm=null 19:01:17,109 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1.jhist dst=null perm=null 19:01:17,171 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:01:17,198 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52958, bytes: 20866, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057, duration: 479000 19:01:17,201 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741883_1059{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:17,204 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059 src: /127.0.0.1:52959 dest: /127.0.0.1:52685 19:01:17,205 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059 src: /127.0.0.1:52960 dest: /127.0.0.1:52682 19:01:17,210 INFO clienttrace:1215 - src: /127.0.0.1:52960, dest: /127.0.0.1:52682, bytes: 20702, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059, duration: 2197000 19:01:17,211 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:17,211 INFO clienttrace:1215 - src: /127.0.0.1:52959, dest: /127.0.0.1:52685, bytes: 20702, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059, duration: 2894000 19:01:17,211 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:17,214 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist_tmp is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:17,216 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:01:17,217 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1_conf.xml dst=null perm=null 19:01:17,219 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml_tmp dst=null perm=null 19:01:17,221 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1_conf.xml dst=null perm=null 19:01:17,222 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml_tmp dst=null perm=null 19:01:17,223 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003/job_1394161202967_0003_1_conf.xml dst=null perm=null 19:01:17,268 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:01:17,290 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52958, bytes: 93648, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741874_1050, duration: 7322000 19:01:17,313 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741884_1060{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:17,318 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060 src: /127.0.0.1:52963 dest: /127.0.0.1:52685 19:01:17,323 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=null 19:01:17,324 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060 src: /127.0.0.1:52964 dest: /127.0.0.1:52682 19:01:17,326 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=null 19:01:17,327 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=null 19:01:17,327 INFO clienttrace:1215 - src: /127.0.0.1:52964, dest: /127.0.0.1:52682, bytes: 92920, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060, duration: 1776000 19:01:17,327 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:17,328 INFO clienttrace:1215 - src: /127.0.0.1:52963, dest: /127.0.0.1:52685, bytes: 92920, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-992373219_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060, duration: 2472000 19:01:17,328 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:17,330 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_-992373219_1 19:01:17,356 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:01:17,359 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary perm=test:supergroup:rwxrwx--- 19:01:17,359 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52965, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_736177494_1602, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054, duration: 205000 19:01:17,382 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml perm=test:supergroup:rwxrwx--- 19:01:17,413 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist perm=test:supergroup:rwxrwx--- 19:01:17,475 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0003_000001 with final state: FINISHING 19:01:17,476 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from RUNNING to FINAL_SAVING 19:01:17,477 INFO RMAppImpl:880 - Updating application application_1394161202967_0003 with final state: FINISHING 19:01:17,477 INFO RMAppImpl:643 - application_1394161202967_0003 State change from RUNNING to FINAL_SAVING 19:01:17,477 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0003 19:01:17,481 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from FINAL_SAVING to FINISHING 19:01:17,481 INFO RMAppImpl:643 - application_1394161202967_0003 State change from FINAL_SAVING to FINISHING 19:01:17,483 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0004/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:01:17,545 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:17,546 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:17,547 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:17,570 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52965, bytes: 283, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_736177494_1602, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053, duration: 237000 19:01:17,621 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0004/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:01:17,699 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:17,701 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:17,702 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:17,727 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52965, bytes: 78444, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_736177494_1602, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055, duration: 398000 19:01:17,778 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0004/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:01:17,778 INFO Container:888 - Container container_1394161202967_0004_01_000002 transitioned from LOCALIZING to LOCALIZED 19:01:17,863 INFO Container:888 - Container container_1394161202967_0004_01_000002 transitioned from LOCALIZED to RUNNING 19:01:17,864 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:18,066 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000002/default_container_executor.sh] 19:01:18,487 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0003 dst=null perm=null 19:01:18,494 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:18,497 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:18,499 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:18,503 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52965, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056, duration: 317000 19:01:18,548 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161278548, maxDate=1394766078548, sequenceNumber=11, masterKeyId=2 19:01:18,549 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 11 19:01:18,554 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741872_1048, blk_1073741873_1049, blk_1073741874_1050, blk_1073741875_1051, blk_1073741881_1057, blk_1073741871_1047] 19:01:18,554 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:01:18,556 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:01:18,558 INFO CompletedJob:95 - Loading job: job_1394161202967_0003 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist 19:01:18,558 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist] 19:01:18,559 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist dst=null perm=null 19:01:18,561 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist dst=null perm=null 19:01:18,563 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52955, bytes: 20866, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059, duration: 282000 19:01:18,591 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary dst=null perm=null 19:01:18,594 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52965, bytes: 425, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058, duration: 292000 19:01:18,594 INFO JobSummary:372 - jobId=job_1394161202967_0003,submitTime=1394161262094,launchTime=1394161268447,firstMapTaskLaunchTime=1394161270626,firstReduceTaskLaunchTime=0,finishTime=1394161276438,resourcesPerMap=1024,resourcesPerReduce=0,numMaps=1,numReduces=0,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=5,reduceSlotSeconds=0,jobName=oozie:launcher:T\=map-reduce:W\=test-wf-cred:A\=map-reduce:ID\=0000000-140306190100960-oozie-rkan-W 19:01:18,594 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary] 19:01:18,596 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003.summary dst=null perm=null 19:01:18,598 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:01:18,599 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist 19:01:18,601 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0003-1394161262094-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dr-1394161276438-1-0-SUCCEEDED-root.test-1394161268447.jhist perm=test:supergroup:rwxrwx--- 19:01:18,601 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0003_conf.xml 19:01:18,603 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0003_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0003_conf.xml perm=test:supergroup:rwxrwx--- 19:01:18,627 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:18,628 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:18,629 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190100960-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:01:18,632 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52955, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-341598792_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056, duration: 265000 19:01:18,708 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161278708, maxDate=1394766078708, sequenceNumber=12, masterKeyId=2 19:01:18,708 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 12 19:01:19,954 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053 19:01:19,955 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056 19:01:19,956 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057 19:01:19,957 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055 19:01:19,958 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059 19:01:19,958 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058 19:01:19,959 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054 19:01:19,960 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060 19:01:19,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741872_1048 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741872 for deletion 19:01:19,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741873_1049 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741873 for deletion 19:01:19,971 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741874_1050 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741874 for deletion 19:01:19,971 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741872_1048 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741872 19:01:19,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741875_1051 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741875 for deletion 19:01:19,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741873_1049 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741873 19:01:19,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741874_1050 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741874 19:01:19,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741881_1057 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741881 for deletion 19:01:19,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741875_1051 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741875 19:01:19,972 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741871_1047 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741871 for deletion 19:01:19,972 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741881_1057 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741881 19:01:19,973 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741871_1047 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741871 19:01:20,561 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741880_1056 19:01:20,562 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741883_1059 19:01:20,562 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741882_1058 19:01:20,563 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741884_1060 19:01:20,563 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741881_1057 19:01:20,673 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:20,706 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/COMMIT_STARTED dst=null perm=null 19:01:20,707 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/COMMIT_SUCCESS dst=null perm=null 19:01:20,709 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/COMMIT_FAIL dst=null perm=null 19:01:20,784 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004 dst=null perm=null 19:01:20,786 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:01:20,787 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:01:21,187 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=null 19:01:21,196 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.splitmetainfo dst=null perm=null 19:01:21,261 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52972, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741878_1054, duration: 261000 19:01:21,555 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741882_1058] 19:01:22,547 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0004_000001 19:01:22,548 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0004 APPATTEMPTID=appattempt_1394161202967_0004_000001 19:01:22,548 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from LAUNCHED to RUNNING 19:01:22,548 INFO RMAppImpl:643 - application_1394161202967_0004 State change from ACCEPTED to RUNNING 19:01:22,578 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:01:22,583 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/output/_temporary/1 dst=null perm=test:testg:rwxrwxrwx 19:01:22,648 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:22,863 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741885_1061{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:22,913 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061 src: /127.0.0.1:52978 dest: /127.0.0.1:52685 19:01:22,915 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061 src: /127.0.0.1:52979 dest: /127.0.0.1:52682 19:01:22,952 INFO clienttrace:1215 - src: /127.0.0.1:52979, dest: /127.0.0.1:52682, bytes: 92231, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061, duration: 35398000 19:01:22,952 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:22,953 INFO clienttrace:1215 - src: /127.0.0.1:52978, dest: /127.0.0.1:52685, bytes: 92231, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061, duration: 36252000 19:01:22,953 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:22,957 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1_conf.xml is closed by DFSClient_NONMAPREDUCE_402611553_1 19:01:22,970 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741882_1058 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741882 for deletion 19:01:22,970 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741882_1058 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741882 19:01:22,983 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004.summary dst=null perm=test:supergroup:rw-rw-rw- 19:01:22,986 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741886_1062{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:22,988 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062 src: /127.0.0.1:52980 dest: /127.0.0.1:52685 19:01:22,990 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062 src: /127.0.0.1:52981 dest: /127.0.0.1:52682 19:01:22,993 INFO clienttrace:1215 - src: /127.0.0.1:52981, dest: /127.0.0.1:52682, bytes: 375, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062, duration: 1438000 19:01:22,993 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:22,993 INFO clienttrace:1215 - src: /127.0.0.1:52980, dest: /127.0.0.1:52685, bytes: 375, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062, duration: 2182000 19:01:22,993 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:22,995 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004.summary is closed by DFSClient_NONMAPREDUCE_402611553_1 19:01:23,987 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000003 Container Transitioned from NEW to ALLOCATED 19:01:23,987 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000003 19:01:23,987 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000003 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:01:24,026 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000004 Container Transitioned from NEW to ALLOCATED 19:01:24,026 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000004 19:01:24,026 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000004 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:01:24,556 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741872_1048, blk_1073741873_1049, blk_1073741874_1050, blk_1073741875_1051, blk_1073741881_1057, blk_1073741882_1058, blk_1073741871_1047] 19:01:24,622 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:01:24,623 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 19:01:24,649 INFO ContainerLaunch:322 - Container container_1394161202967_0003_01_000001 succeeded 19:01:24,649 INFO Container:888 - Container container_1394161202967_0003_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:01:24,649 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0003_01_000001 19:01:24,666 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000001 19:01:24,666 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000001 19:01:24,666 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000001 19:01:24,666 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0003/container_1394161202967_0003_01_000001 19:01:24,667 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000001 19:01:24,667 INFO Container:888 - Container container_1394161202967_0003_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 19:01:24,667 INFO Application:339 - Removing container_1394161202967_0003_01_000001 from application application_1394161202967_0003 19:01:24,667 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:24,667 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0003 19:01:24,679 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:24,680 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:24,692 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:24,695 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:24,697 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:24,706 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:24,707 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:24,709 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:24,710 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:24,711 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:24,712 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/app/lib/test.jar dst=null perm=null 19:01:24,772 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0004_01_000003 by user test 19:01:24,772 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0004 19:01:24,772 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000003 19:01:24,772 INFO Application:450 - Application application_1394161202967_0004 transitioned from NEW to INITING 19:01:24,773 INFO Application:296 - Adding container_1394161202967_0004_01_000003 to application application_1394161202967_0004 19:01:24,773 INFO Application:450 - Application application_1394161202967_0004 transitioned from INITING to RUNNING 19:01:24,773 INFO Container:888 - Container container_1394161202967_0004_01_000003 transitioned from NEW to LOCALIZING 19:01:24,773 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0004 19:01:24,773 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0004 19:01:24,773 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:01:24,773 INFO ShuffleHandler:263 - Added token for job_1394161202967_0004 19:01:24,773 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml transitioned from INIT to DOWNLOADING 19:01:24,774 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0004_01_000003 19:01:24,838 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/nmPrivate/container_1394161202967_0004_01_000003.tokens. Credentials list: 19:01:24,950 INFO DefaultContainerExecutor:469 - Initializing user test 19:01:24,998 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0003_01_000001 19:01:25,007 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061 19:01:25,008 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:01:25,039 INFO RMContainerImpl:220 - container_1394161202967_0003_01_000001 Container Transitioned from RUNNING to COMPLETED 19:01:25,039 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0003_01_000001 in state: COMPLETED event:FINISHED 19:01:25,039 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0003_000001 19:01:25,039 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0003 CONTAINERID=container_1394161202967_0003_01_000001 19:01:25,040 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0003_000001 State change from FINISHING to FINISHED 19:01:25,040 INFO FSSchedulerNode:162 - Released container container_1394161202967_0003_01_000001 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:01:25,040 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0003_000001 released container container_1394161202967_0003_01_000001 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:01:25,040 INFO RMAppImpl:643 - application_1394161202967_0003 State change from FINISHING to FINISHED 19:01:25,040 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062 19:01:25,041 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0003 19:01:25,041 INFO FairScheduler:720 - Application appattempt_1394161202967_0003_000001 is done. finalState=FINISHED 19:01:25,041 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0003,name=oozie:launcher:T\=map-reduce:W\=test-wf-cred:A\=map-reduce:ID\=0000000-140306190100960-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0003/jobhistory/job/job_1394161202967_0003,appMasterHost=rkanter-mbp.local,startTime=1394161262094,finishTime=1394161277477,finalStatus=SUCCEEDED 19:01:25,041 INFO AppSchedulingInfo:108 - Application application_1394161202967_0003 requests cleared 19:01:25,041 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0003_000001 19:01:25,162 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0003_01_000001 19:01:25,371 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/nmPrivate/container_1394161202967_0004_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000003.tokens 19:01:25,371 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0004 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0004 19:01:25,616 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061 19:01:25,617 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741886_1062 19:01:25,619 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:25,621 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:25,622 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml dst=null perm=null 19:01:25,666 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000004 Container Transitioned from ACQUIRED to RELEASED 19:01:25,666 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000004 in state: RELEASED event:RELEASED 19:01:25,666 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000004 19:01:25,669 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000004 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:01:25,669 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000004 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:01:25,669 INFO AppSchedulingInfo:166 - checking for deactivate... 19:01:25,669 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52986, bytes: 78444, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1623570841_1711, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741879_1055, duration: 528000 19:01:25,774 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0004/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:01:25,774 INFO Container:888 - Container container_1394161202967_0004_01_000003 transitioned from LOCALIZING to LOCALIZED 19:01:25,855 INFO Container:888 - Container container_1394161202967_0004_01_000003 transitioned from LOCALIZED to RUNNING 19:01:25,855 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:26,001 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741872_1048 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741872 for deletion 19:01:26,002 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741873_1049 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741873 for deletion 19:01:26,007 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741874_1050 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741874 for deletion 19:01:26,008 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741872_1048 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741872 19:01:26,008 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741875_1051 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741875 for deletion 19:01:26,007 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741873_1049 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741873 19:01:26,020 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741874_1050 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741874 19:01:26,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741875_1051 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741875 19:01:26,019 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741881_1057 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741881 for deletion 19:01:26,028 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741882_1058 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741882 for deletion 19:01:26,028 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741881_1057 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741881 19:01:26,028 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741882_1058 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741882 19:01:26,028 INFO Application:450 - Application application_1394161202967_0003 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:01:26,028 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741871_1047 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741871 for deletion 19:01:26,028 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741871_1047 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741871 19:01:26,035 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0003 19:01:26,035 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0003 19:01:26,035 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0003 19:01:26,036 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0003 19:01:26,036 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0003 19:01:26,036 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000005 Container Transitioned from NEW to ALLOCATED 19:01:26,037 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000005 19:01:26,037 INFO Application:450 - Application application_1394161202967_0003 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:01:26,037 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000005 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:01:26,037 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0003, with delay of 10800 seconds 19:01:26,084 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0004_01_000004 19:01:26,239 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000003/default_container_executor.sh] 19:01:26,679 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000005 Container Transitioned from ALLOCATED to ACQUIRED 19:01:26,687 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0004_01_000005 by user test 19:01:26,687 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000005 19:01:26,687 INFO Application:296 - Adding container_1394161202967_0004_01_000005 to application application_1394161202967_0004 19:01:26,688 INFO Container:888 - Container container_1394161202967_0004_01_000005 transitioned from NEW to LOCALIZING 19:01:26,688 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0004 19:01:26,688 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0004 19:01:26,688 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:01:26,688 INFO ShuffleHandler:263 - Added token for job_1394161202967_0004 19:01:26,689 INFO Container:888 - Container container_1394161202967_0004_01_000005 transitioned from LOCALIZING to LOCALIZED 19:01:26,829 INFO Container:888 - Container container_1394161202967_0004_01_000005 transitioned from LOCALIZED to RUNNING 19:01:26,829 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:27,020 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000005 Container Transitioned from ACQUIRED to RUNNING 19:01:27,253 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000005/default_container_executor.sh] 19:01:27,687 INFO AppSchedulingInfo:166 - checking for deactivate... 19:01:29,444 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:29,538 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52990, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0004_m_000000_0_547822024_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053, duration: 319000 19:01:29,602 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input/data.txt dst=null perm=null 19:01:29,838 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52991, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0004_m_000000_0_547822024_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042, duration: 605000 19:01:29,869 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/output/_temporary/1/_temporary/attempt_1394161202967_0004_m_000000_0 dst=null perm=null 19:01:29,895 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0004_01_000003 19:01:29,895 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000003 19:01:29,895 INFO Container:888 - Container container_1394161202967_0004_01_000003 transitioned from RUNNING to KILLING 19:01:29,895 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0004_01_000003 19:01:29,950 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0004_01_000003 is : 143 19:01:29,951 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741887_1063{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:29,956 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063 src: /127.0.0.1:52992 dest: /127.0.0.1:52685 19:01:29,958 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063 src: /127.0.0.1:52993 dest: /127.0.0.1:52682 19:01:29,964 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1.jhist for DFSClient_NONMAPREDUCE_402611553_1 19:01:29,972 INFO Container:888 - Container container_1394161202967_0004_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:01:29,973 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000003 19:01:29,973 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000003 19:01:29,974 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000003 19:01:29,974 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000003 19:01:29,974 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000003 19:01:29,974 INFO Container:888 - Container container_1394161202967_0004_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:01:29,974 INFO Application:339 - Removing container_1394161202967_0004_01_000003 from application application_1394161202967_0004 19:01:29,975 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:29,975 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0004 19:01:30,414 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job.split dst=null perm=null 19:01:30,495 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52995, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0004_m_000001_1_2112846345_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741877_1053, duration: 523000 19:01:30,541 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/input/data.txt dst=null perm=null 19:01:30,545 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:52995, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0004_m_000001_1_2112846345_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741866_1042, duration: 216000 19:01:30,634 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/output/_temporary/1/_temporary/attempt_1394161202967_0004_m_000001_1 dst=null perm=null 19:01:30,649 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0004_01_000005 19:01:30,649 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000005 19:01:30,649 INFO Container:888 - Container container_1394161202967_0004_01_000005 transitioned from RUNNING to KILLING 19:01:30,650 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0004_01_000005 19:01:30,650 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0004_01_000003 19:01:30,683 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000003 Container Transitioned from RUNNING to COMPLETED 19:01:30,684 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000003 in state: COMPLETED event:FINISHED 19:01:30,684 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000003 19:01:30,685 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000003 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:01:30,685 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000003 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:01:30,721 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0004_01_000005 is : 143 19:01:30,740 INFO Container:888 - Container container_1394161202967_0004_01_000005 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:01:30,740 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000005 19:01:30,740 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000005 19:01:30,740 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000005 19:01:30,741 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000005 19:01:30,741 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000005 19:01:30,741 INFO Container:888 - Container container_1394161202967_0004_01_000005 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:01:30,741 INFO Application:339 - Removing container_1394161202967_0004_01_000005 from application application_1394161202967_0004 19:01:30,741 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:30,741 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0004 19:01:31,660 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0004_01_000005 19:01:31,660 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000005 Container Transitioned from RUNNING to COMPLETED 19:01:31,661 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000005 in state: COMPLETED event:FINISHED 19:01:31,661 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000005 19:01:31,661 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000005 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:01:31,661 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000005 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:01:32,117 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000006 Container Transitioned from NEW to ALLOCATED 19:01:32,117 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000006 19:01:32,118 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000006 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:01:32,736 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000006 Container Transitioned from ALLOCATED to ACQUIRED 19:01:33,740 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000006 Container Transitioned from ACQUIRED to RELEASED 19:01:33,740 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000006 in state: RELEASED event:RELEASED 19:01:33,740 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000006 19:01:33,741 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000006 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:01:33,741 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000006 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:01:33,741 INFO AppSchedulingInfo:166 - checking for deactivate... 19:01:34,119 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0004_01_000006 19:01:35,119 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000007 Container Transitioned from NEW to ALLOCATED 19:01:35,120 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000007 19:01:35,120 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000007 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:01:35,750 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000007 Container Transitioned from ALLOCATED to ACQUIRED 19:01:36,756 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000007 Container Transitioned from ACQUIRED to RELEASED 19:01:36,756 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000007 in state: RELEASED event:RELEASED 19:01:36,757 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000007 19:01:36,757 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000007 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:01:36,757 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000007 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:01:36,757 INFO AppSchedulingInfo:166 - checking for deactivate... 19:01:37,121 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0004_01_000007 19:01:38,123 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000008 Container Transitioned from NEW to ALLOCATED 19:01:38,123 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000008 19:01:38,124 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000008 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:01:38,769 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000008 Container Transitioned from ALLOCATED to ACQUIRED 19:01:39,774 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000008 Container Transitioned from ACQUIRED to RELEASED 19:01:39,774 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000008 in state: RELEASED event:RELEASED 19:01:39,775 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000008 19:01:39,775 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000008 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:01:39,775 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000008 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:01:39,775 INFO AppSchedulingInfo:166 - checking for deactivate... 19:01:40,124 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0004_01_000008 19:01:41,126 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000009 Container Transitioned from NEW to ALLOCATED 19:01:41,126 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000009 19:01:41,126 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0004_01_000009 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:01:41,790 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000009 Container Transitioned from ALLOCATED to ACQUIRED 19:01:41,807 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/b130ab94-f175-4e88-950f-b0bcd615795c/output/_temporary dst=null perm=null 19:01:41,817 INFO clienttrace:1215 - src: /127.0.0.1:52993, dest: /127.0.0.1:52682, bytes: 23547, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063, duration: 11856537000 19:01:41,817 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:41,818 INFO clienttrace:1215 - src: /127.0.0.1:52992, dest: /127.0.0.1:52685, bytes: 23547, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063, duration: 11857636000 19:01:41,818 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:41,820 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1.jhist is closed by DFSClient_NONMAPREDUCE_402611553_1 19:01:41,822 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:01:41,824 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741888_1064{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:41,827 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064 src: /127.0.0.1:52997 dest: /127.0.0.1:52685 19:01:41,829 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064 src: /127.0.0.1:52998 dest: /127.0.0.1:52682 19:01:41,832 INFO clienttrace:1215 - src: /127.0.0.1:52998, dest: /127.0.0.1:52682, bytes: 419, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064, duration: 1499000 19:01:41,832 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:41,833 INFO clienttrace:1215 - src: /127.0.0.1:52997, dest: /127.0.0.1:52685, bytes: 419, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064, duration: 2080000 19:01:41,833 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:41,835 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary_tmp is closed by DFSClient_NONMAPREDUCE_402611553_1 19:01:41,838 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:01:41,844 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1.jhist dst=null perm=null 19:01:41,845 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist_tmp dst=null perm=null 19:01:41,850 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1.jhist dst=null perm=null 19:01:41,851 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist_tmp dst=null perm=null 19:01:41,852 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1.jhist dst=null perm=null 19:01:41,855 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:01:41,857 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52999, bytes: 23731, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063, duration: 229000 19:01:41,859 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741889_1065{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:41,861 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065 src: /127.0.0.1:53000 dest: /127.0.0.1:52685 19:01:41,863 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065 src: /127.0.0.1:53001 dest: /127.0.0.1:52682 19:01:41,866 INFO clienttrace:1215 - src: /127.0.0.1:53001, dest: /127.0.0.1:52682, bytes: 23547, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065, duration: 1591000 19:01:41,866 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:41,867 INFO clienttrace:1215 - src: /127.0.0.1:53000, dest: /127.0.0.1:52685, bytes: 23547, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065, duration: 2288000 19:01:41,867 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:41,869 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist_tmp is closed by DFSClient_NONMAPREDUCE_402611553_1 19:01:41,870 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:01:41,871 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1_conf.xml dst=null perm=null 19:01:41,872 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml_tmp dst=null perm=null 19:01:41,873 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1_conf.xml dst=null perm=null 19:01:41,874 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml_tmp dst=null perm=null 19:01:41,875 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004/job_1394161202967_0004_1_conf.xml dst=null perm=null 19:01:41,877 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:01:41,880 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:52999, bytes: 92955, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741885_1061, duration: 368000 19:01:41,882 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741890_1066{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:41,884 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066 src: /127.0.0.1:53002 dest: /127.0.0.1:52685 19:01:41,886 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066 src: /127.0.0.1:53003 dest: /127.0.0.1:52682 19:01:41,889 INFO clienttrace:1215 - src: /127.0.0.1:53003, dest: /127.0.0.1:52682, bytes: 92231, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066, duration: 1854000 19:01:41,889 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:41,890 INFO clienttrace:1215 - src: /127.0.0.1:53002, dest: /127.0.0.1:52685, bytes: 92231, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_402611553_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066, duration: 2534000 19:01:41,890 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:41,892 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_402611553_1 19:01:41,893 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:01:41,897 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary perm=test:supergroup:rwxrwx--- 19:01:41,902 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml perm=test:supergroup:rwxrwx--- 19:01:41,904 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist perm=test:supergroup:rwxrwx--- 19:01:41,911 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0004_000001 with final state: FINISHING 19:01:41,911 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from RUNNING to FINAL_SAVING 19:01:41,912 INFO RMAppImpl:880 - Updating application application_1394161202967_0004 with final state: FINISHING 19:01:41,912 INFO RMAppImpl:643 - application_1394161202967_0004 State change from RUNNING to FINAL_SAVING 19:01:41,912 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0004 19:01:41,913 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from FINAL_SAVING to FINISHING 19:01:41,913 INFO RMAppImpl:643 - application_1394161202967_0004 State change from FINAL_SAVING to FINISHING 19:01:42,921 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0004 dst=null perm=null 19:01:43,779 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:01:43,785 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741891_1067{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:43,788 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067 src: /127.0.0.1:53006 dest: /127.0.0.1:52682 19:01:43,790 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067 src: /127.0.0.1:53007 dest: /127.0.0.1:52685 19:01:43,793 INFO clienttrace:1215 - src: /127.0.0.1:53007, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067, duration: 1648000 19:01:43,794 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:43,794 INFO clienttrace:1215 - src: /127.0.0.1:53006, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067, duration: 2421000 19:01:43,795 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:43,797 INFO StateChange:2959 - DIR* completeFile: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/input/data.txt is closed by DFSClient_NONMAPREDUCE_1069987893_8 19:01:43,804 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/uber.jar dst=null perm=null 19:01:43,806 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/uber.jar dst=null perm=test:testg:rw-rw-rw- 19:01:43,808 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/uber.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741892_1068{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:43,810 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068 src: /127.0.0.1:53008 dest: /127.0.0.1:52682 19:01:43,812 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068 src: /127.0.0.1:53009 dest: /127.0.0.1:52685 19:01:43,814 INFO clienttrace:1215 - src: /127.0.0.1:53009, dest: /127.0.0.1:52685, bytes: 426, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068, duration: 1414000 19:01:43,815 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:43,815 INFO clienttrace:1215 - src: /127.0.0.1:53008, dest: /127.0.0.1:52682, bytes: 426, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068, duration: 2003000 19:01:43,816 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:43,817 INFO StateChange:2959 - DIR* completeFile: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/uber.jar is closed by DFSClient_NONMAPREDUCE_1069987893_8 19:01:43,830 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:01:43,832 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741893_1069{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:43,834 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069 src: /127.0.0.1:53010 dest: /127.0.0.1:52682 19:01:43,836 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069 src: /127.0.0.1:53011 dest: /127.0.0.1:52685 19:01:43,839 INFO clienttrace:1215 - src: /127.0.0.1:53011, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069, duration: 1514000 19:01:43,839 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:43,840 INFO clienttrace:1215 - src: /127.0.0.1:53010, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069, duration: 2315000 19:01:43,840 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:43,842 INFO StateChange:2959 - DIR* completeFile: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_1069987893_8 19:01:43,844 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:01:43,846 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741894_1070{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:01:43,848 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070 src: /127.0.0.1:53012 dest: /127.0.0.1:52682 19:01:43,850 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070 src: /127.0.0.1:53013 dest: /127.0.0.1:52685 19:01:43,853 INFO clienttrace:1215 - src: /127.0.0.1:53013, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070, duration: 1730000 19:01:43,854 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:43,854 INFO clienttrace:1215 - src: /127.0.0.1:53012, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1069987893_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070, duration: 2449000 19:01:43,855 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:43,858 INFO StateChange:2959 - DIR* completeFile: /user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_1069987893_8 19:01:43,862 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190143455-oozie-rkan-W/map-reduce--java dst=null perm=null 19:01:43,864 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190143455-oozie-rkan-W/map-reduce--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:01:43,865 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190143455-oozie-rkan-W/map-reduce--java.tmp dst=/user/test/oozie-rkan/0000000-140306190143455-oozie-rkan-W/map-reduce--java perm=test:supergroup:rwxrwxrwx 19:01:43,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036/app/workflow.xml dst=null perm=null 19:01:44,539 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=test:testg:rw-rw-rw- 19:01:44,544 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741895_1071{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:44,547 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071 src: /127.0.0.1:53015 dest: /127.0.0.1:52682 19:01:44,549 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071 src: /127.0.0.1:53016 dest: /127.0.0.1:52685 19:01:44,552 INFO clienttrace:1215 - src: /127.0.0.1:53016, dest: /127.0.0.1:52685, bytes: 104954, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071, duration: 1617000 19:01:44,552 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,553 INFO clienttrace:1215 - src: /127.0.0.1:53015, dest: /127.0.0.1:52682, bytes: 104954, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071, duration: 2273000 19:01:44,553 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,555 INFO StateChange:2959 - DIR* completeFile: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar is closed by DFSClient_NONMAPREDUCE_1878067409_8 19:01:44,557 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:01:44,559 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741896_1072{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:44,561 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072 src: /127.0.0.1:53017 dest: /127.0.0.1:52685 19:01:44,562 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072 src: /127.0.0.1:53018 dest: /127.0.0.1:52682 19:01:44,565 INFO clienttrace:1215 - src: /127.0.0.1:53018, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072, duration: 1515000 19:01:44,566 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,566 INFO clienttrace:1215 - src: /127.0.0.1:53017, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072, duration: 2235000 19:01:44,567 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,568 INFO StateChange:2959 - DIR* completeFile: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input/data.txt is closed by DFSClient_NONMAPREDUCE_1878067409_8 19:01:44,577 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:01:44,579 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741897_1073{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:44,581 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073 src: /127.0.0.1:53019 dest: /127.0.0.1:52685 19:01:44,582 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073 src: /127.0.0.1:53020 dest: /127.0.0.1:52682 19:01:44,586 INFO clienttrace:1215 - src: /127.0.0.1:53020, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073, duration: 1563000 19:01:44,586 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,587 INFO clienttrace:1215 - src: /127.0.0.1:53019, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073, duration: 2451000 19:01:44,587 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,589 INFO StateChange:2959 - DIR* completeFile: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_1878067409_8 19:01:44,591 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:01:44,593 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741898_1074{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:44,595 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074 src: /127.0.0.1:53021 dest: /127.0.0.1:52685 19:01:44,597 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074 src: /127.0.0.1:53022 dest: /127.0.0.1:52682 19:01:44,600 INFO clienttrace:1215 - src: /127.0.0.1:53022, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074, duration: 1347000 19:01:44,600 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,601 INFO clienttrace:1215 - src: /127.0.0.1:53021, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1878067409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074, duration: 2011000 19:01:44,601 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,603 INFO StateChange:2959 - DIR* completeFile: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_1878067409_8 19:01:44,608 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java dst=null perm=null 19:01:44,610 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:01:44,612 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java.tmp dst=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java perm=test:supergroup:rwxrwxrwx 19:01:44,612 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/workflow.xml dst=null perm=null 19:01:44,666 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/workflow.xml dst=null perm=null 19:01:44,715 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java dst=null perm=test:supergroup:rwxrwxrwx 19:01:44,718 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:44,726 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741899_1075{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:44,729 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075 src: /127.0.0.1:53024 dest: /127.0.0.1:52685 19:01:44,730 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075 src: /127.0.0.1:53025 dest: /127.0.0.1:52682 19:01:44,733 INFO clienttrace:1215 - src: /127.0.0.1:53025, dest: /127.0.0.1:52682, bytes: 77782, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075, duration: 1672000 19:01:44,733 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,734 INFO clienttrace:1215 - src: /127.0.0.1:53024, dest: /127.0.0.1:52685, bytes: 77782, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075, duration: 2963000 19:01:44,735 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,737 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action.xml is closed by DFSClient_NONMAPREDUCE_943124610_8 19:01:44,738 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input dst=null perm=test:supergroup:rwxrwxrwx 19:01:44,740 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:01:44,741 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741900_1076{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:01:44,743 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076 src: /127.0.0.1:53026 dest: /127.0.0.1:52685 19:01:44,744 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076 src: /127.0.0.1:53027 dest: /127.0.0.1:52682 19:01:44,746 INFO clienttrace:1215 - src: /127.0.0.1:53027, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076, duration: 1021000 19:01:44,746 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,746 INFO clienttrace:1215 - src: /127.0.0.1:53026, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076, duration: 1635000 19:01:44,747 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,750 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_943124610_8 19:01:44,767 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161304767, maxDate=1394766104767, sequenceNumber=13, masterKeyId=2 19:01:44,767 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 13 19:01:44,768 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/0000000-140306190144221-oozie-rkan-W@mr-action dst=null perm=null 19:01:44,769 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161304769, maxDate=1394766104769, sequenceNumber=14, masterKeyId=2 19:01:44,769 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 14 19:01:44,783 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output dst=null perm=null 19:01:44,784 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:44,785 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:44,786 INFO ClientRMService:229 - Allocated new applicationId: 5 19:01:44,787 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005 dst=null perm=null 19:01:44,789 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005 dst=null perm=test:supergroup:rwxrwxrwx 19:01:44,790 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005 dst=null perm=test:supergroup:rwx------ 19:01:44,791 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:44,792 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:44,793 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:44,794 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:44,795 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:44,796 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:01:44,797 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:01:44,798 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:01:44,799 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:44,799 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:44,800 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:44,801 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:44,802 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib dst=null perm=null 19:01:44,803 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app dst=null perm=null 19:01:44,804 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9 dst=null perm=null 19:01:44,804 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:44,806 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:44,807 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar dst=null perm=null 19:01:44,809 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input dst=null perm=null 19:01:44,810 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input dst=null perm=null 19:01:44,811 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input dst=null perm=null 19:01:44,814 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:01:44,815 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=test:supergroup:rw-r--r-- 19:01:44,816 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split 19:01:44,817 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=null 19:01:44,818 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741901_1077{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:44,820 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077 src: /127.0.0.1:53029 dest: /127.0.0.1:52682 19:01:44,821 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077 src: /127.0.0.1:53030 dest: /127.0.0.1:52685 19:01:44,824 INFO clienttrace:1215 - src: /127.0.0.1:53030, dest: /127.0.0.1:52685, bytes: 171, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077, duration: 2096000 19:01:44,824 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,825 INFO clienttrace:1215 - src: /127.0.0.1:53029, dest: /127.0.0.1:52682, bytes: 171, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077, duration: 1740000 19:01:44,825 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,827 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split is closed by DFSClient_NONMAPREDUCE_943124610_8 19:01:44,828 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:01:44,830 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:01:44,831 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741902_1078{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:01:44,833 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078 src: /127.0.0.1:53031 dest: /127.0.0.1:52685 19:01:44,834 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078 src: /127.0.0.1:53032 dest: /127.0.0.1:52682 19:01:44,836 INFO clienttrace:1215 - src: /127.0.0.1:53032, dest: /127.0.0.1:52682, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078, duration: 1203000 19:01:44,836 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,837 INFO clienttrace:1215 - src: /127.0.0.1:53031, dest: /127.0.0.1:52685, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078, duration: 1857000 19:01:44,837 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,839 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_943124610_8 19:01:44,840 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:44,842 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:01:44,848 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741903_1079{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:44,850 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079 src: /127.0.0.1:53033 dest: /127.0.0.1:52682 19:01:44,851 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079 src: /127.0.0.1:53034 dest: /127.0.0.1:52685 19:01:44,854 INFO clienttrace:1215 - src: /127.0.0.1:53034, dest: /127.0.0.1:52685, bytes: 80141, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079, duration: 1507000 19:01:44,854 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:44,855 INFO clienttrace:1215 - src: /127.0.0.1:53033, dest: /127.0.0.1:52682, bytes: 80141, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079, duration: 2108000 19:01:44,855 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:44,857 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml is closed by DFSClient_NONMAPREDUCE_943124610_8 19:01:44,858 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005 dst=null perm=null 19:01:44,859 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:44,860 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:44,861 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=null 19:01:44,861 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=null 19:01:44,862 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=null 19:01:44,863 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=null 19:01:44,865 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:44,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:44,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:44,867 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:44,868 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:44,868 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:44,869 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:44,870 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:44,871 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:44,871 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:44,872 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:44,873 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:44,875 INFO ClientRMService:342 - Application with id 5 submitted by user test 19:01:44,875 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0005 19:01:44,875 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0005 19:01:44,875 INFO RMAppImpl:643 - application_1394161202967_0005 State change from NEW to NEW_SAVING 19:01:44,875 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0005 19:01:44,875 INFO RMAppImpl:643 - application_1394161202967_0005 State change from NEW_SAVING to SUBMITTED 19:01:44,875 INFO FairScheduler:638 - Accepted application application_1394161202967_0005 from user: test, in queue: default, currently num of applications: 2 19:01:44,875 INFO RMAppImpl:643 - application_1394161202967_0005 State change from SUBMITTED to ACCEPTED 19:01:44,876 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0005_000001 19:01:44,876 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from NEW to SUBMITTED 19:01:44,876 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0005_000001 to scheduler from user: test 19:01:44,877 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from SUBMITTED to SCHEDULED 19:01:44,909 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161304909, maxDate=1394766104909, sequenceNumber=15, masterKeyId=2 19:01:44,910 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 15 19:01:45,049 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066 19:01:45,050 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063 19:01:45,051 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079 19:01:45,051 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070 19:01:45,051 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077 19:01:45,052 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071 19:01:45,053 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067 19:01:45,053 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069 19:01:45,053 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068 19:01:45,053 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074 19:01:45,054 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064 19:01:45,054 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073 19:01:45,054 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072 19:01:45,055 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075 19:01:45,132 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0005 19:01:45,132 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000001 Container Transitioned from NEW to RESERVED 19:01:45,132 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0005 reserved container container_1394161202967_0005_01_000001 on node host: 172.16.1.64:52703 #containers=2 available= used=, currently has 1 at priority 0; currentReservation 2048 19:01:45,132 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0005_01_000001 on node host: 172.16.1.64:52703 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@1c14a56f 19:01:45,252 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065 19:01:45,253 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076 19:01:45,254 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078 19:01:45,562 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741877_1053, blk_1073741878_1054, blk_1073741879_1055, blk_1073741885_1061, blk_1073741886_1062, blk_1073741887_1063] 19:01:45,625 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071 19:01:45,626 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741893_1069 19:01:45,627 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075 19:01:45,628 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741892_1068 19:01:45,674 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000002 Container Transitioned from NEW to ALLOCATED 19:01:45,674 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000002 19:01:45,674 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0005_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:01:45,675 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:01:45,675 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0005 AttemptId: appattempt_1394161202967_0005_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0005_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:01:45,676 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:01:45,676 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:01:45,676 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0005_000001 19:01:45,678 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0005_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0005_000001 19:01:45,678 INFO AMLauncher:190 - Command to launch container container_1394161202967_0005_01_000002 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:01:45,684 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0005_01_000002 by user test 19:01:45,684 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0005 19:01:45,685 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000002 19:01:45,685 INFO Application:450 - Application application_1394161202967_0005 transitioned from NEW to INITING 19:01:45,685 INFO Application:296 - Adding container_1394161202967_0005_01_000002 to application application_1394161202967_0005 19:01:45,685 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0005_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0005_000001 19:01:45,685 INFO Application:450 - Application application_1394161202967_0005 transitioned from INITING to RUNNING 19:01:45,685 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from ALLOCATED to LAUNCHED 19:01:45,686 INFO Container:888 - Container container_1394161202967_0005_01_000002 transitioned from NEW to LOCALIZING 19:01:45,686 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0005 19:01:45,686 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:01:45,686 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split transitioned from INIT to DOWNLOADING 19:01:45,686 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml transitioned from INIT to DOWNLOADING 19:01:45,686 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar transitioned from INIT to DOWNLOADING 19:01:45,687 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:01:45,687 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0005_01_000002 19:01:45,687 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar, 1394161304554, FILE, null } 19:01:45,687 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar, 1394161304589, FILE, null } 19:01:45,826 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0005_01_000002.tokens. Credentials list: 19:01:45,833 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741890_1066 19:01:45,912 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077 19:01:45,945 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073 19:01:45,948 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741889_1065 19:01:45,956 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741891_1067 19:01:45,981 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:45,981 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:45,984 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079 19:01:46,007 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064 19:01:46,014 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:46,014 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:46,015 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741887_1063 19:01:46,040 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078 19:01:46,048 INFO DefaultContainerExecutor:469 - Initializing user test 19:01:46,049 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741898_1074 19:01:46,049 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar dst=null perm=null 19:01:46,049 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib dst=null perm=null 19:01:46,056 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072 19:01:46,089 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076 19:01:46,089 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741894_1070 19:01:46,113 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app dst=null perm=null 19:01:46,113 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9 dst=null perm=null 19:01:46,143 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:46,143 INFO FairScheduler:1042 - Releasing reservation that cannot be satisfied for application appattempt_1394161202967_0005_000001 on node host: 172.16.1.64:52703 #containers=2 available= used= 19:01:46,143 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9 dst=null perm=null 19:01:46,143 INFO FSSchedulerApp:123 - Application application_1394161202967_0005 unreserved on node host: 172.16.1.64:52703 #containers=2 available= used=, currently has 0 at priority 0; currentReservation 19:01:46,175 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:46,175 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:46,211 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:46,211 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:46,276 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:46,276 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:46,336 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:46,336 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:46,373 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:46,482 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53037, bytes: 105774, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071, duration: 38763000 19:01:46,580 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53037, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073, duration: 68903000 19:01:46,745 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:01:46,865 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/filecache/14/hadoop-streaming.jar) transitioned from DOWNLOADING to LOCALIZED 19:01:46,866 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/filecache/15/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:01:46,874 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0005_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000002.tokens 19:01:46,874 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0005 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0005 19:01:46,998 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741877_1053 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741877 for deletion 19:01:46,998 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741878_1054 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741878 for deletion 19:01:47,005 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741879_1055 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741879 for deletion 19:01:47,022 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741878_1054 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741878 19:01:47,006 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741877_1053 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741877 19:01:47,033 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741885_1061 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741885 for deletion 19:01:47,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741886_1062 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741886 for deletion 19:01:47,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741887_1063 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741887 for deletion 19:01:47,034 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741879_1055 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741879 19:01:47,034 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741886_1062 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741886 19:01:47,034 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741885_1061 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741885 19:01:47,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741887_1063 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741887 19:01:47,196 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=null 19:01:47,198 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=null 19:01:47,199 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=null 19:01:47,229 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53037, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1436424471_1980, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078, duration: 228000 19:01:47,279 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0005/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:01:47,335 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=null 19:01:47,337 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=null 19:01:47,338 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=null 19:01:47,368 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53040, bytes: 175, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1436424471_1980, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077, duration: 212000 19:01:47,450 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0005/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:01:47,511 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:47,513 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:47,514 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:47,542 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53040, bytes: 80769, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1436424471_1980, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079, duration: 391000 19:01:47,627 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0005/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:01:47,627 INFO Container:888 - Container container_1394161202967_0005_01_000002 transitioned from LOCALIZING to LOCALIZED 19:01:47,699 INFO Container:888 - Container container_1394161202967_0005_01_000002 transitioned from LOCALIZED to RUNNING 19:01:47,699 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:47,974 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000002/default_container_executor.sh] 19:01:48,563 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741877_1053, blk_1073741878_1054, blk_1073741879_1055, blk_1073741885_1061, blk_1073741886_1062, blk_1073741887_1063] 19:01:49,069 INFO ContainerLaunch:322 - Container container_1394161202967_0004_01_000002 succeeded 19:01:49,069 INFO Container:888 - Container container_1394161202967_0004_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:01:49,070 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0004_01_000002 19:01:49,090 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000002 19:01:49,090 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000002 19:01:49,091 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000002 19:01:49,091 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000002 19:01:49,091 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0004/container_1394161202967_0004_01_000002 19:01:49,091 INFO Container:888 - Container container_1394161202967_0004_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 19:01:49,091 INFO Application:339 - Removing container_1394161202967_0004_01_000002 from application application_1394161202967_0004 19:01:49,091 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:49,091 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0004 19:01:49,161 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0004_01_000002 19:01:49,161 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000002 Container Transitioned from RUNNING to COMPLETED 19:01:49,161 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000002 in state: COMPLETED event:FINISHED 19:01:49,161 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0004_000001 19:01:49,161 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000002 19:01:49,161 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0004_000001 State change from FINISHING to FINISHED 19:01:49,162 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:01:49,162 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000002 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:01:49,162 INFO RMAppImpl:643 - application_1394161202967_0004 State change from FINISHING to FINISHED 19:01:49,162 INFO FairScheduler:720 - Application appattempt_1394161202967_0004_000001 is done. finalState=FINISHED 19:01:49,162 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0004 19:01:49,163 INFO RMContainerImpl:220 - container_1394161202967_0004_01_000009 Container Transitioned from ACQUIRED to KILLED 19:01:49,163 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0004_000001 19:01:49,164 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0004_01_000009 in state: KILLED event:KILL 19:01:49,164 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0004 CONTAINERID=container_1394161202967_0004_01_000009 19:01:49,163 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0004,name=oozie:action:T\=map-reduce:W\=test-wf-cred:A\=map-reduce:ID\=0000000-140306190100960-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0004/jobhistory/job/job_1394161202967_0004,appMasterHost=rkanter-mbp.local,startTime=1394161275666,finishTime=1394161301912,finalStatus=FAILED 19:01:49,164 INFO FSSchedulerNode:162 - Released container container_1394161202967_0004_01_000009 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:01:49,164 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0004_000001 released container container_1394161202967_0004_01_000009 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: KILL 19:01:49,164 INFO AppSchedulingInfo:108 - Application application_1394161202967_0004 requests cleared 19:01:49,169 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0004_01_000002 19:01:49,722 INFO Application:450 - Application application_1394161202967_0004 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:01:49,722 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0004 19:01:49,723 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0004 19:01:49,723 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0004 19:01:49,723 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0004 19:01:49,723 INFO Application:450 - Application application_1394161202967_0004 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:01:49,723 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0004 19:01:49,723 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0004, with delay of 10800 seconds 19:01:49,977 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741877_1053 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741877 for deletion 19:01:49,977 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741878_1054 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741878 for deletion 19:01:49,977 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741879_1055 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741879 for deletion 19:01:49,977 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741877_1053 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741877 19:01:49,977 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741885_1061 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741885 for deletion 19:01:49,978 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741878_1054 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741878 19:01:49,978 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741886_1062 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741886 for deletion 19:01:49,978 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741879_1055 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741879 19:01:49,978 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741887_1063 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741887 for deletion 19:01:49,978 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741886_1062 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741886 19:01:49,978 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741885_1061 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741885 19:01:49,978 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741887_1063 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741887 19:01:50,161 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0004_01_000009 19:01:50,162 INFO Application:450 - Application application_1394161202967_0004 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:01:50,162 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0004 19:01:50,162 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0004 19:01:50,162 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0004 19:01:50,163 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0004 19:01:50,163 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0004 19:01:50,163 INFO Application:450 - Application application_1394161202967_0004 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:01:50,163 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0004, with delay of 10800 seconds 19:01:50,373 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:01:50,407 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/COMMIT_STARTED dst=null perm=null 19:01:50,408 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/COMMIT_SUCCESS dst=null perm=null 19:01:50,409 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/COMMIT_FAIL dst=null perm=null 19:01:50,480 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005 dst=null perm=null 19:01:50,482 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:01:50,484 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:01:50,870 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=null 19:01:50,878 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.splitmetainfo dst=null perm=null 19:01:50,943 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53043, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741902_1078, duration: 345000 19:01:51,976 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0005_000001 19:01:51,977 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0005 APPATTEMPTID=appattempt_1394161202967_0005_000001 19:01:51,977 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from LAUNCHED to RUNNING 19:01:51,977 INFO RMAppImpl:643 - application_1394161202967_0005 State change from ACCEPTED to RUNNING 19:01:52,022 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:01:52,027 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1 dst=null perm=test:supergroup:rwxrwxrwx 19:01:52,085 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:01:52,308 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741904_1080{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:52,357 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080 src: /127.0.0.1:53049 dest: /127.0.0.1:52682 19:01:52,359 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080 src: /127.0.0.1:53050 dest: /127.0.0.1:52685 19:01:52,401 INFO clienttrace:1215 - src: /127.0.0.1:53050, dest: /127.0.0.1:52685, bytes: 93127, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080, duration: 41035000 19:01:52,402 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:52,402 INFO clienttrace:1215 - src: /127.0.0.1:53049, dest: /127.0.0.1:52682, bytes: 93127, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080, duration: 41756000 19:01:52,403 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:52,407 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1_conf.xml is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:01:52,436 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005.summary dst=null perm=test:supergroup:rw-rw-rw- 19:01:52,439 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741905_1081{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:01:52,442 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081 src: /127.0.0.1:53051 dest: /127.0.0.1:52682 19:01:52,444 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081 src: /127.0.0.1:53052 dest: /127.0.0.1:52685 19:01:52,447 INFO clienttrace:1215 - src: /127.0.0.1:53052, dest: /127.0.0.1:52685, bytes: 371, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081, duration: 1456000 19:01:52,447 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:01:52,448 INFO clienttrace:1215 - src: /127.0.0.1:53051, dest: /127.0.0.1:52682, bytes: 371, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081, duration: 2444000 19:01:52,448 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:01:52,450 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005.summary is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:01:53,164 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000003 Container Transitioned from NEW to ALLOCATED 19:01:53,165 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000003 19:01:53,165 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0005_01_000003 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:01:54,032 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:01:54,088 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:54,089 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:54,101 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:54,103 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:54,104 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:54,105 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:54,114 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:54,117 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:54,118 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:54,119 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:54,120 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:01:54,120 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:01:54,121 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:54,122 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:54,181 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0005_01_000003 by user test 19:01:54,181 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0005 19:01:54,181 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000003 19:01:54,181 INFO Application:450 - Application application_1394161202967_0005 transitioned from NEW to INITING 19:01:54,182 INFO Application:296 - Adding container_1394161202967_0005_01_000003 to application application_1394161202967_0005 19:01:54,182 INFO Application:450 - Application application_1394161202967_0005 transitioned from INITING to RUNNING 19:01:54,182 INFO Container:888 - Container container_1394161202967_0005_01_000003 transitioned from NEW to LOCALIZING 19:01:54,182 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0005 19:01:54,182 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0005 19:01:54,182 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:01:54,182 INFO ShuffleHandler:263 - Added token for job_1394161202967_0005 19:01:54,183 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml transitioned from INIT to DOWNLOADING 19:01:54,183 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar transitioned from INIT to DOWNLOADING 19:01:54,183 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:01:54,183 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0005_01_000003 19:01:54,183 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar, 1394161304554, FILE, null } 19:01:54,185 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar, 1394161304589, FILE, null } 19:01:54,330 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/nmPrivate/container_1394161202967_0005_01_000003.tokens. Credentials list: 19:01:54,409 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:54,409 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:54,438 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:54,438 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:54,470 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib dst=null perm=null 19:01:54,470 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar dst=null perm=null 19:01:54,496 INFO DefaultContainerExecutor:469 - Initializing user test 19:01:54,518 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9 dst=null perm=null 19:01:54,518 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app dst=null perm=null 19:01:54,563 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9 dst=null perm=null 19:01:54,563 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:54,630 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:54,630 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:01:54,670 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:54,670 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:01:54,753 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:01:54,754 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:54,790 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:54,792 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:01:54,865 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:01:55,036 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53076, bytes: 105774, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741895_1071, duration: 36457000 19:01:55,121 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53077, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741897_1073, duration: 33805000 19:01:55,147 INFO AppSchedulingInfo:166 - checking for deactivate... 19:01:55,311 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:01:55,434 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/filecache/14/hadoop-streaming.jar) transitioned from DOWNLOADING to LOCALIZED 19:01:55,435 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/nmPrivate/container_1394161202967_0005_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000003.tokens 19:01:55,436 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/filecache/15/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:01:55,436 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0005 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0005 19:01:55,436 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080 19:01:55,437 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081 19:01:55,819 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:55,820 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:55,821 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml dst=null perm=null 19:01:55,849 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53076, bytes: 80769, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_2127847636_2105, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741903_1079, duration: 381000 19:01:55,920 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0005/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:01:55,928 INFO Container:888 - Container container_1394161202967_0005_01_000003 transitioned from LOCALIZING to LOCALIZED 19:01:56,005 INFO Container:888 - Container container_1394161202967_0005_01_000003 transitioned from LOCALIZED to RUNNING 19:01:56,006 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:01:56,134 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741905_1081 19:01:56,149 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080 19:01:56,325 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000003/default_container_executor.sh] 19:02:00,147 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job.split dst=null perm=null 19:02:00,235 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53082, bytes: 175, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741901_1077, duration: 265000 19:02:00,318 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/input/dummy.txt dst=null perm=null 19:02:00,342 INFO FSEditLog:674 - Number of transactions: 626 Total time for transactions(ms): 243 Number of transactions batched in Syncs: 13 Number of syncs: 413 SyncTimes(ms): 288 105 19:02:00,343 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/_temporary/attempt_1394161202967_0005_m_000000_0/part-00000 dst=null perm=test:supergroup:rw-rw-rw- 19:02:00,369 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/0000000-140306190144221-oozie-rkan-W@mr-action dst=null perm=null 19:02:00,373 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/0000000-140306190144221-oozie-rkan-W@mr-action dst=null perm=test:supergroup:rw-rw-rw- 19:02:00,385 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/0000000-140306190144221-oozie-rkan-W@mr-action. BP-1007909283-172.16.1.64-1394161187028 blk_1073741906_1082{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:00,434 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082 src: /127.0.0.1:53083 dest: /127.0.0.1:52682 19:02:00,436 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082 src: /127.0.0.1:53084 dest: /127.0.0.1:52685 19:02:00,462 INFO clienttrace:1215 - src: /127.0.0.1:53084, dest: /127.0.0.1:52685, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082, duration: 24662000 19:02:00,462 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:00,463 INFO clienttrace:1215 - src: /127.0.0.1:53083, dest: /127.0.0.1:52682, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082, duration: 25311000 19:02:00,463 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:00,467 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/0000000-140306190144221-oozie-rkan-W@mr-action is closed by DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1 19:02:00,473 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53085, bytes: 9, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741900_1076, duration: 217000 19:02:00,476 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082 19:02:00,529 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action.xml dst=null perm=null 19:02:00,531 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action.xml dst=null perm=null 19:02:00,542 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53082, bytes: 78390, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741899_1075, duration: 334000 19:02:01,177 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741906_1082 19:02:01,188 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output dst=null perm=null 19:02:01,192 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:01,193 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:01,228 INFO ClientRMService:229 - Allocated new applicationId: 6 19:02:01,245 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006 dst=null perm=null 19:02:01,252 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006 dst=null perm=test:supergroup:rwxrwxrwx 19:02:01,262 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006 dst=null perm=test:supergroup:rwx------ 19:02:01,271 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:01,272 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:01,273 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:01,274 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:01,275 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:01,276 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:02:01,276 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:02:01,277 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:02:01,278 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:02:01,280 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:02:01,281 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:01,282 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:01,283 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib dst=null perm=null 19:02:01,284 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app dst=null perm=null 19:02:01,285 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9 dst=null perm=null 19:02:01,286 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:02:01,287 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:01,288 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar dst=null perm=null 19:02:01,303 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input dst=null perm=null 19:02:01,305 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input dst=null perm=null 19:02:01,309 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input dst=null perm=null 19:02:01,337 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:02:01,339 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=test:supergroup:rw-r--r-- 19:02:01,344 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split 19:02:01,345 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:01,353 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741907_1083{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:01,357 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083 src: /127.0.0.1:53087 dest: /127.0.0.1:52682 19:02:01,358 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083 src: /127.0.0.1:53088 dest: /127.0.0.1:52685 19:02:01,361 INFO clienttrace:1215 - src: /127.0.0.1:53088, dest: /127.0.0.1:52685, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083, duration: 1440000 19:02:01,362 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:01,367 INFO clienttrace:1215 - src: /127.0.0.1:53087, dest: /127.0.0.1:52682, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083, duration: 2079000 19:02:01,367 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:01,369 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split is closed by DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1 19:02:01,371 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:02:01,373 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:02:01,378 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741908_1084{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:01,380 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084 src: /127.0.0.1:53089 dest: /127.0.0.1:52682 19:02:01,382 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084 src: /127.0.0.1:53090 dest: /127.0.0.1:52685 19:02:01,384 INFO clienttrace:1215 - src: /127.0.0.1:53090, dest: /127.0.0.1:52685, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084, duration: 1237000 19:02:01,384 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:01,385 INFO clienttrace:1215 - src: /127.0.0.1:53089, dest: /127.0.0.1:52682, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084, duration: 1941000 19:02:01,385 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:01,387 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo is closed by DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1 19:02:01,401 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:02:01,403 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:02:01,437 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741909_1085{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:02:01,440 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085 src: /127.0.0.1:53091 dest: /127.0.0.1:52682 19:02:01,441 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085 src: /127.0.0.1:53092 dest: /127.0.0.1:52685 19:02:01,444 INFO clienttrace:1215 - src: /127.0.0.1:53092, dest: /127.0.0.1:52685, bytes: 79839, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085, duration: 1882000 19:02:01,445 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:01,445 INFO clienttrace:1215 - src: /127.0.0.1:53091, dest: /127.0.0.1:52682, bytes: 79839, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085, duration: 2636000 19:02:01,445 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:01,447 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml is closed by DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1 19:02:01,629 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006 dst=null perm=null 19:02:01,809 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:01,812 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:01,818 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:01,820 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:01,822 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=null 19:02:01,823 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=null 19:02:01,828 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:01,829 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:01,830 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:01,831 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:01,832 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:01,833 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:01,834 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:01,835 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:01,837 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:01,838 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:01,839 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:01,840 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:01,897 INFO ClientRMService:342 - Application with id 6 submitted by user test 19:02:01,897 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0006 19:02:01,897 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0006 19:02:01,897 INFO RMAppImpl:643 - application_1394161202967_0006 State change from NEW to NEW_SAVING 19:02:01,897 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0006 19:02:01,897 INFO RMAppImpl:643 - application_1394161202967_0006 State change from NEW_SAVING to SUBMITTED 19:02:01,897 INFO FairScheduler:638 - Accepted application application_1394161202967_0006 from user: test, in queue: default, currently num of applications: 2 19:02:01,898 INFO RMAppImpl:643 - application_1394161202967_0006 State change from SUBMITTED to ACCEPTED 19:02:01,898 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0006_000001 19:02:01,898 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from NEW to SUBMITTED 19:02:01,898 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0006_000001 to scheduler from user: test 19:02:01,898 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from SUBMITTED to SCHEDULED 19:02:02,175 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=test:supergroup:rw-rw-rw- 19:02:02,207 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq. BP-1007909283-172.16.1.64-1394161187028 blk_1073741910_1086{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:02,210 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086 src: /127.0.0.1:53094 dest: /127.0.0.1:52682 19:02:02,211 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086 src: /127.0.0.1:53095 dest: /127.0.0.1:52685 19:02:02,214 INFO clienttrace:1215 - src: /127.0.0.1:53095, dest: /127.0.0.1:52685, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, duration: 1321000 19:02:02,214 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:02,214 INFO clienttrace:1215 - src: /127.0.0.1:53094, dest: /127.0.0.1:52682, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, duration: 2023000 19:02:02,215 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:02,216 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq is closed by DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1 19:02:02,225 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/_temporary/attempt_1394161202967_0005_m_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0005_m_000000_0_-1214823883_1 19:02:02,245 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/_temporary/attempt_1394161202967_0005_m_000000_0 dst=null perm=null 19:02:02,251 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/_temporary/attempt_1394161202967_0005_m_000000_0 dst=null perm=null 19:02:02,252 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/task_1394161202967_0005_m_000000 dst=null perm=null 19:02:02,255 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000001 Container Transitioned from NEW to ALLOCATED 19:02:02,255 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000001 19:02:02,255 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0006_01_000001 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:02:02,256 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:02:02,256 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0006 AttemptId: appattempt_1394161202967_0006_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0006_01_000001, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] 19:02:02,256 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:02:02,256 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:02:02,257 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/_temporary/attempt_1394161202967_0005_m_000000_0 dst=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/task_1394161202967_0005_m_000000 perm=test:supergroup:rwxrwxrwx 19:02:02,257 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0006_000001 19:02:02,258 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0006_01_000001, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0006_000001 19:02:02,258 INFO AMLauncher:190 - Command to launch container container_1394161202967_0006_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:02:02,264 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0006_01_000001 by user test 19:02:02,264 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0006 19:02:02,264 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000001 19:02:02,265 INFO Application:450 - Application application_1394161202967_0006 transitioned from NEW to INITING 19:02:02,265 INFO Application:296 - Adding container_1394161202967_0006_01_000001 to application application_1394161202967_0006 19:02:02,265 INFO Application:450 - Application application_1394161202967_0006 transitioned from INITING to RUNNING 19:02:02,265 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0006_01_000001, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0006_000001 19:02:02,265 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from ALLOCATED to LAUNCHED 19:02:02,265 INFO Container:888 - Container container_1394161202967_0006_01_000001 transitioned from NEW to LOCALIZING 19:02:02,265 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0006 19:02:02,266 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:02:02,266 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split transitioned from INIT to DOWNLOADING 19:02:02,266 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml transitioned from INIT to DOWNLOADING 19:02:02,266 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0006_01_000001 19:02:02,302 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0005_01_000003 19:02:02,304 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000003 19:02:02,304 INFO Container:888 - Container container_1394161202967_0005_01_000003 transitioned from RUNNING to KILLING 19:02:02,304 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0005_01_000003 19:02:02,351 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:02:02,353 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/nmPrivate/container_1394161202967_0006_01_000001.tokens. Credentials list: 19:02:02,383 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0005_01_000003 is : 143 19:02:02,398 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741911_1087{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:02:02,398 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:02:02,440 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:02:02,467 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087 src: /127.0.0.1:53097 dest: /127.0.0.1:52682 19:02:02,468 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1 dst=null perm=null 19:02:02,498 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087 src: /127.0.0.1:53098 dest: /127.0.0.1:52685 19:02:02,499 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output dst=null perm=null 19:02:02,536 INFO DefaultContainerExecutor:469 - Initializing user test 19:02:02,537 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output dst=null perm=null 19:02:02,567 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/task_1394161202967_0005_m_000000 dst=null perm=null 19:02:02,612 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1.jhist for DFSClient_NONMAPREDUCE_1414878859_1 19:02:02,613 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/part-00000 dst=null perm=null 19:02:02,643 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary/1/task_1394161202967_0005_m_000000/part-00000 dst=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/part-00000 perm=test:supergroup:rw-rw-rw- 19:02:02,721 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_temporary dst=null perm=null 19:02:02,763 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:02:02,810 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:02:02,853 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:02:02,928 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:02:03,033 INFO Container:888 - Container container_1394161202967_0005_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:02:03,040 INFO clienttrace:1215 - src: /127.0.0.1:53098, dest: /127.0.0.1:52685, bytes: 20695, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087, duration: 492947000 19:02:03,047 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:03,076 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000003 19:02:03,078 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000003 19:02:03,079 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000003 19:02:03,079 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000003 19:02:03,080 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000003 19:02:03,080 INFO clienttrace:1215 - src: /127.0.0.1:53097, dest: /127.0.0.1:52682, bytes: 20695, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087, duration: 540188000 19:02:03,080 INFO Container:888 - Container container_1394161202967_0005_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:02:03,080 INFO Application:339 - Removing container_1394161202967_0005_01_000003 from application application_1394161202967_0005 19:02:03,088 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:03,104 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:03,106 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0005 19:02:03,109 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1.jhist is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:02:03,152 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:03,153 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/nmPrivate/container_1394161202967_0006_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000001.tokens 19:02:03,153 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0006 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0006 19:02:03,176 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741912_1088{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:03,204 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088 src: /127.0.0.1:53099 dest: /127.0.0.1:52685 19:02:03,287 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088 src: /127.0.0.1:53100 dest: /127.0.0.1:52682 19:02:03,322 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0005_01_000003 19:02:03,327 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000003 Container Transitioned from RUNNING to COMPLETED 19:02:03,328 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0005_01_000003 in state: COMPLETED event:FINISHED 19:02:03,328 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000003 19:02:03,328 INFO FSSchedulerNode:162 - Released container container_1394161202967_0005_01_000003 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:02:03,328 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0005_000001 released container container_1394161202967_0005_01_000003 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:02:03,359 INFO clienttrace:1215 - src: /127.0.0.1:53100, dest: /127.0.0.1:52682, bytes: 415, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088, duration: 6194000 19:02:03,360 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:03,361 INFO clienttrace:1215 - src: /127.0.0.1:53099, dest: /127.0.0.1:52685, bytes: 415, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088, duration: 38573000 19:02:03,368 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:03,420 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary_tmp is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:02:03,424 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:03,430 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1.jhist dst=null perm=null 19:02:03,432 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist_tmp dst=null perm=null 19:02:03,436 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1.jhist dst=null perm=null 19:02:03,437 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist_tmp dst=null perm=null 19:02:03,438 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1.jhist dst=null perm=null 19:02:03,441 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:03,444 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53101, bytes: 20859, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087, duration: 237000 19:02:03,446 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741913_1089{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:03,457 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089 src: /127.0.0.1:53103 dest: /127.0.0.1:52685 19:02:03,491 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089 src: /127.0.0.1:53104 dest: /127.0.0.1:52682 19:02:03,615 INFO clienttrace:1215 - src: /127.0.0.1:53104, dest: /127.0.0.1:52682, bytes: 20695, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089, duration: 79190000 19:02:03,616 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:03,617 INFO clienttrace:1215 - src: /127.0.0.1:53103, dest: /127.0.0.1:52685, bytes: 20695, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089, duration: 82260000 19:02:03,617 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:03,619 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=null 19:02:03,619 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist_tmp is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:02:03,621 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=null 19:02:03,621 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:03,622 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=null 19:02:03,622 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1_conf.xml dst=null perm=null 19:02:03,624 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml_tmp dst=null perm=null 19:02:03,625 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1_conf.xml dst=null perm=null 19:02:03,647 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml_tmp dst=null perm=null 19:02:03,663 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005/job_1394161202967_0005_1_conf.xml dst=null perm=null 19:02:03,665 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53107, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1796041452_2210, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084, duration: 317000 19:02:03,694 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:03,719 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53101, bytes: 93855, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741904_1080, duration: 1042000 19:02:03,747 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741914_1090{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:02:03,795 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090 src: /127.0.0.1:53109 dest: /127.0.0.1:52682 19:02:03,842 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090 src: /127.0.0.1:53110 dest: /127.0.0.1:52685 19:02:03,847 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0006/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:02:03,873 INFO clienttrace:1215 - src: /127.0.0.1:53110, dest: /127.0.0.1:52685, bytes: 93127, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090, duration: 6018000 19:02:03,884 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:03,916 INFO clienttrace:1215 - src: /127.0.0.1:53109, dest: /127.0.0.1:52682, bytes: 93127, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1414878859_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090, duration: 37397000 19:02:03,917 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:03,944 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_1414878859_1 19:02:03,990 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:03,991 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:03,992 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:03,993 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary perm=test:supergroup:rwxrwx--- 19:02:03,994 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:03,995 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml perm=test:supergroup:rwxrwx--- 19:02:04,003 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist perm=test:supergroup:rwxrwx--- 19:02:04,029 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53107, bytes: 283, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1796041452_2210, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083, duration: 301000 19:02:04,099 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0005_000001 with final state: FINISHING 19:02:04,114 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from RUNNING to FINAL_SAVING 19:02:04,114 INFO RMAppImpl:880 - Updating application application_1394161202967_0005 with final state: FINISHING 19:02:04,114 INFO RMAppImpl:643 - application_1394161202967_0005 State change from RUNNING to FINAL_SAVING 19:02:04,114 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0005 19:02:04,120 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from FINAL_SAVING to FINISHING 19:02:04,135 INFO RMAppImpl:643 - application_1394161202967_0005 State change from FINAL_SAVING to FINISHING 19:02:04,181 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0006/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:02:04,235 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:04,237 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:04,239 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:04,277 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53107, bytes: 80463, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1796041452_2210, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085, duration: 467000 19:02:04,388 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0006/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:02:04,399 INFO Container:888 - Container container_1394161202967_0006_01_000001 transitioned from LOCALIZING to LOCALIZED 19:02:04,516 INFO Container:888 - Container container_1394161202967_0006_01_000001 transitioned from LOCALIZED to RUNNING 19:02:04,516 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:04,770 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000001/default_container_executor.sh] 19:02:05,107 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0005 dst=null perm=null 19:02:05,480 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090 19:02:05,480 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084 19:02:05,481 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083 19:02:05,481 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085 19:02:05,482 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087 19:02:05,482 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086 19:02:05,483 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089 19:02:05,483 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088 19:02:05,525 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:05,529 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:05,531 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:05,534 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53114, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, duration: 279000 19:02:05,579 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161325579, maxDate=1394766125579, sequenceNumber=16, masterKeyId=2 19:02:05,580 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 16 19:02:05,590 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:02:05,592 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:02:05,594 INFO CompletedJob:95 - Loading job: job_1394161202967_0005 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist 19:02:05,595 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist] 19:02:05,595 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary dst=null perm=null 19:02:05,596 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist dst=null perm=null 19:02:05,597 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist dst=null perm=null 19:02:05,598 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53107, bytes: 423, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741888_1064, duration: 253000 19:02:05,598 INFO JobSummary:372 - jobId=job_1394161202967_0004,submitTime=1394161275666,launchTime=1394161282567,firstMapTaskLaunchTime=1394161284778,firstReduceTaskLaunchTime=0,finishTime=1394161301799,resourcesPerMap=1024,resourcesPerReduce=1024,numMaps=2,numReduces=1,user=test,queue=default,status=FAILED,mapSlotSeconds=9,reduceSlotSeconds=0,jobName=oozie:action:T\=map-reduce:W\=test-wf-cred:A\=map-reduce:ID\=0000000-140306190100960-oozie-rkan-W 19:02:05,598 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary] 19:02:05,599 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53114, bytes: 20859, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089, duration: 252000 19:02:05,599 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004.summary dst=null perm=null 19:02:05,601 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:02:05,601 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist 19:02:05,603 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0004-1394161275666-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3Dtest%2Dwf%2Dcred%3AA%3Dmap%2Dred-1394161301799-2-0-FAILED-root.test-1394161282567.jhist perm=test:supergroup:rwxrwx--- 19:02:05,604 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0004_conf.xml 19:02:05,605 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0004_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0004_conf.xml perm=test:supergroup:rwxrwx--- 19:02:05,617 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary dst=null perm=null 19:02:05,619 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53107, bytes: 419, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088, duration: 187000 19:02:05,619 INFO JobSummary:372 - jobId=job_1394161202967_0005,submitTime=1394161304874,launchTime=1394161311997,firstMapTaskLaunchTime=1394161314188,firstReduceTaskLaunchTime=0,finishTime=1394161322947,resourcesPerMap=1024,resourcesPerReduce=0,numMaps=1,numReduces=0,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=8,reduceSlotSeconds=0,jobName=oozie:launcher:T\=map-reduce:W\=testApp:A\=streaming:ID\=0000000-140306190144221-oozie-rkan-W 19:02:05,619 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary] 19:02:05,620 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005.summary dst=null perm=null 19:02:05,624 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:02:05,624 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist 19:02:05,626 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0005-1394161304874-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3A-1394161322947-1-0-SUCCEEDED-root.test-1394161311997.jhist perm=test:supergroup:rwxrwx--- 19:02:05,626 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0005_conf.xml 19:02:05,627 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0005_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0005_conf.xml perm=test:supergroup:rwxrwx--- 19:02:05,647 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:05,648 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:05,649 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:05,651 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53114, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, duration: 183000 19:02:05,707 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161325707, maxDate=1394766125707, sequenceNumber=17, masterKeyId=2 19:02:05,708 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 17 19:02:06,180 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086 19:02:06,180 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741912_1088 19:02:06,181 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741911_1087 19:02:06,181 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741913_1089 19:02:06,182 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741914_1090 19:02:06,183 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085 19:02:06,183 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084 19:02:06,183 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083 19:02:06,614 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741888_1064, blk_1073741904_1080, blk_1073741905_1081, blk_1073741911_1087, blk_1073741912_1088, blk_1073741901_1077, blk_1073741902_1078, blk_1073741903_1079] 19:02:07,267 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:07,319 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/COMMIT_STARTED dst=null perm=null 19:02:07,320 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/COMMIT_SUCCESS dst=null perm=null 19:02:07,321 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/COMMIT_FAIL dst=null perm=null 19:02:07,404 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006 dst=null perm=null 19:02:07,406 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:02:07,407 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:02:07,860 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=null 19:02:07,875 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.splitmetainfo dst=null perm=null 19:02:07,981 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53119, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741908_1084, duration: 268000 19:02:07,992 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741904_1080 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741904 for deletion 19:02:07,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741888_1064 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741888 for deletion 19:02:07,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741905_1081 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741905 for deletion 19:02:07,993 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741904_1080 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741904 19:02:07,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741911_1087 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741911 for deletion 19:02:07,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741912_1088 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741912 for deletion 19:02:07,993 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741888_1064 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741888 19:02:07,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741901_1077 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741901 for deletion 19:02:07,993 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741905_1081 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741905 19:02:07,994 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741902_1078 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741902 for deletion 19:02:07,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741912_1088 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741912 19:02:07,994 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741903_1079 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741903 for deletion 19:02:07,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741911_1087 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741911 19:02:07,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741902_1078 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741902 19:02:07,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741901_1077 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741901 19:02:07,995 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741903_1079 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741903 19:02:09,174 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0006_000001 19:02:09,174 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0006 APPATTEMPTID=appattempt_1394161202967_0006_000001 19:02:09,175 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from LAUNCHED to RUNNING 19:02:09,175 INFO RMAppImpl:643 - application_1394161202967_0006 State change from ACCEPTED to RUNNING 19:02:09,202 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:02:09,205 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1 dst=null perm=test:testg:rwxrwxrwx 19:02:09,269 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:02:09,475 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741915_1091{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:09,529 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091 src: /127.0.0.1:53124 dest: /127.0.0.1:52685 19:02:09,530 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091 src: /127.0.0.1:53125 dest: /127.0.0.1:52682 19:02:09,570 INFO clienttrace:1215 - src: /127.0.0.1:53125, dest: /127.0.0.1:52682, bytes: 94098, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091, duration: 38532000 19:02:09,571 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:09,572 INFO clienttrace:1215 - src: /127.0.0.1:53124, dest: /127.0.0.1:52685, bytes: 94098, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091, duration: 40245000 19:02:09,572 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:09,576 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1_conf.xml is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:09,615 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741888_1064, blk_1073741904_1080, blk_1073741905_1081, blk_1073741911_1087, blk_1073741912_1088, blk_1073741901_1077, blk_1073741902_1078, blk_1073741903_1079] 19:02:09,627 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006.summary dst=null perm=test:supergroup:rw-rw-rw- 19:02:09,630 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741916_1092{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:02:09,632 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092 src: /127.0.0.1:53126 dest: /127.0.0.1:52682 19:02:09,634 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092 src: /127.0.0.1:53127 dest: /127.0.0.1:52685 19:02:09,637 INFO clienttrace:1215 - src: /127.0.0.1:53127, dest: /127.0.0.1:52685, bytes: 369, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092, duration: 1673000 19:02:09,637 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:09,638 INFO clienttrace:1215 - src: /127.0.0.1:53126, dest: /127.0.0.1:52682, bytes: 369, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092, duration: 2572000 19:02:09,638 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:09,640 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006.summary is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:10,338 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000002 Container Transitioned from NEW to ALLOCATED 19:02:10,338 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000002 19:02:10,338 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0006_01_000002 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:02:10,488 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092 19:02:10,489 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091 19:02:10,804 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000003 Container Transitioned from NEW to ALLOCATED 19:02:10,804 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000003 19:02:10,805 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0006_01_000003 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:02:10,992 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741904_1080 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741904 for deletion 19:02:10,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741888_1064 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741888 for deletion 19:02:10,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741905_1081 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741905 for deletion 19:02:10,993 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741904_1080 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741904 19:02:10,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741911_1087 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741911 for deletion 19:02:10,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741912_1088 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741912 for deletion 19:02:10,993 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741888_1064 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741888 19:02:10,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741901_1077 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741901 for deletion 19:02:10,993 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741905_1081 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741905 19:02:10,993 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741912_1088 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741912 19:02:10,993 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741902_1078 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741902 for deletion 19:02:10,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741911_1087 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741911 19:02:10,994 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741903_1079 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741903 for deletion 19:02:10,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741902_1078 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741902 19:02:10,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741901_1077 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741901 19:02:10,994 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741903_1079 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741903 19:02:11,189 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091 19:02:11,190 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741916_1092 19:02:11,237 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:02:11,237 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:02:11,254 INFO ContainerLaunch:322 - Container container_1394161202967_0005_01_000002 succeeded 19:02:11,254 INFO Container:888 - Container container_1394161202967_0005_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:02:11,254 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0005_01_000002 19:02:11,269 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000002 19:02:11,269 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000002 19:02:11,269 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000002 19:02:11,269 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0005/container_1394161202967_0005_01_000002 19:02:11,270 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000002 19:02:11,270 INFO Container:888 - Container container_1394161202967_0005_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 19:02:11,270 INFO Application:339 - Removing container_1394161202967_0005_01_000002 from application application_1394161202967_0005 19:02:11,270 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:11,270 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0005 19:02:11,296 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:11,297 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:11,310 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:11,312 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:11,313 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:11,314 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:11,323 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:11,325 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:11,326 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:11,327 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:11,328 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:11,329 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:11,329 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/app/lib/test.jar dst=null perm=null 19:02:11,330 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/jar/hadoop-streaming.jar dst=null perm=null 19:02:11,399 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0006_01_000003 by user test 19:02:11,400 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0006_01_000002 by user test 19:02:11,400 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0006 19:02:11,400 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000003 19:02:11,400 INFO Application:296 - Adding container_1394161202967_0006_01_000002 to application application_1394161202967_0006 19:02:11,400 INFO Application:450 - Application application_1394161202967_0006 transitioned from NEW to INITING 19:02:11,400 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000002 19:02:11,400 INFO Application:296 - Adding container_1394161202967_0006_01_000003 to application application_1394161202967_0006 19:02:11,401 INFO Container:888 - Container container_1394161202967_0006_01_000002 transitioned from NEW to LOCALIZING 19:02:11,401 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0006 19:02:11,401 INFO Application:450 - Application application_1394161202967_0006 transitioned from INITING to RUNNING 19:02:11,401 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0006 19:02:11,401 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:02:11,401 INFO ShuffleHandler:263 - Added token for job_1394161202967_0006 19:02:11,401 INFO Container:888 - Container container_1394161202967_0006_01_000003 transitioned from NEW to LOCALIZING 19:02:11,401 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0006 19:02:11,401 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0006 19:02:11,401 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:02:11,402 INFO ShuffleHandler:263 - Added token for job_1394161202967_0006 19:02:11,402 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml transitioned from INIT to DOWNLOADING 19:02:11,402 INFO Container:888 - Container container_1394161202967_0006_01_000002 transitioned from LOCALIZING to LOCALIZED 19:02:11,402 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0006_01_000003 19:02:11,464 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/nmPrivate/container_1394161202967_0006_01_000003.tokens. Credentials list: 19:02:11,607 INFO DefaultContainerExecutor:469 - Initializing user test 19:02:11,817 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0005_01_000002 19:02:11,856 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:02:11,866 INFO RMContainerImpl:220 - container_1394161202967_0005_01_000002 Container Transitioned from RUNNING to COMPLETED 19:02:11,866 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0005_01_000002 in state: COMPLETED event:FINISHED 19:02:11,866 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0005_000001 19:02:11,866 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0005 CONTAINERID=container_1394161202967_0005_01_000002 19:02:11,867 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0005_000001 State change from FINISHING to FINISHED 19:02:11,867 INFO FSSchedulerNode:162 - Released container container_1394161202967_0005_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:02:11,867 INFO RMAppImpl:643 - application_1394161202967_0005 State change from FINISHING to FINISHED 19:02:11,867 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0005_000001 released container container_1394161202967_0005_01_000002 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:02:11,868 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0005 19:02:11,875 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0005_000001 19:02:11,875 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0005,name=oozie:launcher:T\=map-reduce:W\=testApp:A\=streaming:ID\=0000000-140306190144221-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0005/jobhistory/job/job_1394161202967_0005,appMasterHost=rkanter-mbp.local,startTime=1394161304874,finishTime=1394161324114,finalStatus=SUCCEEDED 19:02:11,901 INFO FairScheduler:720 - Application appattempt_1394161202967_0005_000001 is done. finalState=FINISHED 19:02:11,911 INFO AppSchedulingInfo:108 - Application application_1394161202967_0005 requests cleared 19:02:12,031 INFO Container:888 - Container container_1394161202967_0006_01_000002 transitioned from LOCALIZED to RUNNING 19:02:12,031 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:12,127 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0005_01_000002 19:02:12,276 INFO AppSchedulingInfo:166 - checking for deactivate... 19:02:12,312 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/nmPrivate/container_1394161202967_0006_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000003.tokens 19:02:12,342 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006 19:02:12,362 INFO Application:450 - Application application_1394161202967_0005 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:02:12,362 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:02:12,362 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0005 19:02:12,398 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0005 19:02:12,386 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0005 19:02:12,399 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0005 19:02:12,399 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0005 19:02:12,436 INFO Application:450 - Application application_1394161202967_0005 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:02:12,446 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0005, with delay of 10800 seconds 19:02:12,730 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000002/default_container_executor.sh] 19:02:12,843 INFO Application:450 - Application application_1394161202967_0005 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:02:12,849 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0005 19:02:12,849 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0005 19:02:12,849 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0005 19:02:12,865 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0005 19:02:12,879 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0005 19:02:12,884 INFO Application:450 - Application application_1394161202967_0005 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:02:12,885 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0005, with delay of 10800 seconds 19:02:12,931 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:12,932 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:12,933 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml dst=null perm=null 19:02:12,960 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53167, bytes: 80463, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_549253801_2348, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741909_1085, duration: 411000 19:02:13,081 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:02:13,082 INFO Container:888 - Container container_1394161202967_0006_01_000003 transitioned from LOCALIZING to LOCALIZED 19:02:13,232 INFO Container:888 - Container container_1394161202967_0006_01_000003 transitioned from LOCALIZED to RUNNING 19:02:13,233 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:13,678 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000003/default_container_executor.sh] 19:02:16,355 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:16,466 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53196, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0006_m_000000_0_1690681588_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083, duration: 325000 19:02:16,533 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input/data.txt dst=null perm=null 19:02:16,764 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53197, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0006_m_000000_0_1690681588_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072, duration: 659000 19:02:16,792 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_m_000000_0 dst=null perm=null 19:02:16,816 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0006_01_000002 19:02:16,817 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000002 19:02:16,817 INFO Container:888 - Container container_1394161202967_0006_01_000002 transitioned from RUNNING to KILLING 19:02:16,817 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0006_01_000002 19:02:16,886 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0006_01_000002 is : 143 19:02:16,887 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741917_1093{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:16,894 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093 src: /127.0.0.1:53198 dest: /127.0.0.1:52682 19:02:16,896 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093 src: /127.0.0.1:53199 dest: /127.0.0.1:52685 19:02:16,901 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1.jhist for DFSClient_NONMAPREDUCE_-712348397_1 19:02:16,913 INFO Container:888 - Container container_1394161202967_0006_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:02:16,914 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000002 19:02:16,914 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000002 19:02:16,914 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000002 19:02:16,915 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000002 19:02:16,915 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000002 19:02:16,915 INFO Container:888 - Container container_1394161202967_0006_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:02:16,915 INFO Application:339 - Removing container_1394161202967_0006_01_000002 from application application_1394161202967_0006 19:02:16,915 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:16,915 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0006 19:02:17,118 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job.split dst=null perm=null 19:02:17,207 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53201, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0006_m_000001_0_-1657692616_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741907_1083, duration: 299000 19:02:17,263 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/input/data.txt dst=null perm=null 19:02:17,267 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53201, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0006_m_000001_0_-1657692616_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741896_1072, duration: 583000 19:02:17,414 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_m_000001_0 dst=null perm=null 19:02:17,427 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0006_01_000003 19:02:17,427 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000003 19:02:17,427 INFO Container:888 - Container container_1394161202967_0006_01_000003 transitioned from RUNNING to KILLING 19:02:17,427 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0006_01_000003 19:02:17,494 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0006_01_000003 is : 143 19:02:17,510 INFO Container:888 - Container container_1394161202967_0006_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:02:17,511 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000003 19:02:17,511 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000003 19:02:17,511 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000003 19:02:17,512 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000003 19:02:17,512 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000003 19:02:17,512 INFO Container:888 - Container container_1394161202967_0006_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:02:17,512 INFO Application:339 - Removing container_1394161202967_0006_01_000003 from application application_1394161202967_0006 19:02:17,512 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:17,512 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0006 19:02:17,818 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0006_01_000002 19:02:17,819 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000002 Container Transitioned from RUNNING to COMPLETED 19:02:17,819 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0006_01_000002 in state: COMPLETED event:FINISHED 19:02:17,819 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000002 19:02:17,820 INFO FSSchedulerNode:162 - Released container container_1394161202967_0006_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:02:17,820 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0006_000001 released container container_1394161202967_0006_01_000002 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:02:18,428 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0006_01_000003 19:02:18,429 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000003 Container Transitioned from RUNNING to COMPLETED 19:02:18,429 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0006_01_000003 in state: COMPLETED event:FINISHED 19:02:18,429 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000003 19:02:18,429 INFO FSSchedulerNode:162 - Released container container_1394161202967_0006_01_000003 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:02:18,429 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0006_000001 released container container_1394161202967_0006_01_000003 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:02:18,429 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000004 Container Transitioned from NEW to ALLOCATED 19:02:18,429 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000004 19:02:18,429 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0006_01_000004 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:02:19,429 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 19:02:19,443 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0006_01_000004 by user test 19:02:19,443 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000004 19:02:19,443 INFO Application:296 - Adding container_1394161202967_0006_01_000004 to application application_1394161202967_0006 19:02:19,443 INFO Container:888 - Container container_1394161202967_0006_01_000004 transitioned from NEW to LOCALIZING 19:02:19,444 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0006 19:02:19,444 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0006 19:02:19,444 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:02:19,444 INFO ShuffleHandler:263 - Added token for job_1394161202967_0006 19:02:19,444 INFO Container:888 - Container container_1394161202967_0006_01_000004 transitioned from LOCALIZING to LOCALIZED 19:02:19,530 INFO Container:888 - Container container_1394161202967_0006_01_000004 transitioned from LOCALIZED to RUNNING 19:02:19,538 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:19,810 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000004/default_container_executor.sh] 19:02:20,430 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000004 Container Transitioned from ACQUIRED to RUNNING 19:02:20,435 INFO AppSchedulingInfo:166 - checking for deactivate... 19:02:23,436 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_r_000000_0/part-00000 dst=null perm=test:testg:rw-rw-rw- 19:02:23,486 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_r_000000_0/part-00000. BP-1007909283-172.16.1.64-1394161187028 blk_1073741918_1094{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:02:23,595 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741918_1094 src: /127.0.0.1:53251 dest: /127.0.0.1:52685 19:02:23,597 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741918_1094 src: /127.0.0.1:53252 dest: /127.0.0.1:52682 19:02:23,633 INFO clienttrace:1215 - src: /127.0.0.1:53252, dest: /127.0.0.1:52682, bytes: 26, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0006_r_000000_0_-1273181067_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741918_1094, duration: 34446000 19:02:23,633 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741918_1094, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:23,634 INFO clienttrace:1215 - src: /127.0.0.1:53251, dest: /127.0.0.1:52685, bytes: 26, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0006_r_000000_0_-1273181067_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741918_1094, duration: 35747000 19:02:23,635 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741918_1094, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:23,639 INFO StateChange:2959 - DIR* completeFile: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_r_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0006_r_000000_0_-1273181067_1 19:02:23,667 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_r_000000_0 dst=null perm=null 19:02:23,674 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_r_000000_0 dst=null perm=null 19:02:23,675 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/task_1394161202967_0006_r_000000 dst=null perm=null 19:02:23,678 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/_temporary/attempt_1394161202967_0006_r_000000_0 dst=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/task_1394161202967_0006_r_000000 perm=test:testg:rwxrwxrwx 19:02:23,690 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0006_01_000004 19:02:23,690 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000004 19:02:23,690 INFO Container:888 - Container container_1394161202967_0006_01_000004 transitioned from RUNNING to KILLING 19:02:23,690 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0006_01_000004 19:02:23,757 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0006_01_000004 is : 143 19:02:23,768 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:02:23,771 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:23,774 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1 dst=null perm=null 19:02:23,780 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output dst=null perm=null 19:02:23,782 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output dst=null perm=null 19:02:23,783 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/task_1394161202967_0006_r_000000 dst=null perm=null 19:02:23,783 INFO Container:888 - Container container_1394161202967_0006_01_000004 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:02:23,784 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000004 19:02:23,784 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000004 19:02:23,784 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000004 19:02:23,784 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000004 19:02:23,784 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000004 19:02:23,784 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/part-00000 dst=null perm=null 19:02:23,785 INFO Container:888 - Container container_1394161202967_0006_01_000004 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:02:23,785 INFO Application:339 - Removing container_1394161202967_0006_01_000004 from application application_1394161202967_0006 19:02:23,785 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:23,785 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0006 19:02:23,788 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary/1/task_1394161202967_0006_r_000000/part-00000 dst=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/part-00000 perm=test:testg:rw-rw-rw- 19:02:23,793 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_temporary dst=null perm=null 19:02:23,798 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_SUCCESS dst=null perm=test:testg:rw-rw-rw- 19:02:23,799 INFO StateChange:2959 - DIR* completeFile: /user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:23,801 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:02:23,803 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:23,822 INFO clienttrace:1215 - src: /127.0.0.1:53199, dest: /127.0.0.1:52685, bytes: 38905, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093, duration: 6924010000 19:02:23,822 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:23,824 INFO clienttrace:1215 - src: /127.0.0.1:53198, dest: /127.0.0.1:52682, bytes: 38905, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093, duration: 6925473000 19:02:23,824 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:23,825 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1.jhist is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:23,827 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:23,829 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741919_1095{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:23,831 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095 src: /127.0.0.1:53253 dest: /127.0.0.1:52685 19:02:23,833 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095 src: /127.0.0.1:53254 dest: /127.0.0.1:52682 19:02:23,836 INFO clienttrace:1215 - src: /127.0.0.1:53254, dest: /127.0.0.1:52682, bytes: 429, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095, duration: 1272000 19:02:23,836 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:23,837 INFO clienttrace:1215 - src: /127.0.0.1:53253, dest: /127.0.0.1:52685, bytes: 429, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095, duration: 1887000 19:02:23,837 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:23,838 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary_tmp is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:23,842 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:23,847 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1.jhist dst=null perm=null 19:02:23,848 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist_tmp dst=null perm=null 19:02:23,851 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1.jhist dst=null perm=null 19:02:23,852 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist_tmp dst=null perm=null 19:02:23,853 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1.jhist dst=null perm=null 19:02:23,856 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:23,858 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53255, bytes: 39209, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093, duration: 228000 19:02:23,859 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741920_1096{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:02:23,862 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096 src: /127.0.0.1:53256 dest: /127.0.0.1:52682 19:02:23,863 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096 src: /127.0.0.1:53257 dest: /127.0.0.1:52685 19:02:23,866 INFO clienttrace:1215 - src: /127.0.0.1:53257, dest: /127.0.0.1:52685, bytes: 38905, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096, duration: 1652000 19:02:23,866 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:23,866 INFO clienttrace:1215 - src: /127.0.0.1:53256, dest: /127.0.0.1:52682, bytes: 38905, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096, duration: 2271000 19:02:23,867 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:23,869 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist_tmp is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:23,870 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:23,871 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1_conf.xml dst=null perm=null 19:02:23,872 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml_tmp dst=null perm=null 19:02:23,873 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1_conf.xml dst=null perm=null 19:02:23,874 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml_tmp dst=null perm=null 19:02:23,875 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006/job_1394161202967_0006_1_conf.xml dst=null perm=null 19:02:23,877 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:23,879 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53255, bytes: 94834, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741915_1091, duration: 321000 19:02:23,881 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741921_1097{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:23,883 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741921_1097 src: /127.0.0.1:53258 dest: /127.0.0.1:52685 19:02:23,884 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741921_1097 src: /127.0.0.1:53259 dest: /127.0.0.1:52682 19:02:23,887 INFO clienttrace:1215 - src: /127.0.0.1:53259, dest: /127.0.0.1:52682, bytes: 94098, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741921_1097, duration: 1765000 19:02:23,887 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741921_1097, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:23,888 INFO clienttrace:1215 - src: /127.0.0.1:53258, dest: /127.0.0.1:52685, bytes: 94098, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-712348397_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741921_1097, duration: 2352000 19:02:23,888 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741921_1097, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:23,890 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_-712348397_1 19:02:23,891 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:23,893 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary perm=test:supergroup:rwxrwx--- 19:02:23,895 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml perm=test:supergroup:rwxrwx--- 19:02:23,897 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist perm=test:supergroup:rwxrwx--- 19:02:23,904 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0006_000001 with final state: FINISHING 19:02:23,904 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from RUNNING to FINAL_SAVING 19:02:23,905 INFO RMAppImpl:880 - Updating application application_1394161202967_0006 with final state: FINISHING 19:02:23,905 INFO RMAppImpl:643 - application_1394161202967_0006 State change from RUNNING to FINAL_SAVING 19:02:23,905 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0006 19:02:23,906 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from FINAL_SAVING to FINISHING 19:02:23,906 INFO RMAppImpl:643 - application_1394161202967_0006 State change from FINAL_SAVING to FINISHING 19:02:24,691 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0006_01_000004 19:02:24,692 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000004 Container Transitioned from RUNNING to COMPLETED 19:02:24,692 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0006_01_000004 in state: COMPLETED event:FINISHED 19:02:24,692 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000004 19:02:24,692 INFO FSSchedulerNode:162 - Released container container_1394161202967_0006_01_000004 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:02:24,693 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0006_000001 released container container_1394161202967_0006_01_000004 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:02:24,913 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0006 dst=null perm=null 19:02:25,012 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161345012, maxDate=1394766145012, sequenceNumber=18, masterKeyId=2 19:02:25,012 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 18 19:02:25,018 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:02:25,019 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:02:25,020 INFO CompletedJob:95 - Loading job: job_1394161202967_0006 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist 19:02:25,020 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist] 19:02:25,021 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist dst=null perm=null 19:02:25,022 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist dst=null perm=null 19:02:25,025 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53269, bytes: 39209, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096, duration: 218000 19:02:25,054 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary dst=null perm=null 19:02:25,056 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53270, bytes: 433, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095, duration: 259000 19:02:25,056 INFO JobSummary:372 - jobId=job_1394161202967_0006,submitTime=1394161321896,launchTime=1394161329193,firstMapTaskLaunchTime=1394161331409,firstReduceTaskLaunchTime=1394161339444,finishTime=1394161343804,resourcesPerMap=1024,resourcesPerReduce=1024,numMaps=2,numReduces=1,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=11,reduceSlotSeconds=4,jobName=oozie:action:T\=map-reduce:W\=testApp:A\=streaming:ID\=0000000-140306190144221-oozie-rkan-W 19:02:25,057 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary] 19:02:25,058 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006.summary dst=null perm=null 19:02:25,060 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:02:25,061 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist 19:02:25,062 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0006-1394161321896-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dstreaming%3AID-1394161343804-2-1-SUCCEEDED-root.test-1394161329193.jhist perm=test:supergroup:rwxrwx--- 19:02:25,062 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0006_conf.xml 19:02:25,064 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0006_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0006_conf.xml perm=test:supergroup:rwxrwx--- 19:02:25,085 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:25,088 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:25,089 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:25,091 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53270, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, duration: 455000 19:02:25,110 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:25,111 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:25,112 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java/action-data.seq dst=null perm=null 19:02:25,114 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53270, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_943124610_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741910_1086, duration: 224000 19:02:25,127 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java dst=null perm=null 19:02:25,129 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190144221-oozie-rkan-W/streaming--java dst=null perm=null 19:02:25,164 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161345164, maxDate=1394766145164, sequenceNumber=19, masterKeyId=2 19:02:25,164 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 19 19:02:25,493 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741918_1094 19:02:25,496 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741921_1097 19:02:25,496 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741917_1093 19:02:25,497 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741920_1096 19:02:25,497 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741919_1095 19:02:26,596 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:02:26,601 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741922_1098{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:26,603 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098 src: /127.0.0.1:53275 dest: /127.0.0.1:52685 19:02:26,605 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098 src: /127.0.0.1:53276 dest: /127.0.0.1:52682 19:02:26,608 INFO clienttrace:1215 - src: /127.0.0.1:53276, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178944540_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098, duration: 1752000 19:02:26,608 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,609 INFO clienttrace:1215 - src: /127.0.0.1:53275, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178944540_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098, duration: 2425000 19:02:26,609 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,612 INFO StateChange:2959 - DIR* completeFile: /user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input/data.txt is closed by DFSClient_NONMAPREDUCE_1178944540_8 19:02:26,621 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:02:26,623 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741923_1099{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:02:26,625 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099 src: /127.0.0.1:53277 dest: /127.0.0.1:52685 19:02:26,626 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099 src: /127.0.0.1:53278 dest: /127.0.0.1:52682 19:02:26,628 INFO clienttrace:1215 - src: /127.0.0.1:53278, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178944540_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099, duration: 1276000 19:02:26,628 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,629 INFO clienttrace:1215 - src: /127.0.0.1:53277, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178944540_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099, duration: 1694000 19:02:26,629 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,633 INFO StateChange:2959 - DIR* completeFile: /user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_1178944540_8 19:02:26,634 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:02:26,636 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741924_1100{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:26,637 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100 src: /127.0.0.1:53279 dest: /127.0.0.1:52682 19:02:26,639 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100 src: /127.0.0.1:53280 dest: /127.0.0.1:52685 19:02:26,641 INFO clienttrace:1215 - src: /127.0.0.1:53280, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178944540_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100, duration: 946000 19:02:26,641 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,641 INFO clienttrace:1215 - src: /127.0.0.1:53279, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178944540_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100, duration: 1555000 19:02:26,641 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,643 INFO StateChange:2959 - DIR* completeFile: /user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_1178944540_8 19:02:26,647 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java dst=null perm=null 19:02:26,649 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:02:26,650 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java.tmp dst=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java perm=test:supergroup:rwxrwxrwx 19:02:26,651 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/workflow.xml dst=null perm=null 19:02:26,688 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/workflow.xml dst=null perm=null 19:02:26,723 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java dst=null perm=test:supergroup:rwxrwxrwx 19:02:26,725 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:02:26,733 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741925_1101{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:26,735 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101 src: /127.0.0.1:53282 dest: /127.0.0.1:52682 19:02:26,736 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101 src: /127.0.0.1:53283 dest: /127.0.0.1:52685 19:02:26,739 INFO clienttrace:1215 - src: /127.0.0.1:53283, dest: /127.0.0.1:52685, bytes: 77267, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101, duration: 1448000 19:02:26,739 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,740 INFO clienttrace:1215 - src: /127.0.0.1:53282, dest: /127.0.0.1:52682, bytes: 77267, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101, duration: 2167000 19:02:26,740 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,742 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action.xml is closed by DFSClient_NONMAPREDUCE_512660409_8 19:02:26,743 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input dst=null perm=test:supergroup:rwxrwxrwx 19:02:26,745 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:02:26,747 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741926_1102{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:02:26,748 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102 src: /127.0.0.1:53284 dest: /127.0.0.1:52685 19:02:26,749 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102 src: /127.0.0.1:53285 dest: /127.0.0.1:52682 19:02:26,751 INFO clienttrace:1215 - src: /127.0.0.1:53285, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102, duration: 1036000 19:02:26,752 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,752 INFO clienttrace:1215 - src: /127.0.0.1:53284, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102, duration: 1560000 19:02:26,752 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,754 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_512660409_8 19:02:26,770 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161346770, maxDate=1394766146770, sequenceNumber=20, masterKeyId=2 19:02:26,770 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 20 19:02:26,771 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/0000000-140306190226247-oozie-rkan-W@mr-action dst=null perm=null 19:02:26,772 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161346772, maxDate=1394766146772, sequenceNumber=21, masterKeyId=2 19:02:26,772 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 21 19:02:26,787 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:02:26,787 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:26,788 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:26,789 INFO ClientRMService:229 - Allocated new applicationId: 7 19:02:26,790 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007 dst=null perm=null 19:02:26,792 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007 dst=null perm=test:supergroup:rwxrwxrwx 19:02:26,793 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007 dst=null perm=test:supergroup:rwx------ 19:02:26,794 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:26,795 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:26,796 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:26,797 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:26,798 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:02:26,798 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:02:26,799 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:02:26,800 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:02:26,800 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:02:26,801 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:26,802 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:26,803 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib dst=null perm=null 19:02:26,803 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app dst=null perm=null 19:02:26,804 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f dst=null perm=null 19:02:26,804 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:02:26,806 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:02:26,807 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:02:26,808 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:02:26,810 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:02:26,812 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=test:supergroup:rw-r--r-- 19:02:26,812 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split 19:02:26,813 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=null 19:02:26,814 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741927_1103{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:02:26,816 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103 src: /127.0.0.1:53287 dest: /127.0.0.1:52682 19:02:26,817 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103 src: /127.0.0.1:53288 dest: /127.0.0.1:52685 19:02:26,819 INFO clienttrace:1215 - src: /127.0.0.1:53288, dest: /127.0.0.1:52685, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103, duration: 1051000 19:02:26,820 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,820 INFO clienttrace:1215 - src: /127.0.0.1:53287, dest: /127.0.0.1:52682, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103, duration: 1704000 19:02:26,820 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,822 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split is closed by DFSClient_NONMAPREDUCE_512660409_8 19:02:26,825 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:02:26,826 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:02:26,827 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741928_1104{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:02:26,829 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104 src: /127.0.0.1:53289 dest: /127.0.0.1:52685 19:02:26,830 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104 src: /127.0.0.1:53290 dest: /127.0.0.1:52682 19:02:26,832 INFO clienttrace:1215 - src: /127.0.0.1:53290, dest: /127.0.0.1:52682, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104, duration: 995000 19:02:26,832 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,833 INFO clienttrace:1215 - src: /127.0.0.1:53289, dest: /127.0.0.1:52685, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104, duration: 1501000 19:02:26,833 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,835 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_512660409_8 19:02:26,836 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:02:26,838 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:02:26,844 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741929_1105{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:02:26,846 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105 src: /127.0.0.1:53291 dest: /127.0.0.1:52685 19:02:26,847 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105 src: /127.0.0.1:53292 dest: /127.0.0.1:52682 19:02:26,850 INFO clienttrace:1215 - src: /127.0.0.1:53292, dest: /127.0.0.1:52682, bytes: 79931, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105, duration: 1440000 19:02:26,850 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:26,850 INFO clienttrace:1215 - src: /127.0.0.1:53291, dest: /127.0.0.1:52685, bytes: 79931, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105, duration: 2012000 19:02:26,851 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:26,852 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml is closed by DFSClient_NONMAPREDUCE_512660409_8 19:02:26,854 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007 dst=null perm=null 19:02:26,855 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=null 19:02:26,856 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=null 19:02:26,857 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=null 19:02:26,857 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=null 19:02:26,858 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=null 19:02:26,859 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=null 19:02:26,860 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:26,861 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:26,862 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:26,863 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:26,863 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:26,864 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:26,865 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:26,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:26,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:26,868 INFO ClientRMService:342 - Application with id 7 submitted by user test 19:02:26,868 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0007 19:02:26,868 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0007 19:02:26,868 INFO RMAppImpl:643 - application_1394161202967_0007 State change from NEW to NEW_SAVING 19:02:26,868 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0007 19:02:26,869 INFO RMAppImpl:643 - application_1394161202967_0007 State change from NEW_SAVING to SUBMITTED 19:02:26,869 INFO FairScheduler:638 - Accepted application application_1394161202967_0007 from user: test, in queue: default, currently num of applications: 2 19:02:26,869 INFO RMAppImpl:643 - application_1394161202967_0007 State change from SUBMITTED to ACCEPTED 19:02:26,869 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0007_000001 19:02:26,869 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from NEW to SUBMITTED 19:02:26,869 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0007_000001 to scheduler from user: test 19:02:26,870 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from SUBMITTED to SCHEDULED 19:02:26,902 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161346902, maxDate=1394766146902, sequenceNumber=22, masterKeyId=2 19:02:26,902 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 22 19:02:27,623 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741906_1082, blk_1073741907_1083, blk_1073741908_1084, blk_1073741909_1085, blk_1073741910_1086, blk_1073741899_1075, blk_1073741915_1091, blk_1073741900_1076, blk_1073741916_1092, blk_1073741917_1093, blk_1073741919_1095] 19:02:27,695 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000001 Container Transitioned from NEW to ALLOCATED 19:02:27,695 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000001 19:02:27,695 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0007_01_000001 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:02:27,695 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:02:27,696 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0007 AttemptId: appattempt_1394161202967_0007_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0007_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:02:27,696 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:02:27,696 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:02:27,696 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0007_000001 19:02:27,697 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0007_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0007_000001 19:02:27,697 INFO AMLauncher:190 - Command to launch container container_1394161202967_0007_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:02:27,702 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0007_01_000001 by user test 19:02:27,702 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0007 19:02:27,702 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000001 19:02:27,702 INFO Application:450 - Application application_1394161202967_0007 transitioned from NEW to INITING 19:02:27,702 INFO Application:296 - Adding container_1394161202967_0007_01_000001 to application application_1394161202967_0007 19:02:27,702 INFO Application:450 - Application application_1394161202967_0007 transitioned from INITING to RUNNING 19:02:27,703 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0007_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0007_000001 19:02:27,703 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from ALLOCATED to LAUNCHED 19:02:27,703 INFO Container:888 - Container container_1394161202967_0007_01_000001 transitioned from NEW to LOCALIZING 19:02:27,703 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0007 19:02:27,703 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:02:27,703 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split transitioned from INIT to DOWNLOADING 19:02:27,703 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml transitioned from INIT to DOWNLOADING 19:02:27,703 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:02:27,703 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0007_01_000001 19:02:27,704 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar, 1394161346632, FILE, null } 19:02:27,800 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0007_01_000001.tokens. Credentials list: 19:02:27,931 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:27,972 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:27,973 INFO DefaultContainerExecutor:469 - Initializing user test 19:02:28,000 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib dst=null perm=null 19:02:28,038 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app dst=null perm=null 19:02:28,068 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f dst=null perm=null 19:02:28,103 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:02:28,140 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:02:28,211 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:02:28,249 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:28,310 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:28,453 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53294, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099, duration: 6542000 19:02:28,676 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0007_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000001.tokens 19:02:28,676 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/filecache/16/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:02:28,676 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0007 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0007 19:02:28,718 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:02:29,019 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741906_1082 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741906 for deletion 19:02:29,019 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741907_1083 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741907 for deletion 19:02:29,019 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741908_1084 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741908 for deletion 19:02:29,019 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741906_1082 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741906 19:02:29,020 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741907_1083 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741907 19:02:29,019 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741909_1085 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741909 for deletion 19:02:29,020 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741908_1084 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741908 19:02:29,020 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741910_1086 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741910 for deletion 19:02:29,020 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741909_1085 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741909 19:02:29,020 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741915_1091 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741915 for deletion 19:02:29,020 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741910_1086 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741910 19:02:29,020 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741899_1075 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741899 for deletion 19:02:29,020 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741915_1091 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741915 19:02:29,020 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741916_1092 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741916 for deletion 19:02:29,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741899_1075 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741899 19:02:29,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741900_1076 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741900 for deletion 19:02:29,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741916_1092 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741916 19:02:29,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741917_1093 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741917 for deletion 19:02:29,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741919_1095 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741919 for deletion 19:02:29,022 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=null 19:02:29,022 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741917_1093 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741917 19:02:29,022 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741900_1076 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741900 19:02:29,022 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741919_1095 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741919 19:02:29,023 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=null 19:02:29,025 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=null 19:02:29,061 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53297, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1862235170_2588, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104, duration: 321000 19:02:29,145 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0007/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:02:29,227 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=null 19:02:29,228 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=null 19:02:29,229 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=null 19:02:29,259 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53297, bytes: 176, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1862235170_2588, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103, duration: 382000 19:02:29,360 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0007/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:02:29,451 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=null 19:02:29,460 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=null 19:02:29,462 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=null 19:02:29,498 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53297, bytes: 80559, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1862235170_2588, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105, duration: 563000 19:02:29,605 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0007/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:02:29,605 INFO Container:888 - Container container_1394161202967_0007_01_000001 transitioned from LOCALIZING to LOCALIZED 19:02:29,780 INFO Container:888 - Container container_1394161202967_0007_01_000001 transitioned from LOCALIZED to RUNNING 19:02:29,781 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:30,174 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000001/default_container_executor.sh] 19:02:30,501 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102 19:02:30,502 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104 19:02:30,503 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101 19:02:30,503 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103 19:02:30,504 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100 19:02:30,504 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098 19:02:30,505 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105 19:02:30,506 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099 19:02:30,623 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741906_1082, blk_1073741907_1083, blk_1073741908_1084, blk_1073741909_1085, blk_1073741910_1086, blk_1073741899_1075, blk_1073741915_1091, blk_1073741900_1076, blk_1073741916_1092, blk_1073741917_1093, blk_1073741919_1095] 19:02:31,072 INFO ContainerLaunch:322 - Container container_1394161202967_0006_01_000001 succeeded 19:02:31,072 INFO Container:888 - Container container_1394161202967_0006_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:02:31,072 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0006_01_000001 19:02:31,090 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000001 19:02:31,090 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000001 19:02:31,091 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000001 19:02:31,091 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0006/container_1394161202967_0006_01_000001 19:02:31,091 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000001 19:02:31,091 INFO Container:888 - Container container_1394161202967_0006_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 19:02:31,091 INFO Application:339 - Removing container_1394161202967_0006_01_000001 from application application_1394161202967_0006 19:02:31,092 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:31,092 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0006 19:02:31,221 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102 19:02:31,222 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104 19:02:31,222 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103 19:02:31,223 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098 19:02:31,223 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741924_1100 19:02:31,224 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099 19:02:31,225 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741929_1105 19:02:31,227 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101 19:02:31,906 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0006_01_000001 19:02:31,907 INFO RMContainerImpl:220 - container_1394161202967_0006_01_000001 Container Transitioned from RUNNING to COMPLETED 19:02:31,907 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0006_000001 19:02:31,907 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0006_01_000001 in state: COMPLETED event:FINISHED 19:02:31,907 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0006_000001 State change from FINISHING to FINISHED 19:02:31,908 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0006 CONTAINERID=container_1394161202967_0006_01_000001 19:02:31,908 INFO RMAppImpl:643 - application_1394161202967_0006 State change from FINISHING to FINISHED 19:02:31,908 INFO FSSchedulerNode:162 - Released container container_1394161202967_0006_01_000001 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:02:31,908 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0006_000001 released container container_1394161202967_0006_01_000001 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: FINISHED 19:02:31,908 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0006 19:02:31,908 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0006_000001 19:02:31,908 INFO FairScheduler:720 - Application appattempt_1394161202967_0006_000001 is done. finalState=FINISHED 19:02:31,908 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0006,name=oozie:action:T\=map-reduce:W\=testApp:A\=streaming:ID\=0000000-140306190144221-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0006/jobhistory/job/job_1394161202967_0006,appMasterHost=rkanter-mbp.local,startTime=1394161321896,finishTime=1394161343905,finalStatus=SUCCEEDED 19:02:31,909 INFO AppSchedulingInfo:108 - Application application_1394161202967_0006 requests cleared 19:02:31,918 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0006_01_000001 19:02:32,004 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741906_1082 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741906 for deletion 19:02:32,004 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741907_1083 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741907 for deletion 19:02:32,004 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741908_1084 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741908 for deletion 19:02:32,004 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741909_1085 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741909 for deletion 19:02:32,004 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741906_1082 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741906 19:02:32,004 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741910_1086 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741910 for deletion 19:02:32,004 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741907_1083 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741907 19:02:32,005 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741915_1091 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741915 for deletion 19:02:32,005 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741908_1084 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741908 19:02:32,005 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741899_1075 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741899 for deletion 19:02:32,005 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741909_1085 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741909 19:02:32,005 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741910_1086 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741910 19:02:32,005 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741916_1092 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741916 for deletion 19:02:32,005 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741915_1091 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741915 19:02:32,005 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741900_1076 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741900 for deletion 19:02:32,006 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741917_1093 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741917 for deletion 19:02:32,006 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741916_1092 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741916 19:02:32,006 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741900_1076 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741900 19:02:32,006 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741899_1075 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741899 19:02:32,006 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741919_1095 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741919 for deletion 19:02:32,006 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741917_1093 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741917 19:02:32,007 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741919_1095 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741919 19:02:32,713 INFO Application:450 - Application application_1394161202967_0006 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:02:32,713 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0006 19:02:32,713 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0006 19:02:32,714 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0006 19:02:32,714 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0006 19:02:32,714 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0006 19:02:32,714 INFO Application:450 - Application application_1394161202967_0006 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:02:32,714 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0006, with delay of 10800 seconds 19:02:32,907 INFO Application:450 - Application application_1394161202967_0006 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:02:32,908 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0006 19:02:32,908 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0006 19:02:32,908 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0006 19:02:32,908 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0006 19:02:32,908 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0006 19:02:32,909 INFO Application:450 - Application application_1394161202967_0006 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:02:32,909 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0006, with delay of 10800 seconds 19:02:32,931 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:32,973 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/COMMIT_STARTED dst=null perm=null 19:02:32,974 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/COMMIT_SUCCESS dst=null perm=null 19:02:32,975 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/COMMIT_FAIL dst=null perm=null 19:02:33,036 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007 dst=null perm=null 19:02:33,038 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:02:33,040 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:02:33,547 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=null 19:02:33,557 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.splitmetainfo dst=null perm=null 19:02:33,627 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53301, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741928_1104, duration: 208000 19:02:34,667 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0007_000001 19:02:34,667 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0007 APPATTEMPTID=appattempt_1394161202967_0007_000001 19:02:34,668 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from LAUNCHED to RUNNING 19:02:34,668 INFO RMAppImpl:643 - application_1394161202967_0007 State change from ACCEPTED to RUNNING 19:02:34,717 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:02:34,720 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=test:supergroup:rwxrwxrwx 19:02:34,779 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:02:34,990 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741930_1106{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:02:35,042 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106 src: /127.0.0.1:53307 dest: /127.0.0.1:52682 19:02:35,044 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106 src: /127.0.0.1:53308 dest: /127.0.0.1:52685 19:02:35,086 INFO clienttrace:1215 - src: /127.0.0.1:53308, dest: /127.0.0.1:52685, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106, duration: 40451000 19:02:35,086 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:35,087 INFO clienttrace:1215 - src: /127.0.0.1:53307, dest: /127.0.0.1:52682, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106, duration: 41819000 19:02:35,088 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:35,093 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1_conf.xml is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:35,124 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007.summary dst=null perm=test:supergroup:rw-rw-rw- 19:02:35,128 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741931_1107{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:35,131 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107 src: /127.0.0.1:53310 dest: /127.0.0.1:52685 19:02:35,132 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107 src: /127.0.0.1:53311 dest: /127.0.0.1:52682 19:02:35,136 INFO clienttrace:1215 - src: /127.0.0.1:53311, dest: /127.0.0.1:52682, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107, duration: 1856000 19:02:35,136 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:35,137 INFO clienttrace:1215 - src: /127.0.0.1:53310, dest: /127.0.0.1:52685, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107, duration: 2773000 19:02:35,137 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:35,140 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007.summary is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:35,510 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107 19:02:35,511 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106 19:02:35,716 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000002 Container Transitioned from NEW to ALLOCATED 19:02:35,716 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000002 19:02:35,716 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0007_01_000002 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:02:36,231 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106 19:02:36,231 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741931_1107 19:02:36,716 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:02:36,783 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=null 19:02:36,784 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.xml dst=null perm=null 19:02:36,797 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:36,800 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:36,802 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:36,812 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:36,814 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:36,816 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:36,817 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:36,819 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:36,820 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:36,880 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0007_01_000002 by user test 19:02:36,880 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000002 19:02:36,880 INFO Application:296 - Adding container_1394161202967_0007_01_000002 to application application_1394161202967_0007 19:02:36,881 INFO Container:888 - Container container_1394161202967_0007_01_000002 transitioned from NEW to LOCALIZING 19:02:36,881 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0007 19:02:36,881 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0007 19:02:36,881 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:02:36,881 INFO ShuffleHandler:263 - Added token for job_1394161202967_0007 19:02:36,882 INFO Container:888 - Container container_1394161202967_0007_01_000002 transitioned from LOCALIZING to LOCALIZED 19:02:36,988 INFO Container:888 - Container container_1394161202967_0007_01_000002 transitioned from LOCALIZED to RUNNING 19:02:36,989 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:37,217 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000002/default_container_executor.sh] 19:02:37,719 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:02:37,730 INFO AppSchedulingInfo:166 - checking for deactivate... 19:02:40,375 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job.split dst=null perm=null 19:02:40,462 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53319, bytes: 176, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741927_1103, duration: 258000 19:02:40,524 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=null 19:02:40,540 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0007_m_000000_0/part-00000 dst=null perm=test:supergroup:rw-rw-rw- 19:02:40,564 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/0000000-140306190226247-oozie-rkan-W@mr-action dst=null perm=null 19:02:40,568 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/0000000-140306190226247-oozie-rkan-W@mr-action dst=null perm=test:supergroup:rw-rw-rw- 19:02:40,579 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/0000000-140306190226247-oozie-rkan-W@mr-action. BP-1007909283-172.16.1.64-1394161187028 blk_1073741932_1108{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:02:40,625 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108 src: /127.0.0.1:53320 dest: /127.0.0.1:52685 19:02:40,627 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108 src: /127.0.0.1:53321 dest: /127.0.0.1:52682 19:02:40,654 INFO clienttrace:1215 - src: /127.0.0.1:53321, dest: /127.0.0.1:52682, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108, duration: 26042000 19:02:40,654 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:40,655 INFO clienttrace:1215 - src: /127.0.0.1:53320, dest: /127.0.0.1:52685, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108, duration: 26755000 19:02:40,655 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:40,659 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/0000000-140306190226247-oozie-rkan-W@mr-action is closed by DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1 19:02:40,666 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53322, bytes: 9, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741926_1102, duration: 450000 19:02:40,724 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:02:40,725 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:02:40,741 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53323, bytes: 77871, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741925_1101, duration: 2858000 19:02:41,235 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108 19:02:41,435 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/output dst=null perm=null 19:02:41,438 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:41,439 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:41,466 INFO ClientRMService:229 - Allocated new applicationId: 8 19:02:41,483 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008 dst=null perm=null 19:02:41,502 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008 dst=null perm=test:supergroup:rwxrwxrwx 19:02:41,514 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008 dst=null perm=test:supergroup:rwx------ 19:02:41,529 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:41,531 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:41,533 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:41,535 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:41,536 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:02:41,538 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:02:41,539 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:02:41,540 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:02:41,542 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:02:41,545 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:41,547 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:41,548 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib dst=null perm=null 19:02:41,549 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app dst=null perm=null 19:02:41,551 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f dst=null perm=null 19:02:41,552 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:02:41,577 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input dst=null perm=null 19:02:41,583 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input dst=null perm=null 19:02:41,590 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input dst=null perm=null 19:02:41,643 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:02:41,646 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=test:supergroup:rw-r--r-- 19:02:41,656 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split 19:02:41,657 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:41,671 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741933_1109{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:41,674 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109 src: /127.0.0.1:53325 dest: /127.0.0.1:52682 19:02:41,676 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109 src: /127.0.0.1:53326 dest: /127.0.0.1:52685 19:02:41,680 INFO clienttrace:1215 - src: /127.0.0.1:53326, dest: /127.0.0.1:52685, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109, duration: 2577000 19:02:41,680 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:41,681 INFO clienttrace:1215 - src: /127.0.0.1:53325, dest: /127.0.0.1:52682, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109, duration: 3284000 19:02:41,681 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:41,683 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split is closed by DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1 19:02:41,686 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:02:41,689 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:02:41,698 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741934_1110{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:02:41,701 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110 src: /127.0.0.1:53327 dest: /127.0.0.1:52685 19:02:41,705 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110 src: /127.0.0.1:53328 dest: /127.0.0.1:52682 19:02:41,709 INFO clienttrace:1215 - src: /127.0.0.1:53328, dest: /127.0.0.1:52682, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110, duration: 1934000 19:02:41,709 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:41,710 INFO clienttrace:1215 - src: /127.0.0.1:53327, dest: /127.0.0.1:52685, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110, duration: 2858000 19:02:41,710 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:41,712 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo is closed by DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1 19:02:41,740 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:02:41,743 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:02:41,792 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741935_1111{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:41,794 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111 src: /127.0.0.1:53329 dest: /127.0.0.1:52682 19:02:41,796 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111 src: /127.0.0.1:53330 dest: /127.0.0.1:52685 19:02:41,799 INFO clienttrace:1215 - src: /127.0.0.1:53330, dest: /127.0.0.1:52685, bytes: 77360, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111, duration: 1753000 19:02:41,800 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:41,800 INFO clienttrace:1215 - src: /127.0.0.1:53329, dest: /127.0.0.1:52682, bytes: 77360, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111, duration: 2606000 19:02:41,801 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:41,802 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml is closed by DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1 19:02:42,030 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008 dst=null perm=null 19:02:42,225 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:42,226 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:42,230 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:42,231 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:42,232 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=null 19:02:42,233 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=null 19:02:42,237 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:42,238 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:42,240 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:42,241 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:42,242 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:42,243 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:42,244 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:42,246 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:42,248 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:42,306 INFO ClientRMService:342 - Application with id 8 submitted by user test 19:02:42,306 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0008 19:02:42,306 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0008 19:02:42,306 INFO RMAppImpl:643 - application_1394161202967_0008 State change from NEW to NEW_SAVING 19:02:42,306 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0008 19:02:42,307 INFO RMAppImpl:643 - application_1394161202967_0008 State change from NEW_SAVING to SUBMITTED 19:02:42,307 INFO FairScheduler:638 - Accepted application application_1394161202967_0008 from user: test, in queue: default, currently num of applications: 2 19:02:42,307 INFO RMAppImpl:643 - application_1394161202967_0008 State change from SUBMITTED to ACCEPTED 19:02:42,307 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0008_000001 19:02:42,307 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from NEW to SUBMITTED 19:02:42,308 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0008_000001 to scheduler from user: test 19:02:42,308 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from SUBMITTED to SCHEDULED 19:02:42,617 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=test:supergroup:rw-rw-rw- 19:02:42,683 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq. BP-1007909283-172.16.1.64-1394161187028 blk_1073741936_1112{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:42,686 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112 src: /127.0.0.1:53332 dest: /127.0.0.1:52682 19:02:42,688 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112 src: /127.0.0.1:53333 dest: /127.0.0.1:52685 19:02:42,691 INFO clienttrace:1215 - src: /127.0.0.1:53333, dest: /127.0.0.1:52685, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112, duration: 2012000 19:02:42,691 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:42,692 INFO clienttrace:1215 - src: /127.0.0.1:53332, dest: /127.0.0.1:52682, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112, duration: 2723000 19:02:42,692 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:42,694 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq is closed by DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1 19:02:42,706 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0007_m_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0007_m_000000_0_-1355839956_1 19:02:42,723 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0008 19:02:42,723 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000001 Container Transitioned from NEW to RESERVED 19:02:42,723 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0008 reserved container container_1394161202967_0008_01_000001 on node host: 172.16.1.64:52707 #containers=2 available= used=, currently has 1 at priority 0; currentReservation 2048 19:02:42,723 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0008_01_000001 on node host: 172.16.1.64:52707 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@77429219 19:02:42,751 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0007_m_000000_0 dst=null perm=null 19:02:42,759 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0007_m_000000_0 dst=null perm=null 19:02:42,760 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0007_m_000000 dst=null perm=null 19:02:42,768 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0007_m_000000_0 dst=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0007_m_000000 perm=test:supergroup:rwxrwxrwx 19:02:42,794 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0007_01_000002 19:02:42,794 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000002 19:02:42,794 INFO Container:888 - Container container_1394161202967_0007_01_000002 transitioned from RUNNING to KILLING 19:02:42,795 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0007_01_000002 19:02:42,849 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0007_01_000002 is : 143 19:02:42,851 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741937_1113{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:42,851 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:02:42,854 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113 src: /127.0.0.1:53334 dest: /127.0.0.1:52682 19:02:42,855 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:42,855 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113 src: /127.0.0.1:53335 dest: /127.0.0.1:52685 19:02:42,860 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1.jhist for DFSClient_NONMAPREDUCE_1219104543_1 19:02:42,860 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=null 19:02:42,868 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:02:42,869 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:02:42,869 INFO Container:888 - Container container_1394161202967_0007_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:02:42,870 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000002 19:02:42,870 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000002 19:02:42,870 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000002 19:02:42,870 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0007_m_000000 dst=null perm=null 19:02:42,870 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000002 19:02:42,870 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000002 19:02:42,871 INFO Container:888 - Container container_1394161202967_0007_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:02:42,871 INFO Application:339 - Removing container_1394161202967_0007_01_000002 from application application_1394161202967_0007 19:02:42,871 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:42,871 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0007 19:02:42,872 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/part-00000 dst=null perm=null 19:02:42,875 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0007_m_000000/part-00000 dst=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/part-00000 perm=test:supergroup:rw-rw-rw- 19:02:42,881 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_temporary dst=null perm=null 19:02:42,886 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:02:42,887 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:42,889 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:02:42,891 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:42,907 INFO clienttrace:1215 - src: /127.0.0.1:53335, dest: /127.0.0.1:52685, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113, duration: 49893000 19:02:42,908 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:42,909 INFO clienttrace:1215 - src: /127.0.0.1:53334, dest: /127.0.0.1:52682, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113, duration: 51371000 19:02:42,909 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:42,911 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1.jhist is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:42,914 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:42,916 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741938_1114{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:42,919 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114 src: /127.0.0.1:53336 dest: /127.0.0.1:52685 19:02:42,920 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114 src: /127.0.0.1:53337 dest: /127.0.0.1:52682 19:02:42,922 INFO clienttrace:1215 - src: /127.0.0.1:53337, dest: /127.0.0.1:52682, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114, duration: 1466000 19:02:42,922 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:42,923 INFO clienttrace:1215 - src: /127.0.0.1:53336, dest: /127.0.0.1:52685, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114, duration: 2048000 19:02:42,923 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:42,924 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary_tmp is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:42,928 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:42,928 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000002 Container Transitioned from NEW to ALLOCATED 19:02:42,928 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000002 19:02:42,929 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:02:42,929 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:02:42,929 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0008 AttemptId: appattempt_1394161202967_0008_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0008_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] 19:02:42,929 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:02:42,930 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:02:42,930 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0008_000001 19:02:42,931 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0008_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0008_000001 19:02:42,931 INFO AMLauncher:190 - Command to launch container container_1394161202967_0008_01_000002 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:02:42,934 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1.jhist dst=null perm=null 19:02:42,935 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist_tmp dst=null perm=null 19:02:42,938 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0008_01_000002 by user test 19:02:42,938 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0008 19:02:42,938 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000002 19:02:42,938 INFO Application:450 - Application application_1394161202967_0008 transitioned from NEW to INITING 19:02:42,938 INFO Application:296 - Adding container_1394161202967_0008_01_000002 to application application_1394161202967_0008 19:02:42,938 INFO Application:450 - Application application_1394161202967_0008 transitioned from INITING to RUNNING 19:02:42,939 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0008_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0008_000001 19:02:42,939 INFO Container:888 - Container container_1394161202967_0008_01_000002 transitioned from NEW to LOCALIZING 19:02:42,939 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from ALLOCATED to LAUNCHED 19:02:42,939 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0008 19:02:42,939 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1.jhist dst=null perm=null 19:02:42,939 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:02:42,939 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split transitioned from INIT to DOWNLOADING 19:02:42,939 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml transitioned from INIT to DOWNLOADING 19:02:42,939 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:02:42,940 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0008_01_000002 19:02:42,940 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar, 1394161346632, FILE, null } 19:02:42,940 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist_tmp dst=null perm=null 19:02:42,941 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1.jhist dst=null perm=null 19:02:43,014 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:43,054 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/nmPrivate/container_1394161202967_0008_01_000002.tokens. Credentials list: 19:02:43,108 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53339, bytes: 20861, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113, duration: 1442000 19:02:43,157 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741939_1115{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:43,185 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:43,205 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115 src: /127.0.0.1:53341 dest: /127.0.0.1:52682 19:02:43,205 INFO DefaultContainerExecutor:469 - Initializing user test 19:02:43,221 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:43,239 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115 src: /127.0.0.1:53342 dest: /127.0.0.1:52685 19:02:43,262 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib dst=null perm=null 19:02:43,293 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app dst=null perm=null 19:02:43,319 INFO clienttrace:1215 - src: /127.0.0.1:53342, dest: /127.0.0.1:52685, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115, duration: 59510000 19:02:43,319 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:43,338 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f dst=null perm=null 19:02:43,348 INFO clienttrace:1215 - src: /127.0.0.1:53341, dest: /127.0.0.1:52682, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115, duration: 66598000 19:02:43,349 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:43,374 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:02:43,380 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist_tmp is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:43,398 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:02:43,398 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:43,423 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:02:43,423 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1_conf.xml dst=null perm=null 19:02:43,453 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml_tmp dst=null perm=null 19:02:43,454 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:43,479 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1_conf.xml dst=null perm=null 19:02:43,481 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:43,488 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml_tmp dst=null perm=null 19:02:43,505 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007/job_1394161202967_0007_1_conf.xml dst=null perm=null 19:02:43,569 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:02:43,603 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53339, bytes: 93643, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741930_1106, duration: 519000 19:02:43,610 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53343, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741923_1099, duration: 657000 19:02:43,640 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741940_1116{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:43,768 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116 src: /127.0.0.1:53344 dest: /127.0.0.1:52685 19:02:43,806 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0007_01_000002 19:02:43,884 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000002 Container Transitioned from RUNNING to COMPLETED 19:02:43,886 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0007_01_000002 in state: COMPLETED event:FINISHED 19:02:43,929 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000002 19:02:43,930 INFO FSSchedulerNode:162 - Released container container_1394161202967_0007_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:02:43,930 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116 src: /127.0.0.1:53345 dest: /127.0.0.1:52682 19:02:43,930 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0007_000001 released container container_1394161202967_0007_01_000002 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:02:43,931 INFO FairScheduler:1042 - Releasing reservation that cannot be satisfied for application appattempt_1394161202967_0008_000001 on node host: 172.16.1.64:52707 #containers=1 available= used= 19:02:43,931 INFO FSSchedulerApp:123 - Application application_1394161202967_0008 unreserved on node host: 172.16.1.64:52707 #containers=1 available= used=, currently has 0 at priority 0; currentReservation 19:02:43,952 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/filecache/16/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:02:43,972 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:02:43,975 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/nmPrivate/container_1394161202967_0008_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000002.tokens 19:02:43,976 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0008 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0008 19:02:44,014 INFO clienttrace:1215 - src: /127.0.0.1:53345, dest: /127.0.0.1:52682, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116, duration: 36903000 19:02:44,015 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:44,046 INFO clienttrace:1215 - src: /127.0.0.1:53344, dest: /127.0.0.1:52685, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1219104543_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116, duration: 48938000 19:02:44,050 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:44,082 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_1219104543_1 19:02:44,118 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:02:44,147 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary perm=test:supergroup:rwxrwx--- 19:02:44,192 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml perm=test:supergroup:rwxrwx--- 19:02:44,224 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist perm=test:supergroup:rwxrwx--- 19:02:44,232 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0007_000001 with final state: FINISHING 19:02:44,233 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from RUNNING to FINAL_SAVING 19:02:44,233 INFO RMAppImpl:880 - Updating application application_1394161202967_0007 with final state: FINISHING 19:02:44,233 INFO RMAppImpl:643 - application_1394161202967_0007 State change from RUNNING to FINAL_SAVING 19:02:44,233 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0007 19:02:44,235 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from FINAL_SAVING to FINISHING 19:02:44,236 INFO RMAppImpl:643 - application_1394161202967_0007 State change from FINAL_SAVING to FINISHING 19:02:44,316 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=null 19:02:44,318 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=null 19:02:44,320 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=null 19:02:44,342 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53348, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_771045170_2768, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110, duration: 205000 19:02:44,433 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0008/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:02:44,538 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:44,539 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:44,540 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:44,566 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53348, bytes: 283, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_771045170_2768, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109, duration: 426000 19:02:44,637 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0008/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:02:44,709 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:44,710 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:44,711 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:44,739 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53348, bytes: 77968, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_771045170_2768, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111, duration: 539000 19:02:44,825 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0008/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:02:44,825 INFO Container:888 - Container container_1394161202967_0008_01_000002 transitioned from LOCALIZING to LOCALIZED 19:02:44,896 INFO Container:888 - Container container_1394161202967_0008_01_000002 transitioned from LOCALIZED to RUNNING 19:02:44,897 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:45,146 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000002/default_container_executor.sh] 19:02:45,241 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0007 dst=null perm=null 19:02:45,349 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:02:45,352 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:02:45,354 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:02:45,358 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53343, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112, duration: 322000 19:02:45,418 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161365418, maxDate=1394766165418, sequenceNumber=23, masterKeyId=2 19:02:45,418 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 23 19:02:45,426 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:02:45,427 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:02:45,430 INFO CompletedJob:95 - Loading job: job_1394161202967_0007 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist 19:02:45,430 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist] 19:02:45,431 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist dst=null perm=null 19:02:45,433 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist dst=null perm=null 19:02:45,435 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53348, bytes: 20861, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115, duration: 269000 19:02:45,462 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary dst=null perm=null 19:02:45,464 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53348, bytes: 420, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114, duration: 243000 19:02:45,464 INFO JobSummary:372 - jobId=job_1394161202967_0007,submitTime=1394161346868,launchTime=1394161354704,firstMapTaskLaunchTime=1394161356887,firstReduceTaskLaunchTime=0,finishTime=1394161362892,resourcesPerMap=1024,resourcesPerReduce=0,numMaps=1,numReduces=0,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=5,reduceSlotSeconds=0,jobName=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190226247-oozie-rkan-W 19:02:45,465 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary] 19:02:45,467 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007.summary dst=null perm=null 19:02:45,469 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:02:45,470 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist 19:02:45,472 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0007-1394161346868-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161362892-1-0-SUCCEEDED-root.test-1394161354704.jhist perm=test:supergroup:rwxrwx--- 19:02:45,472 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0007_conf.xml 19:02:45,474 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0007_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0007_conf.xml perm=test:supergroup:rwxrwx--- 19:02:45,508 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:02:45,509 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:02:45,510 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190226247-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:02:45,512 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53348, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_512660409_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112, duration: 259000 19:02:45,518 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112 19:02:45,520 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116 19:02:45,520 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113 19:02:45,522 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111 19:02:45,522 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110 19:02:45,523 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115 19:02:45,523 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109 19:02:45,524 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114 19:02:45,524 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741932_1108 19:02:45,571 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161365571, maxDate=1394766165571, sequenceNumber=24, masterKeyId=2 19:02:45,572 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 24 19:02:45,629 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741937_1113, blk_1073741938_1114, blk_1073741927_1103, blk_1073741928_1104, blk_1073741929_1105, blk_1073741930_1106, blk_1073741931_1107] 19:02:46,240 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109 19:02:46,241 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111 19:02:46,241 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741936_1112 19:02:46,241 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110 19:02:46,242 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741939_1115 19:02:46,243 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741940_1116 19:02:46,243 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741937_1113 19:02:46,243 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741938_1114 19:02:47,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741937_1113 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741937 for deletion 19:02:47,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741938_1114 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741938 for deletion 19:02:47,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741927_1103 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741927 for deletion 19:02:47,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741928_1104 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741928 for deletion 19:02:47,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741937_1113 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741937 19:02:47,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741938_1114 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741938 19:02:47,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741929_1105 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741929 for deletion 19:02:47,015 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741927_1103 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741927 19:02:47,015 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741930_1106 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741930 for deletion 19:02:47,015 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741928_1104 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741928 19:02:47,015 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741931_1107 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741931 for deletion 19:02:47,015 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741929_1105 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741929 19:02:47,015 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741930_1106 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741930 19:02:47,015 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741931_1107 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741931 19:02:48,148 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:02:48,183 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/COMMIT_STARTED dst=null perm=null 19:02:48,185 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/COMMIT_SUCCESS dst=null perm=null 19:02:48,187 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/COMMIT_FAIL dst=null perm=null 19:02:48,259 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008 dst=null perm=null 19:02:48,261 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:02:48,262 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:02:48,630 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741937_1113, blk_1073741938_1114, blk_1073741927_1103, blk_1073741928_1104, blk_1073741929_1105, blk_1073741930_1106, blk_1073741931_1107] 19:02:48,651 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=null 19:02:48,659 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.splitmetainfo dst=null perm=null 19:02:48,730 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53355, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741934_1110, duration: 189000 19:02:49,930 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0008_000001 19:02:49,930 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0008 APPATTEMPTID=appattempt_1394161202967_0008_000001 19:02:49,930 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from LAUNCHED to RUNNING 19:02:49,930 INFO RMAppImpl:643 - application_1394161202967_0008 State change from ACCEPTED to RUNNING 19:02:49,962 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:02:49,963 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/output/_temporary/1 dst=null perm=test:testg:rwxrwxrwx 19:02:50,013 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741937_1113 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741937 for deletion 19:02:50,013 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741938_1114 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741938 for deletion 19:02:50,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741927_1103 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741927 for deletion 19:02:50,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741937_1113 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741937 19:02:50,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741928_1104 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741928 for deletion 19:02:50,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741929_1105 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741929 for deletion 19:02:50,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741927_1103 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741927 19:02:50,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741930_1106 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741930 for deletion 19:02:50,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741938_1114 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741938 19:02:50,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741929_1105 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741929 19:02:50,014 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741931_1107 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741931 for deletion 19:02:50,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741928_1104 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741928 19:02:50,014 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741931_1107 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741931 19:02:50,015 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741930_1106 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741930 19:02:50,029 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:02:50,246 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741941_1117{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:50,306 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117 src: /127.0.0.1:53360 dest: /127.0.0.1:52682 19:02:50,308 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117 src: /127.0.0.1:53361 dest: /127.0.0.1:52685 19:02:50,343 INFO clienttrace:1215 - src: /127.0.0.1:53361, dest: /127.0.0.1:52685, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117, duration: 34099000 19:02:50,344 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:50,344 INFO clienttrace:1215 - src: /127.0.0.1:53360, dest: /127.0.0.1:52682, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117, duration: 35001000 19:02:50,345 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:50,349 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1_conf.xml is closed by DFSClient_NONMAPREDUCE_1732519755_1 19:02:50,375 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008.summary dst=null perm=test:supergroup:rw-rw-rw- 19:02:50,379 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741942_1118{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:02:50,381 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118 src: /127.0.0.1:53362 dest: /127.0.0.1:52685 19:02:50,383 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118 src: /127.0.0.1:53363 dest: /127.0.0.1:52682 19:02:50,386 INFO clienttrace:1215 - src: /127.0.0.1:53363, dest: /127.0.0.1:52682, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118, duration: 1707000 19:02:50,387 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:02:50,387 INFO clienttrace:1215 - src: /127.0.0.1:53362, dest: /127.0.0.1:52685, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118, duration: 2373000 19:02:50,388 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:02:50,389 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008.summary is closed by DFSClient_NONMAPREDUCE_1732519755_1 19:02:50,530 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117 19:02:50,531 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118 19:02:51,250 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117 19:02:51,251 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741942_1118 19:02:51,390 INFO ContainerLaunch:322 - Container container_1394161202967_0007_01_000001 succeeded 19:02:51,390 INFO Container:888 - Container container_1394161202967_0007_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:02:51,390 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0007_01_000001 19:02:51,404 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000001 19:02:51,404 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000001 19:02:51,404 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000001 19:02:51,404 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0007/container_1394161202967_0007_01_000001 19:02:51,404 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000001 19:02:51,405 INFO Container:888 - Container container_1394161202967_0007_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 19:02:51,405 INFO Application:339 - Removing container_1394161202967_0007_01_000001 from application application_1394161202967_0007 19:02:51,405 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:51,405 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0007 19:02:51,829 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0007_01_000001 19:02:51,829 INFO RMContainerImpl:220 - container_1394161202967_0007_01_000001 Container Transitioned from RUNNING to COMPLETED 19:02:51,830 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0007_01_000001 in state: COMPLETED event:FINISHED 19:02:51,830 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0007_000001 19:02:51,830 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0007 CONTAINERID=container_1394161202967_0007_01_000001 19:02:51,830 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0007_000001 State change from FINISHING to FINISHED 19:02:51,830 INFO FSSchedulerNode:162 - Released container container_1394161202967_0007_01_000001 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:02:51,830 INFO RMAppImpl:643 - application_1394161202967_0007 State change from FINISHING to FINISHED 19:02:51,830 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0007_000001 released container container_1394161202967_0007_01_000001 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:02:51,830 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0007 19:02:51,830 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0007_000001 19:02:51,830 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0007,name=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190226247-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0007/jobhistory/job/job_1394161202967_0007,appMasterHost=rkanter-mbp.local,startTime=1394161346868,finishTime=1394161364233,finalStatus=SUCCEEDED 19:02:51,831 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000003 Container Transitioned from NEW to ALLOCATED 19:02:51,831 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000003 19:02:51,831 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000003 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:02:51,831 INFO FairScheduler:720 - Application appattempt_1394161202967_0007_000001 is done. finalState=FINISHED 19:02:51,831 INFO AppSchedulingInfo:108 - Application application_1394161202967_0007 requests cleared 19:02:51,836 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0007_01_000001 19:02:51,964 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000004 Container Transitioned from NEW to ALLOCATED 19:02:51,964 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000004 19:02:51,964 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000004 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:02:51,985 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:02:51,985 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 19:02:52,035 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:52,037 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:52,049 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:52,051 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:52,053 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:52,062 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:52,063 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:52,065 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:52,066 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:02:52,067 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:02:52,068 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/app/lib/test.jar dst=null perm=null 19:02:52,132 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0008_01_000003 by user test 19:02:52,132 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0008 19:02:52,132 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000003 19:02:52,132 INFO Application:450 - Application application_1394161202967_0008 transitioned from NEW to INITING 19:02:52,133 INFO Application:296 - Adding container_1394161202967_0008_01_000003 to application application_1394161202967_0008 19:02:52,133 INFO Application:450 - Application application_1394161202967_0008 transitioned from INITING to RUNNING 19:02:52,133 INFO Container:888 - Container container_1394161202967_0008_01_000003 transitioned from NEW to LOCALIZING 19:02:52,133 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0008 19:02:52,134 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0008 19:02:52,134 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:02:52,134 INFO ShuffleHandler:263 - Added token for job_1394161202967_0008 19:02:52,134 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml transitioned from INIT to DOWNLOADING 19:02:52,134 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0008_01_000003 19:02:52,170 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0008_01_000003.tokens. Credentials list: 19:02:52,240 INFO DefaultContainerExecutor:469 - Initializing user test 19:02:52,515 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0008_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000003.tokens 19:02:52,522 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008 19:02:52,790 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:52,792 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:52,793 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml dst=null perm=null 19:02:52,831 INFO Application:450 - Application application_1394161202967_0007 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:02:52,832 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:02:52,832 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0007 19:02:52,832 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0007 19:02:52,833 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0007 19:02:52,833 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0007 19:02:52,833 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0007 19:02:52,833 INFO Application:450 - Application application_1394161202967_0007 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:02:52,833 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0007, with delay of 10800 seconds 19:02:52,834 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53369, bytes: 77968, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1612183246_2872, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741935_1111, duration: 499000 19:02:52,893 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0008/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:02:52,903 INFO Container:888 - Container container_1394161202967_0008_01_000003 transitioned from LOCALIZING to LOCALIZED 19:02:53,010 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000004 Container Transitioned from ACQUIRED to RELEASED 19:02:53,024 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000004 in state: RELEASED event:RELEASED 19:02:53,024 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000004 19:02:53,025 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000004 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:02:53,046 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000004 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:02:53,047 INFO AppSchedulingInfo:166 - checking for deactivate... 19:02:53,057 INFO Container:888 - Container container_1394161202967_0008_01_000003 transitioned from LOCALIZED to RUNNING 19:02:53,057 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:53,327 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000003/default_container_executor.sh] 19:02:53,834 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000005 Container Transitioned from NEW to ALLOCATED 19:02:53,834 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000005 19:02:53,834 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000005 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:02:53,981 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0008_01_000004 19:02:54,070 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000005 Container Transitioned from ALLOCATED to ACQUIRED 19:02:54,078 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0008_01_000005 by user test 19:02:54,079 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000005 19:02:54,079 INFO Application:296 - Adding container_1394161202967_0008_01_000005 to application application_1394161202967_0008 19:02:54,079 INFO Container:888 - Container container_1394161202967_0008_01_000005 transitioned from NEW to LOCALIZING 19:02:54,080 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0008 19:02:54,080 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0008 19:02:54,080 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:02:54,080 INFO ShuffleHandler:263 - Added token for job_1394161202967_0008 19:02:54,081 INFO Container:888 - Container container_1394161202967_0008_01_000005 transitioned from LOCALIZING to LOCALIZED 19:02:54,215 INFO Container:888 - Container container_1394161202967_0008_01_000005 transitioned from LOCALIZED to RUNNING 19:02:54,224 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:54,548 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000005/default_container_executor.sh] 19:02:54,835 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000005 Container Transitioned from ACQUIRED to RUNNING 19:02:55,078 INFO AppSchedulingInfo:166 - checking for deactivate... 19:02:56,523 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:56,625 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53373, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0008_m_000000_0_-672306102_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109, duration: 345000 19:02:56,692 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input/data.txt dst=null perm=null 19:02:56,859 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53373, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0008_m_000000_0_-672306102_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098, duration: 629000 19:02:56,889 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/output/_temporary/1/_temporary/attempt_1394161202967_0008_m_000000_0 dst=null perm=null 19:02:56,923 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0008_01_000003 19:02:56,924 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000003 19:02:56,924 INFO Container:888 - Container container_1394161202967_0008_01_000003 transitioned from RUNNING to KILLING 19:02:56,924 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0008_01_000003 19:02:56,977 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741943_1119{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:02:56,977 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0008_01_000003 is : 143 19:02:56,982 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119 src: /127.0.0.1:53374 dest: /127.0.0.1:52682 19:02:56,984 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119 src: /127.0.0.1:53375 dest: /127.0.0.1:52685 19:02:56,989 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1.jhist for DFSClient_NONMAPREDUCE_1732519755_1 19:02:57,005 INFO Container:888 - Container container_1394161202967_0008_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:02:57,005 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000003 19:02:57,006 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000003 19:02:57,006 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000003 19:02:57,006 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000003 19:02:57,007 INFO Container:888 - Container container_1394161202967_0008_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:02:57,007 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000003 19:02:57,007 INFO Application:339 - Removing container_1394161202967_0008_01_000003 from application application_1394161202967_0008 19:02:57,007 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:57,007 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0008 19:02:57,802 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job.split dst=null perm=null 19:02:57,884 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53377, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0008_m_000001_1_741363090_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741933_1109, duration: 231000 19:02:57,925 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0008_01_000003 19:02:57,926 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000003 Container Transitioned from RUNNING to COMPLETED 19:02:57,926 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000003 in state: COMPLETED event:FINISHED 19:02:57,926 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000003 19:02:57,926 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000003 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:02:57,926 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000003 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:02:57,934 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/input/data.txt dst=null perm=null 19:02:57,936 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53377, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0008_m_000001_1_741363090_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741922_1098, duration: 191000 19:02:58,043 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/output/_temporary/1/_temporary/attempt_1394161202967_0008_m_000001_1 dst=null perm=null 19:02:58,056 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0008_01_000005 19:02:58,056 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000005 19:02:58,056 INFO Container:888 - Container container_1394161202967_0008_01_000005 transitioned from RUNNING to KILLING 19:02:58,056 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0008_01_000005 19:02:58,116 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0008_01_000005 is : 143 19:02:58,134 INFO Container:888 - Container container_1394161202967_0008_01_000005 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:02:58,135 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000005 19:02:58,135 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000005 19:02:58,135 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000005 19:02:58,135 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000005 19:02:58,136 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000005 19:02:58,136 INFO Container:888 - Container container_1394161202967_0008_01_000005 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:02:58,136 INFO Application:339 - Removing container_1394161202967_0008_01_000005 from application application_1394161202967_0008 19:02:58,136 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:02:58,136 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0008 19:02:58,987 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000006 Container Transitioned from NEW to ALLOCATED 19:02:58,987 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000006 19:02:58,987 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000006 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:02:59,058 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0008_01_000005 19:02:59,058 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000005 Container Transitioned from RUNNING to COMPLETED 19:02:59,059 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000005 in state: COMPLETED event:FINISHED 19:02:59,059 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000005 19:02:59,059 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000005 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:02:59,059 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000005 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:02:59,125 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000006 Container Transitioned from ALLOCATED to ACQUIRED 19:03:00,131 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000006 Container Transitioned from ACQUIRED to RELEASED 19:03:00,131 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000006 in state: RELEASED event:RELEASED 19:03:00,132 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000006 19:03:00,132 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000006 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:03:00,132 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000006 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:03:00,132 INFO AppSchedulingInfo:166 - checking for deactivate... 19:03:00,989 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0008_01_000006 19:03:01,466 INFO JobHistory:188 - Starting scan to move intermediate done files 19:03:01,469 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:03:01,470 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:03:01,952 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/fail dst=null perm=null 19:03:01,990 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000007 Container Transitioned from NEW to ALLOCATED 19:03:01,991 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000007 19:03:01,991 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000007 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:03:02,139 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000007 Container Transitioned from ALLOCATED to ACQUIRED 19:03:03,144 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000007 Container Transitioned from ACQUIRED to RELEASED 19:03:03,144 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000007 in state: RELEASED event:RELEASED 19:03:03,144 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000007 19:03:03,145 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000007 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:03:03,145 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000007 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:03:03,145 INFO AppSchedulingInfo:166 - checking for deactivate... 19:03:03,992 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0008_01_000007 19:03:04,993 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000008 Container Transitioned from NEW to ALLOCATED 19:03:04,993 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000008 19:03:04,993 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000008 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:03:05,155 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000008 Container Transitioned from ALLOCATED to ACQUIRED 19:03:06,163 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000008 Container Transitioned from ACQUIRED to RELEASED 19:03:06,163 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000008 in state: RELEASED event:RELEASED 19:03:06,163 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000008 19:03:06,164 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000008 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:03:06,164 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000008 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:03:06,164 INFO AppSchedulingInfo:166 - checking for deactivate... 19:03:06,993 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0008_01_000008 19:03:07,995 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000009 Container Transitioned from NEW to ALLOCATED 19:03:07,995 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000009 19:03:07,995 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0008_01_000009 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:03:08,178 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000009 Container Transitioned from ALLOCATED to ACQUIRED 19:03:08,193 INFO FSEditLog:674 - Number of transactions: 927 Total time for transactions(ms): 331 Number of transactions batched in Syncs: 14 Number of syncs: 624 SyncTimes(ms): 370 152 19:03:08,194 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f/output/_temporary dst=null perm=null 19:03:08,206 INFO clienttrace:1215 - src: /127.0.0.1:53375, dest: /127.0.0.1:52685, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119, duration: 11219911000 19:03:08,206 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:08,207 INFO clienttrace:1215 - src: /127.0.0.1:53374, dest: /127.0.0.1:52682, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119, duration: 11221362000 19:03:08,207 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:08,209 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1.jhist is closed by DFSClient_NONMAPREDUCE_1732519755_1 19:03:08,211 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:08,213 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741944_1120{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:08,215 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120 src: /127.0.0.1:53383 dest: /127.0.0.1:52685 19:03:08,216 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120 src: /127.0.0.1:53384 dest: /127.0.0.1:52682 19:03:08,219 INFO clienttrace:1215 - src: /127.0.0.1:53384, dest: /127.0.0.1:52682, bytes: 414, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120, duration: 1462000 19:03:08,219 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:08,219 INFO clienttrace:1215 - src: /127.0.0.1:53383, dest: /127.0.0.1:52685, bytes: 414, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120, duration: 1982000 19:03:08,219 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:08,221 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary_tmp is closed by DFSClient_NONMAPREDUCE_1732519755_1 19:03:08,225 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:08,230 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1.jhist dst=null perm=null 19:03:08,231 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist_tmp dst=null perm=null 19:03:08,234 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1.jhist dst=null perm=null 19:03:08,235 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist_tmp dst=null perm=null 19:03:08,236 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1.jhist dst=null perm=null 19:03:08,238 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:08,241 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53385, bytes: 23548, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119, duration: 240000 19:03:08,242 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741945_1121{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:08,244 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121 src: /127.0.0.1:53386 dest: /127.0.0.1:52682 19:03:08,245 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121 src: /127.0.0.1:53387 dest: /127.0.0.1:52685 19:03:08,248 INFO clienttrace:1215 - src: /127.0.0.1:53387, dest: /127.0.0.1:52685, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121, duration: 1351000 19:03:08,248 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:08,249 INFO clienttrace:1215 - src: /127.0.0.1:53386, dest: /127.0.0.1:52682, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121, duration: 1993000 19:03:08,249 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:08,251 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist_tmp is closed by DFSClient_NONMAPREDUCE_1732519755_1 19:03:08,252 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:08,253 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1_conf.xml dst=null perm=null 19:03:08,254 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml_tmp dst=null perm=null 19:03:08,255 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1_conf.xml dst=null perm=null 19:03:08,256 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml_tmp dst=null perm=null 19:03:08,257 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008/job_1394161202967_0008_1_conf.xml dst=null perm=null 19:03:08,260 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:08,263 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53388, bytes: 92383, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741941_1117, duration: 319000 19:03:08,265 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741946_1122{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:08,268 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122 src: /127.0.0.1:53389 dest: /127.0.0.1:52682 19:03:08,270 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122 src: /127.0.0.1:53390 dest: /127.0.0.1:52685 19:03:08,274 INFO clienttrace:1215 - src: /127.0.0.1:53390, dest: /127.0.0.1:52685, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122, duration: 2578000 19:03:08,274 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:08,274 INFO clienttrace:1215 - src: /127.0.0.1:53389, dest: /127.0.0.1:52682, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1732519755_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122, duration: 3260000 19:03:08,275 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:08,276 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_1732519755_1 19:03:08,278 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:08,281 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary perm=test:supergroup:rwxrwx--- 19:03:08,285 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml perm=test:supergroup:rwxrwx--- 19:03:08,287 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist perm=test:supergroup:rwxrwx--- 19:03:08,294 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0008_000001 with final state: FINISHING 19:03:08,294 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from RUNNING to FINAL_SAVING 19:03:08,294 INFO RMAppImpl:880 - Updating application application_1394161202967_0008 with final state: FINISHING 19:03:08,294 INFO RMAppImpl:643 - application_1394161202967_0008 State change from RUNNING to FINAL_SAVING 19:03:08,295 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0008 19:03:08,296 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from FINAL_SAVING to FINISHING 19:03:08,296 INFO RMAppImpl:643 - application_1394161202967_0008 State change from FINAL_SAVING to FINISHING 19:03:09,302 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0008 dst=null perm=null 19:03:09,633 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741941_1117, blk_1073741942_1118, blk_1073741943_1119, blk_1073741933_1109, blk_1073741934_1110, blk_1073741935_1111] 19:03:10,491 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:03:10,495 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741947_1123{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:10,497 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123 src: /127.0.0.1:53421 dest: /127.0.0.1:52682 19:03:10,498 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123 src: /127.0.0.1:53422 dest: /127.0.0.1:52685 19:03:10,500 INFO clienttrace:1215 - src: /127.0.0.1:53422, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_271293166_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123, duration: 993000 19:03:10,500 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,501 INFO clienttrace:1215 - src: /127.0.0.1:53421, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_271293166_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123, duration: 1544000 19:03:10,501 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,503 INFO StateChange:2959 - DIR* completeFile: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input/data.txt is closed by DFSClient_NONMAPREDUCE_271293166_8 19:03:10,512 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:03:10,514 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741948_1124{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:10,516 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124 src: /127.0.0.1:53423 dest: /127.0.0.1:52685 19:03:10,517 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124 src: /127.0.0.1:53424 dest: /127.0.0.1:52682 19:03:10,519 INFO clienttrace:1215 - src: /127.0.0.1:53424, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_271293166_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124, duration: 1070000 19:03:10,519 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,520 INFO clienttrace:1215 - src: /127.0.0.1:53423, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_271293166_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124, duration: 1704000 19:03:10,520 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,521 INFO StateChange:2959 - DIR* completeFile: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_271293166_8 19:03:10,523 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:03:10,524 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741949_1125{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:10,526 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125 src: /127.0.0.1:53425 dest: /127.0.0.1:52682 19:03:10,527 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125 src: /127.0.0.1:53426 dest: /127.0.0.1:52685 19:03:10,530 INFO clienttrace:1215 - src: /127.0.0.1:53426, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_271293166_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125, duration: 1124000 19:03:10,530 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,530 INFO clienttrace:1215 - src: /127.0.0.1:53425, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_271293166_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125, duration: 1682000 19:03:10,530 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,532 INFO StateChange:2959 - DIR* completeFile: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_271293166_8 19:03:10,536 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java dst=null perm=null 19:03:10,537 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:03:10,538 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120 19:03:10,538 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121 19:03:10,538 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125 19:03:10,538 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java.tmp dst=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java perm=test:supergroup:rwxrwxrwx 19:03:10,539 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123 19:03:10,539 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741943_1119 19:03:10,539 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/workflow.xml dst=null perm=null 19:03:10,540 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122 19:03:10,540 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124 19:03:10,629 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/workflow.xml dst=null perm=null 19:03:10,693 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java dst=null perm=test:supergroup:rwxrwxrwx 19:03:10,695 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:10,707 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741950_1126{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:10,710 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126 src: /127.0.0.1:53428 dest: /127.0.0.1:52685 19:03:10,712 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126 src: /127.0.0.1:53429 dest: /127.0.0.1:52682 19:03:10,716 INFO clienttrace:1215 - src: /127.0.0.1:53429, dest: /127.0.0.1:52682, bytes: 77267, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126, duration: 1971000 19:03:10,716 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,717 INFO clienttrace:1215 - src: /127.0.0.1:53428, dest: /127.0.0.1:52685, bytes: 77267, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126, duration: 2678000 19:03:10,717 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,719 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action.xml is closed by DFSClient_NONMAPREDUCE_1178951046_8 19:03:10,721 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input dst=null perm=test:supergroup:rwxrwxrwx 19:03:10,723 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:03:10,726 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741951_1127{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:10,729 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127 src: /127.0.0.1:53430 dest: /127.0.0.1:52685 19:03:10,730 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127 src: /127.0.0.1:53431 dest: /127.0.0.1:52682 19:03:10,733 INFO clienttrace:1215 - src: /127.0.0.1:53431, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127, duration: 1357000 19:03:10,734 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,734 INFO clienttrace:1215 - src: /127.0.0.1:53430, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127, duration: 2147000 19:03:10,735 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,737 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_1178951046_8 19:03:10,769 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161390769, maxDate=1394766190769, sequenceNumber=25, masterKeyId=2 19:03:10,769 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 25 19:03:10,772 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/0000000-140306190310205-oozie-rkan-W@mr-action dst=null perm=null 19:03:10,773 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161390773, maxDate=1394766190773, sequenceNumber=26, masterKeyId=2 19:03:10,774 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 26 19:03:10,803 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:03:10,804 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:10,806 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:10,808 INFO ClientRMService:229 - Allocated new applicationId: 9 19:03:10,810 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009 dst=null perm=null 19:03:10,812 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009 dst=null perm=test:supergroup:rwxrwxrwx 19:03:10,815 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009 dst=null perm=test:supergroup:rwx------ 19:03:10,817 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:10,819 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:10,821 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:10,823 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:10,824 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:03:10,826 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:03:10,827 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:03:10,830 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:03:10,832 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:03:10,834 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:10,839 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:10,841 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib dst=null perm=null 19:03:10,843 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app dst=null perm=null 19:03:10,845 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c dst=null perm=null 19:03:10,847 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:03:10,849 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:03:10,851 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:03:10,852 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:03:10,857 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:03:10,860 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=test:supergroup:rw-r--r-- 19:03:10,861 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split 19:03:10,862 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=null 19:03:10,864 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741952_1128{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:10,867 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128 src: /127.0.0.1:53454 dest: /127.0.0.1:52685 19:03:10,868 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128 src: /127.0.0.1:53455 dest: /127.0.0.1:52682 19:03:10,872 INFO clienttrace:1215 - src: /127.0.0.1:53455, dest: /127.0.0.1:52682, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128, duration: 2246000 19:03:10,873 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,873 INFO clienttrace:1215 - src: /127.0.0.1:53454, dest: /127.0.0.1:52685, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128, duration: 2710000 19:03:10,874 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,876 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split is closed by DFSClient_NONMAPREDUCE_1178951046_8 19:03:10,879 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:03:10,881 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:03:10,884 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741953_1129{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:10,886 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129 src: /127.0.0.1:53459 dest: /127.0.0.1:52685 19:03:10,888 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129 src: /127.0.0.1:53460 dest: /127.0.0.1:52682 19:03:10,893 INFO clienttrace:1215 - src: /127.0.0.1:53460, dest: /127.0.0.1:52682, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129, duration: 2809000 19:03:10,893 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,895 INFO clienttrace:1215 - src: /127.0.0.1:53459, dest: /127.0.0.1:52685, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129, duration: 4475000 19:03:10,895 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,898 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_1178951046_8 19:03:10,901 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:10,904 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:03:10,915 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741954_1130{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:10,918 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130 src: /127.0.0.1:53466 dest: /127.0.0.1:52685 19:03:10,919 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130 src: /127.0.0.1:53467 dest: /127.0.0.1:52682 19:03:10,924 INFO clienttrace:1215 - src: /127.0.0.1:53467, dest: /127.0.0.1:52682, bytes: 79931, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130, duration: 2645000 19:03:10,924 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:10,925 INFO clienttrace:1215 - src: /127.0.0.1:53466, dest: /127.0.0.1:52685, bytes: 79931, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130, duration: 3508000 19:03:10,925 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:10,928 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml is closed by DFSClient_NONMAPREDUCE_1178951046_8 19:03:10,931 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009 dst=null perm=null 19:03:10,932 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:10,933 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:10,935 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=null 19:03:10,936 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=null 19:03:10,937 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=null 19:03:10,938 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=null 19:03:10,942 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:10,943 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:10,945 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:10,946 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:10,947 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:10,948 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:10,950 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:10,951 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:10,953 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:10,956 INFO ClientRMService:342 - Application with id 9 submitted by user test 19:03:10,956 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0009 19:03:10,956 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0009 19:03:10,956 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0009 19:03:10,956 INFO RMAppImpl:643 - application_1394161202967_0009 State change from NEW to NEW_SAVING 19:03:10,956 INFO RMAppImpl:643 - application_1394161202967_0009 State change from NEW_SAVING to SUBMITTED 19:03:10,957 INFO FairScheduler:638 - Accepted application application_1394161202967_0009 from user: test, in queue: default, currently num of applications: 2 19:03:10,957 INFO RMAppImpl:643 - application_1394161202967_0009 State change from SUBMITTED to ACCEPTED 19:03:10,957 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0009_000001 19:03:10,958 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from NEW to SUBMITTED 19:03:10,958 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0009_000001 to scheduler from user: test 19:03:10,958 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from SUBMITTED to SCHEDULED 19:03:10,999 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0009 19:03:10,999 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000001 Container Transitioned from NEW to RESERVED 19:03:10,999 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0009 reserved container container_1394161202967_0009_01_000001 on node host: 172.16.1.64:52703 #containers=2 available= used=, currently has 1 at priority 0; currentReservation 2048 19:03:10,999 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0009_01_000001 on node host: 172.16.1.64:52703 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@6b1945ee 19:03:11,010 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161391010, maxDate=1394766191010, sequenceNumber=27, masterKeyId=2 19:03:11,010 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 27 19:03:11,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741941_1117 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741941 for deletion 19:03:11,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741942_1118 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741942 for deletion 19:03:11,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741943_1119 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741943 for deletion 19:03:11,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741933_1109 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741933 for deletion 19:03:11,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741941_1117 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741941 19:03:11,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741934_1110 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741934 for deletion 19:03:11,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741942_1118 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741942 19:03:11,021 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741935_1111 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741935 for deletion 19:03:11,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741943_1119 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741943 19:03:11,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741934_1110 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741934 19:03:11,021 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741933_1109 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741933 19:03:11,022 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741935_1111 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741935 19:03:11,068 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000002 Container Transitioned from NEW to ALLOCATED 19:03:11,068 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000002 19:03:11,068 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0009_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:03:11,069 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:03:11,069 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0009 AttemptId: appattempt_1394161202967_0009_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0009_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:03:11,069 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:03:11,069 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:03:11,069 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0009_000001 19:03:11,071 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0009_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0009_000001 19:03:11,071 INFO AMLauncher:190 - Command to launch container container_1394161202967_0009_01_000002 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:03:11,077 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0009_01_000002 by user test 19:03:11,078 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0009 19:03:11,078 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000002 19:03:11,078 INFO Application:450 - Application application_1394161202967_0009 transitioned from NEW to INITING 19:03:11,078 INFO Application:296 - Adding container_1394161202967_0009_01_000002 to application application_1394161202967_0009 19:03:11,078 INFO Application:450 - Application application_1394161202967_0009 transitioned from INITING to RUNNING 19:03:11,079 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0009_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0009_000001 19:03:11,079 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from ALLOCATED to LAUNCHED 19:03:11,079 INFO Container:888 - Container container_1394161202967_0009_01_000002 transitioned from NEW to LOCALIZING 19:03:11,079 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0009 19:03:11,079 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:03:11,079 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split transitioned from INIT to DOWNLOADING 19:03:11,079 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml transitioned from INIT to DOWNLOADING 19:03:11,079 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:03:11,079 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0009_01_000002 19:03:11,080 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar, 1394161390521, FILE, null } 19:03:11,143 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/nmPrivate/container_1394161202967_0009_01_000002.tokens. Credentials list: 19:03:11,226 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:11,249 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:11,270 INFO DefaultContainerExecutor:469 - Initializing user test 19:03:11,275 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib dst=null perm=null 19:03:11,297 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741946_1122 19:03:11,297 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app dst=null perm=null 19:03:11,316 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126 19:03:11,316 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123 19:03:11,339 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c dst=null perm=null 19:03:11,339 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130 19:03:11,346 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120 19:03:11,365 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124 19:03:11,365 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:03:11,365 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741949_1125 19:03:11,371 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129 19:03:11,385 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127 19:03:11,391 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741945_1121 19:03:11,391 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:03:11,391 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128 19:03:11,419 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:03:11,460 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:11,480 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:11,584 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53472, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124, duration: 394000 19:03:11,745 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/nmPrivate/container_1394161202967_0009_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000002.tokens 19:03:11,745 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/filecache/17/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:03:11,745 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0009 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0009 19:03:11,888 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=null 19:03:11,889 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=null 19:03:11,890 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=null 19:03:11,905 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53472, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1051941785_3077, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129, duration: 224000 19:03:11,946 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0009/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:03:11,985 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=null 19:03:11,986 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=null 19:03:11,987 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=null 19:03:11,998 INFO FairScheduler:1042 - Releasing reservation that cannot be satisfied for application appattempt_1394161202967_0009_000001 on node host: 172.16.1.64:52703 #containers=2 available= used= 19:03:11,999 INFO FSSchedulerApp:123 - Application application_1394161202967_0009 unreserved on node host: 172.16.1.64:52703 #containers=2 available= used=, currently has 0 at priority 0; currentReservation 19:03:12,000 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53475, bytes: 176, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1051941785_3077, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128, duration: 293000 19:03:12,072 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:03:12,074 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0009/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:03:12,120 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:12,121 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:12,122 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:12,139 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53472, bytes: 80559, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1051941785_3077, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130, duration: 530000 19:03:12,186 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0009/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:03:12,186 INFO Container:888 - Container container_1394161202967_0009_01_000002 transitioned from LOCALIZING to LOCALIZED 19:03:12,257 INFO Container:888 - Container container_1394161202967_0009_01_000002 transitioned from LOCALIZED to RUNNING 19:03:12,258 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:12,442 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000002/default_container_executor.sh] 19:03:12,634 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741941_1117, blk_1073741942_1118, blk_1073741943_1119, blk_1073741933_1109, blk_1073741934_1110, blk_1073741935_1111] 19:03:14,023 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741941_1117 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741941 for deletion 19:03:14,023 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741942_1118 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741942 for deletion 19:03:14,023 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741943_1119 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741943 for deletion 19:03:14,023 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741941_1117 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741941 19:03:14,023 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741933_1109 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741933 for deletion 19:03:14,023 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741942_1118 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741942 19:03:14,024 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741934_1110 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741934 for deletion 19:03:14,024 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741943_1119 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741943 19:03:14,024 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741935_1111 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741935 for deletion 19:03:14,024 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741934_1110 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741934 19:03:14,024 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741933_1109 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741933 19:03:14,024 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741935_1111 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741935 19:03:15,078 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:15,112 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/COMMIT_STARTED dst=null perm=null 19:03:15,113 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/COMMIT_SUCCESS dst=null perm=null 19:03:15,114 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/COMMIT_FAIL dst=null perm=null 19:03:15,194 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009 dst=null perm=null 19:03:15,196 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:03:15,198 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:03:15,373 INFO ContainerLaunch:322 - Container container_1394161202967_0008_01_000002 succeeded 19:03:15,374 INFO Container:888 - Container container_1394161202967_0008_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:03:15,374 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0008_01_000002 19:03:15,394 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000002 19:03:15,394 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000002 19:03:15,394 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000002 19:03:15,394 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0008/container_1394161202967_0008_01_000002 19:03:15,395 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000002 19:03:15,395 INFO Container:888 - Container container_1394161202967_0008_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 19:03:15,395 INFO Application:339 - Removing container_1394161202967_0008_01_000002 from application application_1394161202967_0008 19:03:15,395 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:15,395 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0008 19:03:15,544 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127 19:03:15,544 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129 19:03:15,544 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128 19:03:15,545 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130 19:03:15,546 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126 19:03:15,671 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=null 19:03:15,680 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.splitmetainfo dst=null perm=null 19:03:15,744 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53507, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741953_1129, duration: 206000 19:03:16,001 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0008_01_000002 19:03:16,001 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000002 Container Transitioned from RUNNING to COMPLETED 19:03:16,001 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0008_000001 19:03:16,001 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000002 in state: COMPLETED event:FINISHED 19:03:16,002 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0008_000001 State change from FINISHING to FINISHED 19:03:16,002 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000002 19:03:16,002 INFO RMAppImpl:643 - application_1394161202967_0008 State change from FINISHING to FINISHED 19:03:16,002 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:03:16,002 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000002 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:03:16,002 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0008 19:03:16,002 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0008_000001 19:03:16,002 INFO FairScheduler:720 - Application appattempt_1394161202967_0008_000001 is done. finalState=FINISHED 19:03:16,002 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0008,name=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190226247-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0008/jobhistory/job/job_1394161202967_0008,appMasterHost=rkanter-mbp.local,startTime=1394161362306,finishTime=1394161388294,finalStatus=FAILED 19:03:16,002 INFO RMContainerImpl:220 - container_1394161202967_0008_01_000009 Container Transitioned from ACQUIRED to KILLED 19:03:16,002 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0008_01_000009 in state: KILLED event:KILL 19:03:16,002 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0008 CONTAINERID=container_1394161202967_0008_01_000009 19:03:16,002 INFO FSSchedulerNode:162 - Released container container_1394161202967_0008_01_000009 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:03:16,002 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0008_000001 released container container_1394161202967_0008_01_000009 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: KILL 19:03:16,003 INFO AppSchedulingInfo:108 - Application application_1394161202967_0008 requests cleared 19:03:16,008 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0008_01_000002 19:03:16,072 INFO Application:450 - Application application_1394161202967_0008 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:03:16,072 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0008 19:03:16,072 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0008 19:03:16,072 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0008 19:03:16,072 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0008 19:03:16,073 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0008 19:03:16,073 INFO Application:450 - Application application_1394161202967_0008 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:03:16,073 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0008, with delay of 10800 seconds 19:03:17,002 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0008_01_000009 19:03:17,002 INFO Application:450 - Application application_1394161202967_0008 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:03:17,002 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0008 19:03:17,002 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0008 19:03:17,003 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0008 19:03:17,003 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0008 19:03:17,003 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0008 19:03:17,003 INFO Application:450 - Application application_1394161202967_0008 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:03:17,003 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0008, with delay of 10800 seconds 19:03:17,295 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0009_000001 19:03:17,296 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0009 APPATTEMPTID=appattempt_1394161202967_0009_000001 19:03:17,296 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from LAUNCHED to RUNNING 19:03:17,296 INFO RMAppImpl:643 - application_1394161202967_0009 State change from ACCEPTED to RUNNING 19:03:17,364 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:03:17,370 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=test:supergroup:rwxrwxrwx 19:03:17,438 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:17,661 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741955_1131{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:17,739 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131 src: /127.0.0.1:53523 dest: /127.0.0.1:52685 19:03:17,741 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131 src: /127.0.0.1:53524 dest: /127.0.0.1:52682 19:03:17,790 INFO clienttrace:1215 - src: /127.0.0.1:53524, dest: /127.0.0.1:52682, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131, duration: 47367000 19:03:17,790 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:17,791 INFO clienttrace:1215 - src: /127.0.0.1:53523, dest: /127.0.0.1:52685, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131, duration: 48296000 19:03:17,791 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:17,796 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1_conf.xml is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:17,836 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009.summary dst=null perm=test:supergroup:rw-rw-rw- 19:03:17,840 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741956_1132{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:17,843 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132 src: /127.0.0.1:53525 dest: /127.0.0.1:52685 19:03:17,844 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132 src: /127.0.0.1:53526 dest: /127.0.0.1:52682 19:03:17,848 INFO clienttrace:1215 - src: /127.0.0.1:53526, dest: /127.0.0.1:52682, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132, duration: 2274000 19:03:17,849 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:17,849 INFO clienttrace:1215 - src: /127.0.0.1:53525, dest: /127.0.0.1:52685, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132, duration: 3036000 19:03:17,850 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:17,852 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009.summary is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:19,004 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000003 Container Transitioned from NEW to ALLOCATED 19:03:19,004 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000003 19:03:19,004 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0009_01_000003 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:03:19,358 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:03:19,441 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:19,442 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:19,466 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:19,469 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:19,471 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:19,486 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:19,488 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:19,490 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:19,491 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:19,492 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:19,493 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:19,551 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0009_01_000003 by user test 19:03:19,552 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0009 19:03:19,552 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000003 19:03:19,552 INFO Application:450 - Application application_1394161202967_0009 transitioned from NEW to INITING 19:03:19,552 INFO Application:296 - Adding container_1394161202967_0009_01_000003 to application application_1394161202967_0009 19:03:19,552 INFO Application:450 - Application application_1394161202967_0009 transitioned from INITING to RUNNING 19:03:19,553 INFO Container:888 - Container container_1394161202967_0009_01_000003 transitioned from NEW to LOCALIZING 19:03:19,553 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0009 19:03:19,553 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0009 19:03:19,553 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:03:19,553 INFO ShuffleHandler:263 - Added token for job_1394161202967_0009 19:03:19,553 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml transitioned from INIT to DOWNLOADING 19:03:19,553 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:03:19,553 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0009_01_000003 19:03:19,554 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar, 1394161390521, FILE, null } 19:03:19,649 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/nmPrivate/container_1394161202967_0009_01_000003.tokens. Credentials list: 19:03:19,684 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:19,710 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:19,762 INFO DefaultContainerExecutor:469 - Initializing user test 19:03:19,762 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib dst=null perm=null 19:03:19,787 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app dst=null perm=null 19:03:19,814 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c dst=null perm=null 19:03:19,843 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:03:19,875 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:03:19,914 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:03:19,951 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:19,995 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:20,030 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:03:20,131 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53534, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741948_1124, duration: 14390000 19:03:20,286 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/filecache/17/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:03:20,299 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/nmPrivate/container_1394161202967_0009_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000003.tokens 19:03:20,300 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0009 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0009 19:03:20,404 INFO AppSchedulingInfo:166 - checking for deactivate... 19:03:20,525 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:20,528 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:20,536 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml dst=null perm=null 19:03:20,571 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53534, bytes: 80559, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-619331797_3188, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741954_1130, duration: 546000 19:03:20,597 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131 19:03:20,618 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132 19:03:20,715 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0009/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:03:20,716 INFO Container:888 - Container container_1394161202967_0009_01_000003 transitioned from LOCALIZING to LOCALIZED 19:03:20,791 INFO Container:888 - Container container_1394161202967_0009_01_000003 transitioned from LOCALIZED to RUNNING 19:03:20,792 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:21,057 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000003/default_container_executor.sh] 19:03:21,401 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741956_1132 19:03:21,402 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131 19:03:24,043 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job.split dst=null perm=null 19:03:24,167 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53544, bytes: 176, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741952_1128, duration: 304000 19:03:24,251 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=null 19:03:24,275 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0009_m_000000_0/part-00000 dst=null perm=test:supergroup:rw-rw-rw- 19:03:24,313 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/0000000-140306190310205-oozie-rkan-W@mr-action dst=null perm=null 19:03:24,319 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/0000000-140306190310205-oozie-rkan-W@mr-action dst=null perm=test:supergroup:rw-rw-rw- 19:03:24,338 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/0000000-140306190310205-oozie-rkan-W@mr-action. BP-1007909283-172.16.1.64-1394161187028 blk_1073741957_1133{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:24,403 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133 src: /127.0.0.1:53545 dest: /127.0.0.1:52682 19:03:24,404 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133 src: /127.0.0.1:53546 dest: /127.0.0.1:52685 19:03:24,429 INFO clienttrace:1215 - src: /127.0.0.1:53546, dest: /127.0.0.1:52685, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133, duration: 23297000 19:03:24,429 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:24,430 INFO clienttrace:1215 - src: /127.0.0.1:53545, dest: /127.0.0.1:52682, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133, duration: 23858000 19:03:24,430 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:24,433 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/0000000-140306190310205-oozie-rkan-W@mr-action is closed by DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1 19:03:24,439 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53544, bytes: 9, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741951_1127, duration: 192000 19:03:24,495 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:03:24,497 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:03:24,508 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53547, bytes: 77871, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741950_1126, duration: 323000 19:03:25,238 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output dst=null perm=null 19:03:25,242 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:25,243 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:25,279 INFO ClientRMService:229 - Allocated new applicationId: 10 19:03:25,296 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010 dst=null perm=null 19:03:25,304 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010 dst=null perm=test:supergroup:rwxrwxrwx 19:03:25,314 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010 dst=null perm=test:supergroup:rwx------ 19:03:25,329 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:25,331 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:25,333 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:25,335 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:25,337 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:03:25,338 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:03:25,339 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:03:25,340 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:03:25,342 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:03:25,344 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:25,346 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:25,347 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib dst=null perm=null 19:03:25,349 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app dst=null perm=null 19:03:25,350 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c dst=null perm=null 19:03:25,351 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:03:25,378 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input dst=null perm=null 19:03:25,382 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input dst=null perm=null 19:03:25,389 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input dst=null perm=null 19:03:25,443 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:03:25,446 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=test:supergroup:rw-r--r-- 19:03:25,455 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split 19:03:25,455 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:25,470 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741958_1134{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:25,474 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134 src: /127.0.0.1:53549 dest: /127.0.0.1:52685 19:03:25,476 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134 src: /127.0.0.1:53550 dest: /127.0.0.1:52682 19:03:25,480 INFO clienttrace:1215 - src: /127.0.0.1:53550, dest: /127.0.0.1:52682, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134, duration: 2033000 19:03:25,480 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:25,481 INFO clienttrace:1215 - src: /127.0.0.1:53549, dest: /127.0.0.1:52685, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134, duration: 2865000 19:03:25,481 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:25,483 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split is closed by DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1 19:03:25,486 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:03:25,488 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:03:25,496 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741959_1135{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:25,499 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135 src: /127.0.0.1:53551 dest: /127.0.0.1:52685 19:03:25,501 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135 src: /127.0.0.1:53552 dest: /127.0.0.1:52682 19:03:25,504 INFO clienttrace:1215 - src: /127.0.0.1:53552, dest: /127.0.0.1:52682, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135, duration: 2036000 19:03:25,505 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:25,505 INFO clienttrace:1215 - src: /127.0.0.1:53551, dest: /127.0.0.1:52685, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135, duration: 2738000 19:03:25,506 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:25,507 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo is closed by DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1 19:03:25,523 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:25,525 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:03:25,561 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741960_1136{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:25,564 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136 src: /127.0.0.1:53553 dest: /127.0.0.1:52682 19:03:25,565 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136 src: /127.0.0.1:53554 dest: /127.0.0.1:52685 19:03:25,569 INFO clienttrace:1215 - src: /127.0.0.1:53554, dest: /127.0.0.1:52685, bytes: 77360, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136, duration: 1927000 19:03:25,569 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:25,569 INFO clienttrace:1215 - src: /127.0.0.1:53553, dest: /127.0.0.1:52682, bytes: 77360, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136, duration: 2703000 19:03:25,569 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:25,571 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml is closed by DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1 19:03:25,629 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136 19:03:25,629 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133 19:03:25,629 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134 19:03:25,630 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135 19:03:25,767 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010 dst=null perm=null 19:03:25,969 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:25,970 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:25,974 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:25,974 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:25,975 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=null 19:03:25,976 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=null 19:03:25,980 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:25,982 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:25,983 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:25,984 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:25,985 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:25,987 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:25,988 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:25,989 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:25,990 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:26,046 INFO ClientRMService:342 - Application with id 10 submitted by user test 19:03:26,047 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0010 19:03:26,047 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0010 19:03:26,047 INFO RMAppImpl:643 - application_1394161202967_0010 State change from NEW to NEW_SAVING 19:03:26,047 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0010 19:03:26,047 INFO RMAppImpl:643 - application_1394161202967_0010 State change from NEW_SAVING to SUBMITTED 19:03:26,048 INFO FairScheduler:638 - Accepted application application_1394161202967_0010 from user: test, in queue: default, currently num of applications: 2 19:03:26,048 INFO RMAppImpl:643 - application_1394161202967_0010 State change from SUBMITTED to ACCEPTED 19:03:26,048 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0010_000001 19:03:26,048 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from NEW to SUBMITTED 19:03:26,048 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0010_000001 to scheduler from user: test 19:03:26,049 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from SUBMITTED to SCHEDULED 19:03:26,114 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000001 Container Transitioned from NEW to ALLOCATED 19:03:26,114 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000001 19:03:26,115 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0010_01_000001 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:03:26,115 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:03:26,115 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0010 AttemptId: appattempt_1394161202967_0010_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0010_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:03:26,115 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:03:26,115 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:03:26,116 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0010_000001 19:03:26,117 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0010_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0010_000001 19:03:26,117 INFO AMLauncher:190 - Command to launch container container_1394161202967_0010_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:03:26,122 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0010_01_000001 by user test 19:03:26,123 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0010 19:03:26,123 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000001 19:03:26,123 INFO Application:450 - Application application_1394161202967_0010 transitioned from NEW to INITING 19:03:26,123 INFO Application:296 - Adding container_1394161202967_0010_01_000001 to application application_1394161202967_0010 19:03:26,123 INFO Application:450 - Application application_1394161202967_0010 transitioned from INITING to RUNNING 19:03:26,123 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0010_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0010_000001 19:03:26,124 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from ALLOCATED to LAUNCHED 19:03:26,124 INFO Container:888 - Container container_1394161202967_0010_01_000001 transitioned from NEW to LOCALIZING 19:03:26,124 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0010 19:03:26,124 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:03:26,124 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split transitioned from INIT to DOWNLOADING 19:03:26,124 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml transitioned from INIT to DOWNLOADING 19:03:26,125 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0010_01_000001 19:03:26,172 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/nmPrivate/container_1394161202967_0010_01_000001.tokens. Credentials list: 19:03:26,216 INFO DefaultContainerExecutor:469 - Initializing user test 19:03:26,321 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=test:supergroup:rw-rw-rw- 19:03:26,386 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq. BP-1007909283-172.16.1.64-1394161187028 blk_1073741961_1137{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:26,440 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137 src: /127.0.0.1:53558 dest: /127.0.0.1:52682 19:03:26,441 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741957_1133 19:03:26,451 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136 19:03:26,471 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135 19:03:26,471 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137 src: /127.0.0.1:53559 dest: /127.0.0.1:52685 19:03:26,471 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134 19:03:26,523 INFO clienttrace:1215 - src: /127.0.0.1:53559, dest: /127.0.0.1:52685, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, duration: 7219000 19:03:26,524 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:26,550 INFO clienttrace:1215 - src: /127.0.0.1:53558, dest: /127.0.0.1:52682, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, duration: 52801000 19:03:26,557 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:26,589 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq is closed by DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1 19:03:26,630 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0009_m_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0009_m_000000_0_837285895_1 19:03:26,651 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/nmPrivate/container_1394161202967_0010_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000001.tokens 19:03:26,651 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0010 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0010 19:03:26,668 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0009_m_000000_0 dst=null perm=null 19:03:26,724 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0009_m_000000_0 dst=null perm=null 19:03:26,731 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0009_m_000000 dst=null perm=null 19:03:26,778 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0009_m_000000_0 dst=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0009_m_000000 perm=test:supergroup:rwxrwxrwx 19:03:26,810 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0009_01_000003 19:03:26,811 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000003 19:03:26,811 INFO Container:888 - Container container_1394161202967_0009_01_000003 transitioned from RUNNING to KILLING 19:03:26,811 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0009_01_000003 19:03:26,867 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0009_01_000003 is : 143 19:03:26,868 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741962_1138{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:26,868 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:03:26,873 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138 src: /127.0.0.1:53560 dest: /127.0.0.1:52685 19:03:26,874 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:26,875 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138 src: /127.0.0.1:53561 dest: /127.0.0.1:52682 19:03:26,880 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1.jhist for DFSClient_NONMAPREDUCE_-969706877_1 19:03:26,880 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=null 19:03:26,890 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:03:26,892 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:03:26,896 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0009_m_000000 dst=null perm=null 19:03:26,898 INFO Container:888 - Container container_1394161202967_0009_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:03:26,899 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/part-00000 dst=null perm=null 19:03:26,899 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000003 19:03:26,899 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000003 19:03:26,900 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000003 19:03:26,900 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000003 19:03:26,900 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000003 19:03:26,908 INFO Container:888 - Container container_1394161202967_0009_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:03:26,939 INFO Application:339 - Removing container_1394161202967_0009_01_000003 from application application_1394161202967_0009 19:03:26,939 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:26,939 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0009 19:03:26,965 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0009_m_000000/part-00000 dst=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/part-00000 perm=test:supergroup:rw-rw-rw- 19:03:27,012 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_temporary dst=null perm=null 19:03:27,024 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=null 19:03:27,024 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:03:27,025 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=null 19:03:27,026 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:27,027 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=null 19:03:27,028 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:03:27,035 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:27,061 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53564, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1822539198_3286, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135, duration: 209000 19:03:27,145 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:03:27,168 INFO clienttrace:1215 - src: /127.0.0.1:53561, dest: /127.0.0.1:52682, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138, duration: 262679000 19:03:27,174 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:27,208 INFO clienttrace:1215 - src: /127.0.0.1:53560, dest: /127.0.0.1:52685, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138, duration: 324564000 19:03:27,227 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:27,231 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1.jhist is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:27,233 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0010/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:03:27,235 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:27,260 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741963_1139{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:27,294 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139 src: /127.0.0.1:53565 dest: /127.0.0.1:52685 19:03:27,334 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139 src: /127.0.0.1:53566 dest: /127.0.0.1:52682 19:03:27,356 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:27,357 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:27,357 INFO clienttrace:1215 - src: /127.0.0.1:53566, dest: /127.0.0.1:52682, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139, duration: 2112000 19:03:27,358 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:27,358 INFO clienttrace:1215 - src: /127.0.0.1:53565, dest: /127.0.0.1:52685, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139, duration: 2775000 19:03:27,359 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:27,359 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:27,361 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary_tmp is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:27,388 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:27,392 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53568, bytes: 283, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1822539198_3286, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134, duration: 215000 19:03:27,401 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1.jhist dst=null perm=null 19:03:27,465 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist_tmp dst=null perm=null 19:03:27,499 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1.jhist dst=null perm=null 19:03:27,536 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist_tmp dst=null perm=null 19:03:27,573 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1.jhist dst=null perm=null 19:03:27,578 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0010/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:03:27,584 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:27,647 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53569, bytes: 20872, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138, duration: 22652000 19:03:27,656 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741964_1140{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:27,706 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140 src: /127.0.0.1:53570 dest: /127.0.0.1:52685 19:03:27,709 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:27,712 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140 src: /127.0.0.1:53571 dest: /127.0.0.1:52682 19:03:27,712 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:27,714 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:27,716 INFO clienttrace:1215 - src: /127.0.0.1:53571, dest: /127.0.0.1:52682, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140, duration: 2559000 19:03:27,716 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:27,717 INFO clienttrace:1215 - src: /127.0.0.1:53570, dest: /127.0.0.1:52685, bytes: 20708, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140, duration: 3495000 19:03:27,717 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:27,728 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist_tmp is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:27,751 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:27,752 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53568, bytes: 77968, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1822539198_3286, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136, duration: 553000 19:03:27,753 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1_conf.xml dst=null perm=null 19:03:27,790 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml_tmp dst=null perm=null 19:03:27,824 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0009_01_000003 19:03:27,833 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1_conf.xml dst=null perm=null 19:03:27,857 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000003 Container Transitioned from RUNNING to COMPLETED 19:03:27,857 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0009_01_000003 in state: COMPLETED event:FINISHED 19:03:27,857 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000003 19:03:27,858 INFO FSSchedulerNode:162 - Released container container_1394161202967_0009_01_000003 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:03:27,858 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0009_000001 released container container_1394161202967_0009_01_000003 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: FINISHED 19:03:27,888 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml_tmp dst=null perm=null 19:03:27,930 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009/job_1394161202967_0009_1_conf.xml dst=null perm=null 19:03:27,935 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0010/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:03:27,935 INFO Container:888 - Container container_1394161202967_0010_01_000001 transitioned from LOCALIZING to LOCALIZED 19:03:27,937 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:27,942 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53573, bytes: 93643, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741955_1131, duration: 562000 19:03:27,946 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741965_1141{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:27,949 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141 src: /127.0.0.1:53574 dest: /127.0.0.1:52682 19:03:27,954 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141 src: /127.0.0.1:53575 dest: /127.0.0.1:52685 19:03:27,959 INFO clienttrace:1215 - src: /127.0.0.1:53575, dest: /127.0.0.1:52685, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141, duration: 2583000 19:03:27,959 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:27,960 INFO clienttrace:1215 - src: /127.0.0.1:53574, dest: /127.0.0.1:52682, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-969706877_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141, duration: 3709000 19:03:27,960 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:27,962 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_-969706877_1 19:03:27,965 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:27,968 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary perm=test:supergroup:rwxrwx--- 19:03:27,971 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml perm=test:supergroup:rwxrwx--- 19:03:28,004 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist perm=test:supergroup:rwxrwx--- 19:03:28,068 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0009_000001 with final state: FINISHING 19:03:28,068 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from RUNNING to FINAL_SAVING 19:03:28,069 INFO RMAppImpl:880 - Updating application application_1394161202967_0009 with final state: FINISHING 19:03:28,069 INFO RMAppImpl:643 - application_1394161202967_0009 State change from RUNNING to FINAL_SAVING 19:03:28,069 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0009 19:03:28,078 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from FINAL_SAVING to FINISHING 19:03:28,079 INFO RMAppImpl:643 - application_1394161202967_0009 State change from FINAL_SAVING to FINISHING 19:03:28,102 INFO Container:888 - Container container_1394161202967_0010_01_000001 transitioned from LOCALIZED to RUNNING 19:03:28,102 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:28,280 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000001/default_container_executor.sh] 19:03:29,078 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0009 dst=null perm=null 19:03:29,340 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:29,342 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:29,344 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:29,346 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53568, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, duration: 204000 19:03:29,382 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161409382, maxDate=1394766209382, sequenceNumber=28, masterKeyId=2 19:03:29,382 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 28 19:03:29,389 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:03:29,391 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:03:29,393 INFO CompletedJob:95 - Loading job: job_1394161202967_0009 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist 19:03:29,393 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist] 19:03:29,393 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary dst=null perm=null 19:03:29,394 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist dst=null perm=null 19:03:29,395 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist dst=null perm=null 19:03:29,396 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53564, bytes: 418, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741944_1120, duration: 248000 19:03:29,396 INFO JobSummary:372 - jobId=job_1394161202967_0008,submitTime=1394161362306,launchTime=1394161369951,firstMapTaskLaunchTime=1394161372142,firstReduceTaskLaunchTime=0,finishTime=1394161388186,resourcesPerMap=1024,resourcesPerReduce=1024,numMaps=2,numReduces=1,user=test,queue=default,status=FAILED,mapSlotSeconds=8,reduceSlotSeconds=0,jobName=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190226247-oozie-rkan-W 19:03:29,396 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary] 19:03:29,397 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008.summary dst=null perm=null 19:03:29,397 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53564, bytes: 20872, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140, duration: 255000 19:03:29,399 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:03:29,399 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist 19:03:29,401 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0008-1394161362306-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161388186-2-0-FAILED-root.test-1394161369951.jhist perm=test:supergroup:rwxrwx--- 19:03:29,401 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0008_conf.xml 19:03:29,404 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0008_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0008_conf.xml perm=test:supergroup:rwxrwx--- 19:03:29,421 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary dst=null perm=null 19:03:29,424 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53564, bytes: 420, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139, duration: 246000 19:03:29,424 INFO JobSummary:372 - jobId=job_1394161202967_0009,submitTime=1394161390955,launchTime=1394161397327,firstMapTaskLaunchTime=1394161399558,firstReduceTaskLaunchTime=0,finishTime=1394161407058,resourcesPerMap=1024,resourcesPerReduce=0,numMaps=1,numReduces=0,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=7,reduceSlotSeconds=0,jobName=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190310205-oozie-rkan-W 19:03:29,424 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary] 19:03:29,425 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009.summary dst=null perm=null 19:03:29,427 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:03:29,427 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist 19:03:29,429 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0009-1394161390955-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161407058-1-0-SUCCEEDED-root.test-1394161397327.jhist perm=test:supergroup:rwxrwx--- 19:03:29,429 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0009_conf.xml 19:03:29,431 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0009_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0009_conf.xml perm=test:supergroup:rwxrwx--- 19:03:29,462 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:29,463 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:29,464 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:29,466 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53568, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, duration: 235000 19:03:29,532 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161409532, maxDate=1394766209532, sequenceNumber=29, masterKeyId=2 19:03:29,532 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 29 19:03:30,636 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137 19:03:30,637 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141 19:03:30,638 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140 19:03:30,639 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139 19:03:30,639 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138 19:03:30,649 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741952_1128, blk_1073741953_1129, blk_1073741954_1130, blk_1073741955_1131, blk_1073741956_1132, blk_1073741944_1120, blk_1073741962_1138, blk_1073741963_1139] 19:03:30,999 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:31,038 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/COMMIT_STARTED dst=null perm=null 19:03:31,039 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/COMMIT_SUCCESS dst=null perm=null 19:03:31,040 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/COMMIT_FAIL dst=null perm=null 19:03:31,125 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010 dst=null perm=null 19:03:31,127 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:03:31,128 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:03:31,478 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741965_1141 19:03:31,479 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137 19:03:31,480 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741963_1139 19:03:31,480 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741962_1138 19:03:31,481 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741964_1140 19:03:31,701 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=null 19:03:31,710 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.splitmetainfo dst=null perm=null 19:03:31,773 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53605, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741959_1135, duration: 186000 19:03:32,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741952_1128 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741952 for deletion 19:03:32,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741953_1129 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741953 for deletion 19:03:32,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741954_1130 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741954 for deletion 19:03:32,030 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741952_1128 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741952 19:03:32,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741955_1131 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741955 for deletion 19:03:32,030 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741953_1129 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741953 19:03:32,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741956_1132 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741956 for deletion 19:03:32,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741954_1130 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741954 19:03:32,031 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741944_1120 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741944 for deletion 19:03:32,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741955_1131 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741955 19:03:32,031 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741962_1138 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741962 for deletion 19:03:32,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741956_1132 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741956 19:03:32,031 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741963_1139 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741963 for deletion 19:03:32,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741944_1120 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741944 19:03:32,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741963_1139 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741963 19:03:32,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741962_1138 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741962 19:03:32,968 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0010_000001 19:03:32,968 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0010 APPATTEMPTID=appattempt_1394161202967_0010_000001 19:03:32,968 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from LAUNCHED to RUNNING 19:03:32,968 INFO RMAppImpl:643 - application_1394161202967_0010 State change from ACCEPTED to RUNNING 19:03:33,009 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:03:33,013 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1 dst=null perm=test:testg:rwxrwxrwx 19:03:33,074 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:33,291 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741966_1142{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:33,346 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142 src: /127.0.0.1:53610 dest: /127.0.0.1:52682 19:03:33,348 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142 src: /127.0.0.1:53611 dest: /127.0.0.1:52685 19:03:33,382 INFO clienttrace:1215 - src: /127.0.0.1:53611, dest: /127.0.0.1:52685, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142, duration: 32213000 19:03:33,382 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:33,382 INFO clienttrace:1215 - src: /127.0.0.1:53610, dest: /127.0.0.1:52682, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142, duration: 33209000 19:03:33,383 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:33,387 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1_conf.xml is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:33,415 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010.summary dst=null perm=test:supergroup:rw-rw-rw- 19:03:33,418 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741967_1143{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:33,421 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143 src: /127.0.0.1:53612 dest: /127.0.0.1:52685 19:03:33,422 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143 src: /127.0.0.1:53613 dest: /127.0.0.1:52682 19:03:33,425 INFO clienttrace:1215 - src: /127.0.0.1:53613, dest: /127.0.0.1:52682, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143, duration: 1476000 19:03:33,425 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:33,426 INFO clienttrace:1215 - src: /127.0.0.1:53612, dest: /127.0.0.1:52685, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143, duration: 2424000 19:03:33,426 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:33,428 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010.summary is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:33,650 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741952_1128, blk_1073741953_1129, blk_1073741954_1130, blk_1073741955_1131, blk_1073741956_1132, blk_1073741944_1120, blk_1073741962_1138, blk_1073741963_1139] 19:03:34,137 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0010 19:03:34,137 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000002 Container Transitioned from NEW to RESERVED 19:03:34,138 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0010 reserved container container_1394161202967_0010_01_000002 on node host: 172.16.1.64:52707 #containers=2 available= used=, currently has 1 at priority 20; currentReservation 1024 19:03:34,138 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0010_01_000002 on node host: 172.16.1.64:52707 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@777d6a6f 19:03:34,841 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000003 Container Transitioned from NEW to ALLOCATED 19:03:34,841 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000003 19:03:34,841 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0010_01_000003 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:03:35,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741952_1128 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741952 for deletion 19:03:35,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741953_1129 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741953 for deletion 19:03:35,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741954_1130 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741954 for deletion 19:03:35,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741955_1131 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741955 for deletion 19:03:35,030 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741952_1128 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741952 19:03:35,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741956_1132 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741956 for deletion 19:03:35,030 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741953_1129 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741953 19:03:35,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741944_1120 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741944 for deletion 19:03:35,030 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741954_1130 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741954 19:03:35,030 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741962_1138 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741962 for deletion 19:03:35,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741955_1131 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741955 19:03:35,031 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741963_1139 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741963 for deletion 19:03:35,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741956_1132 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741956 19:03:35,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741963_1139 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741963 19:03:35,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741944_1120 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741944 19:03:35,031 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741962_1138 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741962 19:03:35,036 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:03:35,092 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:35,094 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:35,116 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:35,118 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:35,121 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:35,135 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:35,137 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:35,140 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:35,143 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:35,145 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:35,147 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/app/lib/test.jar dst=null perm=null 19:03:35,152 INFO ContainerLaunch:322 - Container container_1394161202967_0009_01_000002 succeeded 19:03:35,152 INFO Container:888 - Container container_1394161202967_0009_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:03:35,153 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0009_01_000002 19:03:35,177 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000002 19:03:35,177 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000002 19:03:35,178 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000002 19:03:35,178 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0009/container_1394161202967_0009_01_000002 19:03:35,178 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000002 19:03:35,178 INFO Container:888 - Container container_1394161202967_0009_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 19:03:35,178 INFO Application:339 - Removing container_1394161202967_0009_01_000002 from application application_1394161202967_0009 19:03:35,178 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:35,178 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0009 19:03:35,223 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0010_01_000003 by user test 19:03:35,224 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0010 19:03:35,224 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000003 19:03:35,224 INFO Application:450 - Application application_1394161202967_0010 transitioned from NEW to INITING 19:03:35,224 INFO Application:296 - Adding container_1394161202967_0010_01_000003 to application application_1394161202967_0010 19:03:35,224 INFO Application:450 - Application application_1394161202967_0010 transitioned from INITING to RUNNING 19:03:35,224 INFO Container:888 - Container container_1394161202967_0010_01_000003 transitioned from NEW to LOCALIZING 19:03:35,225 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0010 19:03:35,225 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0010 19:03:35,225 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:03:35,225 INFO ShuffleHandler:263 - Added token for job_1394161202967_0010 19:03:35,225 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml transitioned from INIT to DOWNLOADING 19:03:35,225 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0010_01_000003 19:03:35,271 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/nmPrivate/container_1394161202967_0010_01_000003.tokens. Credentials list: 19:03:35,323 INFO DefaultContainerExecutor:469 - Initializing user test 19:03:35,649 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/nmPrivate/container_1394161202967_0010_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000003.tokens 19:03:35,650 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0010 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0010 19:03:35,668 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142 19:03:35,675 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143 19:03:35,866 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:03:35,867 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000004 Container Transitioned from NEW to ALLOCATED 19:03:35,867 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000004 19:03:35,867 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0010_01_000004 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:03:35,938 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:35,940 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:35,941 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml dst=null perm=null 19:03:35,963 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53619, bytes: 77968, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1113328963_3405, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741960_1136, duration: 352000 19:03:36,039 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0010/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:03:36,040 INFO Container:888 - Container container_1394161202967_0010_01_000003 transitioned from LOCALIZING to LOCALIZED 19:03:36,051 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 19:03:36,057 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0010_01_000004 by user test 19:03:36,058 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000004 19:03:36,058 INFO Application:296 - Adding container_1394161202967_0010_01_000004 to application application_1394161202967_0010 19:03:36,058 INFO Container:888 - Container container_1394161202967_0010_01_000004 transitioned from NEW to LOCALIZING 19:03:36,058 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0010 19:03:36,058 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0010 19:03:36,058 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:03:36,058 INFO ShuffleHandler:263 - Added token for job_1394161202967_0010 19:03:36,093 INFO Container:888 - Container container_1394161202967_0010_01_000004 transitioned from LOCALIZING to LOCALIZED 19:03:36,157 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0009_01_000002 19:03:36,158 INFO RMContainerImpl:220 - container_1394161202967_0009_01_000002 Container Transitioned from RUNNING to COMPLETED 19:03:36,158 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0009_01_000002 in state: COMPLETED event:FINISHED 19:03:36,159 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0009_000001 19:03:36,159 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0009 CONTAINERID=container_1394161202967_0009_01_000002 19:03:36,165 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0009_000001 State change from FINISHING to FINISHED 19:03:36,165 INFO FSSchedulerNode:162 - Released container container_1394161202967_0009_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:03:36,166 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0009_000001 released container container_1394161202967_0009_01_000002 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:03:36,182 INFO RMAppImpl:643 - application_1394161202967_0009 State change from FINISHING to FINISHED 19:03:36,190 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0009 19:03:36,190 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0009_000001 19:03:36,190 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0009,name=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190310205-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0009/jobhistory/job/job_1394161202967_0009,appMasterHost=rkanter-mbp.local,startTime=1394161390955,finishTime=1394161408069,finalStatus=SUCCEEDED 19:03:36,190 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000002 Container Transitioned from NEW to ALLOCATED 19:03:36,190 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000002 19:03:36,191 INFO FSSchedulerApp:123 - Application application_1394161202967_0010 unreserved on node host: 172.16.1.64:52707 #containers=1 available= used=, currently has 0 at priority 20; currentReservation 19:03:36,191 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0010_01_000002 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:03:36,191 INFO Container:888 - Container container_1394161202967_0010_01_000003 transitioned from LOCALIZED to RUNNING 19:03:36,191 INFO FairScheduler:720 - Application appattempt_1394161202967_0009_000001 is done. finalState=FINISHED 19:03:36,191 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:36,194 INFO AppSchedulingInfo:108 - Application application_1394161202967_0009 requests cleared 19:03:36,322 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0009_01_000002 19:03:36,529 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142 19:03:36,564 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741967_1143 19:03:36,666 INFO Container:888 - Container container_1394161202967_0010_01_000004 transitioned from LOCALIZED to RUNNING 19:03:36,667 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:36,844 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000003/default_container_executor.sh] 19:03:36,873 INFO Application:450 - Application application_1394161202967_0009 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:03:36,874 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000004 Container Transitioned from ACQUIRED to RUNNING 19:03:36,879 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0009 19:03:36,880 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0009 19:03:36,900 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0009 19:03:36,901 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0009 19:03:36,903 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0009 19:03:36,912 INFO Application:450 - Application application_1394161202967_0009 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:03:36,912 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0009, with delay of 10800 seconds 19:03:37,084 INFO AppSchedulingInfo:166 - checking for deactivate... 19:03:37,123 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:03:37,190 INFO Application:450 - Application application_1394161202967_0009 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:03:37,191 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0009 19:03:37,197 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0009 19:03:37,198 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0009 19:03:37,198 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0009 19:03:37,198 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0009 19:03:37,225 INFO Application:450 - Application application_1394161202967_0009 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:03:37,226 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0009, with delay of 10800 seconds 19:03:37,226 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000004/default_container_executor.sh] 19:03:38,134 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000002 Container Transitioned from ACQUIRED to RELEASED 19:03:38,134 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0010_01_000002 in state: RELEASED event:RELEASED 19:03:38,134 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000002 19:03:38,135 INFO FSSchedulerNode:162 - Released container container_1394161202967_0010_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:03:38,135 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0010_000001 released container container_1394161202967_0010_01_000002 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: RELEASED 19:03:38,190 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0010_01_000002 19:03:40,526 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:40,612 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53624, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0010_m_000000_0_1185701792_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134, duration: 272000 19:03:40,668 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input/data.txt dst=null perm=null 19:03:40,737 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53624, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0010_m_000000_0_1185701792_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123, duration: 188000 19:03:40,760 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_m_000000_0 dst=null perm=null 19:03:40,788 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0010_01_000003 19:03:40,788 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000003 19:03:40,788 INFO Container:888 - Container container_1394161202967_0010_01_000003 transitioned from RUNNING to KILLING 19:03:40,788 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0010_01_000003 19:03:40,838 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741968_1144{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:40,840 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0010_01_000003 is : 143 19:03:40,844 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144 src: /127.0.0.1:53625 dest: /127.0.0.1:52682 19:03:40,846 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144 src: /127.0.0.1:53626 dest: /127.0.0.1:52685 19:03:40,851 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1.jhist for DFSClient_NONMAPREDUCE_-339006047_1 19:03:40,858 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job.split dst=null perm=null 19:03:40,859 INFO Container:888 - Container container_1394161202967_0010_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:03:40,860 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000003 19:03:40,860 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000003 19:03:40,860 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000003 19:03:40,860 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000003 19:03:40,860 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000003 19:03:40,861 INFO Container:888 - Container container_1394161202967_0010_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:03:40,861 INFO Application:339 - Removing container_1394161202967_0010_01_000003 from application application_1394161202967_0010 19:03:40,861 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:40,861 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0010 19:03:40,943 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53628, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0010_m_000001_0_1424867772_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741958_1134, duration: 287000 19:03:40,989 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/input/data.txt dst=null perm=null 19:03:40,993 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53629, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0010_m_000001_0_1424867772_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741947_1123, duration: 287000 19:03:41,086 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_m_000001_0 dst=null perm=null 19:03:41,138 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0010_01_000004 19:03:41,139 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000004 19:03:41,139 INFO Container:888 - Container container_1394161202967_0010_01_000004 transitioned from RUNNING to KILLING 19:03:41,139 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0010_01_000004 19:03:41,139 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0010_01_000003 19:03:41,164 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000003 Container Transitioned from RUNNING to COMPLETED 19:03:41,165 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0010_01_000003 in state: COMPLETED event:FINISHED 19:03:41,165 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000003 19:03:41,166 INFO FSSchedulerNode:162 - Released container container_1394161202967_0010_01_000003 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:03:41,166 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0010_000001 released container container_1394161202967_0010_01_000003 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:03:41,192 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0010_01_000004 is : 143 19:03:41,210 INFO Container:888 - Container container_1394161202967_0010_01_000004 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:03:41,211 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000004 19:03:41,211 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000004 19:03:41,211 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000004 19:03:41,211 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000004 19:03:41,211 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000004 19:03:41,212 INFO Container:888 - Container container_1394161202967_0010_01_000004 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:03:41,212 INFO Application:339 - Removing container_1394161202967_0010_01_000004 from application application_1394161202967_0010 19:03:41,212 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:41,212 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0010 19:03:42,141 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0010_01_000004 19:03:42,143 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000004 Container Transitioned from RUNNING to COMPLETED 19:03:42,143 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0010_01_000004 in state: COMPLETED event:FINISHED 19:03:42,143 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000004 19:03:42,144 INFO FSSchedulerNode:162 - Released container container_1394161202967_0010_01_000004 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:03:42,144 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0010_000001 released container container_1394161202967_0010_01_000004 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: FINISHED 19:03:43,144 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000005 Container Transitioned from NEW to ALLOCATED 19:03:43,144 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000005 19:03:43,144 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0010_01_000005 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:03:43,206 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000005 Container Transitioned from ALLOCATED to ACQUIRED 19:03:43,219 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0010_01_000005 by user test 19:03:43,219 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000005 19:03:43,219 INFO Application:296 - Adding container_1394161202967_0010_01_000005 to application application_1394161202967_0010 19:03:43,220 INFO Container:888 - Container container_1394161202967_0010_01_000005 transitioned from NEW to LOCALIZING 19:03:43,220 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0010 19:03:43,220 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0010 19:03:43,220 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:03:43,220 INFO ShuffleHandler:263 - Added token for job_1394161202967_0010 19:03:43,220 INFO Container:888 - Container container_1394161202967_0010_01_000005 transitioned from LOCALIZING to LOCALIZED 19:03:43,317 INFO Container:888 - Container container_1394161202967_0010_01_000005 transitioned from LOCALIZED to RUNNING 19:03:43,330 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:43,636 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000005/default_container_executor.sh] 19:03:44,144 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000005 Container Transitioned from ACQUIRED to RUNNING 19:03:44,212 INFO AppSchedulingInfo:166 - checking for deactivate... 19:03:46,401 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_r_000000_0/part-00000 dst=null perm=test:testg:rw-rw-rw- 19:03:46,442 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_r_000000_0/part-00000. BP-1007909283-172.16.1.64-1394161187028 blk_1073741969_1145{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:46,545 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145 src: /127.0.0.1:53634 dest: /127.0.0.1:52685 19:03:46,547 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145 src: /127.0.0.1:53635 dest: /127.0.0.1:52682 19:03:46,580 INFO clienttrace:1215 - src: /127.0.0.1:53635, dest: /127.0.0.1:52682, bytes: 16, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0010_r_000000_0_481452742_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145, duration: 31368000 19:03:46,580 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:46,581 INFO clienttrace:1215 - src: /127.0.0.1:53634, dest: /127.0.0.1:52685, bytes: 16, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0010_r_000000_0_481452742_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145, duration: 32552000 19:03:46,581 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:46,732 INFO StateChange:2959 - DIR* completeFile: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_r_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0010_r_000000_0_481452742_1 19:03:46,747 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_r_000000_0 dst=null perm=null 19:03:46,755 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_r_000000_0 dst=null perm=null 19:03:46,755 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/task_1394161202967_0010_r_000000 dst=null perm=null 19:03:46,759 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/_temporary/attempt_1394161202967_0010_r_000000_0 dst=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/task_1394161202967_0010_r_000000 perm=test:testg:rwxrwxrwx 19:03:46,771 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0010_01_000005 19:03:46,771 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000005 19:03:46,771 INFO Container:888 - Container container_1394161202967_0010_01_000005 transitioned from RUNNING to KILLING 19:03:46,772 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0010_01_000005 19:03:46,821 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:03:46,823 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0010_01_000005 is : 143 19:03:46,826 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:46,831 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1 dst=null perm=null 19:03:46,838 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output dst=null perm=null 19:03:46,839 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output dst=null perm=null 19:03:46,840 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/task_1394161202967_0010_r_000000 dst=null perm=null 19:03:46,841 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/part-00000 dst=null perm=null 19:03:46,842 INFO Container:888 - Container container_1394161202967_0010_01_000005 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:03:46,843 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000005 19:03:46,843 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000005 19:03:46,843 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000005 19:03:46,843 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000005 19:03:46,843 INFO Container:888 - Container container_1394161202967_0010_01_000005 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:03:46,843 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000005 19:03:46,843 INFO Application:339 - Removing container_1394161202967_0010_01_000005 from application application_1394161202967_0010 19:03:46,844 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:46,844 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0010 19:03:46,844 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary/1/task_1394161202967_0010_r_000000/part-00000 dst=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/part-00000 perm=test:testg:rw-rw-rw- 19:03:46,850 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_temporary dst=null perm=null 19:03:46,854 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_SUCCESS dst=null perm=test:testg:rw-rw-rw- 19:03:46,855 INFO StateChange:2959 - DIR* completeFile: /user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:46,857 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:03:46,859 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:46,877 INFO clienttrace:1215 - src: /127.0.0.1:53626, dest: /127.0.0.1:52685, bytes: 38970, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144, duration: 6028964000 19:03:46,877 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:46,882 INFO clienttrace:1215 - src: /127.0.0.1:53625, dest: /127.0.0.1:52682, bytes: 38970, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144, duration: 6033978000 19:03:46,882 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:46,884 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1.jhist is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:46,886 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:46,888 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741970_1146{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:46,890 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146 src: /127.0.0.1:53636 dest: /127.0.0.1:52682 19:03:46,892 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146 src: /127.0.0.1:53637 dest: /127.0.0.1:52685 19:03:46,894 INFO clienttrace:1215 - src: /127.0.0.1:53637, dest: /127.0.0.1:52685, bytes: 430, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146, duration: 1204000 19:03:46,894 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:46,895 INFO clienttrace:1215 - src: /127.0.0.1:53636, dest: /127.0.0.1:52682, bytes: 430, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146, duration: 1860000 19:03:46,895 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:46,896 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary_tmp is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:46,900 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:46,905 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1.jhist dst=null perm=null 19:03:46,906 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist_tmp dst=null perm=null 19:03:46,909 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1.jhist dst=null perm=null 19:03:46,910 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist_tmp dst=null perm=null 19:03:46,911 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1.jhist dst=null perm=null 19:03:46,914 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:46,916 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53638, bytes: 39278, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144, duration: 227000 19:03:46,917 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741971_1147{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:46,920 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147 src: /127.0.0.1:53639 dest: /127.0.0.1:52682 19:03:46,921 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147 src: /127.0.0.1:53640 dest: /127.0.0.1:52685 19:03:46,924 INFO clienttrace:1215 - src: /127.0.0.1:53640, dest: /127.0.0.1:52685, bytes: 38970, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147, duration: 2049000 19:03:46,924 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:46,925 INFO clienttrace:1215 - src: /127.0.0.1:53639, dest: /127.0.0.1:52682, bytes: 38970, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147, duration: 2621000 19:03:46,925 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:46,926 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist_tmp is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:46,928 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:46,929 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1_conf.xml dst=null perm=null 19:03:46,930 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml_tmp dst=null perm=null 19:03:46,930 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1_conf.xml dst=null perm=null 19:03:46,931 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml_tmp dst=null perm=null 19:03:46,932 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010/job_1394161202967_0010_1_conf.xml dst=null perm=null 19:03:46,934 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:03:46,936 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53638, bytes: 92383, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741966_1142, duration: 319000 19:03:46,938 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741972_1148{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:46,940 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148 src: /127.0.0.1:53641 dest: /127.0.0.1:52685 19:03:46,941 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148 src: /127.0.0.1:53642 dest: /127.0.0.1:52682 19:03:46,944 INFO clienttrace:1215 - src: /127.0.0.1:53642, dest: /127.0.0.1:52682, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148, duration: 1728000 19:03:46,944 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:46,945 INFO clienttrace:1215 - src: /127.0.0.1:53641, dest: /127.0.0.1:52685, bytes: 91663, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-339006047_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148, duration: 2417000 19:03:46,945 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:46,947 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_-339006047_1 19:03:46,948 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:03:46,950 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary perm=test:supergroup:rwxrwx--- 19:03:46,951 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml perm=test:supergroup:rwxrwx--- 19:03:46,953 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist perm=test:supergroup:rwxrwx--- 19:03:46,960 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0010_000001 with final state: FINISHING 19:03:46,960 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from RUNNING to FINAL_SAVING 19:03:46,960 INFO RMAppImpl:880 - Updating application application_1394161202967_0010 with final state: FINISHING 19:03:46,960 INFO RMAppImpl:643 - application_1394161202967_0010 State change from RUNNING to FINAL_SAVING 19:03:46,960 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0010 19:03:46,961 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from FINAL_SAVING to FINISHING 19:03:46,961 INFO RMAppImpl:643 - application_1394161202967_0010 State change from FINAL_SAVING to FINISHING 19:03:47,773 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0010_01_000005 19:03:47,775 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000005 Container Transitioned from RUNNING to COMPLETED 19:03:47,775 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0010_01_000005 in state: COMPLETED event:FINISHED 19:03:47,775 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000005 19:03:47,776 INFO FSSchedulerNode:162 - Released container container_1394161202967_0010_01_000005 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:03:47,776 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0010_000001 released container container_1394161202967_0010_01_000005 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: FINISHED 19:03:47,973 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0010 dst=null perm=null 19:03:48,669 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741968_1144, blk_1073741958_1134, blk_1073741959_1135, blk_1073741960_1136, blk_1073741966_1142, blk_1073741967_1143] 19:03:48,907 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161428907, maxDate=1394766228907, sequenceNumber=30, masterKeyId=2 19:03:48,907 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 30 19:03:48,915 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:03:48,916 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:03:48,917 INFO CompletedJob:95 - Loading job: job_1394161202967_0010 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist 19:03:48,917 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist] 19:03:48,918 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist dst=null perm=null 19:03:48,919 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist dst=null perm=null 19:03:48,921 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53647, bytes: 39278, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147, duration: 215000 19:03:48,949 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary dst=null perm=null 19:03:48,953 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53648, bytes: 434, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146, duration: 286000 19:03:48,953 INFO JobSummary:372 - jobId=job_1394161202967_0010,submitTime=1394161406046,launchTime=1394161412997,firstMapTaskLaunchTime=1394161415230,firstReduceTaskLaunchTime=1394161423220,finishTime=1394161426859,resourcesPerMap=1024,resourcesPerReduce=1024,numMaps=2,numReduces=1,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=10,reduceSlotSeconds=3,jobName=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190310205-oozie-rkan-W 19:03:48,953 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary] 19:03:48,956 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010.summary dst=null perm=null 19:03:48,957 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:03:48,958 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist 19:03:48,959 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0010-1394161406046-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161426859-2-1-SUCCEEDED-root.test-1394161412997.jhist perm=test:supergroup:rwxrwx--- 19:03:48,959 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0010_conf.xml 19:03:48,964 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0010_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0010_conf.xml perm=test:supergroup:rwxrwx--- 19:03:49,002 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:49,004 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:49,005 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:49,007 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53648, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, duration: 176000 19:03:49,026 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:49,027 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:49,027 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:03:49,029 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53647, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1178951046_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741961_1137, duration: 206000 19:03:49,043 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java dst=null perm=null 19:03:49,045 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190310205-oozie-rkan-W/map-reduce--java dst=null perm=null 19:03:49,081 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161429081, maxDate=1394766229081, sequenceNumber=31, masterKeyId=2 19:03:49,081 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 31 19:03:49,655 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/be638aa0-afac-418b-b432-10208bc03993/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:03:49,661 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/be638aa0-afac-418b-b432-10208bc03993/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741973_1149{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:49,664 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149 src: /127.0.0.1:53652 dest: /127.0.0.1:52682 19:03:49,666 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149 src: /127.0.0.1:53653 dest: /127.0.0.1:52685 19:03:49,668 INFO clienttrace:1215 - src: /127.0.0.1:53653, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_851270770_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149, duration: 1212000 19:03:49,668 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,669 INFO clienttrace:1215 - src: /127.0.0.1:53652, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_851270770_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149, duration: 1652000 19:03:49,669 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,673 INFO StateChange:2959 - DIR* completeFile: /user/test/be638aa0-afac-418b-b432-10208bc03993/input/data.txt is closed by DFSClient_NONMAPREDUCE_851270770_8 19:03:49,681 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:03:49,683 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741974_1150{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:49,684 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150 src: /127.0.0.1:53654 dest: /127.0.0.1:52682 19:03:49,685 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150 src: /127.0.0.1:53655 dest: /127.0.0.1:52685 19:03:49,687 INFO clienttrace:1215 - src: /127.0.0.1:53655, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_851270770_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150, duration: 1189000 19:03:49,688 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,688 INFO clienttrace:1215 - src: /127.0.0.1:53654, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_851270770_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150, duration: 1720000 19:03:49,688 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,690 INFO StateChange:2959 - DIR* completeFile: /user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_851270770_8 19:03:49,692 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:03:49,693 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/be638aa0-afac-418b-b432-10208bc03993/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741975_1151{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:49,695 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151 src: /127.0.0.1:53656 dest: /127.0.0.1:52682 19:03:49,696 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151 src: /127.0.0.1:53657 dest: /127.0.0.1:52685 19:03:49,698 INFO clienttrace:1215 - src: /127.0.0.1:53657, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_851270770_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151, duration: 1227000 19:03:49,698 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,699 INFO clienttrace:1215 - src: /127.0.0.1:53656, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_851270770_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151, duration: 1601000 19:03:49,699 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,701 INFO StateChange:2959 - DIR* completeFile: /user/test/be638aa0-afac-418b-b432-10208bc03993/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_851270770_8 19:03:49,704 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java dst=null perm=null 19:03:49,706 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:03:49,707 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java.tmp dst=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java perm=test:supergroup:rwxrwxrwx 19:03:49,708 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/workflow.xml dst=null perm=null 19:03:49,748 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/workflow.xml dst=null perm=null 19:03:49,787 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java dst=null perm=test:supergroup:rwxrwxrwx 19:03:49,788 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:49,797 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741976_1152{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:49,799 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152 src: /127.0.0.1:53659 dest: /127.0.0.1:52682 19:03:49,800 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152 src: /127.0.0.1:53660 dest: /127.0.0.1:52685 19:03:49,803 INFO clienttrace:1215 - src: /127.0.0.1:53660, dest: /127.0.0.1:52685, bytes: 77268, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152, duration: 1400000 19:03:49,803 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,803 INFO clienttrace:1215 - src: /127.0.0.1:53659, dest: /127.0.0.1:52682, bytes: 77268, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152, duration: 1936000 19:03:49,803 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,807 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action.xml is closed by DFSClient_NONMAPREDUCE_-215555625_8 19:03:49,809 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input dst=null perm=test:supergroup:rwxrwxrwx 19:03:49,810 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:03:49,811 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741977_1153{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:03:49,813 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153 src: /127.0.0.1:53661 dest: /127.0.0.1:52682 19:03:49,814 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153 src: /127.0.0.1:53662 dest: /127.0.0.1:52685 19:03:49,816 INFO clienttrace:1215 - src: /127.0.0.1:53662, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153, duration: 1022000 19:03:49,816 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,817 INFO clienttrace:1215 - src: /127.0.0.1:53661, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153, duration: 1528000 19:03:49,817 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,819 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-215555625_8 19:03:49,837 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161429837, maxDate=1394766229837, sequenceNumber=32, masterKeyId=2 19:03:49,837 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 32 19:03:49,839 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/0000000-140306190349401-oozie-rkan-W@mr-action dst=null perm=null 19:03:49,840 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161429840, maxDate=1394766229840, sequenceNumber=33, masterKeyId=2 19:03:49,840 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 33 19:03:49,855 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:03:49,856 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:49,857 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:49,858 INFO ClientRMService:229 - Allocated new applicationId: 11 19:03:49,859 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011 dst=null perm=null 19:03:49,860 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011 dst=null perm=test:supergroup:rwxrwxrwx 19:03:49,862 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011 dst=null perm=test:supergroup:rwx------ 19:03:49,862 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:49,863 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:49,864 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:49,865 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:49,865 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:03:49,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:03:49,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:03:49,867 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:03:49,868 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:03:49,868 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:49,869 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:49,869 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib dst=null perm=null 19:03:49,870 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app dst=null perm=null 19:03:49,871 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993 dst=null perm=null 19:03:49,871 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:03:49,873 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:03:49,873 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:03:49,874 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:03:49,877 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:03:49,879 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=test:supergroup:rw-r--r-- 19:03:49,879 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split 19:03:49,880 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=null 19:03:49,881 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741978_1154{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:49,883 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154 src: /127.0.0.1:53664 dest: /127.0.0.1:52685 19:03:49,884 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154 src: /127.0.0.1:53665 dest: /127.0.0.1:52682 19:03:49,886 INFO clienttrace:1215 - src: /127.0.0.1:53665, dest: /127.0.0.1:52682, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154, duration: 1617000 19:03:49,886 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,887 INFO clienttrace:1215 - src: /127.0.0.1:53664, dest: /127.0.0.1:52685, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154, duration: 1728000 19:03:49,887 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,889 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split is closed by DFSClient_NONMAPREDUCE_-215555625_8 19:03:49,890 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:03:49,892 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:03:49,893 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741979_1155{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:03:49,894 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155 src: /127.0.0.1:53666 dest: /127.0.0.1:52682 19:03:49,895 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155 src: /127.0.0.1:53667 dest: /127.0.0.1:52685 19:03:49,897 INFO clienttrace:1215 - src: /127.0.0.1:53667, dest: /127.0.0.1:52685, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155, duration: 1110000 19:03:49,897 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,898 INFO clienttrace:1215 - src: /127.0.0.1:53666, dest: /127.0.0.1:52682, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155, duration: 1516000 19:03:49,898 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,900 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_-215555625_8 19:03:49,902 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:49,903 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:03:49,910 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741980_1156{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:03:49,912 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156 src: /127.0.0.1:53668 dest: /127.0.0.1:52685 19:03:49,913 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156 src: /127.0.0.1:53669 dest: /127.0.0.1:52682 19:03:49,916 INFO clienttrace:1215 - src: /127.0.0.1:53669, dest: /127.0.0.1:52682, bytes: 79931, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156, duration: 1574000 19:03:49,916 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:49,917 INFO clienttrace:1215 - src: /127.0.0.1:53668, dest: /127.0.0.1:52685, bytes: 79931, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156, duration: 2163000 19:03:49,917 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:49,919 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml is closed by DFSClient_NONMAPREDUCE_-215555625_8 19:03:49,921 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011 dst=null perm=null 19:03:49,922 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=null 19:03:49,923 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=null 19:03:49,924 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=null 19:03:49,925 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=null 19:03:49,926 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=null 19:03:49,927 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=null 19:03:49,931 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:49,932 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:49,933 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:49,934 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:49,935 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:49,936 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:49,938 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:49,940 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:49,941 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:49,944 INFO ClientRMService:342 - Application with id 11 submitted by user test 19:03:49,944 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0011 19:03:49,944 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0011 19:03:49,945 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0011 19:03:49,944 INFO RMAppImpl:643 - application_1394161202967_0011 State change from NEW to NEW_SAVING 19:03:49,945 INFO RMAppImpl:643 - application_1394161202967_0011 State change from NEW_SAVING to SUBMITTED 19:03:49,945 INFO FairScheduler:638 - Accepted application application_1394161202967_0011 from user: test, in queue: default, currently num of applications: 2 19:03:49,945 INFO RMAppImpl:643 - application_1394161202967_0011 State change from SUBMITTED to ACCEPTED 19:03:49,946 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0011_000001 19:03:49,946 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from NEW to SUBMITTED 19:03:49,946 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0011_000001 to scheduler from user: test 19:03:49,946 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from SUBMITTED to SCHEDULED 19:03:49,986 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161429986, maxDate=1394766229986, sequenceNumber=34, masterKeyId=2 19:03:49,986 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 34 19:03:50,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741968_1144 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741968 for deletion 19:03:50,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741958_1134 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741958 for deletion 19:03:50,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741959_1135 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741959 for deletion 19:03:50,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741960_1136 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741960 for deletion 19:03:50,034 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741968_1144 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741968 19:03:50,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741966_1142 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741966 for deletion 19:03:50,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741959_1135 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741959 19:03:50,035 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741967_1143 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741967 for deletion 19:03:50,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741958_1134 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741958 19:03:50,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741967_1143 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741967 19:03:50,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741960_1136 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741960 19:03:50,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741966_1142 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741966 19:03:50,206 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000001 Container Transitioned from NEW to ALLOCATED 19:03:50,207 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000001 19:03:50,207 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0011_01_000001 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:03:50,208 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:03:50,208 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0011 AttemptId: appattempt_1394161202967_0011_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0011_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:03:50,208 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:03:50,208 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:03:50,209 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0011_000001 19:03:50,210 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0011_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0011_000001 19:03:50,211 INFO AMLauncher:190 - Command to launch container container_1394161202967_0011_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:03:50,218 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0011_01_000001 by user test 19:03:50,218 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0011 19:03:50,218 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000001 19:03:50,219 INFO Application:450 - Application application_1394161202967_0011 transitioned from NEW to INITING 19:03:50,219 INFO Application:296 - Adding container_1394161202967_0011_01_000001 to application application_1394161202967_0011 19:03:50,219 INFO Application:450 - Application application_1394161202967_0011 transitioned from INITING to RUNNING 19:03:50,219 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0011_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0011_000001 19:03:50,219 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from ALLOCATED to LAUNCHED 19:03:50,219 INFO Container:888 - Container container_1394161202967_0011_01_000001 transitioned from NEW to LOCALIZING 19:03:50,220 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0011 19:03:50,220 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:03:50,220 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split transitioned from INIT to DOWNLOADING 19:03:50,220 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml transitioned from INIT to DOWNLOADING 19:03:50,220 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:03:50,221 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0011_01_000001 19:03:50,221 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar, 1394161429690, FILE, null } 19:03:50,343 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0011_01_000001.tokens. Credentials list: 19:03:50,438 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:50,473 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:50,504 INFO DefaultContainerExecutor:469 - Initializing user test 19:03:50,505 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib dst=null perm=null 19:03:50,567 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app dst=null perm=null 19:03:50,625 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993 dst=null perm=null 19:03:50,661 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:03:50,729 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:03:50,764 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150 19:03:50,765 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:03:50,765 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151 19:03:50,787 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154 19:03:50,801 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155 19:03:50,802 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:50,808 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152 19:03:50,836 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147 19:03:50,871 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149 19:03:50,872 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:50,872 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146 19:03:50,882 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148 19:03:50,910 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145 19:03:50,910 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153 19:03:51,023 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156 19:03:51,033 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53647, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150, duration: 457000 19:03:51,271 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:03:51,271 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0011_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000001.tokens 19:03:51,282 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011 19:03:51,283 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/filecache/18/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:03:51,469 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=null 19:03:51,477 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=null 19:03:51,479 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=null 19:03:51,513 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53647, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1987630131_3647, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155, duration: 219000 19:03:51,571 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0011/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:03:51,572 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741975_1151 19:03:51,573 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155 19:03:51,578 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741969_1145 19:03:51,607 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741971_1147 19:03:51,612 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149 19:03:51,639 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741968_1144 19:03:51,645 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152 19:03:51,672 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741970_1146, blk_1073741957_1133, blk_1073741961_1137, blk_1073741950_1126, blk_1073741951_1127] 19:03:51,672 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741970_1146 19:03:51,706 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156 19:03:51,707 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154 19:03:51,707 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153 19:03:51,708 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=null 19:03:51,709 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=null 19:03:51,710 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=null 19:03:51,733 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53648, bytes: 176, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1987630131_3647, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154, duration: 227000 19:03:51,843 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741972_1148 19:03:51,845 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150 19:03:51,847 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:03:51,931 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=null 19:03:51,932 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=null 19:03:51,933 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=null 19:03:51,959 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53647, bytes: 80559, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1987630131_3647, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741980_1156, duration: 378000 19:03:52,066 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0011/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:03:52,072 INFO Container:888 - Container container_1394161202967_0011_01_000001 transitioned from LOCALIZING to LOCALIZED 19:03:52,175 INFO Container:888 - Container container_1394161202967_0011_01_000001 transitioned from LOCALIZED to RUNNING 19:03:52,175 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:52,479 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000001/default_container_executor.sh] 19:03:53,035 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741970_1146 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741970 for deletion 19:03:53,036 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741957_1133 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741957 for deletion 19:03:53,036 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741961_1137 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741961 for deletion 19:03:53,036 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741950_1126 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741950 for deletion 19:03:53,036 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741970_1146 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741970 19:03:53,036 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741951_1127 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741951 for deletion 19:03:53,037 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741950_1126 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741950 19:03:53,037 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741957_1133 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741957 19:03:53,037 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741961_1137 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741961 19:03:53,038 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741951_1127 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741951 19:03:54,123 INFO ContainerLaunch:322 - Container container_1394161202967_0010_01_000001 succeeded 19:03:54,124 INFO Container:888 - Container container_1394161202967_0010_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:03:54,124 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0010_01_000001 19:03:54,143 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000001 19:03:54,143 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000001 19:03:54,144 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000001 19:03:54,144 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0010/container_1394161202967_0010_01_000001 19:03:54,144 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000001 19:03:54,144 INFO Container:888 - Container container_1394161202967_0010_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 19:03:54,144 INFO Application:339 - Removing container_1394161202967_0010_01_000001 from application application_1394161202967_0010 19:03:54,145 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:54,145 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0010 19:03:54,253 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0010_01_000001 19:03:54,253 INFO RMContainerImpl:220 - container_1394161202967_0010_01_000001 Container Transitioned from RUNNING to COMPLETED 19:03:54,254 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0010_01_000001 in state: COMPLETED event:FINISHED 19:03:54,254 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0010_000001 19:03:54,254 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0010 CONTAINERID=container_1394161202967_0010_01_000001 19:03:54,254 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0010_000001 State change from FINISHING to FINISHED 19:03:54,254 INFO FSSchedulerNode:162 - Released container container_1394161202967_0010_01_000001 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:03:54,254 INFO RMAppImpl:643 - application_1394161202967_0010 State change from FINISHING to FINISHED 19:03:54,254 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0010_000001 released container container_1394161202967_0010_01_000001 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:03:54,254 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0010 19:03:54,254 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0010_000001 19:03:54,254 INFO FairScheduler:720 - Application appattempt_1394161202967_0010_000001 is done. finalState=FINISHED 19:03:54,254 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0010,name=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190310205-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0010/jobhistory/job/job_1394161202967_0010,appMasterHost=rkanter-mbp.local,startTime=1394161406046,finishTime=1394161426960,finalStatus=SUCCEEDED 19:03:54,255 INFO AppSchedulingInfo:108 - Application application_1394161202967_0010 requests cleared 19:03:54,261 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0010_01_000001 19:03:54,655 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:03:54,673 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741968_1144, blk_1073741970_1146, blk_1073741957_1133, blk_1073741958_1134, blk_1073741959_1135, blk_1073741960_1136, blk_1073741961_1137, blk_1073741950_1126, blk_1073741966_1142, blk_1073741951_1127, blk_1073741967_1143] 19:03:54,686 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/COMMIT_STARTED dst=null perm=null 19:03:54,687 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/COMMIT_SUCCESS dst=null perm=null 19:03:54,689 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/COMMIT_FAIL dst=null perm=null 19:03:54,744 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011 dst=null perm=null 19:03:54,747 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:03:54,749 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:03:54,824 INFO Application:450 - Application application_1394161202967_0010 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:03:54,824 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0010 19:03:54,825 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0010 19:03:54,825 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0010 19:03:54,825 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0010 19:03:54,825 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0010 19:03:54,825 INFO Application:450 - Application application_1394161202967_0010 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:03:54,825 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0010, with delay of 10800 seconds 19:03:55,144 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=null 19:03:55,153 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.splitmetainfo dst=null perm=null 19:03:55,230 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53675, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741979_1155, duration: 299000 19:03:55,255 INFO Application:450 - Application application_1394161202967_0010 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:03:55,255 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0010 19:03:55,255 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0010 19:03:55,255 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0010 19:03:55,256 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0010 19:03:55,256 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0010 19:03:55,256 INFO Application:450 - Application application_1394161202967_0010 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:03:55,256 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0010, with delay of 10800 seconds 19:03:56,033 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741968_1144 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741968 for deletion 19:03:56,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741970_1146 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741970 for deletion 19:03:56,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741957_1133 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741957 for deletion 19:03:56,034 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741968_1144 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741968 19:03:56,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741958_1134 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741958 for deletion 19:03:56,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741959_1135 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741959 for deletion 19:03:56,034 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741957_1133 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741957 19:03:56,034 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741960_1136 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741960 for deletion 19:03:56,034 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741970_1146 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741970 19:03:56,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741959_1135 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741959 19:03:56,035 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741961_1137 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741961 for deletion 19:03:56,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741958_1134 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741958 19:03:56,035 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741966_1142 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741966 for deletion 19:03:56,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741961_1137 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741961 19:03:56,035 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741950_1126 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741950 for deletion 19:03:56,035 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741960_1136 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741960 19:03:56,035 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741967_1143 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741967 for deletion 19:03:56,036 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741951_1127 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741951 for deletion 19:03:56,036 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741966_1142 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741966 19:03:56,036 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741967_1143 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741967 19:03:56,036 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741950_1126 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741950 19:03:56,036 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741951_1127 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741951 19:03:56,184 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0011_000001 19:03:56,184 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0011 APPATTEMPTID=appattempt_1394161202967_0011_000001 19:03:56,184 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from LAUNCHED to RUNNING 19:03:56,184 INFO RMAppImpl:643 - application_1394161202967_0011 State change from ACCEPTED to RUNNING 19:03:56,230 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:03:56,232 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=test:supergroup:rwxrwxrwx 19:03:56,297 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:03:56,497 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741981_1157{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:56,549 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157 src: /127.0.0.1:53681 dest: /127.0.0.1:52685 19:03:56,551 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157 src: /127.0.0.1:53682 dest: /127.0.0.1:52682 19:03:56,588 INFO clienttrace:1215 - src: /127.0.0.1:53682, dest: /127.0.0.1:52682, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157, duration: 36051000 19:03:56,588 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:56,589 INFO clienttrace:1215 - src: /127.0.0.1:53681, dest: /127.0.0.1:52685, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157, duration: 36842000 19:03:56,589 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:56,593 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1_conf.xml is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:03:56,625 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011.summary dst=null perm=test:supergroup:rw-rw-rw- 19:03:56,628 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741982_1158{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:03:56,631 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158 src: /127.0.0.1:53683 dest: /127.0.0.1:52685 19:03:56,632 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158 src: /127.0.0.1:53684 dest: /127.0.0.1:52682 19:03:56,635 INFO clienttrace:1215 - src: /127.0.0.1:53684, dest: /127.0.0.1:52682, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158, duration: 1498000 19:03:56,635 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:03:56,636 INFO clienttrace:1215 - src: /127.0.0.1:53683, dest: /127.0.0.1:52685, bytes: 372, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158, duration: 2196000 19:03:56,636 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:03:56,638 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011.summary is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:03:56,849 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158 19:03:56,850 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157 19:03:57,258 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000002 Container Transitioned from NEW to ALLOCATED 19:03:57,258 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000002 19:03:57,258 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0011_01_000002 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:03:58,252 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:03:58,302 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=null 19:03:58,303 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.xml dst=null perm=null 19:03:58,317 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:58,319 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:58,321 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:58,329 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:58,330 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:58,332 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:58,333 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:03:58,333 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:03:58,334 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:03:58,398 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0011_01_000002 by user test 19:03:58,399 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000002 19:03:58,399 INFO Application:296 - Adding container_1394161202967_0011_01_000002 to application application_1394161202967_0011 19:03:58,399 INFO Container:888 - Container container_1394161202967_0011_01_000002 transitioned from NEW to LOCALIZING 19:03:58,399 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0011 19:03:58,399 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0011 19:03:58,399 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:03:58,399 INFO ShuffleHandler:263 - Added token for job_1394161202967_0011 19:03:58,400 INFO Container:888 - Container container_1394161202967_0011_01_000002 transitioned from LOCALIZING to LOCALIZED 19:03:58,479 INFO Container:888 - Container container_1394161202967_0011_01_000002 transitioned from LOCALIZED to RUNNING 19:03:58,479 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:03:58,745 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000002/default_container_executor.sh] 19:03:59,260 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:03:59,268 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:01,039 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741982_1158 19:04:01,040 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157 19:04:01,329 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job.split dst=null perm=null 19:04:01,419 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53688, bytes: 176, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741978_1154, duration: 283000 19:04:01,466 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=null 19:04:01,483 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0011_m_000000_0/part-00000 dst=null perm=test:supergroup:rw-rw-rw- 19:04:01,504 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/0000000-140306190349401-oozie-rkan-W@mr-action dst=null perm=null 19:04:01,508 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/0000000-140306190349401-oozie-rkan-W@mr-action dst=null perm=test:supergroup:rw-rw-rw- 19:04:01,519 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/0000000-140306190349401-oozie-rkan-W@mr-action. BP-1007909283-172.16.1.64-1394161187028 blk_1073741983_1159{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:01,562 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159 src: /127.0.0.1:53689 dest: /127.0.0.1:52682 19:04:01,564 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159 src: /127.0.0.1:53690 dest: /127.0.0.1:52685 19:04:01,598 INFO clienttrace:1215 - src: /127.0.0.1:53690, dest: /127.0.0.1:52685, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159, duration: 33080000 19:04:01,599 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:01,599 INFO clienttrace:1215 - src: /127.0.0.1:53689, dest: /127.0.0.1:52682, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159, duration: 33794000 19:04:01,599 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:01,603 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/0000000-140306190349401-oozie-rkan-W@mr-action is closed by DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1 19:04:01,609 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53691, bytes: 9, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741977_1153, duration: 192000 19:04:01,664 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:04:01,665 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:04:01,676 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53688, bytes: 77872, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741976_1152, duration: 318000 19:04:01,855 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159 19:04:02,349 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/output dst=null perm=null 19:04:02,356 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:02,356 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:02,382 INFO ClientRMService:229 - Allocated new applicationId: 12 19:04:02,404 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012 dst=null perm=null 19:04:02,416 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012 dst=null perm=test:supergroup:rwxrwxrwx 19:04:02,427 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012 dst=null perm=test:supergroup:rwx------ 19:04:02,435 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:02,438 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:02,439 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:02,441 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:02,442 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:04:02,443 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:04:02,444 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:04:02,445 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:04:02,446 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:04:02,448 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:02,449 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:02,450 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib dst=null perm=null 19:04:02,451 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app dst=null perm=null 19:04:02,451 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993 dst=null perm=null 19:04:02,452 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:04:02,465 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/input dst=null perm=null 19:04:02,468 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/input dst=null perm=null 19:04:02,471 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/be638aa0-afac-418b-b432-10208bc03993/input dst=null perm=null 19:04:02,500 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:04:02,503 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=test:supergroup:rw-r--r-- 19:04:02,508 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split 19:04:02,508 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:02,517 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073741984_1160{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:02,519 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160 src: /127.0.0.1:53693 dest: /127.0.0.1:52685 19:04:02,520 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160 src: /127.0.0.1:53694 dest: /127.0.0.1:52682 19:04:02,523 INFO clienttrace:1215 - src: /127.0.0.1:53694, dest: /127.0.0.1:52682, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160, duration: 1466000 19:04:02,523 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:02,524 INFO clienttrace:1215 - src: /127.0.0.1:53693, dest: /127.0.0.1:52685, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160, duration: 2014000 19:04:02,524 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:02,525 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split is closed by DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1 19:04:02,527 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:04:02,529 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:04:02,534 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073741985_1161{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:02,537 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161 src: /127.0.0.1:53695 dest: /127.0.0.1:52685 19:04:02,538 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161 src: /127.0.0.1:53696 dest: /127.0.0.1:52682 19:04:02,540 INFO clienttrace:1215 - src: /127.0.0.1:53696, dest: /127.0.0.1:52682, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161, duration: 1223000 19:04:02,540 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:02,540 INFO clienttrace:1215 - src: /127.0.0.1:53695, dest: /127.0.0.1:52685, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161, duration: 1659000 19:04:02,541 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:02,542 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo is closed by DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1 19:04:02,555 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:04:02,557 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:04:02,591 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741986_1162{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:02,593 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162 src: /127.0.0.1:53697 dest: /127.0.0.1:52685 19:04:02,599 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162 src: /127.0.0.1:53698 dest: /127.0.0.1:52682 19:04:02,602 INFO clienttrace:1215 - src: /127.0.0.1:53698, dest: /127.0.0.1:52682, bytes: 77361, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162, duration: 1774000 19:04:02,602 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:02,602 INFO clienttrace:1215 - src: /127.0.0.1:53697, dest: /127.0.0.1:52685, bytes: 77361, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162, duration: 2101000 19:04:02,603 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:02,604 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml is closed by DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1 19:04:02,784 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012 dst=null perm=null 19:04:02,990 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:02,992 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:02,995 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:02,996 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:02,997 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=null 19:04:02,998 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=null 19:04:03,002 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:03,003 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:03,004 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:03,005 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:03,006 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:03,007 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:03,008 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:03,009 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:03,011 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:03,070 INFO ClientRMService:342 - Application with id 12 submitted by user test 19:04:03,070 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0012 19:04:03,070 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0012 19:04:03,070 INFO RMAppImpl:643 - application_1394161202967_0012 State change from NEW to NEW_SAVING 19:04:03,070 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0012 19:04:03,070 INFO RMAppImpl:643 - application_1394161202967_0012 State change from NEW_SAVING to SUBMITTED 19:04:03,071 INFO FairScheduler:638 - Accepted application application_1394161202967_0012 from user: test, in queue: default, currently num of applications: 2 19:04:03,071 INFO RMAppImpl:643 - application_1394161202967_0012 State change from SUBMITTED to ACCEPTED 19:04:03,071 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0012_000001 19:04:03,071 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from NEW to SUBMITTED 19:04:03,071 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0012_000001 to scheduler from user: test 19:04:03,072 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from SUBMITTED to SCHEDULED 19:04:03,262 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=test:supergroup:rw-rw-rw- 19:04:03,264 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0012 19:04:03,264 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000001 Container Transitioned from NEW to RESERVED 19:04:03,264 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0012 reserved container container_1394161202967_0012_01_000001 on node host: 172.16.1.64:52707 #containers=2 available= used=, currently has 1 at priority 0; currentReservation 2048 19:04:03,264 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0012_01_000001 on node host: 172.16.1.64:52707 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@39ec1ad2 19:04:03,295 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq. BP-1007909283-172.16.1.64-1394161187028 blk_1073741987_1163{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:03,298 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163 src: /127.0.0.1:53700 dest: /127.0.0.1:52685 19:04:03,299 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163 src: /127.0.0.1:53701 dest: /127.0.0.1:52682 19:04:03,301 INFO clienttrace:1215 - src: /127.0.0.1:53701, dest: /127.0.0.1:52682, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163, duration: 1281000 19:04:03,301 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:03,302 INFO clienttrace:1215 - src: /127.0.0.1:53700, dest: /127.0.0.1:52685, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163, duration: 1915000 19:04:03,302 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:03,304 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq is closed by DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1 19:04:03,312 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0011_m_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0011_m_000000_0_1994045683_1 19:04:03,332 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0011_m_000000_0 dst=null perm=null 19:04:03,337 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0011_m_000000_0 dst=null perm=null 19:04:03,338 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0011_m_000000 dst=null perm=null 19:04:03,342 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0011_m_000000_0 dst=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0011_m_000000 perm=test:supergroup:rwxrwxrwx 19:04:03,363 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0011_01_000002 19:04:03,364 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000002 19:04:03,364 INFO Container:888 - Container container_1394161202967_0011_01_000002 transitioned from RUNNING to KILLING 19:04:03,364 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0011_01_000002 19:04:03,426 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0011_01_000002 is : 143 19:04:03,427 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741988_1164{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:03,427 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:04:03,429 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164 src: /127.0.0.1:53702 dest: /127.0.0.1:52685 19:04:03,430 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,431 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164 src: /127.0.0.1:53703 dest: /127.0.0.1:52682 19:04:03,435 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=null 19:04:03,435 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1.jhist for DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,442 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:04:03,443 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:04:03,444 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0011_m_000000 dst=null perm=null 19:04:03,445 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/part-00000 dst=null perm=null 19:04:03,447 INFO Container:888 - Container container_1394161202967_0011_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:04:03,448 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000002 19:04:03,448 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000002 19:04:03,448 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000002 19:04:03,448 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000002 19:04:03,448 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000002 19:04:03,448 INFO Container:888 - Container container_1394161202967_0011_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:04:03,448 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0011_m_000000/part-00000 dst=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/part-00000 perm=test:supergroup:rw-rw-rw- 19:04:03,449 INFO Application:339 - Removing container_1394161202967_0011_01_000002 from application application_1394161202967_0011 19:04:03,449 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:03,449 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0011 19:04:03,454 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_temporary dst=null perm=null 19:04:03,458 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:04:03,460 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,461 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:04:03,463 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,478 INFO clienttrace:1215 - src: /127.0.0.1:53703, dest: /127.0.0.1:52682, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164, duration: 45867000 19:04:03,478 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:03,480 INFO clienttrace:1215 - src: /127.0.0.1:53702, dest: /127.0.0.1:52685, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164, duration: 47363000 19:04:03,480 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:03,482 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1.jhist is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,485 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:03,487 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741989_1165{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:03,489 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165 src: /127.0.0.1:53704 dest: /127.0.0.1:52682 19:04:03,491 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165 src: /127.0.0.1:53705 dest: /127.0.0.1:52685 19:04:03,493 INFO clienttrace:1215 - src: /127.0.0.1:53705, dest: /127.0.0.1:52685, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165, duration: 1237000 19:04:03,493 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:03,494 INFO clienttrace:1215 - src: /127.0.0.1:53704, dest: /127.0.0.1:52682, bytes: 416, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165, duration: 1864000 19:04:03,494 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:03,496 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary_tmp is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,499 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:03,506 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1.jhist dst=null perm=null 19:04:03,507 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist_tmp dst=null perm=null 19:04:03,511 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1.jhist dst=null perm=null 19:04:03,512 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist_tmp dst=null perm=null 19:04:03,513 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1.jhist dst=null perm=null 19:04:03,516 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:03,518 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53706, bytes: 20861, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164, duration: 278000 19:04:03,520 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741990_1166{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:03,522 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166 src: /127.0.0.1:53707 dest: /127.0.0.1:52685 19:04:03,524 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166 src: /127.0.0.1:53708 dest: /127.0.0.1:52682 19:04:03,527 INFO clienttrace:1215 - src: /127.0.0.1:53708, dest: /127.0.0.1:52682, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166, duration: 1450000 19:04:03,527 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:03,527 INFO clienttrace:1215 - src: /127.0.0.1:53707, dest: /127.0.0.1:52685, bytes: 20697, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166, duration: 2122000 19:04:03,528 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:03,529 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist_tmp is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,531 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:03,532 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1_conf.xml dst=null perm=null 19:04:03,532 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml_tmp dst=null perm=null 19:04:03,533 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1_conf.xml dst=null perm=null 19:04:03,534 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml_tmp dst=null perm=null 19:04:03,535 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011/job_1394161202967_0011_1_conf.xml dst=null perm=null 19:04:03,537 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:03,539 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53706, bytes: 93643, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741981_1157, duration: 322000 19:04:03,541 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741991_1167{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:03,543 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167 src: /127.0.0.1:53709 dest: /127.0.0.1:52682 19:04:03,544 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167 src: /127.0.0.1:53710 dest: /127.0.0.1:52685 19:04:03,547 INFO clienttrace:1215 - src: /127.0.0.1:53710, dest: /127.0.0.1:52685, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167, duration: 2119000 19:04:03,547 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:03,548 INFO clienttrace:1215 - src: /127.0.0.1:53709, dest: /127.0.0.1:52682, bytes: 92915, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-120395903_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167, duration: 2799000 19:04:03,548 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:03,550 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_-120395903_1 19:04:03,551 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:03,553 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary perm=test:supergroup:rwxrwx--- 19:04:03,554 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml perm=test:supergroup:rwxrwx--- 19:04:03,556 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist perm=test:supergroup:rwxrwx--- 19:04:03,562 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0011_000001 with final state: FINISHING 19:04:03,562 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from RUNNING to FINAL_SAVING 19:04:03,562 INFO RMAppImpl:880 - Updating application application_1394161202967_0011 with final state: FINISHING 19:04:03,563 INFO RMAppImpl:643 - application_1394161202967_0011 State change from RUNNING to FINAL_SAVING 19:04:03,563 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0011 19:04:03,563 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from FINAL_SAVING to FINISHING 19:04:03,563 INFO RMAppImpl:643 - application_1394161202967_0011 State change from FINAL_SAVING to FINISHING 19:04:03,835 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000002 Container Transitioned from NEW to ALLOCATED 19:04:03,835 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000002 19:04:03,835 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:04:03,836 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:04:03,836 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0012 AttemptId: appattempt_1394161202967_0012_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0012_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] 19:04:03,836 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:04:03,837 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:04:03,837 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0012_000001 19:04:03,838 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0012_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0012_000001 19:04:03,838 INFO AMLauncher:190 - Command to launch container container_1394161202967_0012_01_000002 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:04:03,844 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0012_01_000002 by user test 19:04:03,844 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0012 19:04:03,844 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000002 19:04:03,845 INFO Application:450 - Application application_1394161202967_0012 transitioned from NEW to INITING 19:04:03,845 INFO Application:296 - Adding container_1394161202967_0012_01_000002 to application application_1394161202967_0012 19:04:03,845 INFO Application:450 - Application application_1394161202967_0012 transitioned from INITING to RUNNING 19:04:03,845 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0012_01_000002, NodeId: 172.16.1.64:52703, NodeHttpAddress: 172.16.1.64:52706, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52703 }, ] for AM appattempt_1394161202967_0012_000001 19:04:03,845 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from ALLOCATED to LAUNCHED 19:04:03,845 INFO Container:888 - Container container_1394161202967_0012_01_000002 transitioned from NEW to LOCALIZING 19:04:03,845 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0012 19:04:03,845 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:04:03,846 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split transitioned from INIT to DOWNLOADING 19:04:03,846 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml transitioned from INIT to DOWNLOADING 19:04:03,846 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:04:03,846 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0012_01_000002 19:04:03,846 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar, 1394161429690, FILE, null } 19:04:03,928 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0012_01_000002.tokens. Credentials list: 19:04:04,036 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:04,077 INFO DefaultContainerExecutor:469 - Initializing user test 19:04:04,078 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:04,115 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib dst=null perm=null 19:04:04,145 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app dst=null perm=null 19:04:04,209 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993 dst=null perm=null 19:04:04,239 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:04:04,273 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:04:04,323 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:04:04,391 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0011_01_000002 19:04:04,391 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:04,392 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000002 Container Transitioned from RUNNING to COMPLETED 19:04:04,412 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0011_01_000002 in state: COMPLETED event:FINISHED 19:04:04,414 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000002 19:04:04,415 INFO FSSchedulerNode:162 - Released container container_1394161202967_0011_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:04:04,415 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0011_000001 released container container_1394161202967_0011_01_000002 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:04:04,416 INFO FairScheduler:1042 - Releasing reservation that cannot be satisfied for application appattempt_1394161202967_0012_000001 on node host: 172.16.1.64:52707 #containers=1 available= used= 19:04:04,416 INFO FSSchedulerApp:123 - Application application_1394161202967_0012 unreserved on node host: 172.16.1.64:52707 #containers=1 available= used=, currently has 0 at priority 0; currentReservation 19:04:04,417 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:04,488 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53715, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741974_1150, duration: 277000 19:04:04,617 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/filecache/18/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:04:04,623 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0011 dst=null perm=null 19:04:04,639 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0012_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000002.tokens 19:04:04,639 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0012 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0012 19:04:04,832 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=null 19:04:04,833 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=null 19:04:04,835 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=null 19:04:04,835 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:04:04,851 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53715, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-973653134_3824, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161, duration: 236000 19:04:04,896 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0012/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:04:04,949 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:04,950 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:04,952 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:04,970 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53719, bytes: 283, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-973653134_3824, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160, duration: 206000 19:04:05,015 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0012/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:04:05,075 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:05,076 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:05,077 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:05,100 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53715, bytes: 77969, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-973653134_3824, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162, duration: 453000 19:04:05,153 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0012/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:04:05,153 INFO Container:888 - Container container_1394161202967_0012_01_000002 transitioned from LOCALIZING to LOCALIZED 19:04:05,234 INFO Container:888 - Container container_1394161202967_0012_01_000002 transitioned from LOCALIZED to RUNNING 19:04:05,234 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:05,443 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000002/default_container_executor.sh] 19:04:05,474 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:05,477 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:05,478 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:05,480 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53719, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163, duration: 209000 19:04:05,513 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161445513, maxDate=1394766245513, sequenceNumber=35, masterKeyId=2 19:04:05,514 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 35 19:04:05,522 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:04:05,524 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:04:05,525 INFO CompletedJob:95 - Loading job: job_1394161202967_0011 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist 19:04:05,526 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist] 19:04:05,527 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist dst=null perm=null 19:04:05,528 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist dst=null perm=null 19:04:05,530 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53719, bytes: 20861, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166, duration: 188000 19:04:05,555 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary dst=null perm=null 19:04:05,558 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53719, bytes: 420, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165, duration: 222000 19:04:05,558 INFO JobSummary:372 - jobId=job_1394161202967_0011,submitTime=1394161429944,launchTime=1394161436219,firstMapTaskLaunchTime=1394161438404,firstReduceTaskLaunchTime=0,finishTime=1394161443464,resourcesPerMap=1024,resourcesPerReduce=0,numMaps=1,numReduces=0,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=4,reduceSlotSeconds=0,jobName=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190349401-oozie-rkan-W 19:04:05,558 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary] 19:04:05,561 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011.summary dst=null perm=null 19:04:05,563 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:04:05,564 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist 19:04:05,566 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0011-1394161429944-test-oozie%3Alauncher%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce-1394161443464-1-0-SUCCEEDED-root.test-1394161436219.jhist perm=test:supergroup:rwxrwx--- 19:04:05,566 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0011_conf.xml 19:04:05,568 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0011_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0011_conf.xml perm=test:supergroup:rwxrwx--- 19:04:05,595 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:05,596 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:05,597 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190349401-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:05,599 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53715, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-215555625_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163, duration: 181000 19:04:05,653 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161445653, maxDate=1394766245653, sequenceNumber=36, masterKeyId=2 19:04:05,653 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 36 19:04:06,045 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164 19:04:06,045 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166 19:04:06,045 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741983_1159 19:04:06,046 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165 19:04:06,047 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167 19:04:06,047 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163 19:04:06,048 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162 19:04:06,048 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160 19:04:06,048 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161 19:04:06,677 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741988_1164, blk_1073741989_1165, blk_1073741978_1154, blk_1073741979_1155, blk_1073741980_1156, blk_1073741981_1157, blk_1073741982_1158] 19:04:06,861 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741987_1163 19:04:06,861 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741988_1164 19:04:06,862 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741990_1166 19:04:06,862 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741989_1165 19:04:06,863 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161 19:04:06,864 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741991_1167 19:04:06,864 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160 19:04:06,865 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162 19:04:07,711 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:07,744 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/COMMIT_STARTED dst=null perm=null 19:04:07,746 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/COMMIT_SUCCESS dst=null perm=null 19:04:07,748 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/COMMIT_FAIL dst=null perm=null 19:04:07,819 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012 dst=null perm=null 19:04:07,822 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:04:07,824 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:04:08,045 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741988_1164 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741988 for deletion 19:04:08,045 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741989_1165 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741989 for deletion 19:04:08,045 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741978_1154 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741978 for deletion 19:04:08,046 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741979_1155 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741979 for deletion 19:04:08,046 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741980_1156 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741980 for deletion 19:04:08,046 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741988_1164 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741988 19:04:08,046 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741981_1157 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741981 for deletion 19:04:08,046 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741989_1165 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741989 19:04:08,046 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741982_1158 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741982 for deletion 19:04:08,046 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741978_1154 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741978 19:04:08,046 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741979_1155 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741979 19:04:08,046 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741980_1156 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741980 19:04:08,046 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741981_1157 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741981 19:04:08,047 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741982_1158 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741982 19:04:08,209 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=null 19:04:08,217 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.splitmetainfo dst=null perm=null 19:04:08,277 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53724, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741985_1161, duration: 193000 19:04:09,260 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0012_000001 19:04:09,260 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0012 APPATTEMPTID=appattempt_1394161202967_0012_000001 19:04:09,260 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from LAUNCHED to RUNNING 19:04:09,260 INFO RMAppImpl:643 - application_1394161202967_0012 State change from ACCEPTED to RUNNING 19:04:09,286 INFO FSEditLog:674 - Number of transactions: 1303 Total time for transactions(ms): 394 Number of transactions batched in Syncs: 16 Number of syncs: 884 SyncTimes(ms): 615 202 19:04:09,288 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:04:09,290 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/be638aa0-afac-418b-b432-10208bc03993/output/_temporary/1 dst=null perm=test:testg:rwxrwxrwx 19:04:09,349 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:04:09,554 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073741992_1168{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:09,607 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168 src: /127.0.0.1:53729 dest: /127.0.0.1:52682 19:04:09,609 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168 src: /127.0.0.1:53730 dest: /127.0.0.1:52685 19:04:09,644 INFO clienttrace:1215 - src: /127.0.0.1:53730, dest: /127.0.0.1:52685, bytes: 91664, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168, duration: 34615000 19:04:09,645 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:09,645 INFO clienttrace:1215 - src: /127.0.0.1:53729, dest: /127.0.0.1:52682, bytes: 91664, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168, duration: 35301000 19:04:09,646 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:09,649 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1_conf.xml is closed by DFSClient_NONMAPREDUCE_-1379732878_1 19:04:09,676 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012.summary dst=null perm=test:supergroup:rw-rw-rw- 19:04:09,677 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741988_1164, blk_1073741989_1165, blk_1073741978_1154, blk_1073741979_1155, blk_1073741980_1156, blk_1073741981_1157, blk_1073741982_1158] 19:04:09,678 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073741993_1169{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:09,681 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169 src: /127.0.0.1:53731 dest: /127.0.0.1:52682 19:04:09,682 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169 src: /127.0.0.1:53732 dest: /127.0.0.1:52685 19:04:09,686 INFO clienttrace:1215 - src: /127.0.0.1:53732, dest: /127.0.0.1:52685, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169, duration: 2189000 19:04:09,686 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:09,687 INFO clienttrace:1215 - src: /127.0.0.1:53731, dest: /127.0.0.1:52682, bytes: 370, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169, duration: 3103000 19:04:09,687 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:09,689 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012.summary is closed by DFSClient_NONMAPREDUCE_-1379732878_1 19:04:10,408 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000003 Container Transitioned from NEW to ALLOCATED 19:04:10,409 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000003 19:04:10,409 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000003 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:04:10,788 INFO ContainerLaunch:322 - Container container_1394161202967_0011_01_000001 succeeded 19:04:10,788 INFO Container:888 - Container container_1394161202967_0011_01_000001 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:04:10,788 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0011_01_000001 19:04:10,802 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000001 19:04:10,802 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000001 19:04:10,803 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000001 19:04:10,803 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0011/container_1394161202967_0011_01_000001 19:04:10,803 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000001 19:04:10,803 INFO Container:888 - Container container_1394161202967_0011_01_000001 transitioned from EXITED_WITH_SUCCESS to DONE 19:04:10,803 INFO Application:339 - Removing container_1394161202967_0011_01_000001 from application application_1394161202967_0011 19:04:10,803 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:10,803 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0011 19:04:10,840 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000004 Container Transitioned from NEW to ALLOCATED 19:04:10,841 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000004 19:04:10,841 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000004 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:04:11,046 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741988_1164 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741988 for deletion 19:04:11,047 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741989_1165 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741989 for deletion 19:04:11,047 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741978_1154 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741978 for deletion 19:04:11,047 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741979_1155 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741979 for deletion 19:04:11,047 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741988_1164 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741988 19:04:11,047 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741989_1165 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741989 19:04:11,047 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741980_1156 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741980 for deletion 19:04:11,048 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741978_1154 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741978 19:04:11,048 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741981_1157 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741981 for deletion 19:04:11,048 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741979_1155 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741979 19:04:11,048 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741980_1156 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741980 19:04:11,048 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741982_1158 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741982 for deletion 19:04:11,049 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741981_1157 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741981 19:04:11,049 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741982_1158 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741982 19:04:11,055 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168 19:04:11,055 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169 19:04:11,314 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:04:11,315 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 19:04:11,368 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:11,370 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:11,381 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:11,382 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:11,384 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:11,392 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:11,393 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:11,394 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:11,395 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:11,396 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:11,397 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/app/lib/test.jar dst=null perm=null 19:04:11,408 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0011_01_000001 19:04:11,408 INFO RMContainerImpl:220 - container_1394161202967_0011_01_000001 Container Transitioned from RUNNING to COMPLETED 19:04:11,408 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0011_000001 19:04:11,408 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0011_01_000001 in state: COMPLETED event:FINISHED 19:04:11,408 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0011_000001 State change from FINISHING to FINISHED 19:04:11,408 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0011 CONTAINERID=container_1394161202967_0011_01_000001 19:04:11,408 INFO RMAppImpl:643 - application_1394161202967_0011 State change from FINISHING to FINISHED 19:04:11,409 INFO FSSchedulerNode:162 - Released container container_1394161202967_0011_01_000001 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:04:11,409 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0011_000001 released container container_1394161202967_0011_01_000001 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:04:11,409 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0011 19:04:11,409 INFO FairScheduler:720 - Application appattempt_1394161202967_0011_000001 is done. finalState=FINISHED 19:04:11,409 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0011_000001 19:04:11,409 INFO AppSchedulingInfo:108 - Application application_1394161202967_0011 requests cleared 19:04:11,409 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0011,name=oozie:launcher:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190349401-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0011/jobhistory/job/job_1394161202967_0011,appMasterHost=rkanter-mbp.local,startTime=1394161429944,finishTime=1394161443562,finalStatus=SUCCEEDED 19:04:11,414 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0011_01_000001 19:04:11,457 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0012_01_000003 by user test 19:04:11,458 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0012 19:04:11,458 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000003 19:04:11,458 INFO Application:450 - Application application_1394161202967_0012 transitioned from NEW to INITING 19:04:11,458 INFO Application:296 - Adding container_1394161202967_0012_01_000003 to application application_1394161202967_0012 19:04:11,458 INFO Application:450 - Application application_1394161202967_0012 transitioned from INITING to RUNNING 19:04:11,458 INFO Container:888 - Container container_1394161202967_0012_01_000003 transitioned from NEW to LOCALIZING 19:04:11,458 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0012 19:04:11,459 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0012 19:04:11,459 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:04:11,459 INFO ShuffleHandler:263 - Added token for job_1394161202967_0012 19:04:11,459 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml transitioned from INIT to DOWNLOADING 19:04:11,459 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0012_01_000003 19:04:11,484 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/nmPrivate/container_1394161202967_0012_01_000003.tokens. Credentials list: 19:04:11,513 INFO DefaultContainerExecutor:469 - Initializing user test 19:04:11,717 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/nmPrivate/container_1394161202967_0012_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000003.tokens 19:04:11,718 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012 19:04:11,870 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:11,872 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168 19:04:11,872 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:11,872 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741993_1169 19:04:11,873 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml dst=null perm=null 19:04:11,896 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53738, bytes: 77969, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1780611197_3927, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741986_1162, duration: 339000 19:04:11,950 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:04:11,950 INFO Container:888 - Container container_1394161202967_0012_01_000003 transitioned from LOCALIZING to LOCALIZED 19:04:12,029 INFO Container:888 - Container container_1394161202967_0012_01_000003 transitioned from LOCALIZED to RUNNING 19:04:12,032 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:12,256 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000003/default_container_executor.sh] 19:04:12,333 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000004 Container Transitioned from ACQUIRED to RELEASED 19:04:12,333 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000004 in state: RELEASED event:RELEASED 19:04:12,333 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000004 19:04:12,333 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000004 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:04:12,333 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000004 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:04:12,334 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:12,409 INFO Application:450 - Application application_1394161202967_0011 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:04:12,409 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:04:12,409 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0011 19:04:12,410 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000005 Container Transitioned from NEW to ALLOCATED 19:04:12,410 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000005 19:04:12,410 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0011 19:04:12,410 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0011 19:04:12,410 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0011 19:04:12,410 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0011 19:04:12,410 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000005 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:04:12,410 INFO Application:450 - Application application_1394161202967_0011 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:04:12,410 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0011, with delay of 10800 seconds 19:04:12,847 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0012_01_000004 19:04:13,341 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000005 Container Transitioned from ALLOCATED to ACQUIRED 19:04:13,347 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0012_01_000005 by user test 19:04:13,347 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000005 19:04:13,347 INFO Application:296 - Adding container_1394161202967_0012_01_000005 to application application_1394161202967_0012 19:04:13,348 INFO Container:888 - Container container_1394161202967_0012_01_000005 transitioned from NEW to LOCALIZING 19:04:13,348 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0012 19:04:13,348 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0012 19:04:13,348 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:04:13,348 INFO ShuffleHandler:263 - Added token for job_1394161202967_0012 19:04:13,349 INFO Container:888 - Container container_1394161202967_0012_01_000005 transitioned from LOCALIZING to LOCALIZED 19:04:13,433 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000005 Container Transitioned from ACQUIRED to RUNNING 19:04:13,449 INFO Container:888 - Container container_1394161202967_0012_01_000005 transitioned from LOCALIZED to RUNNING 19:04:13,452 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:13,756 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000005/default_container_executor.sh] 19:04:14,347 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:15,371 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:15,462 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53741, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0012_m_000000_0_-525974952_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160, duration: 219000 19:04:15,512 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/be638aa0-afac-418b-b432-10208bc03993/input/data.txt dst=null perm=null 19:04:15,584 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53741, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0012_m_000000_0_-525974952_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149, duration: 271000 19:04:15,610 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/output/_temporary/1/_temporary/attempt_1394161202967_0012_m_000000_0 dst=null perm=null 19:04:15,636 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0012_01_000003 19:04:15,636 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000003 19:04:15,636 INFO Container:888 - Container container_1394161202967_0012_01_000003 transitioned from RUNNING to KILLING 19:04:15,636 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0012_01_000003 19:04:15,676 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073741994_1170{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:15,699 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0012_01_000003 is : 143 19:04:15,702 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170 src: /127.0.0.1:53743 dest: /127.0.0.1:52685 19:04:15,704 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170 src: /127.0.0.1:53744 dest: /127.0.0.1:52682 19:04:15,709 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1.jhist for DFSClient_NONMAPREDUCE_-1379732878_1 19:04:15,724 INFO Container:888 - Container container_1394161202967_0012_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:04:15,724 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000003 19:04:15,725 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000003 19:04:15,725 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000003 19:04:15,725 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000003 19:04:15,725 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000003 19:04:15,725 INFO Container:888 - Container container_1394161202967_0012_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:04:15,725 INFO Application:339 - Removing container_1394161202967_0012_01_000003 from application application_1394161202967_0012 19:04:15,725 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:15,726 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0012 19:04:16,637 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0012_01_000003 19:04:16,638 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000003 Container Transitioned from RUNNING to COMPLETED 19:04:16,638 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000003 in state: COMPLETED event:FINISHED 19:04:16,638 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000003 19:04:16,639 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000003 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:04:16,639 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000003 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:04:16,733 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job.split dst=null perm=null 19:04:16,812 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53746, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0012_m_000001_1_1686053332_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741984_1160, duration: 520000 19:04:16,861 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/be638aa0-afac-418b-b432-10208bc03993/input/data.txt dst=null perm=null 19:04:16,866 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53747, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0012_m_000001_1_1686053332_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741973_1149, duration: 328000 19:04:16,957 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/be638aa0-afac-418b-b432-10208bc03993/output/_temporary/1/_temporary/attempt_1394161202967_0012_m_000001_1 dst=null perm=null 19:04:16,969 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0012_01_000005 19:04:16,969 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000005 19:04:16,969 INFO Container:888 - Container container_1394161202967_0012_01_000005 transitioned from RUNNING to KILLING 19:04:16,969 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0012_01_000005 19:04:17,015 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0012_01_000005 is : 143 19:04:17,033 INFO Container:888 - Container container_1394161202967_0012_01_000005 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:04:17,033 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000005 19:04:17,034 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000005 19:04:17,034 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000005 19:04:17,034 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000005 19:04:17,034 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000005 19:04:17,034 INFO Container:888 - Container container_1394161202967_0012_01_000005 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:04:17,035 INFO Application:339 - Removing container_1394161202967_0012_01_000005 from application application_1394161202967_0012 19:04:17,035 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:17,035 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0012 19:04:17,854 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000006 Container Transitioned from NEW to ALLOCATED 19:04:17,855 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000006 19:04:17,855 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000006 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:04:17,971 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0012_01_000005 19:04:17,972 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000005 Container Transitioned from RUNNING to COMPLETED 19:04:17,972 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000005 in state: COMPLETED event:FINISHED 19:04:17,973 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000005 19:04:17,973 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000005 of capacity on host 172.16.1.64:52707, which currently has 0 containers, used and available, release resources=true 19:04:17,973 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000005 on node: host: 172.16.1.64:52707 #containers=0 available= used= with event: FINISHED 19:04:18,371 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000006 Container Transitioned from ALLOCATED to ACQUIRED 19:04:19,379 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000006 Container Transitioned from ACQUIRED to RELEASED 19:04:19,379 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000006 in state: RELEASED event:RELEASED 19:04:19,380 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000006 19:04:19,380 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000006 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:04:19,380 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000006 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:04:19,380 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:19,856 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0012_01_000006 19:04:20,857 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000007 Container Transitioned from NEW to ALLOCATED 19:04:20,857 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000007 19:04:20,858 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000007 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:04:21,393 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000007 Container Transitioned from ALLOCATED to ACQUIRED 19:04:22,401 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000007 Container Transitioned from ACQUIRED to RELEASED 19:04:22,401 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000007 in state: RELEASED event:RELEASED 19:04:22,402 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000007 19:04:22,402 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000007 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:04:22,402 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000007 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:04:22,402 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:22,860 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0012_01_000007 19:04:23,862 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000008 Container Transitioned from NEW to ALLOCATED 19:04:23,862 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000008 19:04:23,862 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000008 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:04:24,415 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000008 Container Transitioned from ALLOCATED to ACQUIRED 19:04:25,423 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000008 Container Transitioned from ACQUIRED to RELEASED 19:04:25,424 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000008 in state: RELEASED event:RELEASED 19:04:25,424 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000008 19:04:25,424 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000008 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:04:25,424 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000008 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: RELEASED 19:04:25,424 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:25,864 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0012_01_000008 19:04:26,866 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000009 Container Transitioned from NEW to ALLOCATED 19:04:26,866 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000009 19:04:26,866 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0012_01_000009 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:04:27,435 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000009 Container Transitioned from ALLOCATED to ACQUIRED 19:04:27,449 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/be638aa0-afac-418b-b432-10208bc03993/output/_temporary dst=null perm=null 19:04:27,458 INFO clienttrace:1215 - src: /127.0.0.1:53744, dest: /127.0.0.1:52682, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170, duration: 11752487000 19:04:27,459 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:27,459 INFO clienttrace:1215 - src: /127.0.0.1:53743, dest: /127.0.0.1:52685, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170, duration: 11753770000 19:04:27,460 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:27,461 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1.jhist is closed by DFSClient_NONMAPREDUCE_-1379732878_1 19:04:27,464 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:27,465 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741995_1171{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:27,468 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171 src: /127.0.0.1:53749 dest: /127.0.0.1:52682 19:04:27,469 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171 src: /127.0.0.1:53750 dest: /127.0.0.1:52685 19:04:27,471 INFO clienttrace:1215 - src: /127.0.0.1:53750, dest: /127.0.0.1:52685, bytes: 414, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171, duration: 1212000 19:04:27,472 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:27,472 INFO clienttrace:1215 - src: /127.0.0.1:53749, dest: /127.0.0.1:52682, bytes: 414, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171, duration: 1883000 19:04:27,472 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:27,474 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary_tmp is closed by DFSClient_NONMAPREDUCE_-1379732878_1 19:04:27,477 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:27,482 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1.jhist dst=null perm=null 19:04:27,484 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist_tmp dst=null perm=null 19:04:27,488 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1.jhist dst=null perm=null 19:04:27,489 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist_tmp dst=null perm=null 19:04:27,490 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1.jhist dst=null perm=null 19:04:27,492 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:27,494 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53751, bytes: 23548, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170, duration: 212000 19:04:27,496 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741996_1172{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:27,498 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172 src: /127.0.0.1:53752 dest: /127.0.0.1:52682 19:04:27,499 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172 src: /127.0.0.1:53753 dest: /127.0.0.1:52685 19:04:27,502 INFO clienttrace:1215 - src: /127.0.0.1:53753, dest: /127.0.0.1:52685, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172, duration: 1402000 19:04:27,502 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:27,502 INFO clienttrace:1215 - src: /127.0.0.1:53752, dest: /127.0.0.1:52682, bytes: 23364, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172, duration: 1978000 19:04:27,502 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:27,504 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist_tmp is closed by DFSClient_NONMAPREDUCE_-1379732878_1 19:04:27,506 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:27,506 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1_conf.xml dst=null perm=null 19:04:27,507 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml_tmp dst=null perm=null 19:04:27,508 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1_conf.xml dst=null perm=null 19:04:27,509 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml_tmp dst=null perm=null 19:04:27,510 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012/job_1394161202967_0012_1_conf.xml dst=null perm=null 19:04:27,512 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:27,514 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53751, bytes: 92384, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741992_1168, duration: 318000 19:04:27,516 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073741997_1173{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:27,518 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173 src: /127.0.0.1:53754 dest: /127.0.0.1:52685 19:04:27,519 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173 src: /127.0.0.1:53755 dest: /127.0.0.1:52682 19:04:27,522 INFO clienttrace:1215 - src: /127.0.0.1:53755, dest: /127.0.0.1:52682, bytes: 91664, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173, duration: 2372000 19:04:27,523 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:27,523 INFO clienttrace:1215 - src: /127.0.0.1:53754, dest: /127.0.0.1:52685, bytes: 91664, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1379732878_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173, duration: 2267000 19:04:27,523 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:27,525 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_-1379732878_1 19:04:27,526 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:27,530 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary perm=test:supergroup:rwxrwx--- 19:04:27,533 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml perm=test:supergroup:rwxrwx--- 19:04:27,535 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist perm=test:supergroup:rwxrwx--- 19:04:27,541 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0012_000001 with final state: FINISHING 19:04:27,542 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from RUNNING to FINAL_SAVING 19:04:27,542 INFO RMAppImpl:880 - Updating application application_1394161202967_0012 with final state: FINISHING 19:04:27,542 INFO RMAppImpl:643 - application_1394161202967_0012 State change from RUNNING to FINAL_SAVING 19:04:27,542 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0012 19:04:27,543 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from FINAL_SAVING to FINISHING 19:04:27,543 INFO RMAppImpl:643 - application_1394161202967_0012 State change from FINAL_SAVING to FINISHING 19:04:28,553 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0012 dst=null perm=null 19:04:29,417 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input/data.txt dst=null perm=test:testg:rw-rw-rw- 19:04:29,421 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input/data.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073741998_1174{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:29,423 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174 src: /127.0.0.1:53758 dest: /127.0.0.1:52682 19:04:29,424 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174 src: /127.0.0.1:53759 dest: /127.0.0.1:52685 19:04:29,426 INFO clienttrace:1215 - src: /127.0.0.1:53759, dest: /127.0.0.1:52685, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1884198911_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174, duration: 909000 19:04:29,426 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,426 INFO clienttrace:1215 - src: /127.0.0.1:53758, dest: /127.0.0.1:52682, bytes: 12, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1884198911_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174, duration: 1296000 19:04:29,427 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,428 INFO StateChange:2959 - DIR* completeFile: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input/data.txt is closed by DFSClient_NONMAPREDUCE_1884198911_8 19:04:29,436 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:04:29,438 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073741999_1175{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:29,440 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175 src: /127.0.0.1:53760 dest: /127.0.0.1:52685 19:04:29,441 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175 src: /127.0.0.1:53761 dest: /127.0.0.1:52682 19:04:29,443 INFO clienttrace:1215 - src: /127.0.0.1:53761, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1884198911_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175, duration: 1186000 19:04:29,443 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,443 INFO clienttrace:1215 - src: /127.0.0.1:53760, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1884198911_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175, duration: 1549000 19:04:29,443 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,446 INFO StateChange:2959 - DIR* completeFile: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_1884198911_8 19:04:29,448 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:04:29,449 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073742000_1176{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:29,450 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176 src: /127.0.0.1:53762 dest: /127.0.0.1:52682 19:04:29,451 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176 src: /127.0.0.1:53763 dest: /127.0.0.1:52685 19:04:29,453 INFO clienttrace:1215 - src: /127.0.0.1:53763, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1884198911_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176, duration: 975000 19:04:29,454 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,454 INFO clienttrace:1215 - src: /127.0.0.1:53762, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1884198911_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176, duration: 1479000 19:04:29,454 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,456 INFO StateChange:2959 - DIR* completeFile: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_1884198911_8 19:04:29,460 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java dst=null perm=null 19:04:29,461 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java.tmp dst=null perm=test:supergroup:rwxrwxrwx 19:04:29,463 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java.tmp dst=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java perm=test:supergroup:rwxrwxrwx 19:04:29,463 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/workflow.xml dst=null perm=null 19:04:29,507 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/workflow.xml dst=null perm=null 19:04:29,542 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java dst=null perm=test:supergroup:rwxrwxrwx 19:04:29,545 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=test:supergroup:rw-rw-rw- 19:04:29,555 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073742001_1177{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:29,557 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177 src: /127.0.0.1:53765 dest: /127.0.0.1:52685 19:04:29,558 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177 src: /127.0.0.1:53766 dest: /127.0.0.1:52682 19:04:29,561 INFO clienttrace:1215 - src: /127.0.0.1:53766, dest: /127.0.0.1:52682, bytes: 77446, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177, duration: 1426000 19:04:29,561 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,561 INFO clienttrace:1215 - src: /127.0.0.1:53765, dest: /127.0.0.1:52685, bytes: 77446, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177, duration: 1956000 19:04:29,562 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,564 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action.xml is closed by DFSClient_NONMAPREDUCE_-1007186919_8 19:04:29,565 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input dst=null perm=test:supergroup:rwxrwxrwx 19:04:29,567 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=test:supergroup:rw-rw-rw- 19:04:29,568 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input/dummy.txt. BP-1007909283-172.16.1.64-1394161187028 blk_1073742002_1178{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:29,569 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178 src: /127.0.0.1:53767 dest: /127.0.0.1:52685 19:04:29,570 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178 src: /127.0.0.1:53768 dest: /127.0.0.1:52682 19:04:29,573 INFO clienttrace:1215 - src: /127.0.0.1:53768, dest: /127.0.0.1:52682, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178, duration: 1151000 19:04:29,573 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,573 INFO clienttrace:1215 - src: /127.0.0.1:53767, dest: /127.0.0.1:52685, bytes: 5, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178, duration: 1569000 19:04:29,573 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,575 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input/dummy.txt is closed by DFSClient_NONMAPREDUCE_-1007186919_8 19:04:29,592 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161469592, maxDate=1394766269592, sequenceNumber=37, masterKeyId=2 19:04:29,592 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 37 19:04:29,594 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/0000000-140306190429176-oozie-rkan-W@mr-action dst=null perm=null 19:04:29,594 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161469594, maxDate=1394766269594, sequenceNumber=38, masterKeyId=2 19:04:29,594 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 38 19:04:29,610 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:04:29,612 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:29,614 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:29,615 INFO ClientRMService:229 - Allocated new applicationId: 13 19:04:29,616 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013 dst=null perm=null 19:04:29,618 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013 dst=null perm=test:supergroup:rwxrwxrwx 19:04:29,620 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013 dst=null perm=test:supergroup:rwx------ 19:04:29,620 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:29,621 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:29,622 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:29,623 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:29,623 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:04:29,624 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:04:29,624 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:04:29,625 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:04:29,625 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:04:29,626 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:29,627 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:29,627 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib dst=null perm=null 19:04:29,628 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app dst=null perm=null 19:04:29,629 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d dst=null perm=null 19:04:29,629 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:04:29,630 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:04:29,631 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:04:29,632 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input dst=null perm=null 19:04:29,634 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:04:29,636 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=test:supergroup:rw-r--r-- 19:04:29,637 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split 19:04:29,637 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=null 19:04:29,638 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073742003_1179{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:29,640 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179 src: /127.0.0.1:53770 dest: /127.0.0.1:52685 19:04:29,641 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179 src: /127.0.0.1:53771 dest: /127.0.0.1:52682 19:04:29,644 INFO clienttrace:1215 - src: /127.0.0.1:53771, dest: /127.0.0.1:52682, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179, duration: 1497000 19:04:29,644 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,644 INFO clienttrace:1215 - src: /127.0.0.1:53770, dest: /127.0.0.1:52685, bytes: 172, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179, duration: 1595000 19:04:29,644 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,646 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split is closed by DFSClient_NONMAPREDUCE_-1007186919_8 19:04:29,671 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:04:29,673 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:04:29,674 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073742004_1180{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:29,676 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180 src: /127.0.0.1:53772 dest: /127.0.0.1:52682 19:04:29,677 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180 src: /127.0.0.1:53773 dest: /127.0.0.1:52685 19:04:29,680 INFO clienttrace:1215 - src: /127.0.0.1:53773, dest: /127.0.0.1:52685, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180, duration: 1078000 19:04:29,680 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,680 INFO clienttrace:1215 - src: /127.0.0.1:53772, dest: /127.0.0.1:52682, bytes: 33, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180, duration: 1556000 19:04:29,680 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,682 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_-1007186919_8 19:04:29,684 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:04:29,686 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:04:29,697 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073742005_1181{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:29,700 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181 src: /127.0.0.1:53774 dest: /127.0.0.1:52682 19:04:29,702 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181 src: /127.0.0.1:53775 dest: /127.0.0.1:52685 19:04:29,706 INFO clienttrace:1215 - src: /127.0.0.1:53775, dest: /127.0.0.1:52685, bytes: 80113, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181, duration: 1947000 19:04:29,706 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:29,707 INFO clienttrace:1215 - src: /127.0.0.1:53774, dest: /127.0.0.1:52682, bytes: 80113, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181, duration: 2696000 19:04:29,707 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:29,712 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml is closed by DFSClient_NONMAPREDUCE_-1007186919_8 19:04:29,714 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013 dst=null perm=null 19:04:29,716 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:29,717 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:29,718 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=null 19:04:29,719 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=null 19:04:29,720 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=null 19:04:29,721 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=null 19:04:29,724 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:29,725 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:29,725 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:29,726 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:29,727 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:29,728 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:29,728 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:29,729 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:29,730 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:29,732 INFO ClientRMService:342 - Application with id 13 submitted by user test 19:04:29,732 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0013 19:04:29,732 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0013 19:04:29,732 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0013 19:04:29,732 INFO RMAppImpl:643 - application_1394161202967_0013 State change from NEW to NEW_SAVING 19:04:29,732 INFO RMAppImpl:643 - application_1394161202967_0013 State change from NEW_SAVING to SUBMITTED 19:04:29,732 INFO FairScheduler:638 - Accepted application application_1394161202967_0013 from user: test, in queue: default, currently num of applications: 2 19:04:29,733 INFO RMAppImpl:643 - application_1394161202967_0013 State change from SUBMITTED to ACCEPTED 19:04:29,733 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0013_000001 19:04:29,733 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from NEW to SUBMITTED 19:04:29,733 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0013_000001 to scheduler from user: test 19:04:29,733 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from SUBMITTED to SCHEDULED 19:04:29,775 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161469775, maxDate=1394766269775, sequenceNumber=39, masterKeyId=2 19:04:29,775 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 39 19:04:29,868 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0013 19:04:29,868 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000001 Container Transitioned from NEW to RESERVED 19:04:29,868 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0013 reserved container container_1394161202967_0013_01_000001 on node host: 172.16.1.64:52703 #containers=2 available= used=, currently has 1 at priority 0; currentReservation 2048 19:04:29,868 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0013_01_000001 on node host: 172.16.1.64:52703 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@6865f3cc 19:04:29,986 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000002 Container Transitioned from NEW to ALLOCATED 19:04:29,986 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000002 19:04:29,986 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0013_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available 19:04:29,987 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:04:29,987 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0013 AttemptId: appattempt_1394161202967_0013_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0013_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:04:29,987 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:04:29,988 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:04:29,988 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0013_000001 19:04:29,989 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0013_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0013_000001 19:04:29,989 INFO AMLauncher:190 - Command to launch container container_1394161202967_0013_01_000002 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:04:29,996 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0013_01_000002 by user test 19:04:29,996 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0013 19:04:29,996 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000002 19:04:29,996 INFO Application:450 - Application application_1394161202967_0013 transitioned from NEW to INITING 19:04:29,997 INFO Application:296 - Adding container_1394161202967_0013_01_000002 to application application_1394161202967_0013 19:04:29,997 INFO Application:450 - Application application_1394161202967_0013 transitioned from INITING to RUNNING 19:04:29,997 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0013_01_000002, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0013_000001 19:04:29,997 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from ALLOCATED to LAUNCHED 19:04:29,997 INFO Container:888 - Container container_1394161202967_0013_01_000002 transitioned from NEW to LOCALIZING 19:04:29,997 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0013 19:04:29,997 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:04:29,998 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split transitioned from INIT to DOWNLOADING 19:04:29,998 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml transitioned from INIT to DOWNLOADING 19:04:29,998 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:04:29,998 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0013_01_000002 19:04:29,999 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar, 1394161469445, FILE, null } 19:04:30,087 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0013_01_000002.tokens. Credentials list: 19:04:30,166 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:30,187 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:30,205 INFO DefaultContainerExecutor:469 - Initializing user test 19:04:30,205 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib dst=null perm=null 19:04:30,225 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app dst=null perm=null 19:04:30,248 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d dst=null perm=null 19:04:30,294 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:04:30,316 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:04:30,358 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:04:30,388 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:30,447 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:30,555 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53778, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175, duration: 623000 19:04:30,716 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073741984_1160, blk_1073741985_1161, blk_1073741986_1162, blk_1073741992_1168, blk_1073741993_1169, blk_1073741994_1170] 19:04:30,763 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/filecache/19/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:04:30,772 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/nmPrivate/container_1394161202967_0013_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000002.tokens 19:04:30,772 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0013 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0013 19:04:30,898 INFO FairScheduler:1042 - Releasing reservation that cannot be satisfied for application appattempt_1394161202967_0013_000001 on node host: 172.16.1.64:52703 #containers=2 available= used= 19:04:30,898 INFO FSSchedulerApp:123 - Application application_1394161202967_0013 unreserved on node host: 172.16.1.64:52703 #containers=2 available= used=, currently has 0 at priority 0; currentReservation 19:04:30,992 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:04:31,043 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=null 19:04:31,045 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=null 19:04:31,046 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=null 19:04:31,071 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53781, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1244837950_4132, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180, duration: 209000 19:04:31,071 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174 19:04:31,072 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179 19:04:31,072 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176 19:04:31,078 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180 19:04:31,103 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181 19:04:31,125 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177 19:04:31,131 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172 19:04:31,156 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173 19:04:31,157 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178 19:04:31,162 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170 19:04:31,184 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175 19:04:31,185 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171 19:04:31,197 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0013/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:04:31,270 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=null 19:04:31,271 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=null 19:04:31,272 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=null 19:04:31,304 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53781, bytes: 176, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1244837950_4132, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179, duration: 299000 19:04:31,403 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0013/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:04:31,462 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:31,464 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:31,465 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:31,495 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53781, bytes: 80741, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1244837950_4132, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181, duration: 407000 19:04:31,590 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0013/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:04:31,600 INFO Container:888 - Container container_1394161202967_0013_01_000002 transitioned from LOCALIZING to LOCALIZED 19:04:31,680 INFO Container:888 - Container container_1394161202967_0013_01_000002 transitioned from LOCALIZED to RUNNING 19:04:31,688 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:31,927 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741997_1173 19:04:31,950 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741994_1170 19:04:31,967 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181 19:04:31,982 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174 19:04:31,993 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178 19:04:31,994 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741996_1172 19:04:31,994 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742000_1176 19:04:32,000 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177 19:04:32,027 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171 19:04:32,049 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175 19:04:32,057 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000002/default_container_executor.sh] 19:04:32,057 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179 19:04:32,058 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180 19:04:32,091 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741984_1160 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741984 for deletion 19:04:32,091 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741985_1161 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741985 for deletion 19:04:32,091 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741986_1162 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741986 for deletion 19:04:32,091 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741992_1168 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741992 for deletion 19:04:32,091 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741984_1160 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741984 19:04:32,091 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741993_1169 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741993 for deletion 19:04:32,091 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741985_1161 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741985 19:04:32,092 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741986_1162 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741986 19:04:32,092 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741994_1170 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741994 for deletion 19:04:32,092 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741992_1168 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741992 19:04:32,092 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741993_1169 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741993 19:04:32,092 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741994_1170 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741994 19:04:33,718 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073741984_1160, blk_1073741985_1161, blk_1073741986_1162, blk_1073741992_1168, blk_1073741993_1169, blk_1073741994_1170] 19:04:34,204 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:34,241 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/COMMIT_STARTED dst=null perm=null 19:04:34,244 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/COMMIT_SUCCESS dst=null perm=null 19:04:34,245 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/COMMIT_FAIL dst=null perm=null 19:04:34,304 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013 dst=null perm=null 19:04:34,305 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:04:34,307 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:04:34,704 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=null 19:04:34,707 INFO ContainerLaunch:322 - Container container_1394161202967_0012_01_000002 succeeded 19:04:34,707 INFO Container:888 - Container container_1394161202967_0012_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:04:34,707 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0012_01_000002 19:04:34,714 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.splitmetainfo dst=null perm=null 19:04:34,724 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000002 19:04:34,724 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000002 19:04:34,724 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000002 19:04:34,724 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000002 19:04:34,724 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0012/container_1394161202967_0012_01_000002 19:04:34,724 INFO Container:888 - Container container_1394161202967_0012_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 19:04:34,725 INFO Application:339 - Removing container_1394161202967_0012_01_000002 from application application_1394161202967_0012 19:04:34,725 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:34,725 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0012 19:04:34,775 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53783, bytes: 37, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742004_1180, duration: 219000 19:04:34,904 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0012_01_000002 19:04:34,905 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000002 Container Transitioned from RUNNING to COMPLETED 19:04:34,905 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000002 in state: COMPLETED event:FINISHED 19:04:34,905 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0012_000001 19:04:34,905 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000002 19:04:34,905 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0012_000001 State change from FINISHING to FINISHED 19:04:34,905 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:04:34,905 INFO RMAppImpl:643 - application_1394161202967_0012 State change from FINISHING to FINISHED 19:04:34,906 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000002 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:04:34,906 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0012 19:04:34,906 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0012_000001 19:04:34,906 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0012,name=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190349401-oozie-rkan-W,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0012/jobhistory/job/job_1394161202967_0012,appMasterHost=rkanter-mbp.local,startTime=1394161443070,finishTime=1394161467542,finalStatus=FAILED 19:04:34,906 INFO FairScheduler:720 - Application appattempt_1394161202967_0012_000001 is done. finalState=FINISHED 19:04:34,906 INFO RMContainerImpl:220 - container_1394161202967_0012_01_000009 Container Transitioned from ACQUIRED to KILLED 19:04:34,906 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0012_01_000009 in state: KILLED event:KILL 19:04:34,906 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0012 CONTAINERID=container_1394161202967_0012_01_000009 19:04:34,906 INFO FSSchedulerNode:162 - Released container container_1394161202967_0012_01_000009 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:04:34,907 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0012_000001 released container container_1394161202967_0012_01_000009 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: KILL 19:04:34,907 INFO AppSchedulingInfo:108 - Application application_1394161202967_0012 requests cleared 19:04:34,912 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0012_01_000002 19:04:34,997 INFO Application:450 - Application application_1394161202967_0012 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:04:34,997 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0012 19:04:34,998 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0012 19:04:34,998 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0012 19:04:34,998 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0012 19:04:34,998 INFO Application:450 - Application application_1394161202967_0012 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:04:34,998 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0012, with delay of 10800 seconds 19:04:34,998 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0012 19:04:35,051 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741984_1160 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741984 for deletion 19:04:35,051 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741985_1161 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741985 for deletion 19:04:35,051 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741986_1162 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741986 for deletion 19:04:35,052 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741992_1168 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741992 for deletion 19:04:35,052 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741993_1169 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741993 for deletion 19:04:35,052 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741985_1161 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741985 19:04:35,052 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741994_1170 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741994 for deletion 19:04:35,052 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741984_1160 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741984 19:04:35,052 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741993_1169 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741993 19:04:35,052 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741986_1162 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741986 19:04:35,052 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741992_1168 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741992 19:04:35,052 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741994_1170 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741994 19:04:35,788 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0013_000001 19:04:35,788 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0013 APPATTEMPTID=appattempt_1394161202967_0013_000001 19:04:35,788 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from LAUNCHED to RUNNING 19:04:35,788 INFO RMAppImpl:643 - application_1394161202967_0013 State change from ACCEPTED to RUNNING 19:04:35,847 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:04:35,849 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=test:supergroup:rwxrwxrwx 19:04:35,905 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0012_01_000009 19:04:35,906 INFO Application:450 - Application application_1394161202967_0012 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:04:35,906 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0012 19:04:35,906 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0012 19:04:35,906 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0012 19:04:35,907 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0012 19:04:35,907 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0012 19:04:35,907 INFO Application:450 - Application application_1394161202967_0012 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:04:35,907 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0012, with delay of 10800 seconds 19:04:35,922 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:04:36,181 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073742006_1182{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:36,239 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182 src: /127.0.0.1:53790 dest: /127.0.0.1:52685 19:04:36,240 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182 src: /127.0.0.1:53791 dest: /127.0.0.1:52682 19:04:36,275 INFO clienttrace:1215 - src: /127.0.0.1:53791, dest: /127.0.0.1:52682, bytes: 93003, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182, duration: 33352000 19:04:36,275 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:36,276 INFO clienttrace:1215 - src: /127.0.0.1:53790, dest: /127.0.0.1:52685, bytes: 93003, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182, duration: 34223000 19:04:36,276 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:36,281 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1_conf.xml is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:36,311 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013.summary dst=null perm=test:supergroup:rw-rw-rw- 19:04:36,313 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073742007_1183{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:36,316 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183 src: /127.0.0.1:53792 dest: /127.0.0.1:52685 19:04:36,317 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183 src: /127.0.0.1:53793 dest: /127.0.0.1:52682 19:04:36,320 INFO clienttrace:1215 - src: /127.0.0.1:53793, dest: /127.0.0.1:52682, bytes: 299, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183, duration: 1380000 19:04:36,320 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:36,320 INFO clienttrace:1215 - src: /127.0.0.1:53792, dest: /127.0.0.1:52685, bytes: 299, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183, duration: 2108000 19:04:36,321 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:36,322 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013.summary is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:36,907 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000003 Container Transitioned from NEW to ALLOCATED 19:04:36,907 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000003 19:04:36,907 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0013_01_000003 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:04:37,072 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183 19:04:37,074 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182 19:04:37,850 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 19:04:37,899 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:37,900 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:37,912 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:37,913 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:37,914 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:37,922 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:37,924 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:37,925 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:37,927 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:37,928 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:37,929 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:37,987 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0013_01_000003 by user test 19:04:37,987 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0013 19:04:37,987 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000003 19:04:37,987 INFO Application:450 - Application application_1394161202967_0013 transitioned from NEW to INITING 19:04:37,987 INFO Application:296 - Adding container_1394161202967_0013_01_000003 to application application_1394161202967_0013 19:04:37,987 INFO Application:450 - Application application_1394161202967_0013 transitioned from INITING to RUNNING 19:04:37,988 INFO Container:888 - Container container_1394161202967_0013_01_000003 transitioned from NEW to LOCALIZING 19:04:37,988 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0013 19:04:37,988 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0013 19:04:37,988 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:04:37,988 INFO ShuffleHandler:263 - Added token for job_1394161202967_0013 19:04:37,988 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml transitioned from INIT to DOWNLOADING 19:04:37,988 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar transitioned from INIT to DOWNLOADING 19:04:37,988 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0013_01_000003 19:04:37,989 INFO ResourceLocalizationService:651 - Downloading public rsrc:{ hdfs://localhost:52681/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar, 1394161469445, FILE, null } 19:04:38,107 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0013_01_000003.tokens. Credentials list: 19:04:38,164 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:38,211 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:38,218 INFO DefaultContainerExecutor:469 - Initializing user test 19:04:38,255 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib dst=null perm=null 19:04:38,277 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app dst=null perm=null 19:04:38,328 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d dst=null perm=null 19:04:38,373 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:04:38,397 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:04:38,452 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:04:38,493 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:38,524 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:38,621 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53795, bytes: 1849, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741999_1175, duration: 445000 19:04:38,767 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/nmPrivate/container_1394161202967_0013_01_000003.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000003.tokens 19:04:38,767 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0013 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0013 19:04:38,768 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/filecache/19/test.jar) transitioned from DOWNLOADING to LOCALIZED 19:04:38,886 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:38,911 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000003 Container Transitioned from ACQUIRED to RUNNING 19:04:38,994 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:39,002 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:39,003 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml dst=null perm=null 19:04:39,029 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53795, bytes: 80741, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1051013366_4243, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742005_1181, duration: 434000 19:04:39,118 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0013/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:04:39,118 INFO Container:888 - Container container_1394161202967_0013_01_000003 transitioned from LOCALIZING to LOCALIZED 19:04:39,190 INFO Container:888 - Container container_1394161202967_0013_01_000003 transitioned from LOCALIZED to RUNNING 19:04:39,190 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:39,451 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000003/default_container_executor.sh] 19:04:41,191 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182 19:04:41,191 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742007_1183 19:04:42,185 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job.split dst=null perm=null 19:04:42,268 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53800, bytes: 176, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742003_1179, duration: 300000 19:04:42,319 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/input/dummy.txt dst=null perm=null 19:04:42,334 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0013_m_000000_0/part-00000 dst=null perm=test:supergroup:rw-rw-rw- 19:04:42,361 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/0000000-140306190429176-oozie-rkan-W@mr-action dst=null perm=null 19:04:42,366 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/0000000-140306190429176-oozie-rkan-W@mr-action dst=null perm=test:supergroup:rw-rw-rw- 19:04:42,377 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/0000000-140306190429176-oozie-rkan-W@mr-action. BP-1007909283-172.16.1.64-1394161187028 blk_1073742008_1184{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:42,421 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184 src: /127.0.0.1:53801 dest: /127.0.0.1:52685 19:04:42,422 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184 src: /127.0.0.1:53802 dest: /127.0.0.1:52682 19:04:42,446 INFO clienttrace:1215 - src: /127.0.0.1:53802, dest: /127.0.0.1:52682, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184, duration: 22573000 19:04:42,446 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:42,446 INFO clienttrace:1215 - src: /127.0.0.1:53801, dest: /127.0.0.1:52685, bytes: 22, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184, duration: 23104000 19:04:42,447 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:42,450 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/0000000-140306190429176-oozie-rkan-W@mr-action is closed by DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1 19:04:42,456 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53800, bytes: 9, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742002_1178, duration: 194000 19:04:42,519 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:04:42,521 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action.xml dst=null perm=null 19:04:42,533 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53803, bytes: 78054, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742001_1177, duration: 306000 19:04:43,173 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output dst=null perm=null 19:04:43,177 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:43,178 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:43,203 INFO ClientRMService:229 - Allocated new applicationId: 14 19:04:43,219 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014 dst=null perm=null 19:04:43,228 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014 dst=null perm=test:supergroup:rwxrwxrwx 19:04:43,244 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014 dst=null perm=test:supergroup:rwx------ 19:04:43,252 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:43,254 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:43,256 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:43,259 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:43,261 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir dst=null perm=null 19:04:43,262 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target dst=null perm=null 19:04:43,264 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter dst=null perm=null 19:04:43,265 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user dst=null perm=null 19:04:43,266 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/ dst=null perm=null 19:04:43,268 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:43,270 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:43,270 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib dst=null perm=null 19:04:43,271 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app dst=null perm=null 19:04:43,272 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d dst=null perm=null 19:04:43,273 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test dst=null perm=null 19:04:43,285 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input dst=null perm=null 19:04:43,288 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input dst=null perm=null 19:04:43,292 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input dst=null perm=null 19:04:43,319 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=test:supergroup:rw-rw-rw- 19:04:43,323 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=test:supergroup:rw-r--r-- 19:04:43,329 INFO BlockManager:2576 - Increasing replication from 2 to 10 for /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split 19:04:43,330 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setReplication src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:43,337 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split. BP-1007909283-172.16.1.64-1394161187028 blk_1073742009_1185{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:43,340 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185 src: /127.0.0.1:53805 dest: /127.0.0.1:52685 19:04:43,342 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185 src: /127.0.0.1:53806 dest: /127.0.0.1:52682 19:04:43,345 INFO clienttrace:1215 - src: /127.0.0.1:53806, dest: /127.0.0.1:52682, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185, duration: 1277000 19:04:43,345 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:43,346 INFO clienttrace:1215 - src: /127.0.0.1:53805, dest: /127.0.0.1:52685, bytes: 279, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185, duration: 1865000 19:04:43,346 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:43,347 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split is closed by DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1 19:04:43,349 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=test:supergroup:rw-rw-rw- 19:04:43,351 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=test:supergroup:rw-r--r-- 19:04:43,355 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo. BP-1007909283-172.16.1.64-1394161187028 blk_1073742010_1186{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:43,358 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186 src: /127.0.0.1:53807 dest: /127.0.0.1:52685 19:04:43,359 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186 src: /127.0.0.1:53808 dest: /127.0.0.1:52682 19:04:43,361 INFO clienttrace:1215 - src: /127.0.0.1:53808, dest: /127.0.0.1:52682, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186, duration: 1175000 19:04:43,361 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:43,362 INFO clienttrace:1215 - src: /127.0.0.1:53807, dest: /127.0.0.1:52685, bytes: 57, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186, duration: 1769000 19:04:43,362 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:43,363 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo is closed by DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1 19:04:43,376 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=test:supergroup:rw-rw-rw- 19:04:43,378 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=test:supergroup:rw-r--r-- 19:04:43,412 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073742011_1187{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:43,415 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187 src: /127.0.0.1:53809 dest: /127.0.0.1:52682 19:04:43,416 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187 src: /127.0.0.1:53810 dest: /127.0.0.1:52685 19:04:43,419 INFO clienttrace:1215 - src: /127.0.0.1:53810, dest: /127.0.0.1:52685, bytes: 77604, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187, duration: 1752000 19:04:43,419 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:43,419 INFO clienttrace:1215 - src: /127.0.0.1:53809, dest: /127.0.0.1:52682, bytes: 77604, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187, duration: 2356000 19:04:43,420 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:43,421 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml is closed by DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1 19:04:43,621 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014 dst=null perm=null 19:04:43,853 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:43,854 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:43,858 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:43,858 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:43,859 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=null 19:04:43,860 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=null 19:04:43,864 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:43,865 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:43,866 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:43,867 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:43,868 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:43,869 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:43,871 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:43,872 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:43,873 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:43,932 INFO ClientRMService:342 - Application with id 14 submitted by user test 19:04:43,932 INFO RMAppImpl:861 - Storing application with id application_1394161202967_0014 19:04:43,932 INFO RMAuditLogger:141 - USER=test IP=172.16.1.64 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1394161202967_0014 19:04:43,932 INFO RMAppImpl:643 - application_1394161202967_0014 State change from NEW to NEW_SAVING 19:04:43,933 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0014 19:04:43,933 INFO RMAppImpl:643 - application_1394161202967_0014 State change from NEW_SAVING to SUBMITTED 19:04:43,933 INFO FairScheduler:638 - Accepted application application_1394161202967_0014 from user: test, in queue: default, currently num of applications: 2 19:04:43,933 INFO RMAppImpl:643 - application_1394161202967_0014 State change from SUBMITTED to ACCEPTED 19:04:43,933 INFO ApplicationMasterService:574 - Registering app attempt : appattempt_1394161202967_0014_000001 19:04:43,934 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from NEW to SUBMITTED 19:04:43,934 INFO FairScheduler:676 - Added Application Attempt appattempt_1394161202967_0014_000001 to scheduler from user: test 19:04:43,934 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from SUBMITTED to SCHEDULED 19:04:44,034 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000001 Container Transitioned from NEW to ALLOCATED 19:04:44,034 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000001 19:04:44,034 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0014_01_000001 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:04:44,035 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 19:04:44,035 INFO RMAppAttemptImpl:1573 - Storing attempt: AppId: application_1394161202967_0014 AttemptId: appattempt_1394161202967_0014_000001 MasterContainer: Container: [ContainerId: container_1394161202967_0014_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] 19:04:44,035 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from SCHEDULED to ALLOCATED_SAVING 19:04:44,035 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from ALLOCATED_SAVING to ALLOCATED 19:04:44,035 INFO AMLauncher:248 - Launching masterappattempt_1394161202967_0014_000001 19:04:44,036 INFO AMLauncher:105 - Setting up container Container: [ContainerId: container_1394161202967_0014_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0014_000001 19:04:44,036 INFO AMLauncher:190 - Command to launch container container_1394161202967_0014_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/stdout 2>/stderr 19:04:44,041 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0014_01_000001 by user test 19:04:44,041 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0014 19:04:44,042 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000001 19:04:44,042 INFO Application:450 - Application application_1394161202967_0014 transitioned from NEW to INITING 19:04:44,042 INFO Application:296 - Adding container_1394161202967_0014_01_000001 to application application_1394161202967_0014 19:04:44,042 INFO Application:450 - Application application_1394161202967_0014 transitioned from INITING to RUNNING 19:04:44,042 INFO AMLauncher:126 - Done launching container Container: [ContainerId: container_1394161202967_0014_01_000001, NodeId: 172.16.1.64:52707, NodeHttpAddress: 172.16.1.64:52710, Resource: , Priority: 0, Token: Token { kind: ContainerToken, service: 172.16.1.64:52707 }, ] for AM appattempt_1394161202967_0014_000001 19:04:44,042 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from ALLOCATED to LAUNCHED 19:04:44,042 INFO Container:888 - Container container_1394161202967_0014_01_000001 transitioned from NEW to LOCALIZING 19:04:44,042 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0014 19:04:44,043 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo transitioned from INIT to DOWNLOADING 19:04:44,043 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split transitioned from INIT to DOWNLOADING 19:04:44,043 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml transitioned from INIT to DOWNLOADING 19:04:44,043 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0014_01_000001 19:04:44,106 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/nmPrivate/container_1394161202967_0014_01_000001.tokens. Credentials list: 19:04:44,195 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=test:supergroup:rw-rw-rw- 19:04:44,230 INFO DefaultContainerExecutor:469 - Initializing user test 19:04:44,263 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq. BP-1007909283-172.16.1.64-1394161187028 blk_1073742012_1188{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:44,288 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188 src: /127.0.0.1:53813 dest: /127.0.0.1:52682 19:04:44,319 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188 src: /127.0.0.1:53814 dest: /127.0.0.1:52685 19:04:44,382 INFO clienttrace:1215 - src: /127.0.0.1:53814, dest: /127.0.0.1:52685, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, duration: 24070000 19:04:44,395 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:44,429 INFO clienttrace:1215 - src: /127.0.0.1:53813, dest: /127.0.0.1:52682, bytes: 166, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, duration: 70478000 19:04:44,435 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:44,472 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq is closed by DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1 19:04:44,506 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0013_m_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0013_m_000000_0_-234578063_1 19:04:44,557 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0013_m_000000_0 dst=null perm=null 19:04:44,585 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0013_m_000000_0 dst=null perm=null 19:04:44,620 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0013_m_000000 dst=null perm=null 19:04:44,665 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/_temporary/attempt_1394161202967_0013_m_000000_0 dst=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0013_m_000000 perm=test:supergroup:rwxrwxrwx 19:04:44,699 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0013_01_000003 19:04:44,710 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000003 19:04:44,710 INFO Container:888 - Container container_1394161202967_0013_01_000003 transitioned from RUNNING to KILLING 19:04:44,710 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0013_01_000003 19:04:44,795 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0013_01_000003 is : 143 19:04:44,797 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073742013_1189{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:44,806 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:04:44,842 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/nmPrivate/container_1394161202967_0014_01_000001.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000001.tokens 19:04:44,843 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0014 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0014 19:04:44,844 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189 src: /127.0.0.1:53815 dest: /127.0.0.1:52682 19:04:44,867 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:44,882 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189 src: /127.0.0.1:53816 dest: /127.0.0.1:52685 19:04:44,914 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1 dst=null perm=null 19:04:44,955 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1.jhist for DFSClient_NONMAPREDUCE_1538833463_1 19:04:44,968 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:04:44,978 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output dst=null perm=null 19:04:44,998 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0013_m_000000 dst=null perm=null 19:04:45,034 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/part-00000 dst=null perm=null 19:04:45,090 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000001 Container Transitioned from ACQUIRED to RUNNING 19:04:45,125 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary/1/task_1394161202967_0013_m_000000/part-00000 dst=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/part-00000 perm=test:supergroup:rw-rw-rw- 19:04:45,166 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_temporary dst=null perm=null 19:04:45,179 INFO Container:888 - Container container_1394161202967_0013_01_000003 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:04:45,179 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:04:45,180 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000003 19:04:45,180 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000003 19:04:45,180 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000003 19:04:45,181 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000003 19:04:45,181 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000003 19:04:45,181 INFO Container:888 - Container container_1394161202967_0013_01_000003 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:04:45,181 INFO Application:339 - Removing container_1394161202967_0013_01_000003 from application application_1394161202967_0013 19:04:45,181 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:45,181 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0013 19:04:45,182 INFO StateChange:2959 - DIR* completeFile: /user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:45,184 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:04:45,186 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:45,273 INFO clienttrace:1215 - src: /127.0.0.1:53816, dest: /127.0.0.1:52685, bytes: 20639, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189, duration: 358296000 19:04:45,279 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:45,307 INFO clienttrace:1215 - src: /127.0.0.1:53815, dest: /127.0.0.1:52682, bytes: 20639, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189, duration: 392142000 19:04:45,308 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:45,309 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1.jhist is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:45,311 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=null 19:04:45,312 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:45,312 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=null 19:04:45,313 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=null 19:04:45,313 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073742014_1190{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:45,316 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190 src: /127.0.0.1:53819 dest: /127.0.0.1:52685 19:04:45,342 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190 src: /127.0.0.1:53820 dest: /127.0.0.1:52682 19:04:45,344 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53821, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1585222162_4343, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186, duration: 199000 19:04:45,345 INFO clienttrace:1215 - src: /127.0.0.1:53820, dest: /127.0.0.1:52682, bytes: 343, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190, duration: 1416000 19:04:45,345 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:45,374 INFO clienttrace:1215 - src: /127.0.0.1:53819, dest: /127.0.0.1:52685, bytes: 343, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190, duration: 27712000 19:04:45,412 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:45,464 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary_tmp is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:45,501 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:45,539 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0014/filecache/10/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 19:04:45,540 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1.jhist dst=null perm=null 19:04:45,557 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist_tmp dst=null perm=null 19:04:45,619 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1.jhist dst=null perm=null 19:04:45,622 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist_tmp dst=null perm=null 19:04:45,682 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1.jhist dst=null perm=null 19:04:45,698 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:45,699 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:45,700 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:45,700 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:45,702 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53822, bytes: 20803, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189, duration: 224000 19:04:45,727 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0013_01_000003 19:04:45,728 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073742015_1191{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:45,729 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000003 Container Transitioned from RUNNING to COMPLETED 19:04:45,729 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0013_01_000003 in state: COMPLETED event:FINISHED 19:04:45,729 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000003 19:04:45,729 INFO FSSchedulerNode:162 - Released container container_1394161202967_0013_01_000003 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:04:45,729 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0013_000001 released container container_1394161202967_0013_01_000003 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: FINISHED 19:04:45,730 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53821, bytes: 283, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1585222162_4343, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185, duration: 204000 19:04:45,733 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191 src: /127.0.0.1:53823 dest: /127.0.0.1:52682 19:04:45,769 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191 src: /127.0.0.1:53824 dest: /127.0.0.1:52685 19:04:45,910 INFO clienttrace:1215 - src: /127.0.0.1:53824, dest: /127.0.0.1:52685, bytes: 20639, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191, duration: 41027000 19:04:45,911 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:45,912 INFO clienttrace:1215 - src: /127.0.0.1:53823, dest: /127.0.0.1:52682, bytes: 20639, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191, duration: 80034000 19:04:45,912 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:45,914 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0014/filecache/11/job.split) transitioned from DOWNLOADING to LOCALIZED 19:04:45,915 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist_tmp is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:45,945 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:45,989 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1_conf.xml dst=null perm=null 19:04:46,017 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml_tmp dst=null perm=null 19:04:46,062 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1_conf.xml dst=null perm=null 19:04:46,062 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:46,064 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml_tmp dst=null perm=null 19:04:46,064 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:46,065 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013/job_1394161202967_0013_1_conf.xml dst=null perm=null 19:04:46,065 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:46,092 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:04:46,107 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53822, bytes: 93731, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742006_1182, duration: 392000 19:04:46,107 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53821, bytes: 78212, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1585222162_4343, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187, duration: 480000 19:04:46,132 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073742016_1192{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:04:46,194 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192 src: /127.0.0.1:53825 dest: /127.0.0.1:52685 19:04:46,248 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192 src: /127.0.0.1:53826 dest: /127.0.0.1:52682 19:04:46,284 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191 19:04:46,288 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189 19:04:46,289 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187 19:04:46,289 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186 19:04:46,289 INFO clienttrace:1215 - src: /127.0.0.1:53826, dest: /127.0.0.1:52682, bytes: 93003, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192, duration: 5624000 19:04:46,289 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0014/filecache/12/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:04:46,289 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:46,290 INFO Container:888 - Container container_1394161202967_0014_01_000001 transitioned from LOCALIZING to LOCALIZED 19:04:46,290 INFO clienttrace:1215 - src: /127.0.0.1:53825, dest: /127.0.0.1:52685, bytes: 93003, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1538833463_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192, duration: 6391000 19:04:46,290 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:46,291 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192 19:04:46,291 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184 19:04:46,292 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190 19:04:46,292 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188 19:04:46,292 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_1538833463_1 19:04:46,292 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185 19:04:46,294 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:04:46,296 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary perm=test:supergroup:rwxrwx--- 19:04:46,298 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml perm=test:supergroup:rwxrwx--- 19:04:46,300 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist perm=test:supergroup:rwxrwx--- 19:04:46,347 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0013_000001 with final state: FINISHING 19:04:46,354 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from RUNNING to FINAL_SAVING 19:04:46,355 INFO RMAppImpl:880 - Updating application application_1394161202967_0013 with final state: FINISHING 19:04:46,355 INFO RMAppImpl:643 - application_1394161202967_0013 State change from RUNNING to FINAL_SAVING 19:04:46,355 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0013 19:04:46,377 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from FINAL_SAVING to FINISHING 19:04:46,386 INFO RMAppImpl:643 - application_1394161202967_0013 State change from FINAL_SAVING to FINISHING 19:04:46,425 INFO Container:888 - Container container_1394161202967_0014_01_000001 transitioned from LOCALIZED to RUNNING 19:04:46,425 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:46,666 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000001/default_container_executor.sh] 19:04:47,079 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191 19:04:47,080 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742013_1189 19:04:47,080 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742008_1184 19:04:47,081 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742016_1192 19:04:47,081 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190 19:04:47,279 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187 19:04:47,280 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188 19:04:47,280 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185 19:04:47,281 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186 19:04:47,363 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0013 dst=null perm=null 19:04:48,261 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:48,264 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:48,266 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:48,268 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53830, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, duration: 257000 19:04:48,306 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161488306, maxDate=1394766288306, sequenceNumber=40, masterKeyId=2 19:04:48,306 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 40 19:04:48,312 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:04:48,313 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:04:48,315 INFO CompletedJob:95 - Loading job: job_1394161202967_0013 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist 19:04:48,315 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist] 19:04:48,316 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary dst=null perm=null 19:04:48,316 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist dst=null perm=null 19:04:48,317 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist dst=null perm=null 19:04:48,318 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53830, bytes: 418, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741995_1171, duration: 229000 19:04:48,318 INFO JobSummary:372 - jobId=job_1394161202967_0012,submitTime=1394161443070,launchTime=1394161449279,firstMapTaskLaunchTime=1394161451464,firstReduceTaskLaunchTime=0,finishTime=1394161467440,resourcesPerMap=1024,resourcesPerReduce=1024,numMaps=2,numReduces=1,user=test,queue=default,status=FAILED,mapSlotSeconds=7,reduceSlotSeconds=0,jobName=oozie:action:T\=map-reduce:W\=testApp:A\=map-reduce:ID\=0000000-140306190349401-oozie-rkan-W 19:04:48,318 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary] 19:04:48,319 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53830, bytes: 20803, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742015_1191, duration: 234000 19:04:48,321 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012.summary dst=null perm=null 19:04:48,323 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:04:48,323 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist 19:04:48,325 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0012-1394161443070-test-oozie%3Aaction%3AT%3Dmap%2Dreduce%3AW%3DtestApp%3AA%3Dmap%2Dreduce%3AI-1394161467440-2-0-FAILED-root.test-1394161449279.jhist perm=test:supergroup:rwxrwx--- 19:04:48,325 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0012_conf.xml 19:04:48,327 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0012_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0012_conf.xml perm=test:supergroup:rwxrwx--- 19:04:48,341 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary dst=null perm=null 19:04:48,342 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53830, bytes: 347, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742014_1190, duration: 221000 19:04:48,342 INFO JobSummary:372 - jobId=job_1394161202967_0013,submitTime=1394161469732,launchTime=1394161475816,firstMapTaskLaunchTime=1394161477993,firstReduceTaskLaunchTime=0,finishTime=1394161485187,resourcesPerMap=1024,resourcesPerReduce=0,numMaps=1,numReduces=0,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=6,reduceSlotSeconds=0,jobName=MapReduceLauncherTest 19:04:48,343 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary] 19:04:48,344 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013.summary dst=null perm=null 19:04:48,345 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:04:48,346 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist 19:04:48,347 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0013-1394161469732-test-MapReduceLauncherTest-1394161485187-1-0-SUCCEEDED-root.test-1394161475816.jhist perm=test:supergroup:rwxrwx--- 19:04:48,347 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0013_conf.xml 19:04:48,349 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0013_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0013_conf.xml perm=test:supergroup:rwxrwx--- 19:04:48,371 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:48,372 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:48,373 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:04:48,375 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53830, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, duration: 223000 19:04:48,415 INFO CacheReplicationMonitor:177 - Rescanning after 300001 milliseconds 19:04:48,415 INFO CacheReplicationMonitor:202 - Scanned 0 directive(s) and 0 block(s) in 0 millisecond(s). 19:04:48,454 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161488454, maxDate=1394766288454, sequenceNumber=41, masterKeyId=2 19:04:48,454 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 41 19:04:48,729 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52682 to delete [blk_1073742003_1179, blk_1073742004_1180, blk_1073742005_1181, blk_1073742006_1182, blk_1073742007_1183, blk_1073741995_1171, blk_1073742013_1189, blk_1073742014_1190] 19:04:49,006 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging dst=null perm=null 19:04:49,040 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/COMMIT_STARTED dst=null perm=null 19:04:49,041 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/COMMIT_SUCCESS dst=null perm=null 19:04:49,042 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/COMMIT_FAIL dst=null perm=null 19:04:49,111 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014 dst=null perm=null 19:04:49,113 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:04:49,114 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:04:49,525 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=null 19:04:49,533 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.splitmetainfo dst=null perm=null 19:04:49,594 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53834, bytes: 61, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742010_1186, duration: 182000 19:04:50,057 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742003_1179 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742003 for deletion 19:04:50,057 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742004_1180 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742004 for deletion 19:04:50,057 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742005_1181 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742005 for deletion 19:04:50,058 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742006_1182 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742006 for deletion 19:04:50,058 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742003_1179 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742003 19:04:50,058 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742007_1183 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742007 for deletion 19:04:50,058 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742004_1180 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742004 19:04:50,058 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741995_1171 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741995 for deletion 19:04:50,058 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742005_1181 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742005 19:04:50,058 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742013_1189 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742013 for deletion 19:04:50,058 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742006_1182 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742006 19:04:50,058 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742014_1190 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742014 for deletion 19:04:50,058 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742007_1183 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742007 19:04:50,059 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742014_1190 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data2/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742014 19:04:50,059 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741995_1171 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741995 19:04:50,059 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742013_1189 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data1/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742013 19:04:50,656 INFO ApplicationMasterService:263 - AM registration appattempt_1394161202967_0014_000001 19:04:50,656 INFO RMAuditLogger:120 - USER=test IP=172.16.1.64 OPERATION=Register App Master TARGET=ApplicationMasterService RESULT=SUCCESS APPID=application_1394161202967_0014 APPATTEMPTID=appattempt_1394161202967_0014_000001 19:04:50,656 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from LAUNCHED to RUNNING 19:04:50,657 INFO RMAppImpl:643 - application_1394161202967_0014 State change from ACCEPTED to RUNNING 19:04:50,685 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1.jhist dst=null perm=test:supergroup:rw-rw-rw- 19:04:50,689 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=mkdirs src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1 dst=null perm=test:testg:rwxrwxrwx 19:04:50,746 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1_conf.xml dst=null perm=test:supergroup:rw-rw-rw- 19:04:50,975 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1_conf.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073742017_1193{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:04:51,026 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193 src: /127.0.0.1:53839 dest: /127.0.0.1:52682 19:04:51,027 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193 src: /127.0.0.1:53840 dest: /127.0.0.1:52685 19:04:51,065 INFO clienttrace:1215 - src: /127.0.0.1:53840, dest: /127.0.0.1:52685, bytes: 91955, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193, duration: 36947000 19:04:51,066 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:51,066 INFO clienttrace:1215 - src: /127.0.0.1:53839, dest: /127.0.0.1:52682, bytes: 91955, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193, duration: 37824000 19:04:51,066 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:51,070 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1_conf.xml is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:04:51,094 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014.summary dst=null perm=test:supergroup:rw-rw-rw- 19:04:51,097 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014.summary. BP-1007909283-172.16.1.64-1394161187028 blk_1073742018_1194{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:04:51,100 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194 src: /127.0.0.1:53841 dest: /127.0.0.1:52685 19:04:51,101 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194 src: /127.0.0.1:53842 dest: /127.0.0.1:52682 19:04:51,104 INFO clienttrace:1215 - src: /127.0.0.1:53842, dest: /127.0.0.1:52682, bytes: 291, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194, duration: 1653000 19:04:51,104 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:04:51,105 INFO clienttrace:1215 - src: /127.0.0.1:53841, dest: /127.0.0.1:52685, bytes: 291, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194, duration: 2443000 19:04:51,105 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:04:51,107 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014.summary is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:04:51,300 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194 19:04:51,301 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193 19:04:51,730 INFO StateChange:152 - BLOCK* InvalidateBlocks: ask 127.0.0.1:52685 to delete [blk_1073742003_1179, blk_1073742004_1180, blk_1073742005_1181, blk_1073742006_1182, blk_1073742007_1183, blk_1073741995_1171, blk_1073742013_1189, blk_1073742014_1190] 19:04:51,735 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000002 Container Transitioned from NEW to ALLOCATED 19:04:51,735 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000002 19:04:51,735 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0014_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available 19:04:52,109 INFO AppSchedulable:169 - Making reservation: node=172.16.1.64 app_id=application_1394161202967_0014 19:04:52,110 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000003 Container Transitioned from NEW to RESERVED 19:04:52,110 INFO SchedulerApplicationAttempt:260 - Application application_1394161202967_0014 reserved container container_1394161202967_0014_01_000003 on node host: 172.16.1.64:52707 #containers=2 available= used=, currently has 1 at priority 20; currentReservation 1024 19:04:52,110 INFO FSSchedulerNode:240 - Reserved container container_1394161202967_0014_01_000003 on node host: 172.16.1.64:52707 #containers=2 available= used= for application org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp@2f566e8f 19:04:52,286 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742018_1194 19:04:52,287 INFO BlockPoolSliceScanner:429 - Verification succeeded for BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193 19:04:52,710 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 19:04:52,736 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000004 Container Transitioned from NEW to ALLOCATED 19:04:52,736 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000004 19:04:52,736 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0014_01_000004 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available 19:04:52,759 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:52,760 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:52,773 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:52,775 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:52,778 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:52,789 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:52,790 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:52,791 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:52,792 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/MRAppJar.jar dst=null perm=null 19:04:52,793 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/rkanter/target/MiniMRCluster_1594994783-tmpDir/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0-SNAPSHOT-tests.jar dst=null perm=null 19:04:52,794 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/app/lib/test.jar dst=null perm=null 19:04:52,853 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0014_01_000002 by user test 19:04:52,853 INFO ContainerManagerImpl:611 - Creating a new application reference for app application_1394161202967_0014 19:04:52,854 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000002 19:04:52,854 INFO Application:450 - Application application_1394161202967_0014 transitioned from NEW to INITING 19:04:52,854 INFO Application:296 - Adding container_1394161202967_0014_01_000002 to application application_1394161202967_0014 19:04:52,854 INFO Application:450 - Application application_1394161202967_0014 transitioned from INITING to RUNNING 19:04:52,854 INFO Container:888 - Container container_1394161202967_0014_01_000002 transitioned from NEW to LOCALIZING 19:04:52,854 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0014 19:04:52,854 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0014 19:04:52,854 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:04:52,855 INFO ShuffleHandler:263 - Added token for job_1394161202967_0014 19:04:52,855 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml transitioned from INIT to DOWNLOADING 19:04:52,855 INFO ResourceLocalizationService:590 - Created localizer for container_1394161202967_0014_01_000002 19:04:52,904 INFO ResourceLocalizationService:1015 - Writing credentials to the nmPrivate file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/nmPrivate/container_1394161202967_0014_01_000002.tokens. Credentials list: 19:04:52,956 INFO DefaultContainerExecutor:469 - Initializing user test 19:04:53,098 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742003_1179 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742003 for deletion 19:04:53,098 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742004_1180 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742004 for deletion 19:04:53,105 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742003_1179 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742003 19:04:53,122 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742004_1180 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742004 19:04:53,122 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742005_1181 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742005 for deletion 19:04:53,123 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742006_1182 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742006 for deletion 19:04:53,123 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742007_1183 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742007 for deletion 19:04:53,124 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742006_1182 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742006 19:04:53,124 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073741995_1171 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741995 for deletion 19:04:53,124 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742005_1181 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742005 19:04:53,129 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742013_1189 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742013 for deletion 19:04:53,129 INFO FsDatasetAsyncDiskService:157 - Scheduling blk_1073742014_1190 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742014 for deletion 19:04:53,130 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742007_1183 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742007 19:04:53,130 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742014_1190 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data4/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742014 19:04:53,130 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073741995_1171 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073741995 19:04:53,130 INFO FairScheduler:1042 - Releasing reservation that cannot be satisfied for application appattempt_1394161202967_0014_000001 on node host: 172.16.1.64:52707 #containers=2 available= used= 19:04:53,153 INFO FSSchedulerApp:123 - Application application_1394161202967_0014 unreserved on node host: 172.16.1.64:52707 #containers=2 available= used=, currently has 0 at priority 20; currentReservation 19:04:53,153 INFO FsDatasetAsyncDiskService:233 - Deleted BP-1007909283-172.16.1.64-1394161187028 blk_1073742013_1189 file /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/build/test/data/dfs/data/data3/current/BP-1007909283-172.16.1.64-1394161187028/current/finalized/blk_1073742013 19:04:53,245 INFO DefaultContainerExecutor:103 - Copying from /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/nmPrivate/container_1394161202967_0014_01_000002.tokens to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000002.tokens 19:04:53,245 INFO DefaultContainerExecutor:105 - CWD set to /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0014 = file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0014 19:04:53,437 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:53,438 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:53,439 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml dst=null perm=null 19:04:53,479 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53847, bytes: 78212, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1355672864_4460, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742011_1187, duration: 351000 19:04:53,516 INFO ContainerLaunch:322 - Container container_1394161202967_0013_01_000002 succeeded 19:04:53,547 INFO Container:888 - Container container_1394161202967_0013_01_000002 transitioned from RUNNING to EXITED_WITH_SUCCESS 19:04:53,547 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0013_01_000002 19:04:53,651 INFO LocalizedResource:196 - Resource hdfs://localhost:52681/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.xml(->file:/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0014/filecache/10/job.xml) transitioned from DOWNLOADING to LOCALIZED 19:04:53,651 INFO Container:888 - Container container_1394161202967_0014_01_000002 transitioned from LOCALIZING to LOCALIZED 19:04:53,661 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000002 19:04:53,661 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000002 19:04:53,661 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000002 19:04:53,661 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0013/container_1394161202967_0013_01_000002 19:04:53,661 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Succeeded TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000002 19:04:53,662 INFO Container:888 - Container container_1394161202967_0013_01_000002 transitioned from EXITED_WITH_SUCCESS to DONE 19:04:53,662 INFO Application:339 - Removing container_1394161202967_0013_01_000002 from application application_1394161202967_0013 19:04:53,662 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:53,662 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0013 19:04:53,710 INFO Container:888 - Container container_1394161202967_0014_01_000002 transitioned from LOCALIZED to RUNNING 19:04:53,710 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:53,758 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000004 Container Transitioned from ALLOCATED to ACQUIRED 19:04:53,767 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000002 Container Transitioned from ACQUIRED to RUNNING 19:04:53,767 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000005 Container Transitioned from NEW to ALLOCATED 19:04:53,768 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000005 19:04:53,768 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0014_01_000005 of capacity on host 172.16.1.64:52703, which currently has 3 containers, used and available 19:04:53,793 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0014_01_000004 by user test 19:04:53,830 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000004 19:04:53,830 INFO Application:296 - Adding container_1394161202967_0014_01_000004 to application application_1394161202967_0014 19:04:53,831 INFO Container:888 - Container container_1394161202967_0014_01_000004 transitioned from NEW to LOCALIZING 19:04:53,831 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0014 19:04:53,831 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0014 19:04:53,831 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:04:53,836 INFO ShuffleHandler:263 - Added token for job_1394161202967_0014 19:04:53,852 INFO Container:888 - Container container_1394161202967_0014_01_000004 transitioned from LOCALIZING to LOCALIZED 19:04:54,132 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0013_01_000002 19:04:54,139 INFO RMContainerImpl:220 - container_1394161202967_0013_01_000002 Container Transitioned from RUNNING to COMPLETED 19:04:54,162 INFO ApplicationMasterService:580 - Unregistering app attempt : appattempt_1394161202967_0013_000001 19:04:54,176 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0013_01_000002 in state: COMPLETED event:FINISHED 19:04:54,177 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0013 CONTAINERID=container_1394161202967_0013_01_000002 19:04:54,177 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0013_000001 State change from FINISHING to FINISHED 19:04:54,177 INFO FSSchedulerNode:162 - Released container container_1394161202967_0013_01_000002 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:04:54,178 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0013_000001 released container container_1394161202967_0013_01_000002 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:04:54,178 INFO RMAppImpl:643 - application_1394161202967_0013 State change from FINISHING to FINISHED 19:04:54,178 INFO RMAuditLogger:141 - USER=test OPERATION=Application Finished - Succeeded TARGET=RMAppManager RESULT=SUCCESS APPID=application_1394161202967_0013 19:04:54,178 INFO AMLauncher:262 - Cleaning master appattempt_1394161202967_0013_000001 19:04:54,178 INFO FairScheduler:720 - Application appattempt_1394161202967_0013_000001 is done. finalState=FINISHED 19:04:54,179 INFO RMAppManager$ApplicationSummary:175 - appId=application_1394161202967_0013,name=MapReduceLauncherTest,user=test,queue=root.test,state=FINISHED,trackingUrl=http://rkanter-mbp.local:52701/proxy/application_1394161202967_0013/jobhistory/job/job_1394161202967_0013,appMasterHost=rkanter-mbp.local,startTime=1394161469732,finishTime=1394161486355,finalStatus=SUCCEEDED 19:04:54,179 INFO AppSchedulingInfo:108 - Application application_1394161202967_0013 requests cleared 19:04:54,255 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000002/default_container_executor.sh] 19:04:54,362 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0013_01_000002 19:04:54,391 INFO Container:888 - Container container_1394161202967_0014_01_000004 transitioned from LOCALIZED to RUNNING 19:04:54,391 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:54,788 INFO Application:450 - Application application_1394161202967_0013 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:04:54,788 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000004 Container Transitioned from ACQUIRED to RUNNING 19:04:54,790 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0013 19:04:54,791 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0013 19:04:54,791 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0013 19:04:54,793 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0013 19:04:54,793 INFO Application:450 - Application application_1394161202967_0013 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:04:54,793 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0013 19:04:54,794 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0013, with delay of 10800 seconds 19:04:54,793 INFO AppSchedulingInfo:166 - checking for deactivate... 19:04:54,840 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000005 Container Transitioned from ALLOCATED to ACQUIRED 19:04:54,897 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000004/default_container_executor.sh] 19:04:55,134 INFO Application:450 - Application application_1394161202967_0013 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 19:04:55,135 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0013 19:04:55,136 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0013 19:04:55,136 INFO AuxServices:175 - Got event APPLICATION_STOP for appId application_1394161202967_0013 19:04:55,136 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0013 19:04:55,137 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0013 19:04:55,137 INFO Application:450 - Application application_1394161202967_0013 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 19:04:55,137 INFO NonAggregatingLogHandler:120 - Scheduling Log Deletion for application: application_1394161202967_0013, with delay of 10800 seconds 19:04:55,854 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000005 Container Transitioned from ACQUIRED to RELEASED 19:04:55,854 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0014_01_000005 in state: RELEASED event:RELEASED 19:04:55,854 INFO RMAuditLogger:99 - USER=test IP=172.16.1.64 OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000005 19:04:55,854 INFO FSSchedulerNode:162 - Released container container_1394161202967_0014_01_000005 of capacity on host 172.16.1.64:52703, which currently has 2 containers, used and available, release resources=true 19:04:55,854 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0014_000001 released container container_1394161202967_0014_01_000005 on node: host: 172.16.1.64:52703 #containers=2 available= used= with event: RELEASED 19:04:56,781 WARN ContainerManagerImpl:831 - Event EventType: KILL_CONTAINER sent to absent container container_1394161202967_0014_01_000005 19:04:57,556 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:57,646 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53852, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0014_m_000000_0_548755980_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185, duration: 302000 19:04:57,703 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input/data.txt dst=null perm=null 19:04:57,773 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53852, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0014_m_000000_0_548755980_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174, duration: 182000 19:04:57,797 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_m_000000_0 dst=null perm=null 19:04:57,824 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0014_01_000002 19:04:57,824 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000002 19:04:57,824 INFO Container:888 - Container container_1394161202967_0014_01_000002 transitioned from RUNNING to KILLING 19:04:57,825 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0014_01_000002 19:04:57,878 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0014_01_000002 is : 143 19:04:57,882 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1.jhist. BP-1007909283-172.16.1.64-1394161187028 blk_1073742019_1195{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:04:57,887 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742019_1195 src: /127.0.0.1:53853 dest: /127.0.0.1:52682 19:04:57,889 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742019_1195 src: /127.0.0.1:53854 dest: /127.0.0.1:52685 19:04:57,894 INFO StateChange:3655 - BLOCK* fsync: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1.jhist for DFSClient_NONMAPREDUCE_1556916339_1 19:04:57,906 INFO Container:888 - Container container_1394161202967_0014_01_000002 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:04:57,907 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000002 19:04:57,907 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000002 19:04:57,907 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000002 19:04:57,907 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000002 19:04:57,907 INFO Container:888 - Container container_1394161202967_0014_01_000002 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:04:57,907 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000002 19:04:57,907 INFO Application:339 - Removing container_1394161202967_0014_01_000002 from application application_1394161202967_0014 19:04:57,907 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:57,907 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0014 19:04:58,076 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job.split dst=null perm=null 19:04:58,170 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53856, bytes: 283, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0014_m_000001_0_1874260631_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742009_1185, duration: 284000 19:04:58,222 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/input/data.txt dst=null perm=null 19:04:58,225 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53857, bytes: 16, op: HDFS_READ, cliID: DFSClient_attempt_1394161202967_0014_m_000001_0_1874260631_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073741998_1174, duration: 233000 19:04:58,309 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_m_000001_0 dst=null perm=null 19:04:58,322 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0014_01_000004 19:04:58,322 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000004 19:04:58,322 INFO Container:888 - Container container_1394161202967_0014_01_000004 transitioned from RUNNING to KILLING 19:04:58,322 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0014_01_000004 19:04:58,323 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0014_01_000002 19:04:58,337 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000002 Container Transitioned from RUNNING to COMPLETED 19:04:58,337 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0014_01_000002 in state: COMPLETED event:FINISHED 19:04:58,343 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000002 19:04:58,344 INFO FSSchedulerNode:162 - Released container container_1394161202967_0014_01_000002 of capacity on host 172.16.1.64:52703, which currently has 1 containers, used and available, release resources=true 19:04:58,344 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0014_000001 released container container_1394161202967_0014_01_000002 on node: host: 172.16.1.64:52703 #containers=1 available= used= with event: FINISHED 19:04:58,373 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0014_01_000004 is : 143 19:04:58,389 INFO Container:888 - Container container_1394161202967_0014_01_000004 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:04:58,389 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_0/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000004 19:04:58,389 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_1/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000004 19:04:58,389 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_2/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000004 19:04:58,390 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-0_3/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000004 19:04:58,390 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000004 19:04:58,390 INFO Container:888 - Container container_1394161202967_0014_01_000004 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:04:58,390 INFO Application:339 - Removing container_1394161202967_0014_01_000004 from application application_1394161202967_0014 19:04:58,390 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:04:58,390 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0014 19:04:59,159 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000006 Container Transitioned from NEW to ALLOCATED 19:04:59,160 INFO RMAuditLogger:99 - USER=test OPERATION=AM Allocated Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000006 19:04:59,160 INFO FSSchedulerNode:119 - Assigned container container_1394161202967_0014_01_000006 of capacity on host 172.16.1.64:52707, which currently has 2 containers, used and available 19:04:59,323 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0014_01_000004 19:04:59,325 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000004 Container Transitioned from RUNNING to COMPLETED 19:04:59,325 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0014_01_000004 in state: COMPLETED event:FINISHED 19:04:59,325 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000004 19:04:59,325 INFO FSSchedulerNode:162 - Released container container_1394161202967_0014_01_000004 of capacity on host 172.16.1.64:52703, which currently has 0 containers, used and available, release resources=true 19:04:59,326 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0014_000001 released container container_1394161202967_0014_01_000004 on node: host: 172.16.1.64:52703 #containers=0 available= used= with event: FINISHED 19:04:59,896 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000006 Container Transitioned from ALLOCATED to ACQUIRED 19:04:59,931 INFO ContainerManagerImpl:572 - Start request for container_1394161202967_0014_01_000006 by user test 19:04:59,931 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000006 19:04:59,931 INFO Application:296 - Adding container_1394161202967_0014_01_000006 to application application_1394161202967_0014 19:04:59,932 INFO Container:888 - Container container_1394161202967_0014_01_000006 transitioned from NEW to LOCALIZING 19:04:59,932 INFO AuxServices:175 - Got event CONTAINER_INIT for appId application_1394161202967_0014 19:04:59,932 INFO AuxServices:175 - Got event APPLICATION_INIT for appId application_1394161202967_0014 19:04:59,932 INFO AuxServices:179 - Got APPLICATION_INIT for service mapreduce_shuffle 19:04:59,932 INFO ShuffleHandler:263 - Added token for job_1394161202967_0014 19:04:59,933 INFO Container:888 - Container container_1394161202967_0014_01_000006 transitioned from LOCALIZING to LOCALIZED 19:05:00,004 INFO Container:888 - Container container_1394161202967_0014_01_000006 transitioned from LOCALIZED to RUNNING 19:05:00,004 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:05:00,176 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000006 Container Transitioned from ACQUIRED to RUNNING 19:05:00,334 INFO DefaultContainerExecutor:189 - launchContainer: [nice, -n, 0, bash, /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000006/default_container_executor.sh] 19:05:00,903 INFO AppSchedulingInfo:166 - checking for deactivate... 19:05:03,268 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_r_000000_0/part-00000 dst=null perm=test:testg:rw-rw-rw- 19:05:03,306 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_r_000000_0/part-00000. BP-1007909283-172.16.1.64-1394161187028 blk_1073742020_1196{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:05:03,404 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742020_1196 src: /127.0.0.1:53863 dest: /127.0.0.1:52682 19:05:03,405 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742020_1196 src: /127.0.0.1:53864 dest: /127.0.0.1:52685 19:05:03,438 INFO clienttrace:1215 - src: /127.0.0.1:53864, dest: /127.0.0.1:52685, bytes: 16, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0014_r_000000_0_-1464751727_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742020_1196, duration: 31673000 19:05:03,438 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742020_1196, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:05:03,439 INFO clienttrace:1215 - src: /127.0.0.1:53863, dest: /127.0.0.1:52682, bytes: 16, op: HDFS_WRITE, cliID: DFSClient_attempt_1394161202967_0014_r_000000_0_-1464751727_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742020_1196, duration: 32697000 19:05:03,439 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742020_1196, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:05:03,444 INFO StateChange:2959 - DIR* completeFile: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_r_000000_0/part-00000 is closed by DFSClient_attempt_1394161202967_0014_r_000000_0_-1464751727_1 19:05:03,457 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_r_000000_0 dst=null perm=null 19:05:03,477 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_r_000000_0 dst=null perm=null 19:05:03,478 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/task_1394161202967_0014_r_000000 dst=null perm=null 19:05:03,481 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/_temporary/attempt_1394161202967_0014_r_000000_0 dst=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/task_1394161202967_0014_r_000000 perm=test:testg:rwxrwxrwx 19:05:03,497 INFO ContainerManagerImpl:718 - Stopping container with container Id: container_1394161202967_0014_01_000006 19:05:03,497 INFO NMAuditLogger:89 - USER=test IP=172.16.1.64 OPERATION=Stop Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000006 19:05:03,497 INFO Container:888 - Container container_1394161202967_0014_01_000006 transitioned from RUNNING to KILLING 19:05:03,497 INFO ContainerLaunch:341 - Cleaning up container container_1394161202967_0014_01_000006 19:05:03,569 WARN DefaultContainerExecutor:207 - Exit code from container container_1394161202967_0014_01_000006 is : 143 19:05:03,569 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/COMMIT_STARTED dst=null perm=test:supergroup:rw-rw-rw- 19:05:03,573 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/COMMIT_STARTED is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:05:03,577 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1 dst=null perm=null 19:05:03,585 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output dst=null perm=null 19:05:03,586 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output dst=null perm=null 19:05:03,588 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/task_1394161202967_0014_r_000000 dst=null perm=null 19:05:03,588 INFO Container:888 - Container container_1394161202967_0014_01_000006 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 19:05:03,588 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_0/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000006 19:05:03,589 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_1/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000006 19:05:03,589 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_2/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000006 19:05:03,590 INFO DefaultContainerExecutor:369 - Deleting absolute path : /Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/MiniMRCluster_1594994783/MiniMRCluster_1594994783-localDir-nm-1_3/usercache/test/appcache/application_1394161202967_0014/container_1394161202967_0014_01_000006 19:05:03,590 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/part-00000 dst=null perm=null 19:05:03,590 INFO NMAuditLogger:89 - USER=test OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000006 19:05:03,590 INFO Container:888 - Container container_1394161202967_0014_01_000006 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 19:05:03,590 INFO Application:339 - Removing container_1394161202967_0014_01_000006 from application application_1394161202967_0014 19:05:03,590 INFO ContainersMonitorImpl:159 - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 19:05:03,590 INFO AuxServices:175 - Got event CONTAINER_STOP for appId application_1394161202967_0014 19:05:03,593 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary/1/task_1394161202967_0014_r_000000/part-00000 dst=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/part-00000 perm=test:testg:rw-rw-rw- 19:05:03,601 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_temporary dst=null perm=null 19:05:03,605 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_SUCCESS dst=null perm=test:testg:rw-rw-rw- 19:05:03,607 INFO StateChange:2959 - DIR* completeFile: /user/test/07cbd33e-616b-4996-8bfc-23048d29529d/output/_SUCCESS is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:05:03,609 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/COMMIT_SUCCESS dst=null perm=test:supergroup:rw-rw-rw- 19:05:03,611 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/COMMIT_SUCCESS is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:05:03,630 INFO clienttrace:1215 - src: /127.0.0.1:53854, dest: /127.0.0.1:52685, bytes: 39226, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742019_1195, duration: 5739437000 19:05:03,630 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742019_1195, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:05:03,631 INFO clienttrace:1215 - src: /127.0.0.1:53853, dest: /127.0.0.1:52682, bytes: 39226, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742019_1195, duration: 5740613000 19:05:03,631 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742019_1195, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:05:03,633 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1.jhist is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:05:03,635 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:05:03,637 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073742021_1197{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW]]} 19:05:03,639 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742021_1197 src: /127.0.0.1:53865 dest: /127.0.0.1:52682 19:05:03,641 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742021_1197 src: /127.0.0.1:53866 dest: /127.0.0.1:52685 19:05:03,643 INFO clienttrace:1215 - src: /127.0.0.1:53866, dest: /127.0.0.1:52685, bytes: 350, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742021_1197, duration: 1400000 19:05:03,644 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742021_1197, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:05:03,644 INFO clienttrace:1215 - src: /127.0.0.1:53865, dest: /127.0.0.1:52682, bytes: 350, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742021_1197, duration: 2144000 19:05:03,644 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742021_1197, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:05:03,646 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary_tmp is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:05:03,650 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary_tmp dst=null perm=test:supergroup:rwxrwx--- 19:05:03,654 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1.jhist dst=null perm=null 19:05:03,655 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist_tmp dst=null perm=null 19:05:03,659 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1.jhist dst=null perm=null 19:05:03,660 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist_tmp dst=null perm=null 19:05:03,661 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1.jhist dst=null perm=null 19:05:03,663 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:05:03,666 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53867, bytes: 39534, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742019_1195, duration: 304000 19:05:03,668 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073742022_1198{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:05:03,670 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742022_1198 src: /127.0.0.1:53868 dest: /127.0.0.1:52682 19:05:03,671 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742022_1198 src: /127.0.0.1:53869 dest: /127.0.0.1:52685 19:05:03,676 INFO clienttrace:1215 - src: /127.0.0.1:53869, dest: /127.0.0.1:52685, bytes: 39226, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742022_1198, duration: 2656000 19:05:03,676 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742022_1198, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:05:03,676 INFO clienttrace:1215 - src: /127.0.0.1:53868, dest: /127.0.0.1:52682, bytes: 39226, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742022_1198, duration: 3331000 19:05:03,677 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742022_1198, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:05:03,679 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist_tmp is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:05:03,680 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist_tmp dst=null perm=test:supergroup:rwxrwx--- 19:05:03,682 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1_conf.xml dst=null perm=null 19:05:03,683 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml_tmp dst=null perm=null 19:05:03,684 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1_conf.xml dst=null perm=null 19:05:03,684 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml_tmp dst=null perm=null 19:05:03,685 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014/job_1394161202967_0014_1_conf.xml dst=null perm=null 19:05:03,688 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml_tmp dst=null perm=test:supergroup:rw-rw-rw- 19:05:03,690 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53867, bytes: 92675, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742017_1193, duration: 449000 19:05:03,693 INFO StateChange:3026 - BLOCK* allocateBlock: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml_tmp. BP-1007909283-172.16.1.64-1394161187028 blk_1073742023_1199{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-2ebb2e2d-4942-4815-af28-c14d569871af:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-7be84e8e-426f-440b-b7b5-f2aebf5532d6:NORMAL|RBW]]} 19:05:03,695 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742023_1199 src: /127.0.0.1:53870 dest: /127.0.0.1:52685 19:05:03,697 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742023_1199 src: /127.0.0.1:53871 dest: /127.0.0.1:52682 19:05:03,702 INFO clienttrace:1215 - src: /127.0.0.1:53871, dest: /127.0.0.1:52682, bytes: 91955, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742023_1199, duration: 2259000 19:05:03,702 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742023_1199, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:05:03,702 INFO clienttrace:1215 - src: /127.0.0.1:53870, dest: /127.0.0.1:52685, bytes: 91955, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_1556916339_1, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742023_1199, duration: 3266000 19:05:03,703 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742023_1199, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:05:03,705 INFO StateChange:2959 - DIR* completeFile: /tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml_tmp is closed by DFSClient_NONMAPREDUCE_1556916339_1 19:05:03,707 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=setPermission src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml_tmp dst=null perm=test:supergroup:rwxrwx--- 19:05:03,710 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary perm=test:supergroup:rwxrwx--- 19:05:03,712 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml perm=test:supergroup:rwxrwx--- 19:05:03,715 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=rename src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist_tmp dst=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist perm=test:supergroup:rwxrwx--- 19:05:03,723 INFO RMAppAttemptImpl:962 - Updating application attempt appattempt_1394161202967_0014_000001 with final state: FINISHING 19:05:03,723 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from RUNNING to FINAL_SAVING 19:05:03,724 INFO RMAppImpl:880 - Updating application application_1394161202967_0014 with final state: FINISHING 19:05:03,724 INFO RMAppImpl:643 - application_1394161202967_0014 State change from RUNNING to FINAL_SAVING 19:05:03,724 INFO RMStateStore:596 - Storing info for app: application_1394161202967_0014 19:05:03,725 INFO RMAppAttemptImpl:650 - appattempt_1394161202967_0014_000001 State change from FINAL_SAVING to FINISHING 19:05:03,725 INFO RMAppImpl:643 - application_1394161202967_0014 State change from FINAL_SAVING to FINISHING 19:05:04,498 INFO NodeStatusUpdaterImpl:374 - Removed completed container container_1394161202967_0014_01_000006 19:05:04,499 INFO RMContainerImpl:220 - container_1394161202967_0014_01_000006 Container Transitioned from RUNNING to COMPLETED 19:05:04,499 INFO FSSchedulerApp:90 - Completed container: container_1394161202967_0014_01_000006 in state: COMPLETED event:FINISHED 19:05:04,499 INFO RMAuditLogger:99 - USER=test OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS APPID=application_1394161202967_0014 CONTAINERID=container_1394161202967_0014_01_000006 19:05:04,499 INFO FSSchedulerNode:162 - Released container container_1394161202967_0014_01_000006 of capacity on host 172.16.1.64:52707, which currently has 1 containers, used and available, release resources=true 19:05:04,500 INFO FairScheduler:808 - Application attempt appattempt_1394161202967_0014_000001 released container container_1394161202967_0014_01_000006 on node: host: 172.16.1.64:52707 #containers=1 available= used= with event: FINISHED 19:05:04,735 INFO audit:7724 - allowed=true ugi=test (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/test/.staging/job_1394161202967_0014 dst=null perm=null 19:05:04,789 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161504789, maxDate=1394766304789, sequenceNumber=42, masterKeyId=2 19:05:04,790 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 42 19:05:04,801 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate dst=null perm=null 19:05:04,802 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=listStatus src=/tmp/hadoop-yarn/staging/history/done_intermediate/test dst=null perm=null 19:05:04,803 INFO CompletedJob:95 - Loading job: job_1394161202967_0014 from file: hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist 19:05:04,804 INFO CompletedJob:326 - Loading history file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist] 19:05:04,805 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist dst=null perm=null 19:05:04,805 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist dst=null perm=null 19:05:04,808 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53876, bytes: 39534, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-2110652364_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742022_1198, duration: 219000 19:05:04,837 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary dst=null perm=null 19:05:04,839 INFO clienttrace:734 - src: /127.0.0.1:52682, dest: /127.0.0.1:53877, bytes: 354, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_208821846_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742021_1197, duration: 183000 19:05:04,839 INFO JobSummary:372 - jobId=job_1394161202967_0014,submitTime=1394161483932,launchTime=1394161490676,firstMapTaskLaunchTime=1394161492860,firstReduceTaskLaunchTime=1394161499932,finishTime=1394161503612,resourcesPerMap=1024,resourcesPerReduce=1024,numMaps=2,numReduces=1,user=test,queue=default,status=SUCCEEDED,mapSlotSeconds=9,reduceSlotSeconds=3,jobName=MapReduceTest 19:05:04,840 INFO HistoryFileManager:373 - Deleting JobSummary file: [hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary] 19:05:04,841 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014.summary dst=null perm=null 19:05:04,842 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000 dst=null perm=null 19:05:04,842 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist 19:05:04,844 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0014-1394161483932-test-MapReduceTest-1394161503612-2-1-SUCCEEDED-root.test-1394161490676.jhist perm=test:supergroup:rwxrwx--- 19:05:04,844 INFO HistoryFileManager:947 - Moving hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml to hdfs://localhost:52681/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0014_conf.xml 19:05:04,845 INFO audit:7724 - allowed=true ugi=rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=rename options=0 src=/tmp/hadoop-yarn/staging/history/done_intermediate/test/job_1394161202967_0014_conf.xml dst=/tmp/hadoop-yarn/staging/history/done/2014/03/06/000000/job_1394161202967_0014_conf.xml perm=test:supergroup:rwxrwx--- 19:05:04,863 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:05:04,866 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:05:04,867 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:05:04,870 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53876, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, duration: 226000 19:05:04,889 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:05:04,890 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:05:04,891 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=open src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java/action-data.seq dst=null perm=null 19:05:04,893 INFO clienttrace:734 - src: /127.0.0.1:52685, dest: /127.0.0.1:53876, bytes: 170, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1007186919_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742012_1188, duration: 170000 19:05:04,906 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java dst=null perm=null 19:05:04,908 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=delete src=/user/test/oozie-rkan/0000000-140306190429176-oozie-rkan-W/map-reduce--java dst=null perm=null 19:05:04,936 INFO AbstractDelegationTokenSecretManager:285 - Creating password for identifier: owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161504936, maxDate=1394766304936, sequenceNumber=43, masterKeyId=2 19:05:04,936 INFO RMDelegationTokenSecretManager:108 - storing RMDelegation token with sequence number: 43 19:05:06,037 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/jar.jar dst=null perm=test:testg:rw-rw-rw- 19:05:06,041 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/jar.jar is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,043 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootJar.jar dst=null perm=test:testg:rw-rw-rw- 19:05:06,045 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootJar.jar is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,046 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/file dst=null perm=test:testg:rw-rw-rw- 19:05:06,047 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/file is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,049 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootFile dst=null perm=test:testg:rw-rw-rw- 19:05:06,050 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootFile is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,051 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/soFile.so dst=null perm=test:testg:rw-rw-rw- 19:05:06,053 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/soFile.so is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,054 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootSoFile.so dst=null perm=test:testg:rw-rw-rw- 19:05:06,055 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootSoFile.so is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,057 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/soFile.so.1 dst=null perm=test:testg:rw-rw-rw- 19:05:06,058 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/soFile.so.1 is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,060 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootSoFile.so.1 dst=null perm=test:testg:rw-rw-rw- 19:05:06,061 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootSoFile.so.1 is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,062 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/archive.tar dst=null perm=test:testg:rw-rw-rw- 19:05:06,064 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/archive.tar is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,065 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootArchive.tar dst=null perm=test:testg:rw-rw-rw- 19:05:06,070 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/root/rootArchive.tar is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,077 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/lib/test.jar dst=null perm=test:testg:rw-rw-rw- 19:05:06,079 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/lib/test.jar. BP-1007909283-172.16.1.64-1394161187028 blk_1073742024_1200{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW]]} 19:05:06,081 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742024_1200 src: /127.0.0.1:53882 dest: /127.0.0.1:52682 19:05:06,083 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742024_1200 src: /127.0.0.1:53883 dest: /127.0.0.1:52685 19:05:06,086 INFO clienttrace:1215 - src: /127.0.0.1:53883, dest: /127.0.0.1:52685, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-962038569_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742024_1200, duration: 1282000 19:05:06,114 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742024_1200, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:05:06,115 INFO clienttrace:1215 - src: /127.0.0.1:53882, dest: /127.0.0.1:52682, bytes: 1833, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-962038569_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742024_1200, duration: 30116000 19:05:06,116 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742024_1200, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:05:06,118 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/lib/test.jar is closed by DFSClient_NONMAPREDUCE_-962038569_8 19:05:06,120 INFO audit:7724 - allowed=true ugi=test (auth:PROXY) via rkanter (auth:SIMPLE) ip=/127.0.0.1 cmd=create src=/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/workflow.xml dst=null perm=test:testg:rw-rw-rw- 19:05:06,121 INFO StateChange:3026 - BLOCK* allocateBlock: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/workflow.xml. BP-1007909283-172.16.1.64-1394161187028 blk_1073742025_1201{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-fdea22e5-520f-41a1-ae24-5c89db493ed9:NORMAL|RBW], ReplicaUnderConstruction[[DISK]DS-9d151ffe-72c8-4f4a-8368-39290a0a904c:NORMAL|RBW]]} 19:05:06,123 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742025_1201 src: /127.0.0.1:53884 dest: /127.0.0.1:52685 19:05:06,124 INFO DataNode:569 - Receiving BP-1007909283-172.16.1.64-1394161187028:blk_1073742025_1201 src: /127.0.0.1:53885 dest: /127.0.0.1:52682 19:05:06,126 INFO clienttrace:1215 - src: /127.0.0.1:53885, dest: /127.0.0.1:52682, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-962038569_8, offset: 0, srvID: baef3ef0-1673-49bb-ac59-3f31d58cd534, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742025_1201, duration: 997000 19:05:06,126 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742025_1201, type=LAST_IN_PIPELINE, downstreams=0:[] terminating 19:05:06,127 INFO clienttrace:1215 - src: /127.0.0.1:53884, dest: /127.0.0.1:52685, bytes: 143, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-962038569_8, offset: 0, srvID: a305cec0-5002-4fa9-8de0-6cd548029493, blockid: BP-1007909283-172.16.1.64-1394161187028:blk_1073742025_1201, duration: 1522000 19:05:06,127 INFO DataNode:1196 - PacketResponder: BP-1007909283-172.16.1.64-1394161187028:blk_1073742025_1201, type=HAS_DOWNSTREAM_IN_PIPELINE terminating 19:05:06,129 INFO StateChange:2959 - DIR* completeFile: /user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735/app/workflow.xml is closed by DFSClient_NONMAPREDUCE_-962038569_8 Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduce] 19:00:21,028 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:00:21,028 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:00:21,028 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:00:21,028 INFO XLogService:539 - Log4j reload interval [disabled] 19:00:21,028 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:00:21,028 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduce] 19:00:21,029 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduce/conf] 19:00:21,029 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:00:21,032 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:00:21,033 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:00:21,033 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:00:21,033 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan7904211635080899688.dir] 19:00:21,033 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:00:21,033 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:00:21,033 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:00:21,033 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:00:21,034 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:00:21,304 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:00:21,304 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:00:21,304 INFO Services:539 - Initialized 19:00:21,305 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:00:21,305 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:00:21,317 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:00:21,317 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:00:21,317 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:00:21,317 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:00:21,318 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:00:21,318 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:00:21,318 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:00:21,318 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:00:21,318 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:00:21,318 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/923cd9a9-8f26-4806-b855-dfca70f6bf7a] 19:00:21,358 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:00:21,359 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:00:21,359 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:00:21,359 INFO XLogService:539 - Log4j reload interval [disabled] 19:00:21,359 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:00:21,359 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduce] 19:00:21,359 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduce/conf] 19:00:21,359 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:00:21,363 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:00:21,363 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:00:21,363 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:00:21,364 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:00:21,364 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:00:21,364 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan7801643020234854793.dir] 19:00:21,364 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:00:21,364 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:00:21,365 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:00:21,366 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:00:21,367 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:00:21,368 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:00:21,368 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:00:21,369 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:00:21,371 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:00:21,371 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:00:21,376 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:00:21,376 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:00:21,376 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:00:21,376 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:00:21,377 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:00:21,394 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:00:21,394 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:00:21,687 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190019423-oozie-rkan-W] ACTION[-] All entities initialized 19:00:21,687 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190019423-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:00:21,690 INFO Services:539 - Initialized 19:00:21,690 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:00:21,690 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:00:21,779 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:21,781 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] Setting LibFilesArchives 19:00:21,795 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:21,798 INFO deprecation:1009 - job.end.notification.url is deprecated. Instead, use mapreduce.job.end-notification.url 19:00:21,798 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] Get credential 'null' properties for action : 0000000-140306190021371-oozie-rkan-W@mr-action 19:00:21,801 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] credentials is null for the action 19:00:21,820 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:21,832 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:21,876 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] Creating Job Client for action 0000000-140306190021371-oozie-rkan-W@mr-action 19:00:21,911 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:00:22,002 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] Submitting the job through Job Client for action 0000000-140306190021371-oozie-rkan-W@mr-action 19:00:22,019 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:00:22,059 WARN JobSubmitter:150 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19:00:22,065 WARN JobSubmitter:259 - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 19:00:22,115 INFO FileInputFormat:253 - Total input paths to process : 1 19:00:22,160 INFO JobSubmitter:396 - number of splits:1 19:00:22,170 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:22,188 INFO JobSubmitter:479 - Submitting tokens for job: job_1394161202967_0001 19:00:22,189 INFO JobSubmitter:481 - Kind: RM_DELEGATION_TOKEN, Service: 172.16.1.64:52700, Ident: (owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161222003, maxDate=1394766022003, sequenceNumber=2, masterKeyId=2) 19:00:22,271 INFO YARNRunner:369 - Job jar is not present. Not adding any jar to the list of resources. 19:00:22,580 INFO YarnClientImpl:167 - Submitted application application_1394161202967_0001 19:00:22,615 INFO Job:1299 - The url to track the job: http://rkanter-mbp.local:52701/proxy/application_1394161202967_0001/ 19:00:22,616 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] After submission get the launcherId job_1394161202967_0001 19:00:22,631 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:22,642 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:00:22,692 INFO TestMapReduceActionExecutor:539 - Waiting up to [240,000] msec 19:00:22,694 INFO TestMapReduceActionExecutor:539 - Waiting up to [239,998] msec 19:00:23,750 INFO TestMapReduceActionExecutor:539 - Waiting up to [238,942] msec 19:00:24,754 INFO TestMapReduceActionExecutor:539 - Waiting up to [237,938] msec 19:00:25,827 INFO TestMapReduceActionExecutor:539 - Waiting up to [236,865] msec 19:00:26,842 INFO TestMapReduceActionExecutor:539 - Waiting up to [235,850] msec 19:00:27,845 INFO TestMapReduceActionExecutor:539 - Waiting up to [234,847] msec 19:00:28,848 INFO TestMapReduceActionExecutor:539 - Waiting up to [233,844] msec 19:00:29,850 INFO TestMapReduceActionExecutor:539 - Waiting up to [232,842] msec 19:00:30,966 INFO TestMapReduceActionExecutor:539 - Waiting up to [231,726] msec 19:00:31,692 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 19:00:31,692 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 19:00:31,693 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service first instance 19:00:31,714 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service first instance 19:00:31,723 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking 19:00:31,726 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 19:00:31,727 INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2014-03-07T03:05Z, Num jobs to materialize = 0 19:00:31,742 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null] 19:00:31,742 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days. 19:00:31,742 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinators, [0] bundles 19:00:31,752 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 19:00:31,761 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery 19:00:31,971 INFO TestMapReduceActionExecutor:539 - Waiting up to [230,722] msec 19:00:33,031 INFO TestMapReduceActionExecutor:539 - Waiting up to [229,669] msec 19:00:34,075 INFO TestMapReduceActionExecutor:539 - Waiting up to [228,618] msec 19:00:35,086 INFO TestMapReduceActionExecutor:539 - Waiting up to [227,606] msec 19:00:36,090 INFO TestMapReduceActionExecutor:539 - Waiting up to [226,602] msec 19:00:37,093 INFO TestMapReduceActionExecutor:539 - Waiting up to [225,599] msec 19:00:38,095 INFO TestMapReduceActionExecutor:539 - Waiting up to [224,597] msec 19:00:39,097 INFO TestMapReduceActionExecutor:539 - Waiting up to [223,595] msec 19:00:40,138 INFO TestMapReduceActionExecutor:539 - Waiting up to [222,555] msec 19:00:41,188 INFO TestMapReduceActionExecutor:539 - Waiting up to [221,511] msec 19:00:42,200 INFO TestMapReduceActionExecutor:539 - Waiting up to [220,492] msec 19:00:43,252 INFO CodecPool:179 - Got brand-new decompressor [.deflate] 19:00:43,293 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:43,316 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:00:43,326 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:00:43,910 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0001] new ID [job_1394161202967_0002] 19:00:43,912 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] checking action, external ID [job_1394161202967_0001] status [RUNNING] 19:00:43,936 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:00:43,953 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:00:43,982 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:00:43,983 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,998] msec 19:00:44,986 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,996] msec 19:00:46,113 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,868] msec 19:00:47,115 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,866] msec 19:00:48,117 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,864] msec 19:00:49,159 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,822] msec 19:00:50,163 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,819] msec 19:00:51,166 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,815] msec 19:00:52,169 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,812] msec 19:00:53,174 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,807] msec 19:00:54,179 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,802] msec 19:00:55,208 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,777] msec 19:00:56,211 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,770] msec 19:00:57,213 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,768] msec 19:00:58,216 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,765] msec 19:00:59,219 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,763] msec 19:01:00,254 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:00,273 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:00,280 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:01:00,407 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0001] new ID [job_1394161202967_0002] 19:01:00,415 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] action completed, external ID [job_1394161202967_0001] 19:01:00,500 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:00,519 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:00,524 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:01:00,566 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithCredentials] 19:01:00,626 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:01:00,626 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:01:00,626 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:01:00,626 INFO XLogService:539 - Log4j reload interval [disabled] 19:01:00,626 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:01:00,627 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithCredentials] 19:01:00,627 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithCredentials/conf] 19:01:00,627 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:01:00,631 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:01:00,632 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:01:00,632 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:01:00,632 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan1650318085115875720.dir] 19:01:00,632 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:01:00,632 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:01:00,633 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:01:00,633 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:01:00,633 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:01:00,883 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:01:00,883 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:01:00,883 INFO Services:539 - Initialized 19:01:00,883 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:01:00,883 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:01:00,897 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:01:00,897 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:01:00,897 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:01:00,897 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:01:00,897 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:01:00,897 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:01:00,898 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:01:00,898 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:01:00,898 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:01:00,898 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/b130ab94-f175-4e88-950f-b0bcd615795c] 19:01:00,945 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:01:00,945 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:01:00,945 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:01:00,945 INFO XLogService:539 - Log4j reload interval [disabled] 19:01:00,946 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:01:00,946 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithCredentials] 19:01:00,946 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithCredentials/conf] 19:01:00,946 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:01:00,951 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:01:00,951 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:01:00,951 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:01:00,952 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:01:00,952 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:01:00,952 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan7391420299515693636.dir] 19:01:00,953 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:01:00,953 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:01:00,953 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:01:00,953 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:01:00,953 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:01:00,953 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:01:00,953 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:01:00,953 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:01:00,954 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:01:00,955 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:01:00,956 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:01:00,958 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:01:00,958 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:01:00,958 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:01:00,958 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:01:00,958 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:01:00,958 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:01:00,959 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:01:00,959 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:01:00,959 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:01:00,959 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:01:00,960 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:01:00,961 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:01:00,969 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:01:00,970 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:01:00,970 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:01:00,970 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:01:00,970 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:01:00,998 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:01:00,998 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:01:01,307 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] All entities initialized 19:01:01,307 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190021371-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:01:01,311 INFO Services:539 - Initialized 19:01:01,311 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:01:01,312 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:01:01,389 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:01,391 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Setting LibFilesArchives 19:01:01,406 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:01,408 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Get credential 'testcred' properties for action : 0000000-140306190100960-oozie-rkan-W@mr-action 19:01:01,414 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] getCredProperties: Name: testcred, Type: cred 19:01:01,444 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] getCredProperties: Properties name :'aa', Value : 'AA' 19:01:01,444 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] getCredProperties: Properties name :'bb', Value : 'BB' 19:01:01,445 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Credential Properties set for action : 0000000-140306190100960-oozie-rkan-W@mr-action 19:01:01,445 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] property : 'aa', value : 'AA' 19:01:01,445 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] property : 'bb', value : 'BB' 19:01:01,447 DEBUG CredentialsProvider:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Credentials Provider is created for Type: cred 19:01:01,447 DEBUG CredentialsProvider:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Creating Credential class for : cred=org.apache.oozie.action.hadoop.CredentialForTest 19:01:01,447 DEBUG CredentialsProvider:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Creating Credential type : 'cred', class Name : 'org.apache.oozie.action.hadoop.CredentialForTest' 19:01:01,450 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Retrieved Credential 'testcred' for action 0000000-140306190100960-oozie-rkan-W@mr-action 19:01:01,467 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:01,484 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:01,526 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Creating Job Client for action 0000000-140306190100960-oozie-rkan-W@mr-action 19:01:01,979 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:01,984 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] Submitting the job through Job Client for action 0000000-140306190100960-oozie-rkan-W@mr-action 19:01:01,999 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:02,005 WARN JobSubmitter:150 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19:01:02,009 WARN JobSubmitter:259 - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 19:01:02,027 INFO FileInputFormat:253 - Total input paths to process : 1 19:01:02,056 INFO JobSubmitter:396 - number of splits:1 19:01:02,061 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:02,075 INFO JobSubmitter:479 - Submitting tokens for job: job_1394161202967_0003 19:01:02,075 INFO JobSubmitter:481 - Kind: RM_DELEGATION_TOKEN, Service: 172.16.1.64:52700, Ident: (owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161261984, maxDate=1394766061984, sequenceNumber=9, masterKeyId=2) 19:01:02,079 INFO YARNRunner:369 - Job jar is not present. Not adding any jar to the list of resources. 19:01:02,099 INFO YarnClientImpl:167 - Submitted application application_1394161202967_0003 19:01:02,101 INFO Job:1299 - The url to track the job: http://rkanter-mbp.local:52701/proxy/application_1394161202967_0003/ 19:01:02,101 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] After submission get the launcherId job_1394161202967_0003 19:01:02,118 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:02,132 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:02,153 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:01:02,155 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,998] msec 19:01:03,204 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,949] msec 19:01:04,244 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,910] msec 19:01:05,247 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,906] msec 19:01:06,250 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,903] msec 19:01:07,253 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,900] msec 19:01:08,256 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,897] msec 19:01:09,360 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,793] msec 19:01:10,364 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,790] msec 19:01:11,313 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 19:01:11,314 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-03-07T03:00Z 19:01:11,313 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 19:01:11,352 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking 19:01:11,352 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-03-07T03:00Z 19:01:11,358 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 19:01:11,365 INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2014-03-07T03:06Z, Num jobs to materialize = 0 19:01:11,367 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,786] msec 19:01:11,376 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null] 19:01:11,380 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days. 19:01:11,380 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinators, [0] bundles 19:01:11,394 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 19:01:11,399 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery 19:01:12,370 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,784] msec 19:01:13,372 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,781] msec 19:01:14,375 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,779] msec 19:01:15,377 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,776] msec 19:01:16,411 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,749] msec 19:01:17,481 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,676] msec 19:01:18,525 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:18,546 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:18,551 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:01:18,656 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0003] new ID [job_1394161202967_0004] 19:01:18,658 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] checking action, external ID [job_1394161202967_0003] status [RUNNING] 19:01:18,685 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:18,706 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:18,728 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:01:18,729 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,999] msec 19:01:19,733 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,995] msec 19:01:20,736 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,993] msec 19:01:21,738 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,990] msec 19:01:22,863 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,865] msec 19:01:23,866 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,862] msec 19:01:24,898 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,840] msec 19:01:25,928 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,801] msec 19:01:26,967 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,766] msec 19:01:27,970 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,758] msec 19:01:28,973 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,755] msec 19:01:29,976 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,753] msec 19:01:31,016 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,712] msec 19:01:32,021 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,707] msec 19:01:33,026 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,703] msec 19:01:34,030 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,698] msec 19:01:35,033 INFO TestMapReduceActionExecutor:539 - Waiting up to [103,695] msec 19:01:36,036 INFO TestMapReduceActionExecutor:539 - Waiting up to [102,692] msec 19:01:37,040 INFO TestMapReduceActionExecutor:539 - Waiting up to [101,689] msec 19:01:38,045 INFO TestMapReduceActionExecutor:539 - Waiting up to [100,683] msec 19:01:39,048 INFO TestMapReduceActionExecutor:539 - Waiting up to [99,680] msec 19:01:40,051 INFO TestMapReduceActionExecutor:539 - Waiting up to [98,677] msec 19:01:41,054 INFO TestMapReduceActionExecutor:539 - Waiting up to [97,675] msec 19:01:42,056 INFO TestMapReduceActionExecutor:539 - Waiting up to [96,672] msec 19:01:43,062 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithUberJarDisabled] 19:01:43,098 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:01:43,098 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:01:43,098 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:01:43,098 INFO XLogService:539 - Log4j reload interval [disabled] 19:01:43,099 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:01:43,099 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithUberJarDisabled] 19:01:43,099 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithUberJarDisabled/conf] 19:01:43,099 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:01:43,103 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:01:43,104 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:01:43,104 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:01:43,104 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan5838617367357352700.dir] 19:01:43,104 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:01:43,105 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:01:43,105 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:01:43,105 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:01:43,105 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:01:43,385 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:01:43,386 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:01:43,386 INFO Services:539 - Initialized 19:01:43,386 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:01:43,386 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:01:43,397 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:01:43,398 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:01:43,398 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/8658ee8f-5e90-4b30-b0d7-c9034441c036] 19:01:43,440 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:01:43,440 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:01:43,440 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:01:43,441 INFO XLogService:539 - Log4j reload interval [disabled] 19:01:43,441 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:01:43,441 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithUberJarDisabled] 19:01:43,441 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testMapReduceWithUberJarDisabled/conf] 19:01:43,441 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:01:43,445 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:01:43,445 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:01:43,445 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:01:43,446 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:01:43,446 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:01:43,446 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan6654439169325500290.dir] 19:01:43,447 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:01:43,447 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:01:43,447 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:01:43,447 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:01:43,447 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:01:43,447 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:01:43,448 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:01:43,449 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:01:43,450 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:01:43,450 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:01:43,450 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:01:43,450 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:01:43,450 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:01:43,450 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:01:43,450 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:01:43,451 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:01:43,452 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:01:43,452 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:01:43,452 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:01:43,452 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:01:43,452 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:01:43,452 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:01:43,453 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:01:43,453 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:01:43,453 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:01:43,455 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:01:43,455 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:01:43,460 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:01:43,460 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:01:43,461 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:01:43,461 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:01:43,461 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:01:43,478 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:01:43,478 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:01:43,773 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] All entities initialized 19:01:43,773 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[test-wf-cred] JOB[0000000-140306190100960-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:01:43,777 INFO Services:539 - Initialized 19:01:43,777 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:01:43,777 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:01:43,777 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.action.mapreduce.uber.jar.enable]=[false] 19:01:43,883 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:43,885 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.action.mapreduce.uber.jar.enable]=[false] 19:01:43,888 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testStreaming] 19:01:43,920 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:01:43,920 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:01:43,920 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:01:43,920 INFO XLogService:539 - Log4j reload interval [disabled] 19:01:43,921 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:01:43,921 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testStreaming] 19:01:43,921 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testStreaming/conf] 19:01:43,921 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:01:43,925 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:01:43,926 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:01:43,926 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:01:43,927 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan4013499255807208942.dir] 19:01:43,927 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:01:43,927 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:01:43,927 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:01:43,927 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:01:43,927 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:01:44,154 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:01:44,154 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:01:44,155 INFO Services:539 - Initialized 19:01:44,155 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:01:44,155 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:01:44,169 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:01:44,170 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/89f519b1-00f0-4504-b98f-17ae4592a9d9] 19:01:44,208 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:01:44,208 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:01:44,208 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:01:44,208 INFO XLogService:539 - Log4j reload interval [disabled] 19:01:44,208 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:01:44,208 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testStreaming] 19:01:44,209 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testStreaming/conf] 19:01:44,209 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:01:44,212 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:01:44,213 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:01:44,213 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:01:44,213 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:01:44,213 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:01:44,214 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan5048234864609928970.dir] 19:01:44,214 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:01:44,214 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:01:44,214 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:01:44,214 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:01:44,214 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:01:44,214 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:01:44,215 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:01:44,216 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:01:44,217 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:01:44,217 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:01:44,217 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:01:44,217 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:01:44,217 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:01:44,217 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:01:44,217 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:01:44,218 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:01:44,218 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:01:44,218 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:01:44,219 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:01:44,219 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:01:44,219 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:01:44,219 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:01:44,219 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:01:44,219 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:01:44,219 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:01:44,221 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:01:44,221 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:01:44,226 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:01:44,226 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:01:44,227 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:01:44,227 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:01:44,227 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:01:44,243 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:01:44,244 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:01:44,527 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190143455-oozie-rkan-W] ACTION[-] All entities initialized 19:01:44,527 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190143455-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:01:44,531 INFO Services:539 - Initialized 19:01:44,532 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:01:44,532 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:01:44,630 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,632 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] Setting LibFilesArchives 19:01:44,646 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,660 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,661 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] Get credential 'null' properties for action : 0000000-140306190144221-oozie-rkan-W@mr-action 19:01:44,664 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] credentials is null for the action 19:01:44,681 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,695 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,708 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,751 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] Creating Job Client for action 0000000-140306190144221-oozie-rkan-W@mr-action 19:01:44,764 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:44,769 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] Submitting the job through Job Client for action 0000000-140306190144221-oozie-rkan-W@mr-action 19:01:44,781 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:44,787 WARN JobSubmitter:150 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19:01:44,791 WARN JobSubmitter:259 - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 19:01:44,812 INFO FileInputFormat:253 - Total input paths to process : 1 19:01:44,839 INFO JobSubmitter:396 - number of splits:1 19:01:44,843 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,857 INFO JobSubmitter:479 - Submitting tokens for job: job_1394161202967_0005 19:01:44,857 INFO JobSubmitter:481 - Kind: RM_DELEGATION_TOKEN, Service: 172.16.1.64:52700, Ident: (owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161304769, maxDate=1394766104769, sequenceNumber=14, masterKeyId=2) 19:01:44,860 INFO YARNRunner:369 - Job jar is not present. Not adding any jar to the list of resources. 19:01:44,877 INFO YarnClientImpl:167 - Submitted application application_1394161202967_0005 19:01:44,879 INFO Job:1299 - The url to track the job: http://rkanter-mbp.local:52701/proxy/application_1394161202967_0005/ 19:01:44,880 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] After submission get the launcherId job_1394161202967_0005 19:01:44,896 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:01:44,908 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:01:44,926 INFO TestMapReduceActionExecutor:539 - Waiting up to [240,000] msec 19:01:44,928 INFO TestMapReduceActionExecutor:539 - Waiting up to [239,998] msec 19:01:45,984 INFO TestMapReduceActionExecutor:539 - Waiting up to [238,942] msec 19:01:47,026 INFO TestMapReduceActionExecutor:539 - Waiting up to [237,921] msec 19:01:48,038 INFO TestMapReduceActionExecutor:539 - Waiting up to [236,888] msec 19:01:49,041 INFO TestMapReduceActionExecutor:539 - Waiting up to [235,885] msec 19:01:50,044 INFO TestMapReduceActionExecutor:539 - Waiting up to [234,882] msec 19:01:51,046 INFO TestMapReduceActionExecutor:539 - Waiting up to [233,880] msec 19:01:52,168 INFO TestMapReduceActionExecutor:539 - Waiting up to [232,758] msec 19:01:53,171 INFO TestMapReduceActionExecutor:539 - Waiting up to [231,755] msec 19:01:54,174 INFO TestMapReduceActionExecutor:539 - Waiting up to [230,752] msec 19:01:54,561 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 19:01:54,561 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 19:01:54,563 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-03-07T03:01Z 19:01:54,950 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-03-07T03:01Z 19:01:55,084 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking 19:01:55,311 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 19:01:55,311 INFO TestMapReduceActionExecutor:539 - Waiting up to [229,627] msec 19:01:55,476 INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2014-03-07T03:06Z, Num jobs to materialize = 0 19:01:55,710 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null] 19:01:55,714 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days. 19:01:55,715 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinators, [0] bundles 19:01:55,720 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 19:01:55,729 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery 19:01:56,351 INFO TestMapReduceActionExecutor:539 - Waiting up to [228,578] msec 19:01:57,354 INFO TestMapReduceActionExecutor:539 - Waiting up to [227,573] msec 19:01:58,356 INFO TestMapReduceActionExecutor:539 - Waiting up to [226,570] msec 19:01:59,359 INFO TestMapReduceActionExecutor:539 - Waiting up to [225,567] msec 19:02:00,362 INFO TestMapReduceActionExecutor:539 - Waiting up to [224,564] msec 19:02:01,367 INFO TestMapReduceActionExecutor:539 - Waiting up to [223,559] msec 19:02:02,439 INFO TestMapReduceActionExecutor:539 - Waiting up to [222,487] msec 19:02:03,443 INFO TestMapReduceActionExecutor:539 - Waiting up to [221,483] msec 19:02:04,513 INFO TestMapReduceActionExecutor:539 - Waiting up to [220,413] msec 19:02:05,558 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:05,574 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:05,582 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:02:05,669 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0005] new ID [job_1394161202967_0006] 19:02:05,670 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] checking action, external ID [job_1394161202967_0005] status [RUNNING] 19:02:05,688 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:05,705 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:05,731 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:02:05,732 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,999] msec 19:02:06,735 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,996] msec 19:02:07,737 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,994] msec 19:02:08,739 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,992] msec 19:02:09,847 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,884] msec 19:02:10,849 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,882] msec 19:02:11,912 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,819] msec 19:02:12,921 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,810] msec 19:02:13,934 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,797] msec 19:02:14,937 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,794] msec 19:02:15,940 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,791] msec 19:02:16,951 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,780] msec 19:02:17,954 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,777] msec 19:02:18,956 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,775] msec 19:02:19,960 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,771] msec 19:02:20,963 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,768] msec 19:02:21,966 INFO TestMapReduceActionExecutor:539 - Waiting up to [103,765] msec 19:02:22,971 INFO TestMapReduceActionExecutor:539 - Waiting up to [102,760] msec 19:02:23,974 INFO TestMapReduceActionExecutor:539 - Waiting up to [101,758] msec 19:02:24,994 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:25,008 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:25,015 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:02:25,108 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0005] new ID [job_1394161202967_0006] 19:02:25,114 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] action completed, external ID [job_1394161202967_0005] 19:02:25,146 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:25,162 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:25,167 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:02:25,195 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testPipes] 19:02:25,229 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:02:25,229 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:02:25,229 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:02:25,229 INFO XLogService:539 - Log4j reload interval [disabled] 19:02:25,229 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:02:25,230 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testPipes] 19:02:25,230 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testPipes/conf] 19:02:25,230 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:02:25,234 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:02:25,235 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:02:25,235 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:02:25,235 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan812680431312052279.dir] 19:02:25,235 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:02:25,235 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:02:25,236 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:02:25,236 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:02:25,236 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:02:25,493 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:02:25,493 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:02:25,493 INFO Services:539 - Initialized 19:02:25,494 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:02:25,494 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:02:25,507 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:02:25,507 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:02:25,507 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:02:25,507 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:02:25,508 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:02:25,508 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:02:25,508 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:02:25,508 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:02:25,508 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:02:25,508 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/01ef5ae0-1afd-49ba-9b02-bbd844a78094] 19:02:25,551 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:02:25,551 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:02:25,551 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:02:25,551 INFO XLogService:539 - Log4j reload interval [disabled] 19:02:25,552 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:02:25,552 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testPipes] 19:02:25,552 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testPipes/conf] 19:02:25,552 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:02:25,556 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:02:25,556 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:02:25,556 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:02:25,556 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:02:25,557 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:02:25,557 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan3669523606941280187.dir] 19:02:25,557 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:02:25,557 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:02:25,558 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:02:25,559 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:02:25,560 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:02:25,561 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:02:25,562 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:02:25,564 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:02:25,564 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:02:25,570 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:02:25,570 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:02:25,570 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:02:25,570 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:02:25,570 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:02:25,588 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:02:25,588 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:02:25,838 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] All entities initialized 19:02:25,838 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:02:25,842 INFO Services:539 - Initialized 19:02:25,842 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:02:25,842 INFO Services:539 - Oozie System ID [oozie-rkan] started! SKIPPING TEST: TestMapReduceActionExecutor.testPipes(), binary 'wordcount-simple' not available in the classpath 19:02:25,849 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_TRUE] 19:02:25,880 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:02:25,880 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:02:25,880 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:02:25,880 INFO XLogService:539 - Log4j reload interval [disabled] 19:02:25,880 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:02:25,881 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_TRUE] 19:02:25,881 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_TRUE/conf] 19:02:25,881 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:02:25,887 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:02:25,887 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:02:25,888 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:02:25,888 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan8191455511281790155.dir] 19:02:25,888 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:02:25,889 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:02:25,889 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:02:25,889 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:02:25,889 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:02:26,154 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:02:26,154 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:02:26,154 INFO Services:539 - Initialized 19:02:26,154 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:02:26,155 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:02:26,172 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:02:26,172 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:02:26,173 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:02:26,173 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:02:26,173 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:02:26,173 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:02:26,173 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:02:26,173 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:02:26,173 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:02:26,174 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/0649d175-e1d8-42cf-9c78-9b269d43217f] 19:02:26,232 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:02:26,232 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:02:26,232 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:02:26,233 INFO XLogService:539 - Log4j reload interval [disabled] 19:02:26,233 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:02:26,233 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_TRUE] 19:02:26,233 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_TRUE/conf] 19:02:26,233 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:02:26,238 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:02:26,238 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:02:26,239 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:02:26,239 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:02:26,239 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:02:26,240 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan2241135994745968950.dir] 19:02:26,240 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:02:26,240 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:02:26,240 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:02:26,240 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:02:26,240 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:02:26,241 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:02:26,242 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:02:26,243 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:02:26,244 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:02:26,245 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:02:26,247 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:02:26,247 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:02:26,254 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:02:26,254 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:02:26,254 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:02:26,254 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:02:26,255 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:02:26,278 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:02:26,279 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:02:26,591 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] All entities initialized 19:02:26,591 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190144221-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:02:26,594 INFO Services:539 - Initialized 19:02:26,594 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:02:26,594 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:02:26,666 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:26,667 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] Setting LibFilesArchives 19:02:26,681 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:26,683 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] Get credential 'null' properties for action : 0000000-140306190226247-oozie-rkan-W@mr-action 19:02:26,685 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] credentials is null for the action 19:02:26,705 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:26,719 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:26,754 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] Creating Job Client for action 0000000-140306190226247-oozie-rkan-W@mr-action 19:02:26,767 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:26,771 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] Submitting the job through Job Client for action 0000000-140306190226247-oozie-rkan-W@mr-action 19:02:26,785 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:26,790 WARN JobSubmitter:150 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19:02:26,793 WARN JobSubmitter:259 - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 19:02:26,809 INFO FileInputFormat:253 - Total input paths to process : 1 19:02:26,835 INFO JobSubmitter:396 - number of splits:1 19:02:26,839 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:26,853 INFO JobSubmitter:479 - Submitting tokens for job: job_1394161202967_0007 19:02:26,853 INFO JobSubmitter:481 - Kind: RM_DELEGATION_TOKEN, Service: 172.16.1.64:52700, Ident: (owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161346772, maxDate=1394766146772, sequenceNumber=21, masterKeyId=2) 19:02:26,856 INFO YARNRunner:369 - Job jar is not present. Not adding any jar to the list of resources. 19:02:26,870 INFO YarnClientImpl:167 - Submitted application application_1394161202967_0007 19:02:26,873 INFO Job:1299 - The url to track the job: http://rkanter-mbp.local:52701/proxy/application_1394161202967_0007/ 19:02:26,873 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] After submission get the launcherId job_1394161202967_0007 19:02:26,888 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:26,900 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:26,918 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:02:26,919 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,999] msec 19:02:27,974 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,944] msec 19:02:29,019 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,899] msec 19:02:30,047 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,871] msec 19:02:31,069 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,849] msec 19:02:32,072 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,846] msec 19:02:33,075 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,843] msec 19:02:34,077 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,842] msec 19:02:35,214 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,704] msec 19:02:36,216 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,702] msec 19:02:36,593 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 19:02:36,595 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 19:02:36,595 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-03-07T03:01Z 19:02:36,614 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking 19:02:36,615 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-03-07T03:01Z 19:02:36,619 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 19:02:36,625 INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2014-03-07T03:07Z, Num jobs to materialize = 0 19:02:36,634 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null] 19:02:36,634 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days. 19:02:36,635 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinators, [0] bundles 19:02:36,642 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 19:02:36,647 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery 19:02:37,254 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,664] msec 19:02:38,257 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,661] msec 19:02:39,260 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,658] msec 19:02:40,263 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,656] msec 19:02:41,265 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,653] msec 19:02:42,267 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,651] msec 19:02:43,319 INFO TestMapReduceActionExecutor:539 - Waiting up to [103,600] msec 19:02:44,321 INFO TestMapReduceActionExecutor:539 - Waiting up to [102,597] msec 19:02:45,345 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:45,388 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:02:45,413 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:45,422 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:02:45,540 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0007] new ID [job_1394161202967_0008] 19:02:45,542 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] checking action, external ID [job_1394161202967_0007] status [RUNNING] 19:02:45,569 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:02:45,604 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:02:45,606 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,998] msec 19:02:46,609 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,995] msec 19:02:47,612 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,992] msec 19:02:48,614 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,990] msec 19:02:49,617 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,987] msec 19:02:50,734 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,870] msec 19:02:51,736 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,868] msec 19:02:52,770 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,835] msec 19:02:53,786 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,819] msec 19:02:54,792 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,812] msec 19:02:55,795 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,809] msec 19:02:56,799 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,805] msec 19:02:57,801 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,803] msec 19:02:58,805 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,799] msec 19:02:59,808 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,796] msec 19:03:00,810 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,794] msec 19:03:01,814 INFO TestMapReduceActionExecutor:539 - Waiting up to [103,790] msec 19:03:02,817 INFO TestMapReduceActionExecutor:539 - Waiting up to [102,787] msec 19:03:03,819 INFO TestMapReduceActionExecutor:539 - Waiting up to [101,785] msec 19:03:04,822 INFO TestMapReduceActionExecutor:539 - Waiting up to [100,782] msec 19:03:05,829 INFO TestMapReduceActionExecutor:539 - Waiting up to [99,775] msec 19:03:06,833 INFO TestMapReduceActionExecutor:539 - Waiting up to [98,771] msec 19:03:07,836 INFO TestMapReduceActionExecutor:539 - Waiting up to [97,768] msec 19:03:08,838 INFO TestMapReduceActionExecutor:539 - Waiting up to [96,766] msec 19:03:09,848 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_FALSE] 19:03:09,884 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:03:09,884 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:03:09,884 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:03:09,884 INFO XLogService:539 - Log4j reload interval [disabled] 19:03:09,884 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:03:09,884 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_FALSE] 19:03:09,885 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_FALSE/conf] 19:03:09,885 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:03:09,888 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:03:09,889 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:03:09,889 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:03:09,889 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan5079013354878209536.dir] 19:03:09,890 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:03:09,890 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:03:09,890 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:03:09,890 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:03:09,890 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:03:10,121 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:03:10,121 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:03:10,122 INFO Services:539 - Initialized 19:03:10,122 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:03:10,122 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:03:10,141 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:03:10,141 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:03:10,141 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:03:10,141 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:03:10,142 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:03:10,142 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:03:10,142 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:03:10,142 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:03:10,142 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:03:10,142 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/eb7304e1-3c44-4584-9808-a05e46c29a5c] 19:03:10,190 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:03:10,190 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:03:10,190 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:03:10,190 INFO XLogService:539 - Log4j reload interval [disabled] 19:03:10,191 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:03:10,191 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_FALSE] 19:03:10,191 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetExecutionStats_when_user_has_specified_stats_write_FALSE/conf] 19:03:10,191 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:03:10,195 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:03:10,195 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:03:10,195 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:03:10,196 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:03:10,196 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:03:10,196 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan2367504130702231813.dir] 19:03:10,197 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:03:10,197 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:03:10,197 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:03:10,197 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:03:10,197 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:03:10,197 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:03:10,197 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:03:10,197 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:03:10,197 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:03:10,198 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:03:10,199 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:03:10,200 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:03:10,200 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:03:10,200 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:03:10,200 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:03:10,200 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:03:10,200 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:03:10,202 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:03:10,202 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:03:10,202 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:03:10,202 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:03:10,202 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:03:10,202 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:03:10,203 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:03:10,203 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:03:10,203 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:03:10,203 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:03:10,205 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:03:10,205 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:03:10,211 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:03:10,211 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:03:10,212 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:03:10,212 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:03:10,212 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:03:10,231 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:03:10,232 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:03:10,482 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] All entities initialized 19:03:10,482 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190226247-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:03:10,488 INFO Services:539 - Initialized 19:03:10,489 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:03:10,489 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:03:10,555 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:10,557 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] Setting LibFilesArchives 19:03:10,571 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:10,574 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] Get credential 'null' properties for action : 0000000-140306190310205-oozie-rkan-W@mr-action 19:03:10,625 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] credentials is null for the action 19:03:10,658 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:10,687 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:10,738 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] Creating Job Client for action 0000000-140306190310205-oozie-rkan-W@mr-action 19:03:10,764 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:10,772 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] Submitting the job through Job Client for action 0000000-140306190310205-oozie-rkan-W@mr-action 19:03:10,800 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:10,809 WARN JobSubmitter:150 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19:03:10,816 WARN JobSubmitter:259 - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 19:03:10,854 INFO FileInputFormat:253 - Total input paths to process : 1 19:03:10,899 INFO JobSubmitter:396 - number of splits:1 19:03:10,906 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:10,929 INFO JobSubmitter:479 - Submitting tokens for job: job_1394161202967_0009 19:03:10,929 INFO JobSubmitter:481 - Kind: RM_DELEGATION_TOKEN, Service: 172.16.1.64:52700, Ident: (owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161390773, maxDate=1394766190773, sequenceNumber=26, masterKeyId=2) 19:03:10,934 INFO YARNRunner:369 - Job jar is not present. Not adding any jar to the list of resources. 19:03:10,958 INFO YarnClientImpl:167 - Submitted application application_1394161202967_0009 19:03:10,963 INFO Job:1299 - The url to track the job: http://rkanter-mbp.local:52701/proxy/application_1394161202967_0009/ 19:03:10,964 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] After submission get the launcherId job_1394161202967_0009 19:03:10,992 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:11,008 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:11,036 INFO TestMapReduceActionExecutor:539 - Waiting up to [240,000] msec 19:03:11,037 INFO TestMapReduceActionExecutor:539 - Waiting up to [239,999] msec 19:03:12,073 INFO TestMapReduceActionExecutor:539 - Waiting up to [238,964] msec 19:03:13,077 INFO TestMapReduceActionExecutor:539 - Waiting up to [237,960] msec 19:03:14,079 INFO TestMapReduceActionExecutor:539 - Waiting up to [236,957] msec 19:03:15,081 INFO TestMapReduceActionExecutor:539 - Waiting up to [235,955] msec 19:03:16,083 INFO TestMapReduceActionExecutor:539 - Waiting up to [234,953] msec 19:03:17,085 INFO TestMapReduceActionExecutor:539 - Waiting up to [233,951] msec 19:03:18,191 INFO TestMapReduceActionExecutor:539 - Waiting up to [232,845] msec 19:03:19,195 INFO TestMapReduceActionExecutor:539 - Waiting up to [231,841] msec 19:03:20,234 INFO TestMapReduceActionExecutor:539 - Waiting up to [230,803] msec 19:03:20,506 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 19:03:20,511 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 19:03:20,513 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-03-07T03:02Z 19:03:20,570 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-03-07T03:02Z 19:03:20,570 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking 19:03:20,681 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 19:03:20,714 INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2014-03-07T03:08Z, Num jobs to materialize = 0 19:03:20,724 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null] 19:03:20,725 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days. 19:03:20,725 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinators, [0] bundles 19:03:20,729 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 19:03:20,737 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery 19:03:21,241 INFO TestMapReduceActionExecutor:539 - Waiting up to [229,796] msec 19:03:22,245 INFO TestMapReduceActionExecutor:539 - Waiting up to [228,791] msec 19:03:23,247 INFO TestMapReduceActionExecutor:539 - Waiting up to [227,789] msec 19:03:24,250 INFO TestMapReduceActionExecutor:539 - Waiting up to [226,787] msec 19:03:25,252 INFO TestMapReduceActionExecutor:539 - Waiting up to [225,784] msec 19:03:26,280 INFO TestMapReduceActionExecutor:539 - Waiting up to [224,766] msec 19:03:27,324 INFO TestMapReduceActionExecutor:539 - Waiting up to [223,714] msec 19:03:28,331 INFO TestMapReduceActionExecutor:539 - Waiting up to [222,705] msec 19:03:29,364 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:29,379 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:29,385 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:03:29,494 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0009] new ID [job_1394161202967_0010] 19:03:29,495 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] checking action, external ID [job_1394161202967_0009] status [RUNNING] 19:03:29,513 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:29,529 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:29,565 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:03:29,567 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,997] msec 19:03:30,569 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,995] msec 19:03:31,571 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,993] msec 19:03:32,573 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,991] msec 19:03:33,685 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,879] msec 19:03:34,688 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,876] msec 19:03:35,736 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,828] msec 19:03:36,789 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,776] msec 19:03:37,797 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,767] msec 19:03:38,800 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,764] msec 19:03:39,803 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,761] msec 19:03:40,840 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,724] msec 19:03:41,845 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,719] msec 19:03:42,849 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,715] msec 19:03:43,853 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,712] msec 19:03:44,856 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,709] msec 19:03:45,858 INFO TestMapReduceActionExecutor:539 - Waiting up to [103,706] msec 19:03:46,860 INFO TestMapReduceActionExecutor:539 - Waiting up to [102,704] msec 19:03:47,864 INFO TestMapReduceActionExecutor:539 - Waiting up to [101,700] msec 19:03:48,888 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:48,901 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:48,911 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:03:49,024 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0009] new ID [job_1394161202967_0010] 19:03:49,030 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] action completed, external ID [job_1394161202967_0009] 19:03:49,065 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:49,079 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:49,083 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:03:49,112 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testEndWithoutConfiguration] 19:03:49,137 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:03:49,137 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:03:49,137 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:03:49,137 INFO XLogService:539 - Log4j reload interval [disabled] 19:03:49,137 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:03:49,138 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testEndWithoutConfiguration] 19:03:49,138 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testEndWithoutConfiguration/conf] 19:03:49,138 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:03:49,141 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:03:49,142 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:03:49,142 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:03:49,142 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan1998242984074985298.dir] 19:03:49,142 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:03:49,142 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:03:49,142 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:03:49,142 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:03:49,143 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:03:49,337 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:03:49,338 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:03:49,338 INFO Services:539 - Initialized 19:03:49,338 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:03:49,338 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:03:49,348 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:03:49,349 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:03:49,349 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/be638aa0-afac-418b-b432-10208bc03993] 19:03:49,389 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:03:49,389 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:03:49,389 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:03:49,389 INFO XLogService:539 - Log4j reload interval [disabled] 19:03:49,389 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:03:49,389 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testEndWithoutConfiguration] 19:03:49,389 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testEndWithoutConfiguration/conf] 19:03:49,390 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:03:49,393 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:03:49,393 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:03:49,394 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:03:49,394 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:03:49,394 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:03:49,394 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan8924111544229997148.dir] 19:03:49,395 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:03:49,395 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:03:49,395 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:03:49,395 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:03:49,395 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:03:49,395 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:03:49,395 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:03:49,396 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:03:49,397 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:03:49,398 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:03:49,398 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:03:49,398 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:03:49,398 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:03:49,398 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:03:49,399 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:03:49,399 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:03:49,399 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:03:49,399 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:03:49,399 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:03:49,399 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:03:49,399 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:03:49,400 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:03:49,400 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:03:49,400 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:03:49,401 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:03:49,401 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:03:49,406 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:03:49,406 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:03:49,406 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:03:49,407 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:03:49,407 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:03:49,424 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:03:49,424 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:03:49,646 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] All entities initialized 19:03:49,647 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190310205-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:03:49,651 INFO Services:539 - Initialized 19:03:49,651 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:03:49,651 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:03:49,723 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:49,726 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] Setting LibFilesArchives 19:03:49,741 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:49,743 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] Get credential 'null' properties for action : 0000000-140306190349401-oozie-rkan-W@mr-action 19:03:49,746 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] credentials is null for the action 19:03:49,764 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:49,782 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:49,819 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] Creating Job Client for action 0000000-140306190349401-oozie-rkan-W@mr-action 19:03:49,833 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:49,839 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] Submitting the job through Job Client for action 0000000-140306190349401-oozie-rkan-W@mr-action 19:03:49,853 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:49,858 WARN JobSubmitter:150 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19:03:49,862 WARN JobSubmitter:259 - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 19:03:49,875 INFO FileInputFormat:253 - Total input paths to process : 1 19:03:49,900 INFO JobSubmitter:396 - number of splits:1 19:03:49,905 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:49,919 INFO JobSubmitter:479 - Submitting tokens for job: job_1394161202967_0011 19:03:49,919 INFO JobSubmitter:481 - Kind: RM_DELEGATION_TOKEN, Service: 172.16.1.64:52700, Ident: (owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161429840, maxDate=1394766229840, sequenceNumber=33, masterKeyId=2) 19:03:49,923 INFO YARNRunner:369 - Job jar is not present. Not adding any jar to the list of resources. 19:03:49,946 INFO YarnClientImpl:167 - Submitted application application_1394161202967_0011 19:03:49,949 INFO Job:1299 - The url to track the job: http://rkanter-mbp.local:52701/proxy/application_1394161202967_0011/ 19:03:49,949 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] After submission get the launcherId job_1394161202967_0011 19:03:49,969 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:03:49,984 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:03:50,003 INFO TestMapReduceActionExecutor:539 - Waiting up to [240,000] msec 19:03:50,004 INFO TestMapReduceActionExecutor:539 - Waiting up to [239,999] msec 19:03:51,126 INFO TestMapReduceActionExecutor:539 - Waiting up to [238,878] msec 19:03:52,175 INFO TestMapReduceActionExecutor:539 - Waiting up to [237,828] msec 19:03:53,179 INFO TestMapReduceActionExecutor:539 - Waiting up to [236,824] msec 19:03:54,182 INFO TestMapReduceActionExecutor:539 - Waiting up to [235,821] msec 19:03:55,184 INFO TestMapReduceActionExecutor:539 - Waiting up to [234,819] msec 19:03:56,319 INFO TestMapReduceActionExecutor:539 - Waiting up to [233,684] msec 19:03:57,322 INFO TestMapReduceActionExecutor:539 - Waiting up to [232,681] msec 19:03:58,348 INFO TestMapReduceActionExecutor:539 - Waiting up to [231,655] msec 19:03:59,350 INFO TestMapReduceActionExecutor:539 - Waiting up to [230,653] msec 19:03:59,656 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 19:03:59,656 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 19:03:59,660 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-03-07T03:03Z 19:03:59,679 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-03-07T03:03Z 19:03:59,679 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking 19:03:59,685 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 19:03:59,689 INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2014-03-07T03:08Z, Num jobs to materialize = 0 19:03:59,696 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null] 19:03:59,696 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days. 19:03:59,696 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinators, [0] bundles 19:03:59,703 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 19:03:59,712 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery 19:04:00,353 INFO TestMapReduceActionExecutor:539 - Waiting up to [229,650] msec 19:04:01,355 INFO TestMapReduceActionExecutor:539 - Waiting up to [228,648] msec 19:04:02,357 INFO TestMapReduceActionExecutor:539 - Waiting up to [227,646] msec 19:04:03,360 INFO TestMapReduceActionExecutor:539 - Waiting up to [226,643] msec 19:04:04,416 INFO TestMapReduceActionExecutor:539 - Waiting up to [225,587] msec 19:04:05,497 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:05,511 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:04:05,516 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:04:05,617 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0011] new ID [job_1394161202967_0012] 19:04:05,618 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] checking action, external ID [job_1394161202967_0011] status [RUNNING] 19:04:05,636 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:05,651 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:04:05,673 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:04:05,675 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,998] msec 19:04:06,677 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,996] msec 19:04:07,680 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,994] msec 19:04:08,682 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,992] msec 19:04:09,797 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,876] msec 19:04:10,799 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,874] msec 19:04:11,802 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,871] msec 19:04:12,806 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,867] msec 19:04:13,814 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,859] msec 19:04:14,816 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,857] msec 19:04:15,819 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,854] msec 19:04:16,821 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,852] msec 19:04:17,825 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,848] msec 19:04:18,829 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,844] msec 19:04:19,834 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,839] msec 19:04:20,837 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,836] msec 19:04:21,841 INFO TestMapReduceActionExecutor:539 - Waiting up to [103,832] msec 19:04:22,846 INFO TestMapReduceActionExecutor:539 - Waiting up to [102,827] msec 19:04:23,850 INFO TestMapReduceActionExecutor:539 - Waiting up to [101,823] msec 19:04:24,857 INFO TestMapReduceActionExecutor:539 - Waiting up to [100,816] msec 19:04:25,860 INFO TestMapReduceActionExecutor:539 - Waiting up to [99,813] msec 19:04:26,864 INFO TestMapReduceActionExecutor:539 - Waiting up to [98,809] msec 19:04:27,867 INFO TestMapReduceActionExecutor:539 - Waiting up to [97,806] msec 19:04:28,878 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetMapredJobName] 19:04:28,905 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:04:28,905 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:04:28,905 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:04:28,906 INFO XLogService:539 - Log4j reload interval [disabled] 19:04:28,906 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:04:28,906 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetMapredJobName] 19:04:28,906 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetMapredJobName/conf] 19:04:28,906 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:04:28,910 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:04:28,910 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:04:28,910 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:04:28,911 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan3383393444547661110.dir] 19:04:28,911 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:04:28,911 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:04:28,911 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:04:28,911 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:04:28,911 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:04:29,106 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:04:29,106 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:04:29,106 INFO Services:539 - Initialized 19:04:29,106 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:04:29,106 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:04:29,116 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:04:29,116 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:04:29,117 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:04:29,117 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:04:29,117 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:04:29,117 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:04:29,117 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:04:29,117 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:04:29,117 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:04:29,117 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/07cbd33e-616b-4996-8bfc-23048d29529d] 19:04:29,160 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:04:29,161 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:04:29,161 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:04:29,161 INFO XLogService:539 - Log4j reload interval [disabled] 19:04:29,161 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:04:29,161 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetMapredJobName] 19:04:29,162 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testSetMapredJobName/conf] 19:04:29,162 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:04:29,166 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:04:29,166 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:04:29,166 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:04:29,167 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:04:29,167 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:04:29,167 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan8419843008909515374.dir] 19:04:29,168 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:04:29,168 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:04:29,168 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:04:29,169 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:04:29,170 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:04:29,172 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:04:29,172 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:04:29,172 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:04:29,172 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:04:29,173 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:04:29,173 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:04:29,173 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:04:29,173 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:04:29,173 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:04:29,174 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:04:29,176 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:04:29,176 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:04:29,180 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:04:29,180 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:04:29,180 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:04:29,180 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:04:29,181 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:04:29,197 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:04:29,197 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:04:29,410 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] All entities initialized 19:04:29,411 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190349401-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:04:29,414 INFO Services:539 - Initialized 19:04:29,414 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:04:29,414 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:04:29,479 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:29,480 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] Setting LibFilesArchives 19:04:29,497 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:29,500 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] Get credential 'null' properties for action : 0000000-140306190429176-oozie-rkan-W@mr-action 19:04:29,504 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] credentials is null for the action 19:04:29,522 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:29,537 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:29,576 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] Creating Job Client for action 0000000-140306190429176-oozie-rkan-W@mr-action 19:04:29,589 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:04:29,594 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] Submitting the job through Job Client for action 0000000-140306190429176-oozie-rkan-W@mr-action 19:04:29,608 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:04:29,616 WARN JobSubmitter:150 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 19:04:29,620 WARN JobSubmitter:259 - No job jar file set. User classes may not be found. See Job or Job#setJar(String). 19:04:29,633 INFO FileInputFormat:253 - Total input paths to process : 1 19:04:29,682 INFO JobSubmitter:396 - number of splits:1 19:04:29,688 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:29,713 INFO JobSubmitter:479 - Submitting tokens for job: job_1394161202967_0013 19:04:29,713 INFO JobSubmitter:481 - Kind: RM_DELEGATION_TOKEN, Service: 172.16.1.64:52700, Ident: (owner=test, renewer=oozie mr token, realUser=rkanter, issueDate=1394161469594, maxDate=1394766269594, sequenceNumber=38, masterKeyId=2) 19:04:29,717 INFO YARNRunner:369 - Job jar is not present. Not adding any jar to the list of resources. 19:04:29,733 INFO YarnClientImpl:167 - Submitted application application_1394161202967_0013 19:04:29,736 INFO Job:1299 - The url to track the job: http://rkanter-mbp.local:52701/proxy/application_1394161202967_0013/ 19:04:29,736 DEBUG MapReduceActionExecutor:545 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] After submission get the launcherId job_1394161202967_0013 19:04:29,752 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:29,773 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:04:29,792 INFO TestMapReduceActionExecutor:539 - Waiting up to [240,000] msec 19:04:29,793 INFO TestMapReduceActionExecutor:539 - Waiting up to [239,999] msec 19:04:30,851 INFO TestMapReduceActionExecutor:539 - Waiting up to [238,968] msec 19:04:31,872 INFO TestMapReduceActionExecutor:539 - Waiting up to [237,921] msec 19:04:32,875 INFO TestMapReduceActionExecutor:539 - Waiting up to [236,917] msec 19:04:33,878 INFO TestMapReduceActionExecutor:539 - Waiting up to [235,914] msec 19:04:34,880 INFO TestMapReduceActionExecutor:539 - Waiting up to [234,912] msec 19:04:36,021 INFO TestMapReduceActionExecutor:539 - Waiting up to [233,771] msec 19:04:37,025 INFO TestMapReduceActionExecutor:539 - Waiting up to [232,767] msec 19:04:38,085 INFO TestMapReduceActionExecutor:539 - Waiting up to [231,708] msec 19:04:39,118 INFO TestMapReduceActionExecutor:539 - Waiting up to [230,674] msec 19:04:39,424 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.StatusTransitService] 19:04:39,424 INFO PauseTransitService:539 - USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTransitService] 19:04:39,445 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running coordinator status service from last instance time = 2014-03-07T03:03Z 19:04:39,513 DEBUG ActionCheckerService$ActionCheckRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [] for potential checking 19:04:39,513 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Running bundle status service from last instance time = 2014-03-07T03:03Z 19:04:39,519 INFO StatusTransitService$StatusTransitRunnable:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.StatusTransitService] 19:04:39,522 INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2014-03-07T03:09Z, Num jobs to materialize = 0 19:04:39,530 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Execute command [purge] key [null] 19:04:39,531 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days. 19:04:39,531 DEBUG PurgeXCommand:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinators, [0] bundles 19:04:39,539 INFO PauseTransitService:539 - USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTransitService] 19:04:39,547 DEBUG RecoveryService$RecoveryRunnable:545 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] QUEUING [ WF_ACTIONS 0, COORD_ACTIONS : 0, COORD_READY_JOBS : 0, BUNDLE_ACTIONS : 0] for potential recovery 19:04:40,123 INFO TestMapReduceActionExecutor:539 - Waiting up to [229,669] msec 19:04:41,126 INFO TestMapReduceActionExecutor:539 - Waiting up to [228,667] msec 19:04:42,128 INFO TestMapReduceActionExecutor:539 - Waiting up to [227,665] msec 19:04:43,130 INFO TestMapReduceActionExecutor:539 - Waiting up to [226,662] msec 19:04:44,170 INFO TestMapReduceActionExecutor:539 - Waiting up to [225,623] msec 19:04:45,182 INFO TestMapReduceActionExecutor:539 - Waiting up to [224,610] msec 19:04:46,234 INFO TestMapReduceActionExecutor:539 - Waiting up to [223,559] msec 19:04:47,250 INFO TestMapReduceActionExecutor:539 - Waiting up to [222,543] msec Launcher job name: MapReduceLauncherTest 19:04:48,287 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:48,303 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:04:48,309 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:04:48,402 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0013] new ID [job_1394161202967_0014] 19:04:48,404 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] checking action, external ID [job_1394161202967_0013] status [RUNNING] 19:04:48,429 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:04:48,451 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:04:48,486 INFO TestMapReduceActionExecutor:539 - Waiting up to [120,000] msec 19:04:48,487 INFO TestMapReduceActionExecutor:539 - Waiting up to [119,998] msec 19:04:49,489 INFO TestMapReduceActionExecutor:539 - Waiting up to [118,996] msec 19:04:50,492 INFO TestMapReduceActionExecutor:539 - Waiting up to [117,993] msec 19:04:51,597 INFO TestMapReduceActionExecutor:539 - Waiting up to [116,888] msec 19:04:52,600 INFO TestMapReduceActionExecutor:539 - Waiting up to [115,885] msec 19:04:53,647 INFO TestMapReduceActionExecutor:539 - Waiting up to [114,838] msec 19:04:54,689 INFO TestMapReduceActionExecutor:539 - Waiting up to [113,808] msec 19:04:55,702 INFO TestMapReduceActionExecutor:539 - Waiting up to [112,783] msec 19:04:56,705 INFO TestMapReduceActionExecutor:539 - Waiting up to [111,780] msec 19:04:57,708 INFO TestMapReduceActionExecutor:539 - Waiting up to [110,777] msec 19:04:58,713 INFO TestMapReduceActionExecutor:539 - Waiting up to [109,772] msec 19:04:59,717 INFO TestMapReduceActionExecutor:539 - Waiting up to [108,768] msec 19:05:00,721 INFO TestMapReduceActionExecutor:539 - Waiting up to [107,764] msec 19:05:01,724 INFO TestMapReduceActionExecutor:539 - Waiting up to [106,761] msec 19:05:02,727 INFO TestMapReduceActionExecutor:539 - Waiting up to [105,758] msec 19:05:03,730 INFO TestMapReduceActionExecutor:539 - Waiting up to [104,756] msec 19:05:04,772 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:05:04,785 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:05:04,793 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server 19:05:04,886 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] External ID swap, old ID [job_1394161202967_0013] new ID [job_1394161202967_0014] 19:05:04,894 INFO MapReduceActionExecutor:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] action completed, external ID [job_1394161202967_0013] 19:05:04,922 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:05:04,934 INFO RMProxy:92 - Connecting to ResourceManager at /172.16.1.64:52700 19:05:04,940 INFO ClientServiceDelegate:273 - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server Mapred job name: MapReduceTest 19:05:04,966 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testDefaultShareLibName] 19:05:04,995 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:05:04,995 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:05:04,996 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:05:04,996 INFO XLogService:539 - Log4j reload interval [disabled] 19:05:04,996 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:05:04,996 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testDefaultShareLibName] 19:05:04,996 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testDefaultShareLibName/conf] 19:05:04,997 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:05:05,001 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:05:05,001 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:05:05,001 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:05:05,002 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan4940018602110443290.dir] 19:05:05,002 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:05:05,002 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:05:05,002 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:05:05,002 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:05:05,002 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:05:05,174 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:05:05,174 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:05:05,174 INFO Services:539 - Initialized 19:05:05,175 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:05:05,175 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:05:05,184 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:05:05,184 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:05:05,184 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:05:05,184 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:05:05,184 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:05:05,184 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:05:05,185 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:05:05,185 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:05:05,185 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:05:05,185 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/f645f63b-2b44-4c06-a85f-83d813bc3ee4] 19:05:05,224 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:05:05,225 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:05:05,225 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:05:05,225 INFO XLogService:539 - Log4j reload interval [disabled] 19:05:05,225 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:05:05,225 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testDefaultShareLibName] 19:05:05,225 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testDefaultShareLibName/conf] 19:05:05,225 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:05:05,229 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:05:05,229 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:05:05,229 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:05:05,229 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:05:05,230 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:05:05,230 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan8577732101477071689.dir] 19:05:05,230 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:05:05,230 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:05:05,230 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:05:05,231 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:05:05,232 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:05:05,233 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:05:05,234 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:05:05,234 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:05:05,235 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:05:05,235 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:05:05,235 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:05:05,235 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:05:05,235 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:05:05,236 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:05:05,236 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:05:05,236 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:05:05,239 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:05:05,239 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:05:05,245 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:05:05,245 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:05:05,245 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:05:05,246 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:05:05,246 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:05:05,262 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:05:05,262 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:05:05,478 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] All entities initialized 19:05:05,478 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:05:05,481 INFO Services:539 - Initialized 19:05:05,482 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:05:05,482 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:05:05,486 INFO Services:539 - Shutdown Setting testcase work dir[/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testCommaSeparatedFilesAndArchives] 19:05:05,514 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:05:05,515 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:05:05,515 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:05:05,515 INFO XLogService:539 - Log4j reload interval [disabled] 19:05:05,515 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:05:05,515 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testCommaSeparatedFilesAndArchives] 19:05:05,515 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testCommaSeparatedFilesAndArchives/conf] 19:05:05,515 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:05:05,519 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:05:05,520 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:05:05,520 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:05:05,520 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan3261405668562068451.dir] 19:05:05,520 INFO ConfigurationService:539 - USER[test] GROUP[-] Programmatic configuration change, property[oozie.services]=[org.apache.oozie.service.JPAService,org.apache.oozie.service.StoreService] 19:05:05,521 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:05:05,521 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:05:05,521 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:05:05,521 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:05:05,709 INFO JPAService:539 - USER[test] GROUP[-] All entities initialized 19:05:05,710 INFO JPAService:539 - USER[test] GROUP[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:05:05,710 INFO Services:539 - Initialized 19:05:05,710 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:05:05,710 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:05:05,720 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_JOBS removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in WF_ACTIONS removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_JOBS removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in COORD_ACTIONS removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_JOBS removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in BUNDLE_ACTIONS removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_EVENTS removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_REGISTRATION removed from DB! 19:05:05,721 INFO TestMapReduceActionExecutor:539 - 0 entries in SLA_SUMMARY removed from DB! 19:05:05,721 INFO Services:539 - Shutdown Setting FS testcase work dir[hdfs://localhost:52681/user/test/1fb37c34-ef81-4e6b-8ccf-11716be0c735] 19:05:05,762 INFO XLogService:539 - ******************************************************************************* STARTUP MSG: Oozie BUILD_VERSION [4.0.0-cdh5.1.0-SNAPSHOT] compiled by [rkanter] on [${build.time}] STARTUP MSG: revision [${vc.revision}]@[${vc.url}] ******************************************************************************* 19:05:05,762 INFO XLogService:539 - Log4j configuration file [oozie-log4j.properties] 19:05:05,762 INFO XLogService:539 - Log4j configuration file loaded from [CLASSPATH] 19:05:05,762 INFO XLogService:539 - Log4j reload interval [disabled] 19:05:05,762 WARN XLogService:542 - Oozie WS log will be disabled, missing property 'log4j.appender.oozie.File' for 'oozie' appender 19:05:05,762 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie home dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testCommaSeparatedFilesAndArchives] 19:05:05,763 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf dir [/Users/rkanter/dev/CDH-oozie3/sharelib/streaming/target/test-data/oozietests/org.apache.oozie.action.hadoop.TestMapReduceActionExecutor/testCommaSeparatedFilesAndArchives/conf] 19:05:05,763 INFO ConfigurationService:539 - USER[test] GROUP[-] Oozie conf file [oozie-site.xml] 19:05:05,767 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.credentials.credentialclasses]=[cred=org.apache.oozie.action.hadoop.CredentialForTest] 19:05:05,767 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.ActionService.executor.classes]=[org.apache.oozie.action.hadoop.MapReduceActionExecutor] 19:05:05,767 INFO ConfigurationService:539 - USER[test] GROUP[-] Configuration change via System Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*] 19:05:05,767 WARN Services:542 - USER[test] GROUP[-] System ID [oozie-rkan] exceeds maximum length [10], trimming 19:05:05,768 INFO Services:539 - USER[test] GROUP[-] Exiting null Entering NORMAL 19:05:05,768 INFO Services:539 - USER[test] GROUP[-] Initialized runtime directory [/var/folders/pr/8_m9c5zd4js7tlwpwpfl2ftw0000gp/T/oozie-rkan9221138528728741153.dir] 19:05:05,768 DEBUG Services:545 - Services list obtained from property 'oozie.services' 19:05:05,768 DEBUG Services:545 - Services list obtained from property 'oozie.services.ext' 19:05:05,768 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchedulerService] implementation [class org.apache.oozie.service.SchedulerService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.InstrumentationService] implementation [class org.apache.oozie.service.InstrumentationService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallableQueueService] implementation [class org.apache.oozie.service.CallableQueueService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UUIDService] implementation [class org.apache.oozie.service.UUIDService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ELService] implementation [class org.apache.oozie.service.ELService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.AuthorizationService] implementation [class org.apache.oozie.service.AuthorizationService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.UserGroupInformationService] implementation [class org.apache.oozie.service.UserGroupInformationService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.HadoopAccessorService] implementation [class org.apache.oozie.service.HadoopAccessorService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.URIHandlerService] implementation [class org.apache.oozie.service.URIHandlerService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.MemoryLocksService] implementation [class org.apache.oozie.service.MemoryLocksService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagXLogInfoService] implementation [class org.apache.oozie.service.DagXLogInfoService] 19:05:05,769 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SchemaService] implementation [class org.apache.oozie.service.SchemaService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowAppService] implementation [class org.apache.oozie.service.LiteWorkflowAppService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JPAService] implementation [class org.apache.oozie.service.JPAService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StoreService] implementation [class org.apache.oozie.service.StoreService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorStoreService] implementation [class org.apache.oozie.service.CoordinatorStoreService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.SLAStoreService] implementation [class org.apache.oozie.service.SLAStoreService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.WorkflowStoreService] implementation [class org.apache.oozie.service.DBLiteWorkflowStoreService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CallbackService] implementation [class org.apache.oozie.service.CallbackService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionService] implementation [class org.apache.oozie.service.ActionService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ActionCheckerService] implementation [class org.apache.oozie.service.ActionCheckerService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.RecoveryService] implementation [class org.apache.oozie.service.RecoveryService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PurgeService] implementation [class org.apache.oozie.service.PurgeService] 19:05:05,770 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordinatorEngineService] implementation [class org.apache.oozie.service.CoordinatorEngineService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.BundleEngineService] implementation [class org.apache.oozie.service.BundleEngineService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.DagEngineService] implementation [class org.apache.oozie.service.DagEngineService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.CoordMaterializeTriggerService] implementation [class org.apache.oozie.service.CoordMaterializeTriggerService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.StatusTransitService] implementation [class org.apache.oozie.service.StatusTransitService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.PauseTransitService] implementation [class org.apache.oozie.service.PauseTransitService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.GroupsService] implementation [class org.apache.oozie.service.GroupsService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.ProxyUserService] implementation [class org.apache.oozie.service.ProxyUserService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.XLogStreamingService] implementation [class org.apache.oozie.service.XLogStreamingService] 19:05:05,771 DEBUG Services:545 - Loading service [class org.apache.oozie.service.JobsConcurrencyService] implementation [class org.apache.oozie.service.JobsConcurrencyService] 19:05:05,772 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [kill] 19:05:05,772 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [resume] 19:05:05,772 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [suspend] 19:05:05,773 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_kill] 19:05:05,773 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_resume] 19:05:05,773 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [bundle_suspend] 19:05:05,773 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_kill] 19:05:05,773 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_change] 19:05:05,773 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_resume] 19:05:05,773 DEBUG CallableQueueService:545 - USER[test] GROUP[-] Adding interrupt type [coord_suspend] 19:05:05,775 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.constants.job-submit] not found, using default [] 19:05:05,775 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.ELService.ext.functions.job-submit] not found, using default [] 19:05:05,779 WARN ConfigurationService:542 - USER[test] GROUP[-] Configuration property [oozie.service.AuthorizationService.security.enabled] not found, using default [null] 19:05:05,779 WARN AuthorizationService:542 - USER[test] GROUP[-] Oozie running with authorization disabled 19:05:05,779 INFO HadoopAccessorService:539 - USER[test] GROUP[-] JOB_TRACKER_WHITELIST :[ ], Total entries :0 19:05:05,779 INFO HadoopAccessorService:539 - USER[test] GROUP[-] NAME_NODE_WHITELIST :[ ], Total entries :0 19:05:05,779 INFO HadoopAccessorService:539 - USER[test] GROUP[-] Oozie Kerberos Authentication [disabled] 19:05:05,797 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded urihandlers [org.apache.oozie.dependency.FSURIHandler] 19:05:05,797 INFO URIHandlerService:539 - USER[-] GROUP[-] Loaded default urihandler org.apache.oozie.dependency.FSURIHandler 19:05:06,030 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] All entities initialized 19:05:06,031 INFO JPAService:539 - USER[test] GROUP[-] TOKEN[] APP[testApp] JOB[0000000-140306190429176-oozie-rkan-W] ACTION[-] JPA configuration: DriverClassName=org.hsqldb.jdbcDriver,Url=jdbc:hsqldb:mem:oozie-db;create=true,Username=sa,Password=***,MaxActive=10,TestOnBorrow=false,TestOnReturn=false,TestWhileIdle=false 19:05:06,033 INFO Services:539 - Initialized 19:05:06,033 INFO Services:539 - Running with JARs for Hadoop version [2.3.0-cdh5.1.0-SNAPSHOT] 19:05:06,033 INFO Services:539 - Oozie System ID [oozie-rkan] started! 19:05:06,144 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:05:06,155 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:05:06,167 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:05:06,187 INFO deprecation:1009 - fs.default.name is deprecated. Instead, use fs.defaultFS 19:05:06,200 INFO Services:539 - Shutdown