2016-10-26 16:27:27,496 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 37761: responding to null from 192.168.200.219:55506 Call#-33 Retry#-1 Wrote 166 bytes. 2016-10-26 16:27:27,497 DEBUG org.apache.hadoop.ipc.Server: got #-33 2016-10-26 16:27:27,497 DEBUG org.apache.hadoop.ipc.Server: Have read input token of size 346 for processing by saslServer.evaluateResponse() 2016-10-26 16:27:27,498 DEBUG org.apache.hadoop.yarn.server.security.BaseNMTokenSecretManager: creating password for appattempt_1475850791417_0147_000001 for user shfs3453 to run on NM datanode03.bigdata.fr:37761 2016-10-26 16:27:27,498 DEBUG org.apache.hadoop.yarn.security.NMTokenIdentifier: Writing NMTokenIdentifier to RPC layer: appAttemptId { application_id { id: 147 cluster_timestamp: 1475850791417 } attemptId: 1 } nodeId { host: "datanode03.bigdata.fr" port: 37761 } appSubmitter: "shfs3453" keyId: -646854807 2016-10-26 16:27:27,498 DEBUG org.apache.hadoop.yarn.server.nodemanager.security.NMTokenSecretManagerInNM: NMToken password retrieved successfully!! 2016-10-26 16:27:27,498 DEBUG org.apache.hadoop.security.SaslRpcServer: SASL server DIGEST-MD5 callback: setting password for client: appattempt_1475850791417_0147_000001 (auth:SIMPLE) 2016-10-26 16:27:27,499 DEBUG org.apache.hadoop.security.SaslRpcServer: SASL server DIGEST-MD5 callback: setting canonicalized client ID: appattempt_1475850791417_0147_000001 2016-10-26 16:27:27,499 DEBUG org.apache.hadoop.ipc.Server: Will send SUCCESS token of size 40 from saslServer. 2016-10-26 16:27:27,499 DEBUG org.apache.hadoop.ipc.Server: SASL server context established. Negotiated QoP is auth 2016-10-26 16:27:27,499 DEBUG org.apache.hadoop.ipc.Server: SASL server successfully authenticated client: appattempt_1475850791417_0147_000001 (auth:SIMPLE) 2016-10-26 16:27:27,499 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1475850791417_0147_000001 (auth:SIMPLE) 2016-10-26 16:27:27,499 DEBUG org.apache.hadoop.ipc.Server: Sending sasl message state: SUCCESS token: "rspauth=1da0499993ae7e093df1293cfc6d6112" 2016-10-26 16:27:27,499 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 37761: responding to null from 192.168.200.219:55506 Call#-33 Retry#-1 2016-10-26 16:27:27,499 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 37761: responding to null from 192.168.200.219:55506 Call#-33 Retry#-1 Wrote 64 bytes. 2016-10-26 16:27:27,502 DEBUG org.apache.hadoop.ipc.Server: got #-3 2016-10-26 16:27:27,502 INFO SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for appattempt_1475850791417_0147_000001 (auth:TOKEN) for protocol=interface org.apache.hadoop.yarn.api.ContainerManagementProtocolPB 2016-10-26 16:27:27,502 DEBUG org.apache.hadoop.ipc.Server: Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.api.ContainerManagementProtocolPB" 2016-10-26 16:27:27,502 DEBUG org.apache.hadoop.ipc.Server: got #2165 2016-10-26 16:27:27,502 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 37761: org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 192.168.200.219:55506 Call#2165 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:27,503 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:appattempt_1475850791417_0147_000001 (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:27,505 DEBUG org.apache.hadoop.yarn.server.security.BaseContainerTokenSecretManager: Retrieving password for container_1475850791417_0147_01_000001 for user container_1475850791417_0147_01_000001 (auth:SIMPLE) to be run on NM datanode03.bigdata.fr:37761 2016-10-26 16:27:27,505 DEBUG org.apache.hadoop.yarn.security.ContainerTokenIdentifier: Writing ContainerTokenIdentifier to RPC layer: containerId { app_attempt_id { application_id { id: 147 cluster_timestamp: 1475850791417 } attemptId: 1 } id: 1 } nmHostAddr: "datanode03.bigdata.fr:37761" appSubmitter: "shfs3453" resource { memory: 4096 virtual_cores: 1 } expiryTimeStamp: 1477492647489 masterKeyId: -1972402786 rmIdentifier: 1475850791417 priority { priority: 0 } creationTime: 1477492047489 2016-10-26 16:27:27,506 DEBUG org.apache.hadoop.yarn.server.nodemanager.security.NMTokenSecretManagerInNM: NMToken key updated for application attempt : appattempt_1475850791417_0147_000001 2016-10-26 16:27:27,506 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1475850791417_0147_01_000001 by user shfs3453 2016-10-26 16:27:27,506 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: ha-hdfs:sandbox = Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:sandbox, Ident: (HDFS_DELEGATION_TOKEN token 12521 for shfs3453) 2016-10-26 16:27:27,506 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: = Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 147 cluster_timestamp: 1475850791417 } attemptId: 1 } keyId: -998955696) 2016-10-26 16:27:27,506 WARN org.apache.hadoop.security.token.Token: Cannot find class for token kind HBASE_AUTH_TOKEN 2016-10-26 16:27:27,507 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: e7142baa-7bed-4c65-8ed5-6594470a05bd = Kind: HBASE_AUTH_TOKEN, Service: e7142baa-7bed-4c65-8ed5-6594470a05bd, Ident: 00 00 00 2e 08 00 12 17 73 68 66 73 33 34 35 33 40 53 41 4e 44 42 4f 58 2e 48 41 44 4f 4f 50 18 c6 04 20 8b e4 8d 8b 80 2b 28 8b ec bf ab 82 2b 30 59 2016-10-26 16:27:27,507 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Creating a new application reference for app application_1475850791417_0147 2016-10-26 16:27:27,507 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationInitEvent.EventType: INIT_APPLICATION 2016-10-26 16:27:27,507 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=shfs3453 IP=192.168.200.219 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1475850791417_0147 CONTAINERID=container_1475850791417_0147_01_000001 2016-10-26 16:27:27,507 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Processing application_1475850791417_0147 of type INIT_APPLICATION 2016-10-26 16:27:27,507 DEBUG org.apache.hadoop.ipc.Server: Served: startContainers queueTime= 2 procesingTime= 3 2016-10-26 16:27:27,507 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1475850791417_0147 transitioned from NEW to INITING 2016-10-26 16:27:27,507 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerInitEvent.EventType: INIT_CONTAINER 2016-10-26 16:27:27,508 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Processing application_1475850791417_0147 of type INIT_CONTAINER 2016-10-26 16:27:27,508 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 37761: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 192.168.200.219:55506 Call#2165 Retry#0 2016-10-26 16:27:27,508 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Adding container_1475850791417_0147_01_000001 to application application_1475850791417_0147 2016-10-26 16:27:27,508 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerAppStartedEvent.EventType: APPLICATION_STARTED 2016-10-26 16:27:27,508 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 37761: responding to org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers from 192.168.200.219:55506 Call#2165 Retry#0 Wrote 80 bytes. 2016-10-26 16:27:27,508 DEBUG org.apache.hadoop.ipc.Client: The ping interval is 60000 ms. 2016-10-26 16:27:27,508 DEBUG org.apache.hadoop.ipc.Client: Connecting to namenode01.bigdata.fr/192.168.200.23:8020 2016-10-26 16:27:27,509 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:nm/datanode03.bigdata.fr@SANDBOX.HADOOP (auth:KERBEROS) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:724) 2016-10-26 16:27:27,510 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: NEGOTIATE 2016-10-26 16:27:27,510 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: NEGOTIATE auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" challenge: "realm=\"default\",nonce=\"NLxrhI/9pCLZMtc8tsaw7Ecf/7zLesrlndZHlk/z\",qop=\"auth\",charset=utf-8,algorithm=md5-sess" } auths { method: "KERBEROS" mechanism: "GSSAPI" protocol: "nn" serverId: "namenode01.bigdata.fr" } 2016-10-26 16:27:27,510 DEBUG org.apache.hadoop.security.SaslRpcClient: Get token info proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB info:@org.apache.hadoop.security.token.TokenInfo(value=class org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSelector) 2016-10-26 16:27:27,511 DEBUG org.apache.hadoop.security.SaslRpcClient: Get kerberos info proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB info:@org.apache.hadoop.security.KerberosInfo(clientPrincipal=, serverPrincipal=dfs.namenode.kerberos.principal) 2016-10-26 16:27:27,512 DEBUG org.apache.hadoop.security.SaslRpcClient: RPC Server's Kerberos principal name for protocol=org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB is nn/namenode01.bigdata.fr@SANDBOX.HADOOP 2016-10-26 16:27:27,512 DEBUG org.apache.hadoop.security.SaslRpcClient: Creating SASL GSSAPI(KERBEROS) client to authenticate to service at namenode01.bigdata.fr 2016-10-26 16:27:27,512 DEBUG org.apache.hadoop.security.SaslRpcClient: Use KERBEROS authentication for protocol ClientNamenodeProtocolPB 2016-10-26 16:27:27,515 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: INITIATE token: "`\202\002\263\006\t*\206H\206\367\022\001\002\002\001\000n\202\002\2420\202\002\236\240\003\002\001\005\241\003\002\001\016\242\a\003\005\000 \000\000\000\243\202\001\222a\202\001\2160\202\001\212\240\003\002\001\005\241\020\033\016SANDBOX.HADOOP\242301\240\003\002\001\000\241*0(\033\002nn\033\"namenode01.bigdata.fr\243\202\001:0\202\0016\240\003\002\001\022\241\003\002\001\v\242\202\001(\004\202\001$\000-\324?\266QMr6Rf\277\v\266\256l\353\231\005\375\240O\n\226mC\316\237L\214xbu\223\267\315#\274\202R[\267\034\316\303\n\361\232z\233\031\021E\233y\021ra\031\306\354\271D\r\006\334R\206Rr\257\004\005\326V\000\344\363xg\242\351L\250\f6\316" auths { method: "KERBEROS" mechanism: "GSSAPI" protocol: "nn" serverId: "namenode01.bigdata.fr" } 2016-10-26 16:27:27,517 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: CHALLENGE token: "`j\006\t*\206H\206\367\022\001\002\002\002\000o[0Y\240\003\002\001\005\241\003\002\001\017\242M0K\240\003\002\001\022\242D\004B\252(\353\344\232\274\021\274\207\223\213\265\204\353\366K\277R\024v\226j\244=\246V&\271\351\324\366\361\f\025oS,a\317\'\233\212\324J\005\237\231\034\266\242H@\214y\253BgB\370\207\254\006\n\356\375\334" 2016-10-26 16:27:27,518 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: RESPONSE token: "" 2016-10-26 16:27:27,519 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: CHALLENGE token: "\005\004\001\377\000\f\000\000\000\000\000\000:\023ec\001\001\000\000%\266\033\270w\216\217\372h\245w" 2016-10-26 16:27:27,520 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: RESPONSE token: "\005\004\000\377\000\f\000\000\000\000\000\000#\221iX\001\001\000\000cl\354\343Qa\t\341Kf=4" 2016-10-26 16:27:27,520 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: SUCCESS 2016-10-26 16:27:27,521 DEBUG org.apache.hadoop.ipc.Client: Negotiated QOP is :auth 2016-10-26 16:27:27,521 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #502 2016-10-26 16:27:27,521 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP: starting, having connections 2 2016-10-26 16:27:27,536 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #502 2016-10-26 16:27:27,536 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getFileInfo took 28ms 2016-10-26 16:27:27,537 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:nm/datanode03.bigdata.fr@SANDBOX.HADOOP (auth:KERBEROS) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331) 2016-10-26 16:27:27,537 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished. 2016-10-26 16:27:27,537 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:shfs3453 (auth:SIMPLE) from:org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.createAppDir(LogAggregationService.java:261) 2016-10-26 16:27:27,538 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false 2016-10-26 16:27:27,538 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false 2016-10-26 16:27:27,538 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false 2016-10-26 16:27:27,538 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.domain.socket.path = 2016-10-26 16:27:27,540 DEBUG org.apache.hadoop.security.SecurityUtil: Acquired token Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.200.23:8020, Ident: (HDFS_DELEGATION_TOKEN token 12521 for shfs3453) 2016-10-26 16:27:27,540 DEBUG org.apache.hadoop.hdfs.HAUtil: Mapped HA service delegation token for logical URI hdfs://sandbox to namenode namenode01.bigdata.fr/192.168.200.23:8020 2016-10-26 16:27:27,540 DEBUG org.apache.hadoop.security.SecurityUtil: Acquired token Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.200.24:8020, Ident: (HDFS_DELEGATION_TOKEN token 12521 for shfs3453) 2016-10-26 16:27:27,541 DEBUG org.apache.hadoop.hdfs.HAUtil: Mapped HA service delegation token for logical URI hdfs://sandbox to namenode namenode02.bigdata.fr/192.168.200.24:8020 2016-10-26 16:27:27,541 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false 2016-10-26 16:27:27,541 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false 2016-10-26 16:27:27,541 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false 2016-10-26 16:27:27,541 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.domain.socket.path = 2016-10-26 16:27:27,541 DEBUG org.apache.hadoop.io.retry.RetryUtils: multipleLinearRandomRetry = null 2016-10-26 16:27:27,541 DEBUG org.apache.hadoop.ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@40f129db 2016-10-26 16:27:27,542 DEBUG org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil: DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection 2016-10-26 16:27:27,543 DEBUG org.apache.hadoop.ipc.Client: The ping interval is 60000 ms. 2016-10-26 16:27:27,543 DEBUG org.apache.hadoop.ipc.Client: Connecting to namenode01.bigdata.fr/192.168.200.23:8020 2016-10-26 16:27:27,543 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:shfs3453 (auth:SIMPLE) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:724) 2016-10-26 16:27:27,543 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: NEGOTIATE 2016-10-26 16:27:27,544 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: NEGOTIATE auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" challenge: "realm=\"default\",nonce=\"ZTM0cIHz0jgP4YlEt4wCw6qRBZ7R4t/+my1izt3Q\",qop=\"auth\",charset=utf-8,algorithm=md5-sess" } auths { method: "KERBEROS" mechanism: "GSSAPI" protocol: "nn" serverId: "namenode01.bigdata.fr" } 2016-10-26 16:27:27,544 DEBUG org.apache.hadoop.security.SaslRpcClient: Get token info proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB info:@org.apache.hadoop.security.token.TokenInfo(value=class org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSelector) 2016-10-26 16:27:27,545 DEBUG org.apache.hadoop.security.SaslRpcClient: Creating SASL DIGEST-MD5(TOKEN) client to authenticate to service at default 2016-10-26 16:27:27,545 DEBUG org.apache.hadoop.security.SaslRpcClient: Use TOKEN authentication for protocol ClientNamenodeProtocolPB 2016-10-26 16:27:27,545 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting username: ABdzaGZzMzQ1M0BTQU5EQk9YLkhBRE9PUAJybQCKAVgBY3KMigFYJW/2jI4w6Y4GHg== 2016-10-26 16:27:27,545 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting userPassword 2016-10-26 16:27:27,545 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting realm: default 2016-10-26 16:27:27,546 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: INITIATE token: "charset=utf-8,username=\"ABdzaGZzMzQ1M0BTQU5EQk9YLkhBRE9PUAJybQCKAVgBY3KMigFYJW/2jI4w6Y4GHg==\",realm=\"default\",nonce=\"ZTM0cIHz0jgP4YlEt4wCw6qRBZ7R4t/+my1izt3Q\",nc=00000001,cnonce=\"5cCBbBQEBsh0oN0Pz86HADdhA20SCH4PXwhv75b3\",digest-uri=\"/default\",maxbuf=65536,response=99d6a832af59402869f843f8a5e52761,qop=auth" auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" } 2016-10-26 16:27:27,547 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: SUCCESS token: "rspauth=bf15f9c6b9385f2fcf9b15a185c75d06" 2016-10-26 16:27:27,547 DEBUG org.apache.hadoop.ipc.Client: Negotiated QOP is :auth 2016-10-26 16:27:27,551 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 sending #503 2016-10-26 16:27:27,551 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453: starting, having connections 3 2016-10-26 16:27:27,552 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 got value #503 2016-10-26 16:27:27,552 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getFileInfo took 9ms 2016-10-26 16:27:27,553 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 sending #504 2016-10-26 16:27:27,553 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 got value #504 2016-10-26 16:27:27,553 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getFileInfo took 0ms 2016-10-26 16:27:27,554 DEBUG org.apache.hadoop.hdfs.DFSClient: /Products/YARN/logs/shfs3453/logs/application_1475850791417_0147: masked=rwxr-x--- 2016-10-26 16:27:27,554 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 sending #505 2016-10-26 16:27:27,583 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 got value #505 2016-10-26 16:27:27,583 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: mkdirs took 29ms 2016-10-26 16:27:27,583 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 sending #506 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 got value #506 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: setPermission took 3ms 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEvent.EventType: APPLICATION_LOG_HANDLING_INITED 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Processing application_1475850791417_0147 of type APPLICATION_LOG_HANDLING_INITED 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ApplicationLocalizationEvent.EventType: INIT_APPLICATION_RESOURCES 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationInitedEvent.EventType: APPLICATION_INITED 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Processing application_1475850791417_0147 of type APPLICATION_INITED 2016-10-26 16:27:27,586 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1475850791417_0147 transitioned from INITING to RUNNING 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerInitEvent.EventType: INIT_CONTAINER 2016-10-26 16:27:27,586 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type INIT_CONTAINER 2016-10-26 16:27:27,587 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475850791417_0147_01_000001 transitioned from NEW to LOCALIZING 2016-10-26 16:27:27,587 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_INIT 2016-10-26 16:27:27,587 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_INIT for appId application_1475850791417_0147 2016-10-26 16:27:27,587 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationRequestEvent.EventType: INIT_CONTAINER_RESOURCES 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-hbase-handler-2.1.0.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-hbase-handler-2.1.0.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/metrics-core-2.2.0.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/metrics-core-2.2.0.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-exec-2.1.0.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-exec-2.1.0.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-hadoop-compat-1.1.2.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-hadoop-compat-1.1.2.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-common-1.1.2.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-common-1.1.2.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/netty-all-4.0.23.Final.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/netty-all-4.0.23.Final.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-client-1.1.2.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-client-1.1.2.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-jdbc-2.1.0-standalone.jar of type REQUEST 2016-10-26 16:27:27,588 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-jdbc-2.1.0-standalone.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,588 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/guava-14.0.1.jar of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/guava-14.0.1.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-protocol-1.1.2.jar of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-protocol-1.1.2.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/spark-assembly-1.4.1-hadoop2.7.1.jar of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/spark-assembly-1.4.1-hadoop2.7.1.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-server-1.1.2.jar of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-server-1.1.2.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.jar of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.jar transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.splitmetainfo of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.splitmetainfo transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.split of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.split transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.xml of type REQUEST 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.xml transitioned from INIT to DOWNLOADING 2016-10-26 16:27:27,589 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,589 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Created localizer for container_1475850791417_0147_01_000001 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerResourceRequestEvent.EventType: REQUEST_RESOURCE_LOCALIZATION 2016-10-26 16:27:27,590 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:nm/datanode03.bigdata.fr@SANDBOX.HADOOP (auth:KERBEROS) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331) 2016-10-26 16:27:27,591 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Writing credentials to the nmPrivate file /mnt/hd11/hadoop/yarn/local/nmPrivate/container_1475850791417_0147_01_000001.tokens. Credentials list: 2016-10-26 16:27:27,592 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: ha-hdfs:sandbox : MQAXc2hmczM0NTNAU0FOREJPWC5IQURPT1ACcm0AigFYAWNyjIoBWCVv9oyOMOmOBh4UoU6kOA3YV-0Uq2TESPSVJsujJNIVSERGU19ERUxFR0FUSU9OX1RPS0VOD2hhLWhkZnM6c2FuZGJveA 2016-10-26 16:27:27,592 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: : GwoOCgoIkwEQ-dO__PkqEAEQ0MrUo_z_____ARQM2lT_3dx6ahLV5ZOWrJ9bdcvqURBZQVJOX0FNX1JNX1RPS0VOAA 2016-10-26 16:27:27,592 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: e7142baa-7bed-4c65-8ed5-6594470a05bd : MgAAAC4IABIXc2hmczM0NTNAU0FOREJPWC5IQURPT1AYxgQgi-SNi4ArKIvsv6uCKzBZFBEQe7buSwL4V2cbFsO-Ih-blVVFEEhCQVNFX0FVVEhfVE9LRU4kZTcxNDJiYWEtN2JlZC00YzY1LThlZDUtNjU5NDQ3MGEwNWJk 2016-10-26 16:27:28,048 DEBUG org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: initApplication: [/opt/application/Hadoop/current/bin/container-executor, shfs3453, shfs3453, 0, application_1475850791417_0147, /mnt/hd11/hadoop/yarn/local/nmPrivate/container_1475850791417_0147_01_000001.tokens, /mnt/hd6/hadoop/yarn/local,/mnt/hd8/hadoop/yarn/local,/mnt/hd9/hadoop/yarn/local,/mnt/hd4/hadoop/yarn/local,/mnt/hd5/hadoop/yarn/local,/mnt/hd10/hadoop/yarn/local,/mnt/hd0/hadoop/yarn/local,/mnt/hd2/hadoop/yarn/local,/mnt/hd11/hadoop/yarn/local,/mnt/hd7/hadoop/yarn/local,/mnt/hd3/hadoop/yarn/local,/mnt/hd1/hadoop/yarn/local, /mnt/hd6/hadoop/yarn/log,/mnt/hd8/hadoop/yarn/log,/mnt/hd9/hadoop/yarn/log,/mnt/hd4/hadoop/yarn/log,/mnt/hd5/hadoop/yarn/log,/mnt/hd10/hadoop/yarn/log,/mnt/hd0/hadoop/yarn/log,/mnt/hd2/hadoop/yarn/log,/mnt/hd11/hadoop/yarn/log,/mnt/hd7/hadoop/yarn/log,/mnt/hd3/hadoop/yarn/log,/mnt/hd1/hadoop/yarn/log, /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/jre/bin/java, -classpath, /opt/application/Hadoop/current/etc/hadoop/:/opt/application/Hadoop/current/etc/hadoop/:/opt/application/Hadoop/current/etc/hadoop/:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-framework.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/gson-2.2.4.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/httpclient-4.2.5.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/zookeeper-3.4.6.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/slf4j-api-1.7.10.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jetty-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-auth.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/paranamer-2.3.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/httpcore-4.2.5.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jettison-1.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/guava-11.0.2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jsch-0.1.42.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/junit-4.11.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-recipes.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/asm-3.2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-net-3.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/xz-1.0.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-auth-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/activation-1.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-annotations.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/avro-1.7.4.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-client.jar:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-io-2.4.jar:/opt/application/Hadoop/current/share/hadoop/common/hadoop-common-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/common/hadoop-common.jar:/opt/application/Hadoop/current/share/hadoop/common/hadoop-nfs-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/common/hadoop-common-2.7.2-tests.jar:/opt/application/Hadoop/current/share/hadoop/hdfs:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/guava-11.0.2.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-2.7.2-tests.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-3.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guava-11.0.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/asm-3.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/xz-1.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/activation-1.1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-common.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-common.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-registry.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-resourcemanager.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-api.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-distributedshell.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-nodemanager.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-web-proxy.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-client.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-app.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-core.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-common.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar::/opt/application/Tez/current/conf:/opt/application/Tez/current/tez-api-0.5.1.jar:/opt/application/Tez/current/tez-dag-0.5.1.jar:/opt/application/Tez/current/tez-yarn-timeline-history-0.5.1.jar:/opt/application/Tez/current/tez-common-0.5.1.jar:/opt/application/Tez/current/tez-runtime-library-0.5.1.jar:/opt/application/Tez/current/tez-examples-0.5.1.jar:/opt/application/Tez/current/tez-tests-0.5.1.jar:/opt/application/Tez/current/tez-mapreduce-0.5.1.jar:/opt/application/Tez/current/tez-runtime-internals-0.5.1.jar:/opt/application/Tez/current/lib/commons-collections4-4.0.jar:/opt/application/Tez/current/lib/commons-codec-1.4.jar:/opt/application/Tez/current/lib/protobuf-java-2.5.0.jar:/opt/application/Tez/current/lib/commons-cli-1.2.jar:/opt/application/Tez/current/lib/jsr305-2.0.3.jar:/opt/application/Tez/current/lib/commons-collections-3.2.1.jar:/opt/application/Tez/current/lib/guava-11.0.2.jar:/opt/application/Tez/current/lib/jettison-1.3.4.jar:/opt/application/Tez/current/lib/hadoop-mapreduce-client-common-2.4.1.jar:/opt/application/Tez/current/lib/hadoop-mapreduce-client-core-2.4.1.jar:/opt/application/Tez/current/lib/commons-lang-2.6.jar:/opt/application/Tez/current/lib/log4j-1.2.17.jar:/opt/application/Tez/current/lib/commons-io-2.4.jar:/opt/application/Tez/current/lib/commons-math3-3.1.1.jar:/opt/application/Tez/current/lib/hadoop-annotations-2.4.1.jar:/opt/application/Tez/current/lib/commons-logging-1.1.3.jar:/opt/application/Hadoop/current/contrib/capacity-scheduler/*.jar:/opt/application/Hadoop/current/contrib/capacity-scheduler/*.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-common.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-common.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-registry.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-resourcemanager.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-api.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-distributedshell.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-nodemanager.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-web-proxy.jar:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-client.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-3.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guava-11.0.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/asm-3.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/xz-1.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/activation-1.1.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/application/Hadoop/current/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/application/Hadoop/current/etc/hadoop//nm-config/log4j.properties, -Djava.library.path=/opt/application/Hadoop/hadoop-2.7.2/lib/native, org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer, shfs3453, application_1475850791417_0147, container_1475850791417_0147_01_000001, datanode03.bigdata.fr, 8040, /mnt/hd6/hadoop/yarn/local, /mnt/hd8/hadoop/yarn/local, /mnt/hd9/hadoop/yarn/local, /mnt/hd4/hadoop/yarn/local, /mnt/hd5/hadoop/yarn/local, /mnt/hd10/hadoop/yarn/local, /mnt/hd0/hadoop/yarn/local, /mnt/hd2/hadoop/yarn/local, /mnt/hd11/hadoop/yarn/local, /mnt/hd7/hadoop/yarn/local, /mnt/hd3/hadoop/yarn/local, /mnt/hd1/hadoop/yarn/local] 2016-10-26 16:27:28,489 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:28,490 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:28,490 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #507 2016-10-26 16:27:28,491 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #507 2016-10-26 16:27:28,491 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:29,251 DEBUG org.apache.hadoop.ipc.Server: IPC Server idle connection scanner for port 37761: task running 2016-10-26 16:27:29,304 DEBUG org.apache.hadoop.ipc.Server: IPC Server idle connection scanner for port 8040: task running 2016-10-26 16:27:29,492 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:29,492 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:29,492 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #508 2016-10-26 16:27:29,493 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #508 2016-10-26 16:27:29,493 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:29,756 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:29,756 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:29,774 DEBUG org.apache.hadoop.ipc.Server: Server connection from 192.168.200.27:46140; # active connections: 1; # queued calls: 0 2016-10-26 16:27:29,788 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:29,789 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.3 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:29,792 DEBUG org.apache.hadoop.ipc.Server: got #-33 2016-10-26 16:27:29,792 DEBUG org.apache.hadoop.security.SaslRpcServer: Created SASL server with mechanism = DIGEST-MD5 2016-10-26 16:27:29,792 DEBUG org.apache.hadoop.ipc.Server: Sending sasl message state: NEGOTIATE auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" challenge: "realm=\"default\",nonce=\"u1LlduTlkrQZyQcuuIefSPX35pDTB4zARNygQMVD\",qop=\"auth\",charset=utf-8,algorithm=md5-sess" } auths { method: "KERBEROS" mechanism: "GSSAPI" protocol: "nm" serverId: "datanode03.bigdata.fr" } 2016-10-26 16:27:29,793 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8040: responding to null from 192.168.200.27:46140 Call#-33 Retry#-1 2016-10-26 16:27:29,793 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8040: responding to null from 192.168.200.27:46140 Call#-33 Retry#-1 Wrote 226 bytes. 2016-10-26 16:27:29,823 DEBUG org.apache.hadoop.ipc.Server: got #-33 2016-10-26 16:27:29,823 DEBUG org.apache.hadoop.ipc.Server: Have read input token of size 246 for processing by saslServer.evaluateResponse() 2016-10-26 16:27:29,824 DEBUG org.apache.hadoop.security.SaslRpcServer: SASL server DIGEST-MD5 callback: setting password for client: testing (auth:SIMPLE) 2016-10-26 16:27:29,824 DEBUG org.apache.hadoop.security.SaslRpcServer: SASL server DIGEST-MD5 callback: setting canonicalized client ID: testing 2016-10-26 16:27:29,824 DEBUG org.apache.hadoop.ipc.Server: Will send SUCCESS token of size 40 from saslServer. 2016-10-26 16:27:29,824 DEBUG org.apache.hadoop.ipc.Server: SASL server context established. Negotiated QoP is auth 2016-10-26 16:27:29,824 DEBUG org.apache.hadoop.ipc.Server: SASL server successfully authenticated client: testing (auth:SIMPLE) 2016-10-26 16:27:29,824 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for testing (auth:SIMPLE) 2016-10-26 16:27:29,824 DEBUG org.apache.hadoop.ipc.Server: Sending sasl message state: SUCCESS token: "rspauth=08ca46effeba049bd0cf2c0b01944b30" 2016-10-26 16:27:29,825 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8040: responding to null from 192.168.200.27:46140 Call#-33 Retry#-1 2016-10-26 16:27:29,825 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8040: responding to null from 192.168.200.27:46140 Call#-33 Retry#-1 Wrote 64 bytes. 2016-10-26 16:27:29,835 DEBUG org.apache.hadoop.ipc.Server: got #-3 2016-10-26 16:27:29,835 INFO SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization successful for testing (auth:TOKEN) for protocol=interface org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB 2016-10-26 16:27:29,835 DEBUG org.apache.hadoop.ipc.Server: Successfully authorized userInfo { } protocol: "org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB" 2016-10-26 16:27:29,835 DEBUG org.apache.hadoop.ipc.Server: got #0 2016-10-26 16:27:29,836 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#0 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:29,836 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:29,838 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 1 2016-10-26 16:27:29,839 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#0 Retry#0 2016-10-26 16:27:29,839 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#0 Retry#0 Wrote 254 bytes. 2016-10-26 16:27:30,494 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:30,494 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:30,495 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #509 2016-10-26 16:27:30,495 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #509 2016-10-26 16:27:30,495 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:30,972 DEBUG org.apache.hadoop.ipc.Server: got #1 2016-10-26 16:27:30,972 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#1 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:30,973 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:30,974 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 1 2016-10-26 16:27:30,975 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#1 Retry#0 2016-10-26 16:27:30,975 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#1 Retry#0 Wrote 247 bytes. 2016-10-26 16:27:31,283 DEBUG org.apache.hadoop.ipc.Server: got #7 2016-10-26 16:27:31,283 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#7 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,283 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,284 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-hbase-handler-2.1.0.jar of type LOCALIZED 2016-10-26 16:27:31,284 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-hbase-handler-2.1.0.jar(->/mnt/hd11/hadoop/yarn/local/usercache/shfs3453/filecache/11/hive-hbase-handler-2.1.0.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,284 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,284 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,284 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/metrics-core-2.2.0.jar of type LOCALIZED 2016-10-26 16:27:31,284 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/metrics-core-2.2.0.jar(->/mnt/hd7/hadoop/yarn/local/usercache/shfs3453/filecache/12/metrics-core-2.2.0.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,284 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,284 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,285 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 1 2016-10-26 16:27:31,285 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#7 Retry#0 2016-10-26 16:27:31,286 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#7 Retry#0 Wrote 244 bytes. 2016-10-26 16:27:31,287 DEBUG org.apache.hadoop.ipc.Server: got #8 2016-10-26 16:27:31,287 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#8 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,287 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,288 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 1 2016-10-26 16:27:31,288 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#8 Retry#0 2016-10-26 16:27:31,288 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#8 Retry#0 Wrote 254 bytes. 2016-10-26 16:27:31,496 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:31,496 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:31,497 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #510 2016-10-26 16:27:31,497 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #510 2016-10-26 16:27:31,497 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:31,622 DEBUG org.apache.hadoop.ipc.Server: got #11 2016-10-26 16:27:31,622 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#11 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,622 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,622 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-exec-2.1.0.jar of type LOCALIZED 2016-10-26 16:27:31,623 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-exec-2.1.0.jar(->/mnt/hd3/hadoop/yarn/local/usercache/shfs3453/filecache/13/hive-exec-2.1.0.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,623 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,623 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,624 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 2 2016-10-26 16:27:31,624 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#11 Retry#0 2016-10-26 16:27:31,624 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#11 Retry#0 Wrote 247 bytes. 2016-10-26 16:27:31,639 DEBUG org.apache.hadoop.ipc.Server: got #14 2016-10-26 16:27:31,639 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#14 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,639 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,639 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-hadoop-compat-1.1.2.jar of type LOCALIZED 2016-10-26 16:27:31,639 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-hadoop-compat-1.1.2.jar(->/mnt/hd1/hadoop/yarn/local/usercache/shfs3453/filecache/14/hbase-hadoop-compat-1.1.2.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,639 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,640 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,640 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 1 2016-10-26 16:27:31,640 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#14 Retry#0 2016-10-26 16:27:31,641 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#14 Retry#0 Wrote 251 bytes. 2016-10-26 16:27:31,657 DEBUG org.apache.hadoop.ipc.Server: got #17 2016-10-26 16:27:31,657 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#17 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,657 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,658 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-common-1.1.2.jar of type LOCALIZED 2016-10-26 16:27:31,658 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-common-1.1.2.jar(->/mnt/hd6/hadoop/yarn/local/usercache/shfs3453/filecache/15/hbase-common-1.1.2.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,658 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,658 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,659 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 2 2016-10-26 16:27:31,659 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#17 Retry#0 2016-10-26 16:27:31,659 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#17 Retry#0 Wrote 247 bytes. 2016-10-26 16:27:31,682 DEBUG org.apache.hadoop.ipc.Server: got #20 2016-10-26 16:27:31,682 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#20 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,682 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,682 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/netty-all-4.0.23.Final.jar of type LOCALIZED 2016-10-26 16:27:31,682 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/netty-all-4.0.23.Final.jar(->/mnt/hd8/hadoop/yarn/local/usercache/shfs3453/filecache/16/netty-all-4.0.23.Final.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,682 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,683 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,684 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 2 2016-10-26 16:27:31,684 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#20 Retry#0 2016-10-26 16:27:31,684 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#20 Retry#0 Wrote 256 bytes. 2016-10-26 16:27:31,702 DEBUG org.apache.hadoop.ipc.Server: got #23 2016-10-26 16:27:31,702 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#23 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,702 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,703 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-client-1.1.2.jar of type LOCALIZED 2016-10-26 16:27:31,703 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-client-1.1.2.jar(->/mnt/hd9/hadoop/yarn/local/usercache/shfs3453/filecache/17/hbase-client-1.1.2.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,703 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,703 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,704 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 1 2016-10-26 16:27:31,704 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#23 Retry#0 2016-10-26 16:27:31,704 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#23 Retry#0 Wrote 240 bytes. 2016-10-26 16:27:31,790 DEBUG org.apache.hadoop.ipc.Server: got #27 2016-10-26 16:27:31,790 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#27 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,790 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,791 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-jdbc-2.1.0-standalone.jar of type LOCALIZED 2016-10-26 16:27:31,791 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-jdbc-2.1.0-standalone.jar(->/mnt/hd4/hadoop/yarn/local/usercache/shfs3453/filecache/18/hive-jdbc-2.1.0-standalone.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,791 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,791 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,792 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 1 2016-10-26 16:27:31,792 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#27 Retry#0 2016-10-26 16:27:31,792 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#27 Retry#0 Wrote 250 bytes. 2016-10-26 16:27:31,815 DEBUG org.apache.hadoop.ipc.Server: got #29 2016-10-26 16:27:31,816 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#29 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,816 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,816 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/guava-14.0.1.jar of type LOCALIZED 2016-10-26 16:27:31,816 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/guava-14.0.1.jar(->/mnt/hd5/hadoop/yarn/local/usercache/shfs3453/filecache/19/guava-14.0.1.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,816 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,816 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,817 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 1 2016-10-26 16:27:31,817 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#29 Retry#0 2016-10-26 16:27:31,817 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#29 Retry#0 Wrote 262 bytes. 2016-10-26 16:27:31,850 DEBUG org.apache.hadoop.ipc.Server: got #32 2016-10-26 16:27:31,850 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#32 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:31,850 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:31,851 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-protocol-1.1.2.jar of type LOCALIZED 2016-10-26 16:27:31,851 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-protocol-1.1.2.jar(->/mnt/hd10/hadoop/yarn/local/usercache/shfs3453/filecache/20/hbase-protocol-1.1.2.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:31,851 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:31,851 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:31,852 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 2 2016-10-26 16:27:31,852 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#32 Retry#0 2016-10-26 16:27:31,852 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#32 Retry#0 Wrote 247 bytes. 2016-10-26 16:27:32,498 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:32,498 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:32,499 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #511 2016-10-26 16:27:32,500 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #511 2016-10-26 16:27:32,500 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 2ms 2016-10-26 16:27:32,789 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:32,789 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:32,799 DEBUG org.apache.hadoop.ipc.Server: got #35 2016-10-26 16:27:32,799 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#35 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:32,799 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:32,800 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/spark-assembly-1.4.1-hadoop2.7.1.jar of type LOCALIZED 2016-10-26 16:27:32,800 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/spark-assembly-1.4.1-hadoop2.7.1.jar(->/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/filecache/21/spark-assembly-1.4.1-hadoop2.7.1.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:32,800 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:32,800 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:32,801 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 2 2016-10-26 16:27:32,802 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#35 Retry#0 2016-10-26 16:27:32,802 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#35 Retry#0 Wrote 277 bytes. 2016-10-26 16:27:32,823 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:32,824 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.3 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:32,834 DEBUG org.apache.hadoop.ipc.Server: got #38 2016-10-26 16:27:32,834 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#38 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:32,834 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:32,835 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-server-1.1.2.jar of type LOCALIZED 2016-10-26 16:27:32,835 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-server-1.1.2.jar(->/mnt/hd2/hadoop/yarn/local/usercache/shfs3453/filecache/22/hbase-server-1.1.2.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:32,835 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:32,835 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:32,836 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 2 2016-10-26 16:27:32,836 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#38 Retry#0 2016-10-26 16:27:32,836 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#38 Retry#0 Wrote 265 bytes. 2016-10-26 16:27:33,100 DEBUG org.apache.hadoop.ipc.Server: got #41 2016-10-26 16:27:33,101 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#41 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:33,101 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:33,101 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.jar of type LOCALIZED 2016-10-26 16:27:33,102 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.jar(->/mnt/hd11/hadoop/yarn/local/usercache/shfs3453/appcache/application_1475850791417_0147/filecache/10/job.jar) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:33,102 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:33,102 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:33,102 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 1 2016-10-26 16:27:33,103 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#41 Retry#0 2016-10-26 16:27:33,103 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#41 Retry#0 Wrote 257 bytes. 2016-10-26 16:27:33,119 DEBUG org.apache.hadoop.ipc.Server: got #45 2016-10-26 16:27:33,120 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#45 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:33,120 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:33,120 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.splitmetainfo of type LOCALIZED 2016-10-26 16:27:33,120 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.splitmetainfo(->/mnt/hd7/hadoop/yarn/local/usercache/shfs3453/appcache/application_1475850791417_0147/filecache/11/job.splitmetainfo) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:33,120 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:33,120 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:33,121 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 1 2016-10-26 16:27:33,121 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#45 Retry#0 2016-10-26 16:27:33,121 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 0 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#45 Retry#0 Wrote 255 bytes. 2016-10-26 16:27:33,500 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:33,501 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:33,501 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #512 2016-10-26 16:27:33,502 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #512 2016-10-26 16:27:33,502 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:33,548 DEBUG org.apache.hadoop.ipc.Server: got #47 2016-10-26 16:27:33,549 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#47 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:33,549 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:33,549 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.split of type LOCALIZED 2016-10-26 16:27:33,549 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.split(->/mnt/hd3/hadoop/yarn/local/usercache/shfs3453/appcache/application_1475850791417_0147/filecache/12/job.split) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:33,549 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:33,549 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:33,550 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 1 2016-10-26 16:27:33,550 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#47 Retry#0 2016-10-26 16:27:33,550 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 3 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#47 Retry#0 Wrote 34 bytes. 2016-10-26 16:27:33,569 DEBUG org.apache.hadoop.ipc.Server: got #50 2016-10-26 16:27:33,569 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#50 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:33,569 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.xml of type LOCALIZED 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerResourceLocalizedEvent.EventType: RESOURCE_LOCALIZED 2016-10-26 16:27:33,570 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.xml(->/mnt/hd1/hadoop/yarn/local/usercache/shfs3453/appcache/application_1475850791417_0147/filecache/13/job.xml) transitioned from DOWNLOADING to LOCALIZED 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type RESOURCE_LOCALIZED 2016-10-26 16:27:33,570 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475850791417_0147_01_000001 transitioned from LOCALIZING to LOCALIZED 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 1 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationEvent.EventType: CONTAINER_RESOURCES_LOCALIZED 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: LAUNCH_CONTAINER 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#50 Retry#0 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#50 Retry#0 Wrote 34 bytes. 2016-10-26 16:27:33,570 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEvent.EventType: UPLOAD 2016-10-26 16:27:33,572 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:nm/datanode03.bigdata.fr@SANDBOX.HADOOP (auth:KERBEROS) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331) 2016-10-26 16:27:33,599 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_LAUNCHED 2016-10-26 16:27:33,599 DEBUG org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [/opt/application/Hadoop/current/bin/container-executor, shfs3453, shfs3453, 1, application_1475850791417_0147, container_1475850791417_0147_01_000001, /mnt/hd8/hadoop/yarn/local/usercache/shfs3453/appcache/application_1475850791417_0147/container_1475850791417_0147_01_000001, /mnt/hd5/hadoop/yarn/local/nmPrivate/application_1475850791417_0147/container_1475850791417_0147_01_000001/launch_container.sh, /mnt/hd3/hadoop/yarn/local/nmPrivate/application_1475850791417_0147/container_1475850791417_0147_01_000001/container_1475850791417_0147_01_000001.tokens, /mnt/hd0/hadoop/yarn/local/nmPrivate/application_1475850791417_0147/container_1475850791417_0147_01_000001/container_1475850791417_0147_01_000001.pid, /mnt/hd6/hadoop/yarn/local,/mnt/hd8/hadoop/yarn/local,/mnt/hd9/hadoop/yarn/local,/mnt/hd4/hadoop/yarn/local,/mnt/hd5/hadoop/yarn/local,/mnt/hd10/hadoop/yarn/local,/mnt/hd0/hadoop/yarn/local,/mnt/hd2/hadoop/yarn/local,/mnt/hd11/hadoop/yarn/local,/mnt/hd7/hadoop/yarn/local,/mnt/hd3/hadoop/yarn/local,/mnt/hd1/hadoop/yarn/local, /mnt/hd6/hadoop/yarn/log,/mnt/hd8/hadoop/yarn/log,/mnt/hd9/hadoop/yarn/log,/mnt/hd4/hadoop/yarn/log,/mnt/hd5/hadoop/yarn/log,/mnt/hd10/hadoop/yarn/log,/mnt/hd0/hadoop/yarn/log,/mnt/hd2/hadoop/yarn/log,/mnt/hd11/hadoop/yarn/log,/mnt/hd7/hadoop/yarn/log,/mnt/hd3/hadoop/yarn/log,/mnt/hd1/hadoop/yarn/log, cgroups=none] 2016-10-26 16:27:33,599 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type CONTAINER_LAUNCHED 2016-10-26 16:27:33,599 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475850791417_0147_01_000001 transitioned from LOCALIZED to RUNNING 2016-10-26 16:27:33,599 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStartMonitoringEvent.EventType: START_MONITORING_CONTAINER 2016-10-26 16:27:34,502 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:34,503 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:34,504 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #513 2016-10-26 16:27:34,504 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #513 2016-10-26 16:27:34,505 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 2ms 2016-10-26 16:27:34,571 DEBUG org.apache.hadoop.ipc.Server: got #51 2016-10-26 16:27:34,571 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#51 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:34,572 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:34,572 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 1 procesingTime= 0 2016-10-26 16:27:34,572 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#51 Retry#0 2016-10-26 16:27:34,572 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 4 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#51 Retry#0 Wrote 34 bytes. 2016-10-26 16:27:34,573 DEBUG org.apache.hadoop.ipc.Server: got #52 2016-10-26 16:27:34,573 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#52 Retry#0 for RpcKind RPC_PROTOCOL_BUFFER 2016-10-26 16:27:34,573 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:testing (auth:TOKEN) from:org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043) 2016-10-26 16:27:34,573 DEBUG org.apache.hadoop.ipc.Server: Served: heartbeat queueTime= 0 procesingTime= 0 2016-10-26 16:27:34,573 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#52 Retry#0 2016-10-26 16:27:34,573 DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8040: responding to org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB.heartbeat from 192.168.200.27:46140 Call#52 Retry#0 Wrote 34 bytes. 2016-10-26 16:27:34,924 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8040: disconnecting client 192.168.200.27:46140. Number of active connections: 0 2016-10-26 16:27:34,925 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : command provided 0 2016-10-26 16:27:34,925 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : user is shfs3453 2016-10-26 16:27:34,925 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : requested yarn user is shfs3453 2016-10-26 16:27:35,505 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:35,505 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:35,506 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #514 2016-10-26 16:27:35,507 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #514 2016-10-26 16:27:35,507 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:35,608 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container container_1475850791417_0147_01_000001 is : 1 2016-10-26 16:27:35,608 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch with container ID: container_1475850791417_0147_01_000001 and exit code: 1 ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:297) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2016-10-26 16:27:35,610 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception from container-launch. 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Container id: container_1475850791417_0147_01_000001 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exit code: 1 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Stack trace: ExitCodeException exitCode=1: 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.util.Shell.run(Shell.java:456) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:297) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.util.concurrent.FutureTask.run(FutureTask.java:262) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.lang.Thread.run(Thread.java:745) 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Shell output: main : command provided 1 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : user is shfs3453 2016-10-26 16:27:35,611 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: main : requested yarn user is shfs3453 2016-10-26 16:27:35,612 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type UPDATE_DIAGNOSTICS_MSG 2016-10-26 16:27:35,612 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container container_1475850791417_0147_01_000001 completed with exit code 1 2016-10-26 16:27:35,612 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container exited with a non-zero exit code 1 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerExitEvent.EventType: CONTAINER_EXITED_WITH_FAILURE 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type CONTAINER_EXITED_WITH_FAILURE 2016-10-26 16:27:35,613 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475850791417_0147_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEvent.EventType: CLEANUP_CONTAINER 2016-10-26 16:27:35,613 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_1475850791417_0147_01_000001 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Marking container container_1475850791417_0147_01_000001 as inactive 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Getting pid for container container_1475850791417_0147_01_000001 to kill from pid file /mnt/hd0/hadoop/yarn/local/nmPrivate/application_1475850791417_0147/container_1475850791417_0147_01_000001/container_1475850791417_0147_01_000001.pid 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Accessing pid for container container_1475850791417_0147_01_000001 from pid file /mnt/hd0/hadoop/yarn/local/nmPrivate/application_1475850791417_0147/container_1475850791417_0147_01_000001/container_1475850791417_0147_01_000001.pid 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.util.ProcessIdFileReader: Accessing pid from pid file /mnt/hd0/hadoop/yarn/local/nmPrivate/application_1475850791417_0147/container_1475850791417_0147_01_000001/container_1475850791417_0147_01_000001.pid 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.util.ProcessIdFileReader: Got pid 29260 from path /mnt/hd0/hadoop/yarn/local/nmPrivate/application_1475850791417_0147/container_1475850791417_0147_01_000001/container_1475850791417_0147_01_000001.pid 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Got pid 29260 for container container_1475850791417_0147_01_000001 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Sending signal to pid 29260 as user shfs3453 for container container_1475850791417_0147_01_000001 2016-10-26 16:27:35,613 DEBUG org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: signalContainer: [/opt/application/Hadoop/current/bin/container-executor, shfs3453, shfs3453, 2, 29260, 15] 2016-10-26 16:27:35,644 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Sent signal SIGTERM to pid 29260 as user shfs3453 for container container_1475850791417_0147_01_000001, result=failed 2016-10-26 16:27:35,644 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:nm/datanode03.bigdata.fr@SANDBOX.HADOOP (auth:KERBEROS) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331) 2016-10-26 16:27:35,663 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ContainerLocalizationCleanupEvent.EventType: CLEANUP_CONTAINER_RESOURCES 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.splitmetainfo of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.split of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox:8020/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/job.xml of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-hbase-handler-2.1.0.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/metrics-core-2.2.0.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-exec-2.1.0.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-hadoop-compat-1.1.2.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-common-1.1.2.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/netty-all-4.0.23.Final.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-client-1.1.2.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hive-jdbc-2.1.0-standalone.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/guava-14.0.1.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-protocol-1.1.2.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/spark-assembly-1.4.1-hadoop2.7.1.jar of type RELEASE 2016-10-26 16:27:35,664 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Processing hdfs://sandbox/Products/MR/staging/shfs3453/.staging/job_1475850791417_0147/libjars/hbase-server-1.1.2.jar of type RELEASE 2016-10-26 16:27:35,669 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEvent.EventType: CONTAINER_RESOURCES_CLEANEDUP 2016-10-26 16:27:35,669 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Processing container_1475850791417_0147_01_000001 of type CONTAINER_RESOURCES_CLEANEDUP 2016-10-26 16:27:35,669 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=shfs3453 OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1475850791417_0147 CONTAINERID=container_1475850791417_0147_01_000001 2016-10-26 16:27:35,669 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475850791417_0147_01_000001 transitioned from EXITED_WITH_FAILURE to DONE 2016-10-26 16:27:35,669 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationContainerFinishedEvent.EventType: APPLICATION_CONTAINER_FINISHED 2016-10-26 16:27:35,669 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Processing application_1475850791417_0147 of type APPLICATION_CONTAINER_FINISHED 2016-10-26 16:27:35,669 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1475850791417_0147_01_000001 from application application_1475850791417_0147 2016-10-26 16:27:35,669 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerStopMonitoringEvent.EventType: STOP_MONITORING_CONTAINER 2016-10-26 16:27:35,669 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerContainerFinishedEvent.EventType: CONTAINER_FINISHED 2016-10-26 16:27:35,669 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1475850791417_0147_01_000001 for log-aggregation 2016-10-26 16:27:35,669 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: CONTAINER_STOP 2016-10-26 16:27:35,670 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1475850791417_0147 2016-10-26 16:27:35,824 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:35,824 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1475850791417_0147_01_000001 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ContainerResource_container_1475850791417_0147_01_000001, Metrics for container: container_1475850791417_0147_01_000001 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsConfig: poking parent 'PropertiesConfiguration' for key: source.source.start_mbeans 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsConfig: poking parent 'MetricsConfig' for key: source.start_mbeans 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsConfig: poking parent 'PropertiesConfiguration' for key: *.source.start_mbeans 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Updating attr cache... 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Done. # tags & metrics=0 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Updating info cache... 2016-10-26 16:27:35,825 DEBUG org.apache.hadoop.metrics2.impl.MetricsSystemImpl: [] 2016-10-26 16:27:35,826 DEBUG org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: Done 2016-10-26 16:27:35,826 DEBUG org.apache.hadoop.metrics2.util.MBeans: Registered Hadoop:service=NodeManager,name=ContainerResource_container_1475850791417_0147_01_000001 2016-10-26 16:27:35,826 DEBUG org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ContainerResource_container_1475850791417_0147_01_000001 registered. 2016-10-26 16:27:35,826 DEBUG org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Registered source ContainerResource_container_1475850791417_0147_01_000001 2016-10-26 16:27:35,826 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1475850791417_0147_01_000001 2016-10-26 16:27:35,826 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:35,858 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:35,859 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.4 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:35,894 DEBUG org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: signalContainer: [/opt/application/Hadoop/current/bin/container-executor, shfs3453, shfs3453, 2, 29260, 9] 2016-10-26 16:27:36,507 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:36,508 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 2 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ], ContainerStatus: [ContainerId: container_1475850791417_0147_01_000001, State: COMPLETE, Diagnostics: Exception from container-launch. Container id: container_1475850791417_0147_01_000001 Exit code: 1 Stack trace: ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:297) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Shell output: main : command provided 1 main : user is shfs3453 main : requested yarn user is shfs3453 Container exited with a non-zero exit code 1 , ExitStatus: 1, ]] 2016-10-26 16:27:36,508 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #515 2016-10-26 16:27:36,509 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #515 2016-10-26 16:27:36,510 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 2ms 2016-10-26 16:27:37,501 DEBUG org.apache.hadoop.ipc.Server: Socket Reader #1 for port 37761: disconnecting client 192.168.200.219:55506. Number of active connections: 0 2016-10-26 16:27:37,510 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:37,510 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:37,511 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #516 2016-10-26 16:27:37,511 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #516 2016-10-26 16:27:37,511 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:37,512 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1475850791417_0147_01_000001] 2016-10-26 16:27:37,523 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP: closed 2016-10-26 16:27:37,523 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP: stopped, remaining connections 2 2016-10-26 16:27:37,583 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453: closed 2016-10-26 16:27:37,583 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453: stopped, remaining connections 1 2016-10-26 16:27:38,512 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:38,512 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:38,514 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #517 2016-10-26 16:27:38,514 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #517 2016-10-26 16:27:38,514 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:38,859 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:38,859 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:38,892 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:38,892 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.4 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:39,251 DEBUG org.apache.hadoop.ipc.Server: IPC Server idle connection scanner for port 37761: task running 2016-10-26 16:27:39,304 DEBUG org.apache.hadoop.ipc.Server: IPC Server idle connection scanner for port 8040: task running 2016-10-26 16:27:39,515 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:39,515 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:39,515 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #518 2016-10-26 16:27:39,516 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #518 2016-10-26 16:27:39,516 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:40,517 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:40,517 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:40,517 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #519 2016-10-26 16:27:40,518 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #519 2016-10-26 16:27:40,518 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:41,518 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:41,519 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:41,519 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #520 2016-10-26 16:27:41,520 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #520 2016-10-26 16:27:41,520 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:41,893 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:41,893 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:41,926 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:41,926 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.4 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:42,520 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:42,520 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:42,521 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #521 2016-10-26 16:27:42,521 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #521 2016-10-26 16:27:42,522 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 2ms 2016-10-26 16:27:43,522 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:43,522 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:43,524 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #522 2016-10-26 16:27:43,524 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #522 2016-10-26 16:27:43,524 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:44,525 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:44,525 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:44,525 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #523 2016-10-26 16:27:44,526 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #523 2016-10-26 16:27:44,526 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:44,926 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:44,927 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:44,960 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:44,960 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.4 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:45,526 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:45,527 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:45,527 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #524 2016-10-26 16:27:45,528 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #524 2016-10-26 16:27:45,528 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:45,826 DEBUG org.apache.hadoop.metrics2.util.MBeans: Unregistering Hadoop:service=NodeManager,name=ContainerResource_container_1475850791417_0147_01_000001 2016-10-26 16:27:46,528 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:46,528 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:46,529 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #525 2016-10-26 16:27:46,529 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #525 2016-10-26 16:27:46,530 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 2ms 2016-10-26 16:27:47,530 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:47,530 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:47,530 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #526 2016-10-26 16:27:47,531 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #526 2016-10-26 16:27:47,531 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:47,960 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:47,961 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:47,994 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:47,994 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.4 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:48,532 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:48,532 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:48,533 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #527 2016-10-26 16:27:48,534 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #527 2016-10-26 16:27:48,534 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:49,251 DEBUG org.apache.hadoop.ipc.Server: IPC Server idle connection scanner for port 37761: task running 2016-10-26 16:27:49,304 DEBUG org.apache.hadoop.ipc.Server: IPC Server idle connection scanner for port 8040: task running 2016-10-26 16:27:49,534 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:49,534 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:49,535 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #528 2016-10-26 16:27:49,535 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #528 2016-10-26 16:27:49,536 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:50,536 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:50,536 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:50,536 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #529 2016-10-26 16:27:50,537 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #529 2016-10-26 16:27:50,537 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:50,995 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Current ProcessTree list : [ 28218 ] 2016-10-26 16:27:50,995 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Constructing ProcessTree for : PID = 28218 ContainerId = container_1475850791417_0142_01_000001 2016-10-26 16:27:51,042 DEBUG org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 28218 28223 ] 2016-10-26 16:27:51,043 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 28218 for container-id container_1475850791417_0142_01_000001: 300.4 MB of 1 GB physical memory used; 1.7 GB of 2.1 GB virtual memory used 2016-10-26 16:27:51,538 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's health-status : true, 2016-10-26 16:27:51,538 DEBUG org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 1 container statuses: [ContainerStatus: [ContainerId: container_1475850791417_0142_01_000001, State: RUNNING, Diagnostics: , ExitStatus: -1000, ]] 2016-10-26 16:27:51,538 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP sending #530 2016-10-26 16:27:51,539 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to resourcemanager02.bigdata.fr/192.168.200.219:8031 from nm/datanode03.bigdata.fr@SANDBOX.HADOOP got value #530 2016-10-26 16:27:51,539 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: nodeHeartbeat took 1ms 2016-10-26 16:27:51,539 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.CMgrCompletedAppsEvent.EventType: FINISH_APPS 2016-10-26 16:27:51,540 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationFinishEvent.EventType: FINISH_APPLICATION 2016-10-26 16:27:51,540 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Processing application_1475850791417_0147 of type FINISH_APPLICATION 2016-10-26 16:27:51,540 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1475850791417_0147 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2016-10-26 16:27:51,540 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.ApplicationLocalizationEvent.EventType: DESTROY_APPLICATION_RESOURCES 2016-10-26 16:27:51,544 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEvent.EventType: APPLICATION_STOP 2016-10-26 16:27:51,544 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1475850791417_0147 2016-10-26 16:27:51,544 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEvent.EventType: APPLICATION_RESOURCES_CLEANEDUP 2016-10-26 16:27:51,544 DEBUG org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Processing application_1475850791417_0147 of type APPLICATION_RESOURCES_CLEANEDUP 2016-10-26 16:27:51,544 DEBUG org.apache.hadoop.yarn.server.nodemanager.security.NMTokenSecretManagerInNM: Removing application attempts NMToken keys for application application_1475850791417_0147 2016-10-26 16:27:51,544 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1475850791417_0147 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2016-10-26 16:27:51,544 DEBUG org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerAppFinishedEvent.EventType: APPLICATION_FINISHED 2016-10-26 16:27:51,544 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Application just finished : application_1475850791417_0147 2016-10-26 16:27:51,545 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:shfs3453 (auth:SIMPLE) from:org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogWriter.(AggregatedLogFormat.java:378) 2016-10-26 16:27:51,546 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:shfs3453 (auth:SIMPLE) from:org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:331) 2016-10-26 16:27:51,546 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false 2016-10-26 16:27:51,546 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false 2016-10-26 16:27:51,546 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false 2016-10-26 16:27:51,546 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.domain.socket.path = 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.security.SecurityUtil: Acquired token Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.200.23:8020, Ident: (HDFS_DELEGATION_TOKEN token 12521 for shfs3453) 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.hdfs.HAUtil: Mapped HA service delegation token for logical URI hdfs://sandbox to namenode namenode01.bigdata.fr/192.168.200.23:8020 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.security.SecurityUtil: Acquired token Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.200.24:8020, Ident: (HDFS_DELEGATION_TOKEN token 12521 for shfs3453) 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.hdfs.HAUtil: Mapped HA service delegation token for logical URI hdfs://sandbox to namenode namenode02.bigdata.fr/192.168.200.24:8020 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.hdfs.BlockReaderLocal: dfs.domain.socket.path = 2016-10-26 16:27:51,548 DEBUG org.apache.hadoop.io.retry.RetryUtils: multipleLinearRandomRetry = null 2016-10-26 16:27:51,549 DEBUG org.apache.hadoop.ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@40f129db 2016-10-26 16:27:51,550 DEBUG org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil: DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection 2016-10-26 16:27:51,550 DEBUG org.apache.hadoop.ipc.Client: The ping interval is 60000 ms. 2016-10-26 16:27:51,551 DEBUG org.apache.hadoop.ipc.Client: Connecting to namenode01.bigdata.fr/192.168.200.23:8020 2016-10-26 16:27:51,551 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:shfs3453 (auth:SIMPLE) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:724) 2016-10-26 16:27:51,551 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: NEGOTIATE 2016-10-26 16:27:51,553 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: NEGOTIATE auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" challenge: "realm=\"default\",nonce=\"xmRDNpw0jDG+3ydHzSlNfrXuKwwj6jEL0VIj+aWR\",qop=\"auth\",charset=utf-8,algorithm=md5-sess" } auths { method: "KERBEROS" mechanism: "GSSAPI" protocol: "nn" serverId: "namenode01.bigdata.fr" } 2016-10-26 16:27:51,553 DEBUG org.apache.hadoop.security.SaslRpcClient: Get token info proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB info:@org.apache.hadoop.security.token.TokenInfo(value=class org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSelector) 2016-10-26 16:27:51,553 DEBUG org.apache.hadoop.security.SaslRpcClient: Creating SASL DIGEST-MD5(TOKEN) client to authenticate to service at default 2016-10-26 16:27:51,553 DEBUG org.apache.hadoop.security.SaslRpcClient: Use TOKEN authentication for protocol ClientNamenodeProtocolPB 2016-10-26 16:27:51,554 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting username: ABdzaGZzMzQ1M0BTQU5EQk9YLkhBRE9PUAJybQCKAVgBY3KMigFYJW/2jI4w6Y4GHg== 2016-10-26 16:27:51,554 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting userPassword 2016-10-26 16:27:51,554 DEBUG org.apache.hadoop.security.SaslRpcClient: SASL client callback: setting realm: default 2016-10-26 16:27:51,554 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: INITIATE token: "charset=utf-8,username=\"ABdzaGZzMzQ1M0BTQU5EQk9YLkhBRE9PUAJybQCKAVgBY3KMigFYJW/2jI4w6Y4GHg==\",realm=\"default\",nonce=\"xmRDNpw0jDG+3ydHzSlNfrXuKwwj6jEL0VIj+aWR\",nc=00000001,cnonce=\"3NaYZ0TkG6vFVo++5v24l630Kf+I6cI6ZrIY5ASA\",digest-uri=\"/default\",maxbuf=65536,response=02dbb174960ca1f717a7294301c6cb15,qop=auth" auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" } 2016-10-26 16:27:51,556 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: SUCCESS token: "rspauth=33d63471d8fb4d7c938c6aed67a4e8fd" 2016-10-26 16:27:51,556 DEBUG org.apache.hadoop.ipc.Client: Negotiated QOP is :auth 2016-10-26 16:27:51,556 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 sending #531 2016-10-26 16:27:51,556 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453: starting, having connections 2 2016-10-26 16:27:51,558 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 got value #531 2016-10-26 16:27:51,558 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getServerDefaults took 8ms 2016-10-26 16:27:51,558 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 sending #532 2016-10-26 16:27:51,561 DEBUG org.apache.hadoop.ipc.Client: IPC Client (2061583323) connection to namenode01.bigdata.fr/192.168.200.23:8020 from shfs3453 got value #532 2016-10-26 16:27:51,561 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: create took 3ms 2016-10-26 16:27:51,561 DEBUG org.apache.hadoop.hdfs.DFSClient: computePacketChunkSize: src=/Products/YARN/logs/shfs3453/logs/application_1475850791417_0147/datanode03.bigdata.fr_37761.tmp, chunkSize=516, chunksPerPacket=126, packetSize=65016 2016-10-26 16:27:51,562 DEBUG org.apache.hadoop.hdfs.LeaseRenewer: Lease renewer daemon for [DFSClient_NONMAPREDUCE_1408813199_457] with renew id 1 started 2016-10-26 16:27:51,563 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_1475850791417_0147_01_000001. Current good log dirs are /mnt/hd6/hadoop/yarn/log,/mnt/hd8/hadoop/yarn/log,/mnt/hd9/hadoop/yarn/log,/mnt/hd4/hadoop/yarn/log,/mnt/hd5/hadoop/yarn/log,/mnt/hd10/hadoop/yarn/log,/mnt/hd0/hadoop/yarn/log,/mnt/hd2/hadoop/yarn/log,/mnt/hd11/hadoop/yarn/log,/mnt/hd7/hadoop/yarn/log,/mnt/hd3/hadoop/yarn/log,/mnt/hd1/hadoop/yarn/log 2016-10-26 16:27:51,565 DEBUG org.apache.hadoop.ipc.Client: The ping interval is 60000 ms. 2016-10-26 16:27:51,565 DEBUG org.apache.hadoop.ipc.Client: Connecting to namenode01.bigdata.fr/192.168.200.23:8020 2016-10-26 16:27:51,565 DEBUG org.apache.hadoop.security.UserGroupInformation: PrivilegedAction as:nm/datanode03.bigdata.fr@SANDBOX.HADOOP (auth:KERBEROS) from:org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:724) 2016-10-26 16:27:51,566 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: NEGOTIATE 2016-10-26 16:27:51,566 DEBUG org.apache.hadoop.security.SaslRpcClient: Received SASL message state: NEGOTIATE auths { method: "TOKEN" mechanism: "DIGEST-MD5" protocol: "" serverId: "default" challenge: "realm=\"default\",nonce=\"ikeAzxbv8WMRNIxkQVIw0N8MF3d9pXL6JbaKckzp\",qop=\"auth\",charset=utf-8,algorithm=md5-sess" } auths { method: "KERBEROS" mechanism: "GSSAPI" protocol: "nn" serverId: "namenode01.bigdata.fr" } 2016-10-26 16:27:51,566 DEBUG org.apache.hadoop.security.SaslRpcClient: Get token info proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB info:@org.apache.hadoop.security.token.TokenInfo(value=class org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenSelector) 2016-10-26 16:27:51,567 DEBUG org.apache.hadoop.security.SaslRpcClient: Get kerberos info proto:interface org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB info:@org.apache.hadoop.security.KerberosInfo(clientPrincipal=, serverPrincipal=dfs.namenode.kerberos.principal) 2016-10-26 16:27:51,567 DEBUG org.apache.hadoop.security.SaslRpcClient: RPC Server's Kerberos principal name for protocol=org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolPB is nn/namenode01.bigdata.fr@SANDBOX.HADOOP 2016-10-26 16:27:51,567 DEBUG org.apache.hadoop.security.SaslRpcClient: Creating SASL GSSAPI(KERBEROS) client to authenticate to service at namenode01.bigdata.fr 2016-10-26 16:27:51,568 DEBUG org.apache.hadoop.security.SaslRpcClient: Use KERBEROS authentication for protocol ClientNamenodeProtocolPB 2016-10-26 16:27:51,570 DEBUG org.apache.hadoop.security.SaslRpcClient: Sending sasl message state: INITIATE token: "`\202\002\263\006\t*\206H\206\367\022\001\002\002\001\000n\202\002\2420\202\002\236\240\003\002\001\005\241\003\002\001\016\242\a\003\005\000 \000\000\000\243\202\001\222a\202\001\2160\202\001\212\240\003\002\001\005\241\020\033\016SANDBOX.HADOOP\242301\240\003\002\001\000\241*0(\033\002nn\033\"namenode01.bigdata.fr\243\202\001:0\202\0016\240\003\002\001\022\241\003\002\001\v\242\202\001(\004\202\001$\000-