2013-11-27 10:54:41,161 INFO [main] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:setup(51)) - Starting up YARN cluster 2013-11-27 10:54:41,710 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2013-11-27 10:54:42,398 INFO [main] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:rollMasterKey(107)) - Rolling master-key for amrm-tokens 2013-11-27 10:54:42,420 INFO [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:(75)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms 2013-11-27 10:54:42,425 INFO [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:(77)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms 2013-11-27 10:54:42,435 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler 2013-11-27 10:54:42,436 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreOperationFailedEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMStateStoreOperationFailedEventDispatcher 2013-11-27 10:54:42,440 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager 2013-11-27 10:54:42,441 INFO [main] resourcemanager.ResourceManager (ResourceManager.java:createScheduler(242)) - Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler 2013-11-27 10:54:42,537 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher 2013-11-27 10:54:42,539 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher 2013-11-27 10:54:42,540 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher 2013-11-27 10:54:42,542 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher 2013-11-27 10:54:42,658 INFO [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(111)) - loaded properties from hadoop-metrics2.properties 2013-11-27 10:54:42,740 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(341)) - Scheduled snapshot period at 10 second(s). 2013-11-27 10:54:42,740 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(183)) - ResourceManager metrics system started 2013-11-27 10:54:42,846 INFO [main] capacity.ParentQueue (ParentQueue.java:setupQueueConfigs(189)) - root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=SUBMIT_APPLICATIONS:*ADMINISTER_QUEUE:* 2013-11-27 10:54:42,847 INFO [main] capacity.ParentQueue (ParentQueue.java:(151)) - Initialized parent-queue root name=root, fullname=root 2013-11-27 10:54:42,862 INFO [main] capacity.LeafQueue (LeafQueue.java:setupQueueConfigs(271)) - Initializing default capacity = 1.0 [= (float) configuredCapacity / 100 ] asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ] maxCapacity = 1.0 [= configuredMaxCapacity ] absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ] userLimit = 100 [= configuredUserLimit ] userLimitFactor = 1.0 [= configuredUserLimitFactor ] maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)] maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ] maxActiveApplications = 1 [= max((int)ceil((clusterResourceMemory / minimumAllocation) * maxAMResourcePerQueuePercent * absoluteMaxCapacity),1) ] maxActiveAppsUsingAbsCap = 1 [= max((int)ceil((clusterResourceMemory / minimumAllocation) *maxAMResourcePercent * absoluteCapacity),1) ] maxActiveApplicationsPerUser = 1 [= max((int)(maxActiveApplications * (userLimit / 100.0f) * userLimitFactor),1) ] usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)] absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory] maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ] minimumAllocationFactor = 0.984375 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ] numContainers = 0 [= currentNumContainers ] state = RUNNING [= configuredState ] acls = SUBMIT_APPLICATIONS:*ADMINISTER_QUEUE:* [= configuredAcls ] nodeLocalityDelay = 0 2013-11-27 10:54:42,863 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(410)) - Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0 2013-11-27 10:54:42,864 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:parseQueue(410)) - Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2013-11-27 10:54:42,864 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:initializeQueues(308)) - Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, numApps=0, numContainers=0 2013-11-27 10:54:42,864 INFO [main] capacity.CapacityScheduler (CapacityScheduler.java:reinitialize(272)) - Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<>, maximumAllocation=<> 2013-11-27 10:54:42,903 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager 2013-11-27 10:54:42,917 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher 2013-11-27 10:54:42,920 INFO [main] resourcemanager.RMNMInfo (RMNMInfo.java:(63)) - Registered RMNMInfo MBean 2013-11-27 10:54:42,921 INFO [main] util.HostsFileReader (HostsFileReader.java:refresh(84)) - Refreshing hosts (include/exclude) list 2013-11-27 10:54:42,935 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper$2 2013-11-27 10:54:42,937 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(363)) - Created localDir in /Users/aajisaka/git/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/target/TestUnmanagedAMLauncher/TestUnmanagedAMLauncher-localDir-nm-0_0 2013-11-27 10:54:42,937 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:prepareDirs(363)) - Created logDir in /Users/aajisaka/git/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/target/TestUnmanagedAMLauncher/TestUnmanagedAMLauncher-logDir-nm-0_0 2013-11-27 10:54:42,939 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceInit(345)) - Starting NM: 0 2013-11-27 10:54:43,063 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher 2013-11-27 10:54:43,064 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher 2013-11-27 10:54:43,065 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService 2013-11-27 10:54:43,066 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices 2013-11-27 10:54:43,066 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl 2013-11-27 10:54:43,067 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher 2013-11-27 10:54:43,079 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl 2013-11-27 10:54:43,080 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.MiniYARNCluster$CustomNodeManager 2013-11-27 10:54:43,080 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - NodeManager metrics system started (again) 2013-11-27 10:54:43,116 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler 2013-11-27 10:54:43,117 INFO [main] localizer.ResourceLocalizationService (ResourceLocalizationService.java:validateConf(196)) - per directory file limit = 8192 2013-11-27 10:54:43,198 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:register(182)) - Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker 2013-11-27 10:54:43,202 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(98)) - Using ResourceCalculatorPlugin : null 2013-11-27 10:54:43,203 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(103)) - Using ResourceCalculatorProcessTree : null 2013-11-27 10:54:43,204 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(127)) - Physical memory check enabled: false 2013-11-27 10:54:43,204 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:serviceInit(128)) - Virtual memory check enabled: false 2013-11-27 10:54:43,205 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:serviceInit(163)) - Initialized nodemanager for null: physical-memory=4096 virtual-memory=8602 virtual-cores=8 2013-11-27 10:54:43,207 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(248)) - Waiting for RM to start... 2013-11-27 10:54:43,207 INFO [Thread-16] resourcemanager.RMHAProtocolService (RMHAProtocolService.java:transitionToActive(186)) - Transitioning to active 2013-11-27 10:54:43,208 INFO [Thread-16] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:rollMasterKey(103)) - Rolling master-key for container-tokens 2013-11-27 10:54:43,208 INFO [Timer-0] security.AMRMTokenSecretManager (AMRMTokenSecretManager.java:rollMasterKey(107)) - Rolling master-key for amrm-tokens 2013-11-27 10:54:43,212 INFO [Thread-16] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:rollMasterKey(97)) - Rolling master-key for nm-tokens 2013-11-27 10:54:43,278 INFO [Thread-16] http.HttpServer (HttpServer.java:(345)) - SSL is enabled on HttpServer at http://macaa:-1/ - not live 2013-11-27 10:54:43,287 INFO [Thread-16] mortbay.log (Slf4jLog.java:info(67)) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2013-11-27 10:54:43,301 INFO [Thread-16] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(70)) - Http request log for http.requests.resourcemanager is not defined 2013-11-27 10:54:43,374 INFO [Thread-16] http.HttpServer (HttpServer.java:addGlobalFilter(648)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2013-11-27 10:54:43,377 INFO [Thread-16] http.HttpServer (HttpServer.java:addFilter(626)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context cluster 2013-11-27 10:54:43,377 INFO [Thread-16] http.HttpServer (HttpServer.java:addFilter(633)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2013-11-27 10:54:43,380 INFO [Thread-16] http.HttpServer (HttpServer.java:(401)) - adding path spec: /cluster/* 2013-11-27 10:54:43,381 INFO [Thread-16] http.HttpServer (HttpServer.java:(401)) - adding path spec: /ws/* 2013-11-27 10:54:43,389 INFO [Thread-16] http.HttpServer (HttpServer.java:start(823)) - Jetty bound to port 51120 2013-11-27 10:54:43,389 INFO [Thread-16] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 2013-11-27 10:54:43,530 INFO [Thread-16] mortbay.log (Slf4jLog.java:info(67)) - Started SelectChannelConnector@macaa:51120 2013-11-27 10:54:43,530 INFO [Thread-16] webapp.WebApps (WebApps.java:start(258)) - Web app /cluster started at 51120 2013-11-27 10:54:43,932 INFO [Thread-16] webapp.WebApps (WebApps.java:start(272)) - Registered webapp guice modules 2013-11-27 10:54:43,934 INFO [Thread-16] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(223)) - Updating the current master key for generating delegation tokens 2013-11-27 10:54:43,935 INFO [Thread-16] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(85)) - storing master key with keyID 1 2013-11-27 10:54:43,936 INFO [Thread[Thread-22,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(536)) - Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 2013-11-27 10:54:43,936 INFO [Thread[Thread-22,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(223)) - Updating the current master key for generating delegation tokens 2013-11-27 10:54:43,936 INFO [Thread[Thread-22,5,main]] security.RMDelegationTokenSecretManager (RMDelegationTokenSecretManager.java:storeNewMasterKey(85)) - storing master key with keyID 2 2013-11-27 10:54:43,969 INFO [Socket Reader #1 for port 51121] ipc.Server (Server.java:run(554)) - Starting Socket Reader #1 for port 51121 2013-11-27 10:54:43,994 INFO [Thread-16] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceTrackerPB to the server 2013-11-27 10:54:43,994 INFO [IPC Server Responder] ipc.Server (Server.java:run(776)) - IPC Server Responder: starting 2013-11-27 10:54:43,994 INFO [IPC Server listener on 51121] ipc.Server (Server.java:run(623)) - IPC Server listener on 51121: starting 2013-11-27 10:54:44,014 INFO [Socket Reader #1 for port 51122] ipc.Server (Server.java:run(554)) - Starting Socket Reader #1 for port 51122 2013-11-27 10:54:44,019 INFO [Thread-16] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB to the server 2013-11-27 10:54:44,020 INFO [IPC Server Responder] ipc.Server (Server.java:run(776)) - IPC Server Responder: starting 2013-11-27 10:54:44,020 INFO [IPC Server listener on 51122] ipc.Server (Server.java:run(623)) - IPC Server listener on 51122: starting 2013-11-27 10:54:44,045 INFO [Socket Reader #1 for port 51123] ipc.Server (Server.java:run(554)) - Starting Socket Reader #1 for port 51123 2013-11-27 10:54:44,048 INFO [Thread-16] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ApplicationClientProtocolPB to the server 2013-11-27 10:54:44,048 INFO [IPC Server Responder] ipc.Server (Server.java:run(776)) - IPC Server Responder: starting 2013-11-27 10:54:44,048 INFO [IPC Server listener on 51123] ipc.Server (Server.java:run(623)) - IPC Server listener on 51123: starting 2013-11-27 10:54:44,078 INFO [Socket Reader #1 for port 51124] ipc.Server (Server.java:run(554)) - Starting Socket Reader #1 for port 51124 2013-11-27 10:54:44,081 INFO [Thread-16] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB to the server 2013-11-27 10:54:44,081 INFO [IPC Server Responder] ipc.Server (Server.java:run(776)) - IPC Server Responder: starting 2013-11-27 10:54:44,082 INFO [IPC Server listener on 51124] ipc.Server (Server.java:run(623)) - IPC Server listener on 51124: starting 2013-11-27 10:54:44,082 INFO [Thread-16] resourcemanager.RMHAProtocolService (RMHAProtocolService.java:transitionToActive(189)) - Transitioned to active 2013-11-27 10:54:44,708 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(261)) - MiniYARN ResourceManager address: macaa:51123 2013-11-27 10:54:44,709 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(263)) - MiniYARN ResourceManager web address: macaa:51120 2013-11-27 10:54:44,710 INFO [main] server.MiniYARNCluster (MiniYARNCluster.java:serviceStart(380)) - Waiting for NM 0 to start... 2013-11-27 10:54:44,718 INFO [Socket Reader #1 for port 51125] ipc.Server (Server.java:run(554)) - Starting Socket Reader #1 for port 51125 2013-11-27 10:54:44,722 INFO [Thread-192] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server 2013-11-27 10:54:44,722 INFO [Thread-192] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(283)) - Blocking new container-requests as container manager rpc server is still starting. 2013-11-27 10:54:44,723 INFO [IPC Server Responder] ipc.Server (Server.java:run(776)) - IPC Server Responder: starting 2013-11-27 10:54:44,723 INFO [IPC Server listener on 51125] ipc.Server (Server.java:run(623)) - IPC Server listener on 51125: starting 2013-11-27 10:54:44,729 INFO [Thread-192] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setNodeId(189)) - Updating node address : macaa:51125 2013-11-27 10:54:44,729 INFO [Thread-192] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:serviceStart(294)) - ContainerManager started at /192.168.2.36:51125 2013-11-27 10:54:44,734 INFO [Socket Reader #1 for port 51126] ipc.Server (Server.java:run(554)) - Starting Socket Reader #1 for port 51126 2013-11-27 10:54:44,739 INFO [Thread-192] pb.RpcServerFactoryPBImpl (RpcServerFactoryPBImpl.java:createServer(174)) - Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server 2013-11-27 10:54:44,739 INFO [IPC Server Responder] ipc.Server (Server.java:run(776)) - IPC Server Responder: starting 2013-11-27 10:54:44,739 INFO [IPC Server listener on 51126] ipc.Server (Server.java:run(623)) - IPC Server listener on 51126: starting 2013-11-27 10:54:44,741 INFO [Thread-192] localizer.ResourceLocalizationService (ResourceLocalizationService.java:serviceStart(263)) - Localizer started on port 51126 2013-11-27 10:54:44,741 INFO [Thread-192] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 2013-11-27 10:54:44,742 INFO [Thread-192] webapp.WebServer (WebServer.java:serviceStart(60)) - Instantiating NMWebApp at macaa:0 2013-11-27 10:54:44,742 INFO [Thread-192] http.HttpServer (HttpServer.java:(345)) - SSL is enabled on HttpServer at http://macaa:-1/ - not live 2013-11-27 10:54:44,743 INFO [Thread-192] http.HttpRequestLog (HttpRequestLog.java:getRequestLog(70)) - Http request log for http.requests.nodemanager is not defined 2013-11-27 10:54:44,743 INFO [Thread-192] http.HttpServer (HttpServer.java:addGlobalFilter(648)) - Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2013-11-27 10:54:44,744 INFO [Thread-192] http.HttpServer (HttpServer.java:addFilter(626)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node 2013-11-27 10:54:44,745 INFO [Thread-192] http.HttpServer (HttpServer.java:addFilter(633)) - Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2013-11-27 10:54:44,746 INFO [Thread-192] http.HttpServer (HttpServer.java:(401)) - adding path spec: /node/* 2013-11-27 10:54:44,746 INFO [Thread-192] http.HttpServer (HttpServer.java:(401)) - adding path spec: /ws/* 2013-11-27 10:54:44,746 INFO [Thread-192] http.HttpServer (HttpServer.java:start(823)) - Jetty bound to port 51127 2013-11-27 10:54:44,747 INFO [Thread-192] mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26 11 27, 2013 10:54:44 午前 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register 情報: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class 11 27, 2013 10:54:44 午前 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register 情報: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class 11 27, 2013 10:54:44 午前 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register 情報: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class 11 27, 2013 10:54:44 午前 com.sun.jersey.server.impl.application.WebApplicationImpl _initiate 情報: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' 11 27, 2013 10:54:45 午前 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider 情報: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton" 11 27, 2013 10:54:45 午前 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider 情報: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" 2013-11-27 10:54:45,710 INFO [main] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:setup(60)) - MiniYARN ResourceManager published address: macaa:51123 2013-11-27 10:54:45,711 INFO [main] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:setup(62)) - MiniYARN ResourceManager published web address: macaa:51120 2013-11-27 10:54:45,711 INFO [main] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:setup(67)) - Yarn webapp is at macaa:51120 11 27, 2013 10:54:46 午前 com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider 情報: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton" 2013-11-27 10:54:46,113 INFO [Thread-192] mortbay.log (Slf4jLog.java:info(67)) - Started SelectChannelConnector@macaa:51127 2013-11-27 10:54:46,114 INFO [Thread-192] webapp.WebApps (WebApps.java:start(258)) - Web app /node started at 51127 11 27, 2013 10:54:46 午前 com.google.inject.servlet.GuiceFilter setPipeline 警告: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected. 2013-11-27 10:54:46,134 INFO [Thread-192] webapp.WebApps (WebApps.java:start(272)) - Registered webapp guice modules 2013-11-27 10:54:46,141 INFO [Thread-192] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(244)) - Registering with RM using finished containers :[] 2013-11-27 10:54:46,152 INFO [Thread-192] util.RackResolver (RackResolver.java:coreResolve(109)) - Resolved macaa to /default-rack 2013-11-27 10:54:46,157 INFO [Thread-192] resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(294)) - NodeManager from node macaa(cmPort: 51125 httpPort: 51127) registered with capability: , assigned nodeId macaa:51125 2013-11-27 10:54:46,158 INFO [Thread-192] security.NMContainerTokenSecretManager (NMContainerTokenSecretManager.java:setMasterKey(71)) - Rolling master-key for container-tokens, got key with id -1372090981 2013-11-27 10:54:46,158 INFO [Thread-192] security.NMTokenSecretManagerInNM (NMTokenSecretManagerInNM.java:setMasterKey(65)) - Rolling master-key for nm-tokens, got key with id :-1256177644 2013-11-27 10:54:46,159 INFO [Thread-192] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(294)) - Registered with ResourceManager as macaa:51125 with total resource of 2013-11-27 10:54:46,159 INFO [Thread-192] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(296)) - Notifying ContainerManager to unblock new container-requests 2013-11-27 10:54:46,160 INFO [AsyncDispatcher event handler] rmnode.RMNodeImpl (RMNodeImpl.java:handle(384)) - macaa:51125 Node Transitioned from NEW to RUNNING 2013-11-27 10:54:46,162 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addNode(779)) - Added node macaa:51125 clusterResource: 2013-11-27 10:54:47,957 INFO [Thread-230] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:getTestRuntimeClasspath(103)) - Trying to generate classpath for app master from current thread's classpath 2013-11-27 10:54:47,957 INFO [Thread-230] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:testDSShell(139)) - Initializing Launcher 2013-11-27 10:54:48,004 INFO [Thread-230] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:testDSShell(144)) - Running Launcher 2013-11-27 10:54:48,005 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(307)) - Starting Client 2013-11-27 10:54:48,080 INFO [Thread-230] client.RMProxy (RMProxy.java:createRMProxy(56)) - Connecting to ResourceManager at macaa/192.168.2.36:51123 2013-11-27 10:54:48,086 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(313)) - Setting up application submission context for ASM 2013-11-27 10:54:48,273 INFO [IPC Server handler 0 on 51123] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(210)) - Allocated new applicationId: 1 2013-11-27 10:54:48,291 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(336)) - Setting unmanaged AM 2013-11-27 10:54:48,291 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(339)) - Submitting application to ASM 2013-11-27 10:54:48,311 WARN [IPC Server handler 1 on 51123] rmapp.RMAppImpl (RMAppImpl.java:(293)) - The specific max attempts: 0 for application: 1 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2013-11-27 10:54:48,312 INFO [IPC Server handler 1 on 51123] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(323)) - Application with id 1 submitted by user aajisaka 2013-11-27 10:54:48,312 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(772)) - Storing application with id application_1385517283208_0001 2013-11-27 10:54:48,314 INFO [IPC Server handler 1 on 51123] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(140)) - USER=aajisaka IP=192.168.2.36 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1385517283208_0001 2013-11-27 10:54:48,314 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(598)) - application_1385517283208_0001 State change from NEW to NEW_SAVING 2013-11-27 10:54:48,322 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(552)) - Storing info for app: application_1385517283208_0001 2013-11-27 10:54:48,357 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(598)) - application_1385517283208_0001 State change from NEW_SAVING to SUBMITTED 2013-11-27 10:54:48,357 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(542)) - Registering app attempt : appattempt_1385517283208_0001_000001 2013-11-27 10:54:48,363 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(646)) - appattempt_1385517283208_0001_000001 State change from NEW to SUBMITTED 2013-11-27 10:54:48,380 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(716)) - Application application_1385517283208_0001 from user: aajisaka activated in queue: default 2013-11-27 10:54:48,380 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplication(732)) - Application added - appId: application_1385517283208_0001 user: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User@3ad8d1f9, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 1 #queue-pending-applications: 0 #queue-active-applications: 1 2013-11-27 10:54:48,380 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(480)) - Application added - appId: application_1385517283208_0001 user: aajisaka leaf-queue of parent: root #applications: 1 2013-11-27 10:54:48,380 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(458)) - Application Submission: appattempt_1385517283208_0001_000001, user: aajisaka queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0, currently active: 1 2013-11-27 10:54:48,381 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(1550)) - Storing attempt: AppId: application_1385517283208_0001 AttemptId: appattempt_1385517283208_0001_000001 MasterContainer: null 2013-11-27 10:54:48,382 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(646)) - appattempt_1385517283208_0001_000001 State change from SUBMITTED to LAUNCHED_UNMANAGED_SAVING 2013-11-27 10:54:48,393 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(646)) - appattempt_1385517283208_0001_000001 State change from LAUNCHED_UNMANAGED_SAVING to LAUNCHED 2013-11-27 10:54:48,393 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(598)) - application_1385517283208_0001 State change from SUBMITTED to ACCEPTED 2013-11-27 10:54:49,348 INFO [Thread-230] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(174)) - Submitted application application_1385517283208_0001 to ResourceManager at macaa/192.168.2.36:51123 2013-11-27 10:54:50,352 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:50,352 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(346)) - Launching application with id: appattempt_1385517283208_0001_000001 Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8 2013-11-27 10:54:50,729 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:main(244)) - Initializing ApplicationMaster 2013-11-27 10:54:50,750 FATAL [main] distributedshell.ApplicationMaster (ApplicationMaster.java:main(251)) - Error running ApplicationMaster org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: --shell_command at org.apache.commons.cli.Parser.processOption(Parser.java:363) at org.apache.commons.cli.Parser.parse(Parser.java:199) at org.apache.commons.cli.Parser.parse(Parser.java:85) at org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:325) at org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:245) 2013-11-27 10:54:50,760 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:launchAM(282)) - AM process exited with value: 1 2013-11-27 10:54:51,766 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:52,770 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:53,774 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:54,778 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:55,785 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:56,789 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:57,792 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:58,797 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:54:59,801 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:00,805 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:01,812 INFO [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=1, appAttemptId=appattempt_1385517283208_0001_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517288311, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:01,812 WARN [Thread-230] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(442)) - Waited 10 seconds after process completed for AppReport to reach desired final state. Not waiting anymore.CurrentState = ACCEPTED, ExpectedStates = FINISHED,FAILED,KILLED 2013-11-27 10:55:01,928 INFO [Thread-235] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:getTestRuntimeClasspath(103)) - Trying to generate classpath for app master from current thread's classpath 2013-11-27 10:55:01,929 INFO [Thread-235] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:testDSShellError(173)) - Initializing Launcher 2013-11-27 10:55:01,949 INFO [Thread-235] unmanagedamlauncher.TestUnmanagedAMLauncher (TestUnmanagedAMLauncher.java:testDSShellError(178)) - Running Launcher 2013-11-27 10:55:01,949 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(307)) - Starting Client 2013-11-27 10:55:01,951 INFO [Thread-235] client.RMProxy (RMProxy.java:createRMProxy(56)) - Connecting to ResourceManager at macaa/192.168.2.36:51123 2013-11-27 10:55:01,951 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(313)) - Setting up application submission context for ASM 2013-11-27 10:55:01,955 INFO [IPC Server handler 18 on 51123] resourcemanager.ClientRMService (ClientRMService.java:getNewApplicationId(210)) - Allocated new applicationId: 2 2013-11-27 10:55:01,956 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(336)) - Setting unmanaged AM 2013-11-27 10:55:01,956 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(339)) - Submitting application to ASM 2013-11-27 10:55:01,958 WARN [IPC Server handler 19 on 51123] rmapp.RMAppImpl (RMAppImpl.java:(293)) - The specific max attempts: 0 for application: 2 is invalid, because it is out of the range [1, 2]. Use the global max attempts instead. 2013-11-27 10:55:01,959 INFO [IPC Server handler 19 on 51123] resourcemanager.ClientRMService (ClientRMService.java:submitApplication(323)) - Application with id 2 submitted by user aajisaka 2013-11-27 10:55:01,959 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:transition(772)) - Storing application with id application_1385517283208_0002 2013-11-27 10:55:01,959 INFO [IPC Server handler 19 on 51123] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(140)) - USER=aajisaka IP=192.168.2.36 OPERATION=Submit Application Request TARGET=ClientRMService RESULT=SUCCESS APPID=application_1385517283208_0002 2013-11-27 10:55:01,959 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(598)) - application_1385517283208_0002 State change from NEW to NEW_SAVING 2013-11-27 10:55:01,960 INFO [AsyncDispatcher event handler] recovery.RMStateStore (RMStateStore.java:handleStoreEvent(552)) - Storing info for app: application_1385517283208_0002 2013-11-27 10:55:01,961 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(598)) - application_1385517283208_0002 State change from NEW_SAVING to SUBMITTED 2013-11-27 10:55:01,961 INFO [AsyncDispatcher event handler] resourcemanager.ApplicationMasterService (ApplicationMasterService.java:registerAppAttempt(542)) - Registering app attempt : appattempt_1385517283208_0002_000001 2013-11-27 10:55:01,961 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(646)) - appattempt_1385517283208_0002_000001 State change from NEW to SUBMITTED 2013-11-27 10:55:01,962 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:activateApplications(716)) - Application application_1385517283208_0002 from user: aajisaka activated in queue: default 2013-11-27 10:55:01,962 INFO [ResourceManager Event Processor] capacity.LeafQueue (LeafQueue.java:addApplication(732)) - Application added - appId: application_1385517283208_0002 user: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue$User@3ad8d1f9, leaf-queue: default #user-pending-applications: 0 #user-active-applications: 2 #queue-pending-applications: 0 #queue-active-applications: 2 2013-11-27 10:55:01,963 INFO [ResourceManager Event Processor] capacity.ParentQueue (ParentQueue.java:addApplication(480)) - Application added - appId: application_1385517283208_0002 user: aajisaka leaf-queue of parent: root #applications: 2 2013-11-27 10:55:01,963 INFO [ResourceManager Event Processor] capacity.CapacityScheduler (CapacityScheduler.java:addApplication(458)) - Application Submission: appattempt_1385517283208_0002_000001, user: aajisaka queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=2, numContainers=0, currently active: 2 2013-11-27 10:55:01,963 INFO [Thread-235] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(174)) - Submitted application application_1385517283208_0002 to ResourceManager at macaa/192.168.2.36:51123 2013-11-27 10:55:01,963 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:storeAttempt(1550)) - Storing attempt: AppId: application_1385517283208_0002 AttemptId: appattempt_1385517283208_0002_000001 MasterContainer: null 2013-11-27 10:55:01,964 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(646)) - appattempt_1385517283208_0002_000001 State change from SUBMITTED to LAUNCHED_UNMANAGED_SAVING 2013-11-27 10:55:01,964 INFO [AsyncDispatcher event handler] attempt.RMAppAttemptImpl (RMAppAttemptImpl.java:handle(646)) - appattempt_1385517283208_0002_000001 State change from LAUNCHED_UNMANAGED_SAVING to LAUNCHED 2013-11-27 10:55:01,964 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(598)) - application_1385517283208_0002 State change from SUBMITTED to ACCEPTED 2013-11-27 10:55:02,967 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:02,968 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:run(346)) - Launching application with id: appattempt_1385517283208_0002_000001 Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8 2013-11-27 10:55:03,862 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:main(244)) - Initializing ApplicationMaster 2013-11-27 10:55:04,244 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:init(385)) - Application master for app, appId=2, clustertimestamp=1385517283208, attemptId=1 2013-11-27 10:55:04,245 FATAL [main] distributedshell.ApplicationMaster (ApplicationMaster.java:main(251)) - Error running ApplicationMaster java.lang.IllegalArgumentException: No shell command specified to be executed by application master at org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:391) at org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:245) 2013-11-27 10:55:04,270 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:launchAM(282)) - AM process exited with value: 1 2013-11-27 10:55:05,278 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:06,284 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:07,287 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:08,290 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:09,294 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:10,297 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:11,301 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:12,305 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:13,308 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:14,312 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:15,318 INFO [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(418)) - Got application report from ASM for, appId=2, appAttemptId=appattempt_1385517283208_0002_000001, clientToAMToken=null, appDiagnostics=, appMasterHost=N/A, appQueue=default, appMasterRpcPort=0, appStartTime=1385517301958, yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=aajisaka 2013-11-27 10:55:15,318 WARN [Thread-235] unmanagedamlauncher.UnmanagedAMLauncher (UnmanagedAMLauncher.java:monitorApplication(442)) - Waited 10 seconds after process completed for AppReport to reach desired final state. Not waiting anymore.CurrentState = ACCEPTED, ExpectedStates = FINISHED,FAILED,KILLED 2013-11-27 10:55:15,430 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped SelectChannelConnector@macaa:0 2013-11-27 10:55:15,532 INFO [main] ipc.Server (Server.java:stop(2341)) - Stopping server on 51125 2013-11-27 10:55:15,535 INFO [IPC Server listener on 51125] ipc.Server (Server.java:run(655)) - Stopping IPC Server listener on 51125 2013-11-27 10:55:15,536 INFO [IPC Server Responder] ipc.Server (Server.java:run(781)) - Stopping IPC Server Responder 2013-11-27 10:55:15,536 INFO [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled. 2013-11-27 10:55:15,539 INFO [main] ipc.Server (Server.java:stop(2341)) - Stopping server on 51126 2013-11-27 10:55:15,540 INFO [IPC Server listener on 51126] ipc.Server (Server.java:run(655)) - Stopping IPC Server listener on 51126 2013-11-27 10:55:15,540 INFO [IPC Server Responder] ipc.Server (Server.java:run(781)) - Stopping IPC Server Responder 2013-11-27 10:55:15,541 INFO [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(728)) - Public cache exiting 2013-11-27 10:55:15,541 INFO [main] resourcemanager.RMHAProtocolService (RMHAProtocolService.java:transitionToStandby(219)) - Transitioning to standby 2013-11-27 10:55:15,542 INFO [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped SelectChannelConnector@macaa:0 2013-11-27 10:55:15,644 ERROR [Thread[Thread-22,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(557)) - InterruptedExcpetion recieved for ExpiredTokenRemover thread java.lang.InterruptedException: sleep interrupted 2013-11-27 10:55:15,645 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(200)) - Stopping NodeManager metrics system... 2013-11-27 10:55:15,647 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(206)) - NodeManager metrics system stopped. 2013-11-27 10:55:15,648 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(567)) - NodeManager metrics system shutdown complete. 2013-11-27 10:55:15,648 INFO [main] event.AsyncDispatcher (AsyncDispatcher.java:serviceStop(127)) - AsyncDispatcher is draining to stop, igonring any new events. 2013-11-27 10:55:15,649 WARN [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(98)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning. 2013-11-27 10:55:15,649 INFO [main] ipc.Server (Server.java:stop(2341)) - Stopping server on 51124 2013-11-27 10:55:15,650 INFO [IPC Server listener on 51124] ipc.Server (Server.java:run(655)) - Stopping IPC Server listener on 51124 2013-11-27 10:55:15,650 INFO [IPC Server Responder] ipc.Server (Server.java:run(781)) - Stopping IPC Server Responder 2013-11-27 10:55:15,650 INFO [main] ipc.Server (Server.java:stop(2341)) - Stopping server on 51123 2013-11-27 10:55:15,654 INFO [IPC Server listener on 51123] ipc.Server (Server.java:run(655)) - Stopping IPC Server listener on 51123 2013-11-27 10:55:15,654 INFO [IPC Server Responder] ipc.Server (Server.java:run(781)) - Stopping IPC Server Responder 2013-11-27 10:55:15,654 INFO [main] ipc.Server (Server.java:stop(2341)) - Stopping server on 51122 2013-11-27 10:55:15,662 INFO [main] ipc.Server (Server.java:stop(2341)) - Stopping server on 51121 2013-11-27 10:55:15,689 INFO [IPC Server Responder] ipc.Server (Server.java:run(781)) - Stopping IPC Server Responder 2013-11-27 10:55:15,689 INFO [IPC Server listener on 51122] ipc.Server (Server.java:run(655)) - Stopping IPC Server listener on 51122 2013-11-27 10:55:15,696 INFO [IPC Server listener on 51121] ipc.Server (Server.java:run(655)) - Stopping IPC Server listener on 51121 2013-11-27 10:55:15,697 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(591)) - Returning, interrupted : java.lang.InterruptedException 2013-11-27 10:55:15,697 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - NMLivelinessMonitor thread interrupted 2013-11-27 10:55:15,699 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - AMLivelinessMonitor thread interrupted 2013-11-27 10:55:15,699 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted 2013-11-27 10:55:15,697 INFO [IPC Server Responder] ipc.Server (Server.java:run(781)) - Stopping IPC Server Responder 2013-11-27 10:55:15,700 INFO [main] resourcemanager.RMHAProtocolService (RMHAProtocolService.java:transitionToStandby(227)) - Transitioned to standby 2013-11-27 10:55:15,699 INFO [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - AMLivelinessMonitor thread interrupted