Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: 2.1.0-beta
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Target Version/s:

      Description

      509 2013-07-19 15:53:55,569 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 54313: readAndProcess from client 127.0.0.1 threw exception [org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN]]
      510 org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN]
      511 at org.apache.hadoop.ipc.Server$Connection.initializeAuthContext(Server.java:1531)
      512 at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1482)
      513 at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:788)
      514 at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:587)
      515 at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:562)

      1. YARN-945-20130729.txt
        34 kB
        Vinod Kumar Vavilapalli
      2. YARN-945-20130728.txt
        29 kB
        Vinod Kumar Vavilapalli
      3. yarn-site.xml
        4 kB
        Bikas Saha
      4. rm.log
        97 kB
        Bikas Saha
      5. nm.log
        136 kB
        Bikas Saha

        Issue Links

          Activity

          Bikas Saha created issue -
          Bikas Saha made changes -
          Field Original Value New Value
          Attachment nm.log [ 12593288 ]
          Attachment rm.log [ 12593289 ]
          Attachment yarn-site.xml [ 12593290 ]
          Hide
          Bikas Saha added a comment -

          This happens for TEZ and MR AMs on single node cluster. Repro on a different machine too.

          Show
          Bikas Saha added a comment - This happens for TEZ and MR AMs on single node cluster. Repro on a different machine too.
          Hide
          Hitesh Shah added a comment -

          am log has:

          2013-07-19 17:26:28,837 DEBUG [main] org.apache.hadoop.service.AbstractService: noteFailure org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN]
          2013-07-19 17:26:28,837 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN]
          org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN]
                  at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
                  at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:107)
                  at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:213)
                  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
                  at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:789)
                  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
                  at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
                  at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1019)
                  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
                  at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1401)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at javax.security.auth.Subject.doAs(Subject.java:396)
                  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1493)
                  at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1397)
                  at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1330)
          Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN]
                  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
                  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
                  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
                  at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
                  at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
                  at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104)
                  at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109)
                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                  at java.lang.reflect.Method.invoke(Method.java:597)
                  at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:175)
                  at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:94)
                  at com.sun.proxy.$Proxy29.registerApplicationMaster(Unknown Source)
                  at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:147)
                  ... 14 more
          Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled.  Available:[TOKEN]
                  at org.apache.hadoop.ipc.Client.call(Client.java:1433)
                  at org.apache.hadoop.ipc.Client.call(Client.java:1385)
                  at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
                  at com.sun.proxy.$Proxy28.registerApplicationMaster(Unknown Source)
                  at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106)
          
          
          Show
          Hitesh Shah added a comment - am log has: 2013-07-19 17:26:28,837 DEBUG [main] org.apache.hadoop.service.AbstractService: noteFailure org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN] 2013-07-19 17:26:28,837 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN] org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN] at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159) at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:107) at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:213) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:789) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1019) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1401) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1493) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1397) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1330) Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53) at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:175) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:94) at com.sun.proxy.$Proxy29.registerApplicationMaster(Unknown Source) at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:147) ... 14 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled. Available:[TOKEN] at org.apache.hadoop.ipc.Client.call(Client.java:1433) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206) at com.sun.proxy.$Proxy28.registerApplicationMaster(Unknown Source) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106)
          Hide
          Omkar Vinit Joshi added a comment -

          thanks Vinod Kumar Vavilapalli .. does this help? Should we really fix this ....
          Setting conf parameter

          Show
          Omkar Vinit Joshi added a comment - thanks Vinod Kumar Vavilapalli .. does this help? Should we really fix this .... Setting conf parameter
          Hide
          Bikas Saha added a comment -

          Pasting comment from YARN-944 to this jira where its relevant.

          By default resolve ip is set to true...
          boolean useIp = conf.getBoolean(
          CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP,
          CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP_DEFAULT);
          setTokenServiceUseIp(useIp);
          can you try setting below parameter?
          <property>
          <name>yarn.resourcemanager.scheduler.address</name>
          <value>localhost:54313</value> <!-- your resource manager scheduler address -->
          <description>host is the hostname of the resourcemanager and port is the port
          on which the Applications in the cluster talk to the Resource Manager.
          </description>
          </property>
          I don't know whether we need to fix this..

          Show
          Bikas Saha added a comment - Pasting comment from YARN-944 to this jira where its relevant. By default resolve ip is set to true... boolean useIp = conf.getBoolean( CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP, CommonConfigurationKeys.HADOOP_SECURITY_TOKEN_SERVICE_USE_IP_DEFAULT); setTokenServiceUseIp(useIp); can you try setting below parameter? <property> <name>yarn.resourcemanager.scheduler.address</name> <value>localhost:54313</value> <!-- your resource manager scheduler address --> <description>host is the hostname of the resourcemanager and port is the port on which the Applications in the cluster talk to the Resource Manager. </description> </property> I don't know whether we need to fix this..
          Hide
          Bikas Saha added a comment -

          I have tried with the above setting. It works. However, we need to verify if this needs to be fixed. Its common practice to use 0.0.0.0 instead of localhost in the settings. In fact, its the default value in yarn-defaults.xml as of today. Ideally it should just work. If we decide to not do so then we should change yarn-defaults.xml and also document this extremely clearly.

          Show
          Bikas Saha added a comment - I have tried with the above setting. It works. However, we need to verify if this needs to be fixed. Its common practice to use 0.0.0.0 instead of localhost in the settings. In fact, its the default value in yarn-defaults.xml as of today. Ideally it should just work. If we decide to not do so then we should change yarn-defaults.xml and also document this extremely clearly.
          Hide
          Bikas Saha added a comment -

          Making this a blocker because there have been multiple reports of user setups failing. In the minimum, yarn-defaults.xml should be changed to use localhost for RM instead of 0.0.0.0.

          Show
          Bikas Saha added a comment - Making this a blocker because there have been multiple reports of user setups failing. In the minimum, yarn-defaults.xml should be changed to use localhost for RM instead of 0.0.0.0.
          Bikas Saha made changes -
          Priority Major [ 3 ] Blocker [ 1 ]
          Alejandro Abdelnur made changes -
          Fix Version/s 2.1.0-beta [ 12324029 ]
          Affects Version/s 2.1.0-beta [ 12324029 ]
          Target Version/s 2.1.0-beta [ 12324029 ]
          Hide
          Alejandro Abdelnur added a comment -

          Bikas Saha 0.0.0.0 will listen in ALL interfaces, localhost will listen only on localhost. So I think we should not change it to localhost. Instead, we should get the receiver of the token to set the service using the address used for the RPC.

          Show
          Alejandro Abdelnur added a comment - Bikas Saha 0.0.0.0 will listen in ALL interfaces, localhost will listen only on localhost. So I think we should not change it to localhost. Instead, we should get the receiver of the token to set the service using the address used for the RPC.
          Hide
          Daryn Sharp added a comment -

          Please be sure I get a chance to look at the patch.

          Show
          Daryn Sharp added a comment - Please be sure I get a chance to look at the patch.
          Vinod Kumar Vavilapalli made changes -
          Link This issue is duplicated by MAPREDUCE-5426 [ MAPREDUCE-5426 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Here's a patch that fixes this

          • Removes service-set for AMRMToken on the server side completely
          • Changes ClientRMProxy to set the token-service address with the RM bind-address that it intends to connect to. Once we have FailOverProviders for HA, this code will likely move there.
          • Did some minor code style updates to RMProxy and ServerRMProxy
          • Fixed recovery code to store and retrieve AMRMToken based on a constant. This will have to be changed in future to some kind of a service-ID/cluster-ID
          • Fixed tests that were failing before the patch because of multiple local addresses on my Mac. They pass after the patch.

          I can run jobs successfully with 0.0.0.0 as bind-addresses on a single node cluster.

          Not that this bug was always the case (since MAPREDUCE-4162) with single node secure installs. YARN-701 just made it common even for non-secure install.

          Show
          Vinod Kumar Vavilapalli added a comment - Here's a patch that fixes this Removes service-set for AMRMToken on the server side completely Changes ClientRMProxy to set the token-service address with the RM bind-address that it intends to connect to. Once we have FailOverProviders for HA, this code will likely move there. Did some minor code style updates to RMProxy and ServerRMProxy Fixed recovery code to store and retrieve AMRMToken based on a constant. This will have to be changed in future to some kind of a service-ID/cluster-ID Fixed tests that were failing before the patch because of multiple local addresses on my Mac. They pass after the patch. I can run jobs successfully with 0.0.0.0 as bind-addresses on a single node cluster. Not that this bug was always the case (since MAPREDUCE-4162 ) with single node secure installs. YARN-701 just made it common even for non-secure install.
          Vinod Kumar Vavilapalli made changes -
          Attachment YARN-945-20130728.txt [ 12594608 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12594608/YARN-945-20130728.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.client.api.impl.TestNMClient

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1598//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1598//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12594608/YARN-945-20130728.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.client.api.impl.TestNMClient +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1598//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1598//console This message is automatically generated.
          Hide
          Bikas Saha added a comment -

          Looks good overall.

          To be clear, this works for cases in which the UGI already contains the token and then, when we first initialize the protocol proxy via the ClientRMProxy, the ClientRMProxy layer adds the correct service addr to the token. Is that correct? The token is obtained by either the client (unmanaged AM) or set directly by the RM (managed AM) and gets populated in the AM's UGI before the AM creates the AM->RM proxy. When the AM->RM proxy is created then the service is initialized in the token by ClientRMProxy.
          In that case, does YARNClientImpl need to do setTokenService (with static RM address lookup) on the token in YarnClient.getAMRMToken()? The unmanagedAMLauncher or other user of YarnClient.getAMRMToken() needs to simply populate the AM's UGI correctly without doing any translation right? In that case, can we remove the YarnClient side translation code

          Could you please open jiras for the TODO's so that we dont lose them. And refer the jira numbers in the TODO's.

          I wish we could leverage RMProxy or something like that in the tests instead of having to replicate the setTokenService() code in so many places. Can leave it for a separate jira.

          Show
          Bikas Saha added a comment - Looks good overall. To be clear, this works for cases in which the UGI already contains the token and then, when we first initialize the protocol proxy via the ClientRMProxy, the ClientRMProxy layer adds the correct service addr to the token. Is that correct? The token is obtained by either the client (unmanaged AM) or set directly by the RM (managed AM) and gets populated in the AM's UGI before the AM creates the AM->RM proxy. When the AM->RM proxy is created then the service is initialized in the token by ClientRMProxy. In that case, does YARNClientImpl need to do setTokenService (with static RM address lookup) on the token in YarnClient.getAMRMToken()? The unmanagedAMLauncher or other user of YarnClient.getAMRMToken() needs to simply populate the AM's UGI correctly without doing any translation right? In that case, can we remove the YarnClient side translation code Could you please open jiras for the TODO's so that we dont lose them. And refer the jira numbers in the TODO's. I wish we could leverage RMProxy or something like that in the tests instead of having to replicate the setTokenService() code in so many places. Can leave it for a separate jira.
          Vinod Kumar Vavilapalli made changes -
          Link This issue relates to YARN-986 [ YARN-986 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          The unmanagedAMLauncher or other user of YarnClient.getAMRMToken() needs to simply populate the AM's UGI correctly without doing any translation right? In that case, can we remove the YarnClient side translation code

          That's correct. Removing it.

          Could you please open jiras for the TODO's so that we dont lose them. And refer the jira numbers in the TODO's.

          Done. YARN-986.

          I wish we could leverage RMProxy or something like that in the tests instead of having to replicate the setTokenService() code in so many places. Can leave it for a separate jira.

          I tried a lot but RMProxy/ClientRMProxy dependency graphs didn't let me do it easily. IMO, We can correctly fix it correctly once we do fail-over etc.

          Show
          Vinod Kumar Vavilapalli added a comment - The unmanagedAMLauncher or other user of YarnClient.getAMRMToken() needs to simply populate the AM's UGI correctly without doing any translation right? In that case, can we remove the YarnClient side translation code That's correct. Removing it. Could you please open jiras for the TODO's so that we dont lose them. And refer the jira numbers in the TODO's. Done. YARN-986 . I wish we could leverage RMProxy or something like that in the tests instead of having to replicate the setTokenService() code in so many places. Can leave it for a separate jira. I tried a lot but RMProxy/ClientRMProxy dependency graphs didn't let me do it easily. IMO, We can correctly fix it correctly once we do fail-over etc.
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Updated patch addressing review comments.

          Show
          Vinod Kumar Vavilapalli added a comment - Updated patch addressing review comments.
          Vinod Kumar Vavilapalli made changes -
          Attachment YARN-945-20130729.txt [ 12594734 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Bikas Saha made changes -
          Link This issue is related to YARN-701 [ YARN-701 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12594734/YARN-945-20130729.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

          org.apache.hadoop.yarn.client.api.impl.TestNMClient
          org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1603//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1603//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12594734/YARN-945-20130729.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: org.apache.hadoop.yarn.client.api.impl.TestNMClient org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1603//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1603//console This message is automatically generated.
          Hide
          Bikas Saha added a comment -

          From what I see, your patch makes it fairly straightforward to add a YARNConfiguration.ClusterName that can be used as the amrmtoken's service at the RM. Then the state store can use it as the lookup key instead of its own key. And ClientRMProxy can use that to find the amrmtoken from the UGI and add a new entry with the correct token service. We can do that via YARN-986.
          +1 pending Jenkins.

          Show
          Bikas Saha added a comment - From what I see, your patch makes it fairly straightforward to add a YARNConfiguration.ClusterName that can be used as the amrmtoken's service at the RM. Then the state store can use it as the lookup key instead of its own key. And ClientRMProxy can use that to find the amrmtoken from the UGI and add a new entry with the correct token service. We can do that via YARN-986 . +1 pending Jenkins.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          The test issue with TestAMAuthorization is being fixed at YARN-961. TestNMClient failure is tracked at YARN-906.

          Show
          Vinod Kumar Vavilapalli added a comment - The test issue with TestAMAuthorization is being fixed at YARN-961 . TestNMClient failure is tracked at YARN-906 .
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Rekicking Jenkins after YARN-961.

          Show
          Vinod Kumar Vavilapalli added a comment - Rekicking Jenkins after YARN-961 .
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12594734/YARN-945-20130729.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1610//testReport/
          Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1610//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12594734/YARN-945-20130729.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 5 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/1610//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1610//console This message is automatically generated.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Committed this to trunk, branch-2 and branch-2.1.

          Tx for the reviews, Bikas Saha!

          Show
          Vinod Kumar Vavilapalli added a comment - Committed this to trunk, branch-2 and branch-2.1. Tx for the reviews, Bikas Saha !
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Bikas Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development