Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20659

YARNSessionCapacitySchedulerITCase.perJobYarnClusterOffHeap test failed with NPE

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.11.0, 1.12.0, 1.13.0
    • Fix Version/s: None
    • Component/s: Deployment / YARN
    • Labels:

      Description

      https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10989&view=logs&j=fc5181b0-e452-5c8f-68de-1097947f6483&t=6b04ca5f-0b52-511d-19c9-52bf0d9fbdfa

      2020-12-17T22:57:58.1994352Z Test perJobYarnClusterOffHeap(org.apache.flink.yarn.YARNSessionCapacitySchedulerITCase) failed with:
      2020-12-17T22:57:58.1994893Z java.lang.NullPointerException: java.lang.NullPointerException
      2020-12-17T22:57:58.1995439Z 	at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptMetrics.getAggregateAppResourceUsage(RMAppAttemptMetrics.java:128)
      2020-12-17T22:57:58.1996185Z 	at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.getApplicationResourceUsageReport(RMAppAttemptImpl.java:900)
      2020-12-17T22:57:58.1996919Z 	at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.createAndGetApplicationReport(RMAppImpl.java:660)
      2020-12-17T22:57:58.1997526Z 	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplications(ClientRMService.java:930)
      2020-12-17T22:57:58.1998193Z 	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplications(ApplicationClientProtocolPBServiceImpl.java:273)
      2020-12-17T22:57:58.1998960Z 	at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:507)
      2020-12-17T22:57:58.1999876Z 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
      2020-12-17T22:57:58.2000346Z 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
      2020-12-17T22:57:58.2000744Z 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
      2020-12-17T22:57:58.2001532Z 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
      2020-12-17T22:57:58.2001915Z 	at java.security.AccessController.doPrivileged(Native Method)
      2020-12-17T22:57:58.2002286Z 	at javax.security.auth.Subject.doAs(Subject.java:422)
      2020-12-17T22:57:58.2002734Z 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
      2020-12-17T22:57:58.2003185Z 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
      2020-12-17T22:57:58.2003447Z 
      2020-12-17T22:57:58.2003708Z 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      2020-12-17T22:57:58.2004233Z 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      2020-12-17T22:57:58.2004810Z 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      2020-12-17T22:57:58.2005468Z 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      2020-12-17T22:57:58.2005907Z 	at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
      2020-12-17T22:57:58.2006387Z 	at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
      2020-12-17T22:57:58.2006920Z 	at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
      2020-12-17T22:57:58.2007515Z 	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplications(ApplicationClientProtocolPBClientImpl.java:291)
      2020-12-17T22:57:58.2008082Z 	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
      2020-12-17T22:57:58.2008518Z 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      2020-12-17T22:57:58.2008964Z 	at java.lang.reflect.Method.invoke(Method.java:498)
      2020-12-17T22:57:58.2009430Z 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
      2020-12-17T22:57:58.2010002Z 	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
      2020-12-17T22:57:58.2010554Z 	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
      2020-12-17T22:57:58.2011301Z 	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
      2020-12-17T22:57:58.2011857Z 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
      2020-12-17T22:57:58.2012288Z 	at com.sun.proxy.$Proxy111.getApplications(Unknown Source)
      2020-12-17T22:57:58.2012739Z 	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplications(YarnClientImpl.java:528)
      2020-12-17T22:57:58.2013298Z 	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplications(YarnClientImpl.java:505)
      2020-12-17T22:57:58.2013829Z 	at org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:270)
      2020-12-17T22:57:58.2014312Z 	at org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:262)
      2020-12-17T22:57:58.2014879Z 	at org.apache.flink.yarn.YARNSessionCapacitySchedulerITCase.perJobYarnClusterOffHeap(YARNSessionCapacitySchedulerITCase.java:205)
      2020-12-17T22:57:58.2015399Z 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      2020-12-17T22:57:58.2015825Z 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      2020-12-17T22:57:58.2016341Z 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      2020-12-17T22:57:58.2016825Z 	at java.lang.reflect.Method.invoke(Method.java:498)
      2020-12-17T22:57:58.2017268Z 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      2020-12-17T22:57:58.2017776Z 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      2020-12-17T22:57:58.2018358Z 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      2020-12-17T22:57:58.2018857Z 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      2020-12-17T22:57:58.2019340Z 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      2020-12-17T22:57:58.2019808Z 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      2020-12-17T22:57:58.2020275Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      2020-12-17T22:57:58.2020729Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      2020-12-17T22:57:58.2021350Z 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
      2020-12-17T22:57:58.2021755Z 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
      2020-12-17T22:57:58.2022149Z 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      2020-12-17T22:57:58.2022598Z 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      2020-12-17T22:57:58.2023100Z 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      2020-12-17T22:57:58.2023540Z 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      2020-12-17T22:57:58.2024038Z 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      2020-12-17T22:57:58.2024514Z 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      2020-12-17T22:57:58.2024944Z 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      2020-12-17T22:57:58.2025360Z 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      2020-12-17T22:57:58.2025815Z 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      2020-12-17T22:57:58.2026293Z 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      2020-12-17T22:57:58.2026789Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      2020-12-17T22:57:58.2027249Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
      2020-12-17T22:57:58.2027665Z 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
      2020-12-17T22:57:58.2028046Z 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      2020-12-17T22:57:58.2028499Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
      2020-12-17T22:57:58.2029013Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
      2020-12-17T22:57:58.2029532Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
      2020-12-17T22:57:58.2030050Z 	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
      2020-12-17T22:57:58.2030589Z 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
      2020-12-17T22:57:58.2031381Z 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
      2020-12-17T22:57:58.2031904Z 	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
      2020-12-17T22:57:58.2032377Z 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
      2020-12-17T22:57:58.2032893Z Caused by: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException
      2020-12-17T22:57:58.2033563Z 	at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptMetrics.getAggregateAppResourceUsage(RMAppAttemptMetrics.java:128)
      2020-12-17T22:57:58.2034291Z 	at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl.getApplicationResourceUsageReport(RMAppAttemptImpl.java:900)
      2020-12-17T22:57:58.2034964Z 	at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.createAndGetApplicationReport(RMAppImpl.java:660)
      2020-12-17T22:57:58.2035575Z 	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplications(ClientRMService.java:930)
      2020-12-17T22:57:58.2036257Z 	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplications(ApplicationClientProtocolPBServiceImpl.java:273)
      2020-12-17T22:57:58.2037129Z 	at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:507)
      2020-12-17T22:57:58.2037801Z 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:447)
      2020-12-17T22:57:58.2038286Z 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
      2020-12-17T22:57:58.2038667Z 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:847)
      2020-12-17T22:57:58.2039072Z 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:790)
      2020-12-17T22:57:58.2039456Z 	at java.security.AccessController.doPrivileged(Native Method)
      2020-12-17T22:57:58.2039812Z 	at javax.security.auth.Subject.doAs(Subject.java:422)
      2020-12-17T22:57:58.2040253Z 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
      2020-12-17T22:57:58.2040724Z 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2486)
      2020-12-17T22:57:58.2041132Z 
      2020-12-17T22:57:58.2041435Z 	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1489)
      2020-12-17T22:57:58.2041824Z 	at org.apache.hadoop.ipc.Client.call(Client.java:1435)
      2020-12-17T22:57:58.2042282Z 	at org.apache.hadoop.ipc.Client.call(Client.java:1345)
      2020-12-17T22:57:58.2042721Z 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
      2020-12-17T22:57:58.2043216Z 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
      2020-12-17T22:57:58.2043651Z 	at com.sun.proxy.$Proxy110.getApplications(Unknown Source)
      2020-12-17T22:57:58.2044195Z 	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplications(ApplicationClientProtocolPBClientImpl.java:288)
      2020-12-17T22:57:58.2044672Z 	... 50 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mapohl Matthias
                Reporter:
                hxbks2ks Huang Xingbo
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: