[YARN-5721] NPE at AMRMClientImpl.getMatchingRequests

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Component/s: api

Environment

Tested on Windows 10, in Dockerized Linux containers & Ubuntu 16.04 with Java 7, Java 8.

• Flags: Important
• Assignee: Unassigned
• Reporter: Zoltán Zvara
• Created: 11/Oct/16 12:43
• Updated: 23/Jan/19 07:31
• Resolved: 02/Nov/16 14:00

Description

The following NPE was thrown using a Spark 2.1.0-SNAPSHOT (as client) by changing Hadoop dependency to the latest (by the time the ERROR has been generated).

{{2016-10-10 11:33:53,392 ERROR yarn.ApplicationMaster: Uncaught exception:
java.lang.NullPointerException
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.getMatchingRequests(AMRMClientImpl.java:668)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.getMatchingRequests(AMRMClientImpl.java:651)
at org.apache.spark.deploy.yarn.YarnAllocator.getPendingAtLocation(YarnAllocator.scala:210)
at org.apache.spark.deploy.yarn.YarnAllocator.getPendingAllocate(YarnAllocator.scala:203)
at org.apache.spark.deploy.yarn.YarnAllocator.updateResourceRequests(YarnAllocator.scala:318)
at org.apache.spark.deploy.yarn.YarnAllocator.allocateResources(YarnAllocator.scala:278)
at org.apache.spark.deploy.yarn.ApplicationMaster.registerAM(ApplicationMaster.scala:350)
at org.apache.spark.deploy.yarn.ApplicationMaster.runExecutorLauncher(ApplicationMaster.scala:418)
at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:250)}}

We've also pulled the latest code (1 hour ago) from the repository, and ran a test for getMatchingRequests. Same NPE has been encountered.

getMatchingRequests should never throw an NPE even if it has been called right after the client has been started.

Issue Links

• duplicates: YARN-5753 fix NPE in AMRMClientImpl.getMatchingRequests()
• links to: GitHub Pull Request #139
• links to: GitHub Pull Request #139