[HDFS-14550] RBF: Failed to get statistics from NameNodes before 2.9.0

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0, HDFS-13891
• Hadoop Flags: Reviewed
• Assignee: Xiaoqiao He
• Reporter: Akira Ajisaka
• Created: 06/Jun/19 07:51
• Updated: 12/Sep/19 20:06
• Resolved: 07/Jun/19 06:46

Description

DFSRouter fails to get stats from NameNodes that do not have HDFS-7877

2019-06-03 17:40:15,407 ERROR org.apache.hadoop.hdfs.server.federation.router.NamenodeHeartbeatService: Cannot get stat from nn1:nn01:8022 using JMX
org.codehaus.jettison.json.JSONException: JSONObject["NumInMaintenanceLiveDataNodes"] not found.
        at org.codehaus.jettison.json.JSONObject.get(JSONObject.java:360)
        at org.codehaus.jettison.json.JSONObject.getInt(JSONObject.java:421)
        at org.apache.hadoop.hdfs.server.federation.router.NamenodeHeartbeatService.updateJMXParameters(NamenodeHeartbeatService.java:345)
        at org.apache.hadoop.hdfs.server.federation.router.NamenodeHeartbeatService.getNamenodeStatusReport(NamenodeHeartbeatService.java:278)
        at org.apache.hadoop.hdfs.server.federation.router.NamenodeHeartbeatService.updateState(NamenodeHeartbeatService.java:206)
        at org.apache.hadoop.hdfs.server.federation.router.NamenodeHeartbeatService.periodicInvoke(NamenodeHeartbeatService.java:160)
        at org.apache.hadoop.hdfs.server.federation.router.PeriodicService$1.run(PeriodicService.java:178)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Issue Links

• is related to: HDFS-14024 RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService