[YARN-10810] YARN Native Service Definition is not backward compatible

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.0
• Component/s: yarn-native-services
• Assignee: Prabhu Joseph
• Reporter: Prabhu Joseph
• Created: 08/Jun/21 12:15
• Updated: 17/Jun/21 08:15

Description

YARN Native Service Spec PlacementScope value was NODE in hadoop-3.1 version but got changed to node in hadoop-3.3. This causes older Service Client (hadoop-3.1) to fail while getting the status from new Api Server (hadoop-3.3). This looks caused due to jackson upgrade.

 

2021-06-07 06:08:40,095 INFO utils.ServiceApiUtil: Loading service definition from hdfs://prabhuhdfs/user/root/.yarn/services/llap0/llap0.json
2021-06-07 06:08:40,798 ERROR utils.JsonSerDeser: Exception while parsing json : org.codehaus.jackson.map.JsonMappingException: Can not construct instance of org.apache.hadoop.yarn.service.api.records.PlacementScope from String value 'node': value not one of declared Enum instance names
 at [Source: java.io.StringReader@72c927f1; line: 27, column: 33] (through reference chain: org.apache.hadoop.yarn.service.api.records.Service["components"]->org.apache.hadoop.yarn.service.api.records.Component["placement_policy"]->org.apache.hadoop.yarn.service.api.records.PlacementPolicy["constraints"]->org.apache.hadoop.yarn.service.api.records.PlacementConstraint["scope"])    "placement_policy" : {
      "constraints" : [ {
        "name" : null,
        "type" : "ANTI_AFFINITY",
        "scope" : "node",
        "target_tags" : [ "llap" ],
        "node_attributes" : { },
        "node_partitions" : [ ],
        "min_cardinality" : null,
        "max_cardinality" : null
      } ]
    },org.codehaus.jackson.map.JsonMappingException: Can not construct instance of org.apache.hadoop.yarn.service.api.records.PlacementScope from String value 'node': value not one of declared Enum instance names
 at [Source: java.io.StringReader@72c927f1; line: 27, column: 33] (through reference chain: org.apache.hadoop.yarn.service.api.records.Service["components"]->org.apache.hadoop.yarn.service.api.records.Component["placement_policy"]->org.apache.hadoop.yarn.service.api.records.PlacementPolicy["constraints"]->org.apache.hadoop.yarn.service.api.records.PlacementConstraint["scope"])
        at org.codehaus.jackson.map.JsonMappingException.from(JsonMappingException.java:163)
        at org.codehaus.jackson.map.deser.StdDeserializationContext.weirdStringException(StdDeserializationContext.java:243)
        at org.codehaus.jackson.map.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:80)
        at org.codehaus.jackson.map.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:23)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:217)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:194)
        at org.codehaus.jackson.map.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30)
        at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:299)
        at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:414)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:697)
        at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
        at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732)
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863)
        at org.apache.hadoop.yarn.service.utils.JsonSerDeser.fromJson(JsonSerDeser.java:87)
        at org.apache.hadoop.yarn.service.utils.JsonSerDeser.fromBytes(JsonSerDeser.java:168)
        at org.apache.hadoop.yarn.service.utils.JsonSerDeser.load(JsonSerDeser.java:190)
        at org.apache.hadoop.yarn.service.utils.ServiceApiUtil.loadService(ServiceApiUtil.java:360)
        at org.apache.hadoop.yarn.service.client.ServiceClient.getAppId(ServiceClient.java:1409)
        at org.apache.hadoop.hive.llap.cli.status.LlapStatusServiceDriver.getAppReport(LlapStatusServiceDriver.java:263)
        at org.apache.hadoop.hive.llap.cli.status.LlapStatusServiceDriver.run(LlapStatusServiceDriver.java:208)
        at org.apache.hadoop.hive.llap.cli.status.LlapStatusServiceDriver.main(LlapStatusServiceDriver.java:530)
2021-06-07 06:08:40,811 INFO status.LlapStatusServiceDriver: No Application Found