[METRON-1336] Patching Can Result in Bad Configuration

• Type: Bug
• Status: Done
• Priority: Major
• Resolution: Done
• Affects Version/s: 0.4.1
• Fix Version/s: 0.4.2
• Assignee: Nick Allen
• Reporter: Nick Allen
• Created: 28/Nov/17 13:57
• Updated: 15/Dec/17 17:04
• Resolved: 15/Dec/17 17:04

Description

When applying a patch with `zk_load_configs` the resulting configuration can be invalid. The resulting configuration should be validated so that a patch can never result in an invalid configuration.

For example, applying the following patch with `zk_load_config` to the Profiler yields a broken Profiler configuration.

[  
   {  
      "path":"profiles",
      "value":{  
         "profile":"sketchy_mad",
         "onlyif":"true",
         "update":{  
            "s":"OUTLIER_MAD_ADD(s, value)"
         },
         "init":{  
            "s":"OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad','global', PROFILE_FIXED(5, 'MINUTES')))"
         },
         "foreach":"'global'",
         "result":"s"
      },
      "op":"add"
   }
]

The broken configuration is only discovered after dumping the configuration.

$ bin/zk_load_configs.sh -z $ZOOKEEPER -m DUMP -c PROFILER
Exception in thread "main" java.lang.RuntimeException: Unable to load {
  "profiles" : {
    "profile" : "sketchy_mad",
    "onlyif" : "true",
    "update" : {
      "s" : "OUTLIER_MAD_ADD(s, value)"
    },
    "init" : {
      "s" : "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad','global', PROFILE_FIXED(5, 'MINUTES')))"
    },
    "foreach" : "'global'",
    "result" : "s"
  }
}
	at org.apache.metron.common.configuration.ConfigurationType.lambda$static$4(ConfigurationType.java:68)
	at org.apache.metron.common.configuration.ConfigurationType.deserialize(ConfigurationType.java:93)
	at org.apache.metron.common.configuration.ConfigurationsUtils.lambda$dumpConfigs$6(ConfigurationsUtils.java:621)
	at org.apache.metron.common.configuration.ConfigurationsUtils.visitConfigs(ConfigurationsUtils.java:575)
	at org.apache.metron.common.configuration.ConfigurationsUtils.dumpConfigs(ConfigurationsUtils.java:619)
	at org.apache.metron.common.cli.ConfigurationManager.dump(ConfigurationManager.java:189)
	at org.apache.metron.common.cli.ConfigurationManager.run(ConfigurationManager.java:268)
	at org.apache.metron.common.cli.ConfigurationManager.run(ConfigurationManager.java:243)
	at org.apache.metron.common.cli.ConfigurationManager.main(ConfigurationManager.java:355)
Caused by: org.apache.metron.jackson.databind.JsonMappingException: Can not deserialize instance of java.util.ArrayList out of START_OBJECT token
 at [Source: {
  "profiles" : {
    "profile" : "sketchy_mad",
    "onlyif" : "true",
    "update" : {
      "s" : "OUTLIER_MAD_ADD(s, value)"
    },
    "init" : {
      "s" : "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad','global', PROFILE_FIXED(5, 'MINUTES')))"
    },
    "foreach" : "'global'",
    "result" : "s"
  }
}; line: 2, column: 16] (through reference chain: org.apache.metron.common.configuration.profiler.ProfilerConfig["profiles"])
	at org.apache.metron.jackson.databind.JsonMappingException.from(JsonMappingException.java:255)
	at org.apache.metron.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:971)
	at org.apache.metron.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:967)
	at org.apache.metron.jackson.databind.deser.std.CollectionDeserializer.handleNonArray(CollectionDeserializer.java:327)
	at org.apache.metron.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:259)
	at org.apache.metron.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:249)
	at org.apache.metron.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:26)
	at org.apache.metron.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490)
	at org.apache.metron.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
	at org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
	at org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125)
	at org.apache.metron.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3807)
	at org.apache.metron.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2797)
	at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:79)
	at org.apache.metron.common.configuration.ConfigurationType.lambda$static$4(ConfigurationType.java:66)
	... 8 more

Issue Links

• links to: GitHub Pull Request #851