[METRON-1501] Parser messages that fail to validate are dropped silently

• Type: Bug
• Status: Done
• Priority: Major
• Resolution: Done
• Fix Version/s: 0.5.0
• Assignee: Casey Stella
• Reporter: Casey Stella
• Created: 23/Mar/18 18:56
• Updated: 22/May/18 19:34
• Resolved: 22/May/18 19:19

Description

Currently we have two concepts of dealing with messages that are not going to be passed through:

• Messages that fail global validation
• Messages that fail the parser's validation function
• Messages that are filtered out

 

Currently in the case of messages that fail global validation, we send them to the error queue.

In the case of messages that fail the parser's validation function, we drop them silently.

In the case of messages that are filtered out, we drop them silently as well.

 

Given the use-cases behind filtering (e.g. a kafka topic with multiple types of messages where we want to pass along only one type), it makes sense to drop them silently.  However, in the case of the messages that fail the parser's validation function, we should like to know that and treat those similarly to those messages that fail global validation.

Issue Links

• links to: GitHub Pull Request #972