Details
-
Improvement
-
Status: Done
-
Major
-
Resolution: Done
-
None
-
None
Description
Currently we have only one layer of parsing prior to enrichment, but often real data is more complex. For instance, often data is wrapped in an envelope (e.g. syslog data which contains a field which needs to be parsed).
This Jira allows us to support a DAG of parsers prior to enrichment by allowing us to provide a strategy for interpreting what is data and what is metadata in the parser bolt. This enables upstream parsers to pass in a JSON Blob which contains the metadata and have the parser bolt choose which field is the data to be parsed, the remaining fields would be considered metadata (and merged into the resulting data or not depending on our existing rules for handling metadata).
To illustrate this better, I've provided a use-case with an example. Note, this PR depends on METRON-1643 and METRON-1642, so those should be reviewed prior to this.
Attachments
Issue Links
- links to
