[NUTCH-3028] WARCExported to support filtering by JEXL

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.19
• Fix Version/s: 1.21
• Assignee: Markus Jelsma
• Reporter: Markus Jelsma
• Created: 01/Feb/24 14:19
• Updated: 30/Apr/24 13:29

Description

Filtering segment data to WARC is now possible using JEXL expressions. In the next example, all records with SOME_KEY=SOME_VALUE in their parseData metadata are exported to WARC.

-expr 'parseData.getParseMeta().get("SOME_KEY").equals("SOME_VALUE")'

or

-expr 'content.getMetadata().get("SOME_KEY").equals("SOME_VALUE")'