[DRILL-7852] Handle PCAP/PCAPNG Better

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.18.0
• Fix Version/s: 1.19.0
• Component/s: Storage - PCAP
• Labels: pcap pcapng plugin
• Assignee: Vitalii Diravka
• Reporter: Vitalii Diravka
• Created: 01/Feb/21 16:39
• Updated: 25/Mar/21 08:19
• Resolved: 25/Mar/21 08:19

Description

SUEE8 from https://github.com/vs-uulm/2017-SUEE-data-set is a pcapng labeled with pcap extension. Initially this causes a magic number error where Drill can't read the file because the extension and magic numbers don't match. We need to handle that better.
 
After renaming the file with the pcapng extension Drill can read it and display it on the CLI but not in the GUI. You get constant errors for running out of heap memory.
 
Example Errors:

org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: There is not enough heap memory to run this query using the web interface.

 
 

org.apache.drill.common.exceptions.UserRemoteException: EXECUTION_ERROR ERROR: Java heap space Open failed for reader: PcapngBatchReader Fragment: 0:0

 

Issue Links

• is duplicated by: DRILL-7828 Refactor Pcap and Pcapng format plugin
• is related to: DRILL-6179 Added pcapng-format support