[DRILL-7021] HTTPD Throws NPE and Doesn't Recognize Timeformat

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0
• Component/s: Storage - Other
• Labels: ready-to-commit
• Assignee: Charles Givre
• Reporter: Charles Givre
• Reviewer: Bohdan Kazydub
• Created: 31/Jan/19 20:04
• Updated: 18/Mar/19 16:05
• Resolved: 17/Mar/19 07:38

Description

The HTTPD format plugin throws a NPE when you attempt to query all fields and enumerate them in the query.

The query below will throw the NPE:

```
SELECT `request_referer_query`,
. . . . . . .semicolon> `request_referer_ref`,
. . . . . . .semicolon> `request_firstline_uri_port`,
. . . . . . .semicolon> `request_firstline_method`,
. . . . . . .semicolon> `request_firstline_uri_protocol`,
. . . . . . .semicolon> `request_receive_time_millisecond`,
. . . . . . .semicolon> `request_receive_time_day__utc`,
. . . . . . .semicolon> `request_receive_time_year__utc`,
. . . . . . .semicolon> `request_receive_time_weekofweekyear__utc`,
. . . . . . .semicolon> `request_receive_time_second`,
. . . . . . .semicolon> `connection_client_host`,
. . . . . . .semicolon> `request_referer_userinfo`,
. . . . . . .semicolon> `request_referer_path`,
. . . . . . .semicolon> `request_referer_host`,
. . . . . . .semicolon> `request_receive_time_monthname`,
. . . . . . .semicolon> `request_receive_time_month__utc`,
. . . . . . .semicolon> `request_firstline_uri_ref`,
. . . . . . .semicolon> `request_firstline_protocol_version`,
. . . . . . .semicolon> `request_receive_time_minute`,
. . . . . . .semicolon> `request_firstline`,
. . . . . . .semicolon> `request_receive_time_day`,
. . . . . . .semicolon> `request_referer_port`,
. . . . . . .semicolon> `request_receive_time_year`,
. . . . . . .semicolon> `request_referer_query_$`,
. . . . . . .semicolon> `request_firstline_uri_query_$`,
. . . . . . .semicolon> `request_firstline_uri`,
. . . . . . .semicolon> `request_receive_time_month`,
. . . . . . .semicolon> `request_receive_time_weekofweekyear`,
. . . . . . .semicolon> `request_firstline_uri_userinfo`,
. . . . . . .semicolon> `request_referer`,
. . . . . . .semicolon> `request_receive_time_epoch`,
. . . . . . .semicolon> `request_referer_protocol`,
. . . . . . .semicolon> `request_receive_time_monthname__utc`,
. . . . . . .semicolon> `connection_client_logname`,
. . . . . . .semicolon> `request_receive_time`,
. . . . . . .semicolon> `request_firstline_protocol`,
. . . . . . .semicolon> `request_receive_time_hour`,
. . . . . . .semicolon> `request_firstline_uri_host`,
. . . . . . .semicolon> `request_firstline_uri_path`,
. . . . . . .semicolon> `request_user-agent`,
. . . . . . .semicolon> `request_receive_time_hour__utc`,
. . . . . . .semicolon> `request_receive_time_second__utc`,
. . . . . . .semicolon> `request_receive_time_weekyear`,
. . . . . . .semicolon> `request_receive_time_timezone`,
. . . . . . .semicolon> `request_receive_time_weekyear__utc`,
. . . . . . .semicolon> `response_body_bytesclf`,
. . . . . . .semicolon> `connection_client_user`,
. . . . . . .semicolon> `request_receive_time_millisecond__utc`,
. . . . . . .semicolon> `request_status_last`,
. . . . . . .semicolon> `request_firstline_uri_query`,
. . . . . . .semicolon> `request_receive_time_minute__utc`
. . . . . . .semicolon> FROM `dfs.drillclass`.`hackers-access.httpd`

```

The cause for the NPE is that several fields were missing from a type map in the format plugin.

Separately, the format plugin is not recognizing the time stamp and is not parsing dates as time formats.

Oh... and the unit tests suck. Sorry.
 

 

Issue Links

• links to: GitHub Pull Request #1635