[DRILL-1575] kvgen function should eliminate keys with null values

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Component/s: Functions - Drill
• Assignee: Mehant Baid
• Reporter: Neeraja
• Created: 23/Oct/14 17:42
• Updated: 29/Apr/15 19:47
• Resolved: 28/Oct/14 20:59

Description

Here is the same JSON I am trying to query and use kvgen function with.
{
"name":"test",
"myarray":

{ "key1":2, "key4":3, "key2":4 }

}{
"name":"mytest",
"myarray":

{ "key3":2, "key5":3, "key6":4 }

}

Currently , when I apply kvgen a list of 'unioned' columns are returned as keys in the result. For ex: both the result rows below have keys 1-6.
The expected result is only the keys that actually have values associated with them.

0: jdbc:drill:zk=local> SELECT name, kvgen(myarray) FROM dfs.`default`.`/Users/nrentachintala/Downloads/apache-drill-0.7.0-incubating-SNAPSHOT/sample-data/mytest.json`;
----------------------+

name

EXPR$1

----------------------+

test	[{"key":"key1","value":2},{"key":"key4","value":3},{"key":"key2","value":4},{"key":"key3"},{"key":"key5"},{"key":"key6"}]
mytest	[{"key":"key1"},{"key":"key4"},{"key":"key2"},{"key":"key3","value":2},{"key":"key5","value":3},{"key":"key6","value":4}]

----------------------+