[DRILL-6242] Output format for nested date, time, timestamp values in an object hierarchy

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.14.0
• Component/s: Execution - Data Types
• Labels: ready-to-commit
• Assignee: Jiang Wu
• Reporter: Jiang Wu
• Reviewer: Parth Chandra
• Created: 14/Mar/18 16:40
• Updated: 26/Dec/18 07:37
• Resolved: 12/May/18 00:08

Description

Some storages (mapr db, mongo db, etc.) have hierarchical objects that contain nested fields of date, time, timestamp types.  When a query returns these objects, the output format for the nested date, time, timestamp, are showing the internal object (org.joda.time.DateTime), rather than the logical data value.

For example.  Suppose in MongoDB, we have a single object that looks like this:

> db.test.findOne();
{
    "_id" : ObjectId("5aa8487d470dd39a635a12f5"),
    "name" : "orange",
    "context" : {
        "date" : ISODate("2018-03-13T21:52:54.940Z"),
        "user" : "jack"
    }
}

Then connect Drill to the above MongoDB storage, and run the following query within Drill:

> select t.context.`date`, t.context from test t; 
+--------+---------+ 
| EXPR$0 | context | 
+--------+---------+ 
| 2018-03-13 | {"date":{"dayOfYear":72,"year":2018,"dayOfMonth":13,"dayOfWeek":2,"era":1,"millisOfDay":78774940,"weekOfWeekyear":11,"weekyear":2018,"monthOfYear":3,"yearOfEra":2018,"yearOfCentury":18,"centuryOfEra":20,"millisOfSecond":940,"secondOfMinute":54,"secondOfDay":78774,"minuteOfHour":52,"minuteOfDay":1312,"hourOfDay":21,"zone":{"fixed":true,"id":"UTC"},"millis":1520977974940,"chronology":{"zone":{"fixed":true,"id":"UTC"}},"afterNow":false,"beforeNow":true,"equalNow":false},"user":"jack"} |

We can see that from the above output, when the date field is retrieved as a top level column, Drill outputs a logical date value.  But when the same field is within an object hierarchy, Drill outputs the internal object used to hold the date value.

The expected output is the same display for whether the date field is shown as a top level column or when it is within an object hierarchy:

> select t.context.`date`, t.context from test t; 
+--------+---------+ 
| EXPR$0 | context | 
+--------+---------+ 
| 2018-03-13 | {"date":"2018-03-13","user":"jack"} |

Issue Links

• is depended upon by: DRILL-5360 Timestamp type documented as UTC, implemented as local time
• is related to: DRILL-5332 DateVector support uses questionable conversions to a time
• is related to: DRILL-5334 Questionable code in the TimeVector class
• links to: GitHub Pull Request #1184
• links to: GitHub Pull Request #1247