Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4824

Null maps / lists and non-provided state support for JSON fields. Numeric types promotion.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: Storage - JSON
    • Labels:
      None

      Description

      There is incorrect output in case of JSON file with complex nested data.

      JSON:

      example.json
      {
              "Field1" : {
              }
      }
      
      {
              "Field1" : {
                      "InnerField1": {"key1":"value1"},
                      "InnerField2": {"key2":"value2"}
              }
      }
      
      {
              "Field1" : {
                      "InnerField3" : ["value3", "value4"],
                      "InnerField4" : ["value5", "value6"]
              }
      }
      

      Query:

      select Field1 from dfs.`/tmp/example.json`
      

      Incorrect result:

      +---------------------------+
      |          Field1           |
      +---------------------------+
      {"InnerField1":{},"InnerField2":{},"InnerField3":[],"InnerField4":[]}
      {"InnerField1":{"key1":"value1"},"InnerField2" {"key2":"value2"},"InnerField3":[],"InnerField4":[]}
      {"InnerField1":{},"InnerField2":{},"InnerField3":["value3","value4"],"InnerField4":["value5","value6"]}
      +--------------------------+
      

      Theres is no need to output missing fields. In case of deeply nested structure we will get unreadable result for user.

      Correct result:

      +--------------------------+
      |         Field1           |
      +--------------------------+
      |{}                                                                     
      {"InnerField1":{"key1":"value1"},"InnerField2":{"key2":"value2"}}
      {"InnerField3":["value3","value4"],"InnerField4":["value5","value6"]}
      +--------------------------+
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                RomanKulyk Roman Kulyk
                Reviewer:
                Paul Rogers
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: