[DRILL-4574] Avro Plugin: Flatten does not work correctly on record items

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.8.0
• Component/s: Storage - Avro
• Assignee: Unassigned
• Reporter: Johannes Schulte
• Created: 03/Apr/16 09:01
• Updated: 24/Jun/16 22:14
• Resolved: 24/Jun/16 22:14

Description

If the schema looks like this:

{
  "type" : "record",
  "name" : "MainRecord",
  "namespace" : "drizz.WriteAvroTestFileForDrill$",
  "fields" : [ {
    "name" : "elements",
    "type" : {
      "type" : "array",
      "items" : {
        "type" : "record",
        "name" : "NestedRecord",
        "fields" : [ {
          "name" : "field1",
          "type" : "int"
        } ]
      },
      "java-class" : "java.util.List"
    }
  } ]
}

and the contents looks like this (according to avro tojson command line utility)

{"elements":[{"field1":0},{"field1":1},{"field1":2},{"field1":3},{"field1":4},{"field1":5},{"field1":6},{"field1":7},{"field1":8},{"field1":9}]}
{"elements":[{"field1":0},{"field1":1},{"field1":2},{"field1":3},{"field1":4},{"field1":5},{"field1":6},{"field1":7},{"field1":8},{"field1":9}]}

a query like

select flatten(elements) from dfs.`/Users/j.schulte/data/avro-drill/no-union/`;

yields exactly two rows:

+---------------+
|    EXPR$0     |
+---------------+
| {"field1":9}  |
| {"field1":9}  |
+---------------+

as if only the last element in the array would survive.

It works for primitive arrays.

Issue Links

• links to: GitHub Pull Request #459