Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6129

Query fails on nested data type schema change

    Details

      Description

      Use-Case -

      • Assume two parquet files with similar schemas except for a nested column
      • Schema file1
        • int64 field1
        • optional group field2
          • optional group field2.1 (LIST)
            • repeated group list
              • optional group element
                • optional int64 child_field
      • Schema file2
        • int64 field1
        • optional group field2
          • optional group field2.1 (LIST)
            • repeated group list
              • optional group element
                • optional group child_field
                  • optional int64 child_field_f1
                  • optional int64 child_field_f1
      • Essentially child_field changed from an int64 to a group of fields

       

      Observed Query Failure

      select * from <file1 and file2>;
      Error: Unexpected RuntimeException: java.lang.IllegalArgumentException: The field $bits$(UINT1:REQUIRED) doesn't match the provided metadata major_type {
      minor_type: MAP
      mode: REQUIRED
      Note that selecting one file at a time succeeds which seems to indicate the issue has to do with the schema change logic. 

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sachouche salim achouche
                Reporter:
                sachouche salim achouche
                Reviewer:
                Aman Sinha
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: