[DRILL-6129] Query fails on nested data type schema change

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.13.0
• Component/s: Client - CLI
• Labels: ready-to-commit
• Assignee: Salim Achouche
• Reporter: Salim Achouche
• Reviewer: Aman Sinha
• Created: 01/Feb/18 01:31
• Updated: 09/Feb/18 09:57
• Resolved: 09/Feb/18 09:57

Description

Use-Case -

• Assume two parquet files with similar schemas except for a nested column
• Schema file1
  • int64 field1
  • optional group field2

• • • optional group field2.1 (LIST)
      • repeated group list

• • • • • optional group element

• • • • • • optional int64 child_field
• Schema file2
  • int64 field1
  • optional group field2

• • • optional group field2.1 (LIST)
      • repeated group list

• • • • • optional group element

• • • • • • optional group child_field
            • optional int64 child_field_f1
            • optional int64 child_field_f1
• Essentially child_field changed from an int64 to a group of fields

 

Observed Query Failure

select * from <file1 and file2>;
Error: Unexpected RuntimeException: java.lang.IllegalArgumentException: The field $bits$(UINT1:REQUIRED) doesn't match the provided metadata major_type {
minor_type: MAP
mode: REQUIRED
Note that selecting one file at a time succeeds which seems to indicate the issue has to do with the schema change logic. 

 

 

Issue Links

• links to: GitHub Pull Request #1106