[ORC-54] Evolve schemas based on field name rather than index

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Assignee: Mark Wagner
• Reporter: Mark Wagner
• Created: 20/May/16 22:43
• Updated: 06/Oct/16 17:31
• Resolved: 17/Aug/16 15:50

Description

Schema evolution as it stands today allows adding fields to the end of schemas or removing them from the end. However, because it is based on the index of the column, you can only ever add or remove – not both.

ORC files have the full schema information of their contents, so there's actually enough metadata to support changing columns anywhere in the schema.

Issue Links

• is related to: HIVE-13974 ORC Schema Evolution doesn't support add columns to non-last STRUCT columns
• is related to: HIVE-4243 Fix column names in FileSinkOperator
• links to: GitHub Pull Request #40
• links to: GitHub Pull Request #55