[CARBONDATA-3287] Remove the validation of same chema data files in location for external table and file format

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.3
• Assignee: Akash R Nilugal
• Reporter: Akash R Nilugal
• Created: 04/Feb/19 10:05
• Updated: 14/Feb/19 07:28
• Resolved: 14/Feb/19 07:28

Description

Currently we have a validation that if there are two carbondata files in a location with different schema, then we fail the query. I think there is no need to fail. If you see the parquet behavior also we cna understand. 
 
Here i think failing is not good, we can read the latets schema from latest carbondata file in the given location and based on that read all the files and give query output. For the columns which are not present in some data files, it wil have null values for the new column.
 
But here basically we do not merge schema. we can maintain the same now also, only thing is can take latest schma.
 
for example:
1. one data file with columns a,b and c. 2nd file is with columns a,b,c,d,e. then can read and create table with 5 columns or 3 columns which ever is latest and create table(This will be when user does not specify schema). If he species table will be created with specified schema.

Issue Links

• links to: GitHub Pull Request #3121