Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-3436

update pre insert into rule as per spark

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: None
    • Labels:
      None

      Description

      Problem: Carbon is not following hive syntax for one insert into strictly.

      Cause: For insert into, carbon has its own rule. CarbonAnalysisRule[CarbonPreInsertionCasts]
      In CarbonPreInsertionCasts, data type cast is missed. Hence carbon was not following hive syntax.

      solution:
      Add data type cast rule for insert into.

      Note:

      1. Change syntax for insert single record(for binary and complex type(array,struct,map))
      2. Binary cannot accept value as integer, need to pass in quotes only.
      3. Earlier complex type was accepting null values for complex datatypes, now it won't accept null values for complex types as per Hive rule.
      4. Need to pass in default format only for date type and timestamp type for Complex type in insert command.
      5. For boolean type 't' is a valid entry for true and 'f' is a valid entry for false.
      6. For boolean type , any integer greater than 0 will be considered as true and 0 will be considered as false.
      7. In case of map,for same key. As per hive, the latter value should be inserted for the same key.

      Steps to reproduce:

      create table test(name string,id binary) stored by 'carbondata';
      insert into test select 'a',1;

      insert should throw cast exception as 1 is not binary. But it is passing now.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Ajantha_Bhat Ajantha Bhat
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 10m
                  3h 10m