Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-437

Make acid schema checks case insensitive

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.5.3
    • 1.5.4
    • Java
    • None

    Description

      When reading from an Orc file, SchemaEvolution evolution tries to determine if this is an Acid compliant format by comparing field names with Acid event names in SchemaEvolution.checkAcidSchema. Would be good to make this comparison case insensitive.
      This requirement comes in from HIVE-20699 where a Hive query is being used to run compaction (and hence write the compacted data to the bucket files via a HiveQL query). Since hive converts all column names to lower case, the compacted files end up with lower case Acid schema columns. The change is much simpler when made in Orc.

      Attachments

        1. ORC-437.2.patch
          0.9 kB
          Vaibhav Gumashta
        2. ORC-437.1.patch
          0.8 kB
          Vaibhav Gumashta

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. HIVE-20699 Query based compactor for full CRUD Acid tables

          • Major - Major loss of function.
          • Resolved
          relates to

          Task - A task that needs to be done. ORC-553 Add test case to check that SchemaEvolution checkAcidSchema works well

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          links to

          Web Link GitHub Pull Request #339

          Web Link GitHub Pull Request #340

          Activity

            People

              vgumashta Vaibhav Gumashta
              vgumashta Vaibhav Gumashta
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: