Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-613

OrcMapredRecordReader mis-reuse struct object when actual children schema differs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.5.10, 1.6.3
    • Java
    • None

    Description

      When reading from schema like following:  

      uniontype <struct<field0, field1, ..., fieldN>, struct<>> 

      `org.apache.orc.mapreduce.OrcMapreduceRecordReader#nextStruct` will determine if previous object's schema can be reused or not. The determination of this is problematic, since it only checks the top-level type (OrcStruct) but not the schema of OrcStruct. Therefore, if encountering schema like above, and when struct at tag_0 is processed followed with a struct at tag_1, it will reuse the tag_0's struct schema which results in in correct result. 

      Attachments

        Issue Links

          Activity

            People

              lesun Lei Sun
              lesun Lei Sun
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m