Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13285

Orc concatenation may drop old files from moving to final path

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.14.0, 1.0.0, 1.1.0, 1.2.0, 1.3.0, 2.0.0, 2.1.0
    • Fix Version/s: 1.3.0, 2.0.1, 2.1.0
    • Component/s: ORC
    • Labels:
      None

      Description

      ORC concatenation uses combine hive input format for merging files. Under specific case where all files within a combine split are incompatible for merge (old files without stripe statistics) then these files are added to incompatible file set. But this file set is not processed as closeOp() will not be called (no output file writer will exist which will skip super.closeOp()). As a result, these incompatible files are not moved to final path.

        Attachments

        1. HIVE-13285.3.patch
          11 kB
          Prasanth Jayachandran
        2. HIVE-13285.2.patch
          11 kB
          Prasanth Jayachandran
        3. HIVE-13285.1.patch
          10 kB
          Prasanth Jayachandran

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              prasanth_j Prasanth Jayachandran Assign to me
              Reporter:
              prasanth_j Prasanth Jayachandran

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment