Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6913

Hive unable to find the hashtable file during complex multi-staged map join

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      If a query has multiple mapjoins and one of the tables to be mapjoined is empty, the query can result in a "no such file or directory" when looking for the hashtable.

      This is because when we generate a dummy hash table, we do not close the TableScan (TS) operator for that table. Additionally, HashTableSinkOperator (HTSO) outputs it's hash tables in the closeOp method. However, when close is called on HTSO we check to ensure that all parents are closed: https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java#L333

      which is not true on this case, because the TS operator for the empty table was never closed.

        Attachments

        1. HIVE-6913.patch
          5 kB
          Brock Noland
        2. HIVE-6913.patch
          5 kB
          Brock Noland

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              brocknoland Brock Noland Assign to me
              Reporter:
              brocknoland Brock Noland

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment