Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-10989

HoS can't control number of map tasks for runtime skew join [Spark Branch]

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: spark-branch, 1.3.0, 2.0.0
    • Component/s: Spark
    • Labels:
      None

      Description

      Flags hive.skewjoin.mapjoin.map.tasks and hive.skewjoin.mapjoin.min.split are used to control the number of map tasks for the map join of runtime skew join. They work well for MR but have no effect for spark.
      This makes runtime skew join less useful, i.e. we just end up with slow mappers instead of reducers.

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment