Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-3784

de-emphasize mapjoin hint

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.11.0
    • Query Processor
    • None
    • Map join hint will no longer be valid for some queries. Drop the hint in those cases. Hive will automatically try to convert join to map-join with config hive.auto.convert.join set to true.

    Description

      hive.auto.convert.join has been around for a long time, and is pretty stable.
      When mapjoin hint was created, the above parameter did not exist.

      The only reason for the user to specify a mapjoin currently is if they want
      it to be converted to a bucketed-mapjoin or a sort-merge bucketed mapjoin.
      Eventually, that should also go away, but that may take some time to stabilize.

      There are many rules in SemanticAnalyzer to handle the following trees:

      ReduceSink -> MapJoin
      Union -> MapJoin
      MapJoin -> MapJoin

      This should not be supported anymore. In any of the above scenarios, the
      user can get the mapjoin behavior by setting hive.auto.convert.join to true
      and not specifying the hint. This will simplify the code a lot.

      What does everyone think ?

      Attachments

        1. hive.3784.1.patch
          209 kB
          Namit Jain
        2. hive.3784.10.patch
          504 kB
          Namit Jain
        3. hive.3784.11.patch
          547 kB
          Namit Jain
        4. hive.3784.12.patch
          550 kB
          Namit Jain
        5. hive.3784.13.patch
          617 kB
          Namit Jain
        6. hive.3784.14.patch
          617 kB
          Namit Jain
        7. hive.3784.15.patch
          657 kB
          Namit Jain
        8. hive.3784.16.patch
          658 kB
          Namit Jain
        9. hive.3784.17.patch
          659 kB
          Namit Jain
        10. hive.3784.18.patch
          659 kB
          Namit Jain
        11. hive.3784.19.patch
          659 kB
          Namit Jain
        12. hive.3784.2.patch
          209 kB
          Namit Jain
        13. hive.3784.21.patch
          659 kB
          Namit Jain
        14. hive.3784.22.patch
          665 kB
          Namit Jain
        15. hive.3784.3.patch
          434 kB
          Namit Jain
        16. hive.3784.4.patch
          650 kB
          Namit Jain
        17. hive.3784.5.patch
          650 kB
          Namit Jain
        18. hive.3784.6.patch
          528 kB
          Namit Jain
        19. hive.3784.7.patch
          571 kB
          Namit Jain
        20. hive.3784.8.patch
          578 kB
          Namit Jain
        21. hive.3784.9.patch
          581 kB
          Namit Jain

        Issue Links

          Activity

            People

              namit Namit Jain
              namit Namit Jain
              Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: