Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16546

LLAP: Fail map join tasks if hash table memory exceeds threshold

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • llap

    Description

      When map join task is running in llap, it can potentially use lot more memory than its limit which could be memory per executor or no conditional task size. If it uses more memory, it can adversely affect other query performance or it can even bring down the daemon. In such cases, it is better to fail the query than to bring down the daemon.

      Attachments

        1. HIVE-16546.WIP.patch
          54 kB
          Prasanth Jayachandran
        2. HIVE-16546.1.patch
          57 kB
          Prasanth Jayachandran
        3. HIVE-16546.2.patch
          65 kB
          Prasanth Jayachandran
        4. HIVE-16546.3.patch
          66 kB
          Prasanth Jayachandran
        5. HIVE-16546.4.patch
          66 kB
          Prasanth Jayachandran
        6. HIVE-16546.5.patch
          92 kB
          Prasanth Jayachandran

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-16582 HashTableLoader should log info about the input, rows, size etc.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          links to

          Web Link RB

          Activity

            People

              prasanth_j Prasanth Jayachandran
              prasanth_j Prasanth Jayachandran
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: