Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7926 long-lived daemons for query fragment execution, I/O and caching
  3. HIVE-10406

LLAP: Make use of additional information to determine run/preemption order

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: llap
    • Component/s: llap
    • Labels:
      None

      Description

      The preemption will evolve as it's tested.
      Had a discussion offline with Gunther Hagleitner. The initial policy will likely be the following.

      Within a running DAG, the priority / topo order decides which fragment runs / is a candidate for preemption.
      Beyond this, the number of tasks in the current vertex + upstream vertices will be used as a measure of the size of the query to determine which fragment gets to run, if there's multiple fragments queued up. Fragments with a lower count will be preferred to push through what are expected to be shorter running queries.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sseth Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: