Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12558

LLAP: output QueryFragmentCounters somewhere

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0
    • llap
    • None

    Description

      Right now, LLAP logs counters for every fragment; most of them are IO related and could be very useful, they also include table names so that things like cache hit ratio, etc., could be calculated for every table.
      We need to output them to some metrics system (preserving the breakdown by table, possibly also adding query ID or even stage) so that they'd be usable without grep/sed/awk.

      Attachments

        1. HIVE-12558.1.patch
          40 kB
          Prasanth Jayachandran
        2. HIVE-12558.2.patch
          40 kB
          Prasanth Jayachandran
        3. HIVE-12558.3.patch
          41 kB
          Prasanth Jayachandran
        4. HIVE-12558.3.patch
          41 kB
          Prasanth Jayachandran
        5. HIVE-12558.4.patch
          41 kB
          Prasanth Jayachandran
        6. HIVE-12558.wip.patch
          47 kB
          Prasanth Jayachandran
        7. sample-output.png
          84 kB
          Prasanth Jayachandran
        8. sample-output2.png
          220 kB
          Prasanth Jayachandran

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. TEZ-3090 MRInput should make dagIdentifier, vertexIdentifier, etc available to the InputFormat jobConf

          • Major - Major loss of function.
          • Closed
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-13226 Improve tez print summary to print query execution breakdown

          • Major - Major loss of function.
          • Closed
          is duplicated by

          Sub-task - The sub-task of the issue HIVE-12163 LLAP: Tez counters for LLAP 2

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-12626 Publish some of the LLAP cache counters via Tez

          • Major - Major loss of function.
          • Resolved
          is part of

          Task - A task that needs to be done. HIVE-13097 [Umbrella] Changes dependent on Tez 0.8.3

          • Major - Major loss of function.
          • Closed
          links to

          Web Link RB

          Activity

            People

              prasanth_j Prasanth Jayachandran
              sershe Sergey Shelukhin
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: