Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-27639 Query level performance counters
  3. HIVE-27693

FileSystem counters summary to include durations

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      given this:

      INFO  : FileSystem Counters Summary
      INFO  : 
      INFO  : Scheme: HDFS
      INFO  : ----------------------------------------------------------------------------------------------
      INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
      INFO  : ----------------------------------------------------------------------------------------------
      INFO  :      Map 1        350.69MB           622                  0                 0B             0
      INFO  :     Map 10              0B             0                  0                 0B             0
      INFO  :     Map 11              0B             1                  0                 0B             0
      INFO  :     Map 12              0B             1                  0                 0B             0
      INFO  :     Map 13              0B             1                  0                 0B             0
      INFO  :     Map 14            206B             1                  0                 0B             0
      INFO  :     Map 15            206B             1                  0                 0B             0
      INFO  :     Map 16              0B             1                  0                 0B             0
      INFO  :     Map 17              0B             1                  0                 0B             0
      

      I want to be able to see how much time has actually been spent on filesystem operation on a specific scheme, as by looking at that, I could easily tell if file reads were slow

      Attachments

        Activity

          People

            Unassigned Unassigned
            abstractdog László Bodor
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: