Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2127

Improve stats gathering reliability by retries on failures with hive.stats.retries.max and hive.stats.retries.wait

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Query Processor, Statistics
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Stats publishing and aggregation only try once and if there is any exception it will fail and return. If many mappers/reducers updating stats at the same time, it is very common to get lock timeout. We should make stats more reliable by retry when there is an SQLException.

        Attachments

        1. HIVE-2127.patch
          19 kB
          Ning Zhang
        2. HIVE-2127.2.patch
          30 kB
          Ning Zhang

          Activity

            People

            • Assignee:
              nzhang Ning Zhang
              Reporter:
              nzhang Ning Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: