HBase
  1. HBase
  2. HBASE-3336

Make GC pauses more obvious to users

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Implemented
    • Affects Version/s: 0.92.0
    • Fix Version/s: None
    • Component/s: regionserver
    • Labels:
      None

      Description

      We often see users hit GC pauses, but it's not obvious to them what happened. I propose we add a "GCMonitor" thread which wakes up every few seconds and checks (a) that it slept for around the right amount of time, and (b) uses the JVM GC metrics to see if there was a long pause. If a long pause is detected we should output a very loud and obnoxious message to the logs indicating that a length GC pause occurred, and point to a wiki page on the topic.

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        1472d 11h 47m 1 Andrew Purtell 24/Dec/14 19:48
        Andrew Purtell made changes -
        Field Original Value New Value
        Resolution Implemented [ 10 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Hide
        Andrew Purtell added a comment -

        Recent versions of HBase have a sleeper that will warn if it misses scheduled wakeup substantially, and will print GC collection times by collector.

        Show
        Andrew Purtell added a comment - Recent versions of HBase have a sleeper that will warn if it misses scheduled wakeup substantially, and will print GC collection times by collector.
        Hide
        Todd Lipcon added a comment -

        Another similar idea: this thread could run a quick ping to the host's default gateway, and include warning messages if it appears the network has gone down/flaked

        Show
        Todd Lipcon added a comment - Another similar idea: this thread could run a quick ping to the host's default gateway, and include warning messages if it appears the network has gone down/flaked
        Todd Lipcon created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development