Bigtop
  1. Bigtop
  2. BIGTOP-721

improve the package daemon status check, check twice by some delay if status doesn't match expected value

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Component/s: tests
    • Labels:
      None

      Description

      the current package test check the daemon can be correctly started, stopped, restarted and so on. It first changes the daemon status (start it, or stop it), then sleep 3001, then use function checkThat to check if the daemon status satisfy the matcher.

      However, there is a case that daemon status update speed is slightly different: some daemon status change a little bit slower than others. In this case, we want the test can give a delay and check the daemon status again if it doesn't match the matcher at the first place.

      Example is:

      service flume-ng-agent start
      __EOT__
      12/09/28 14:10:19 TRACE shell.Shell: 
      <stdout>
      Flume NG agent is not running..failed
      Starting Flume NG agent daemon (flume-ng-agent): ..done
      </stdout>
      12/09/28 14:10:22 TRACE shell.Shell: /bin/bash -s << __EOT__
      service flume-ng-agent status
      __EOT__
      12/09/28 14:10:22 TRACE shell.Shell: return: 3
      12/09/28 14:10:22 TRACE shell.Shell: 
      <stdout>
      Flume NG agent is not running..failed
      </stdout>
      12/09/28 14:10:25 TRACE shell.Shell: /bin/bash -s << __EOT__
      service flume-ng-agent status
      __EOT__
      12/09/28 14:10:25 TRACE shell.Shell: 
      <stdout>
      Flume NG agent is running..done
      

      in above example (real case) is in SLES11, the flume-ng-agent daemon status update is a little bit slower, after sleep 3001, the daemon doesn't shown as "running" yet, then we give another 3001 delay and check again, and it is shown as "running".

      1. BIGTOP_721-1.txt
        4 kB
        Johnny Zhang
      2. BIGTOP_721.txt
        4 kB
        Johnny Zhang

        Activity

        Hide
        Roman Shaposhnik added a comment -

        +1 and committed!

        Show
        Roman Shaposhnik added a comment - +1 and committed!
        Hide
        Johnny Zhang added a comment -

        this is a better patch.
        Please let me know if you think the function checkEquals() in PackageTestErrorProxy.java is necessary (we can call method matches() directly in PackageTestCommon.groovy). I am fine with both.

        Show
        Johnny Zhang added a comment - this is a better patch. Please let me know if you think the function checkEquals() in PackageTestErrorProxy.java is necessary (we can call method matches() directly in PackageTestCommon.groovy). I am fine with both.
        Hide
        Johnny Zhang added a comment -

        found some room to improve the patch, will submit a new one later

        Show
        Johnny Zhang added a comment - found some room to improve the patch, will submit a new one later
        Hide
        Johnny Zhang added a comment -

        of course, for case daemon status doesn't match expected by 3rd try, test will record the failure as usual

        service hadoop-yarn-proxyserver status
        __EOT__
        12/09/28 14:17:33 TRACE shell.Shell: return: 1
        12/09/28 14:17:33 TRACE shell.Shell: 
        <stdout>
        Hadoop proxyserver is dead and pid file exists ... failed!
        </stdout>
        12/09/28 14:17:36 TRACE shell.Shell: /bin/bash -s << __EOT__
        service hadoop-yarn-proxyserver status
        __EOT__
        12/09/28 14:17:36 TRACE shell.Shell: return: 1
        12/09/28 14:17:36 TRACE shell.Shell: 
        <stdout>
        Hadoop proxyserver is dead and pid file exists ... failed!
        </stdout>
        12/09/28 14:17:39 TRACE shell.Shell: /bin/bash -s << __EOT__
        service hadoop-yarn-proxyserver status
        __EOT__
        12/09/28 14:17:39 TRACE shell.Shell: return: 1
        12/09/28 14:17:39 TRACE shell.Shell: 
        <stdout>
        Hadoop proxyserver is dead and pid file exists ... failed!
        

        I have been tested both the positive and negative case already!

        Show
        Johnny Zhang added a comment - of course, for case daemon status doesn't match expected by 3rd try, test will record the failure as usual service hadoop-yarn-proxyserver status __EOT__ 12/09/28 14:17:33 TRACE shell.Shell: return: 1 12/09/28 14:17:33 TRACE shell.Shell: <stdout> Hadoop proxyserver is dead and pid file exists ... failed! </stdout> 12/09/28 14:17:36 TRACE shell.Shell: /bin/bash -s << __EOT__ service hadoop-yarn-proxyserver status __EOT__ 12/09/28 14:17:36 TRACE shell.Shell: return: 1 12/09/28 14:17:36 TRACE shell.Shell: <stdout> Hadoop proxyserver is dead and pid file exists ... failed! </stdout> 12/09/28 14:17:39 TRACE shell.Shell: /bin/bash -s << __EOT__ service hadoop-yarn-proxyserver status __EOT__ 12/09/28 14:17:39 TRACE shell.Shell: return: 1 12/09/28 14:17:39 TRACE shell.Shell: <stdout> Hadoop proxyserver is dead and pid file exists ... failed! I have been tested both the positive and negative case already!

          People

          • Assignee:
            Johnny Zhang
            Reporter:
            Johnny Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development