Hadoop Common
  1. Hadoop Common
  2. HADOOP-3992

Synthetic Load Generator for NameNode testing

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Added a synthetic load generation facility to the test directory.

      Description

      For performance and correctness testing. The goal is to faithfully simulate the load of a 5000-node cluster with a (comparatively) tiny number of nodes.

      1. LoadGeneratorUserGuide.pdf
        79 kB
        Hairong Kuang
      2. loadGenerator4.patch
        54 kB
        Hairong Kuang
      3. loadGenerator3.patch
        54 kB
        Hairong Kuang
      4. loadGenerator2.patch
        54 kB
        Hairong Kuang
      5. loadGenerator1.patch
        43 kB
        Hairong Kuang
      6. loadGenerator.patch
        31 kB
        Hairong Kuang

        Issue Links

          Activity

          Hide
          Hairong Kuang added a comment -

          This issue will focus on providing a tool for testing NameNode behavior under different client loads. It allows the user to generate different mixes of read, write, and list requests by specifying the probabilities of read and write. The user controls the the intensity of the load by adjusting parameters for the number of worker threads and the delay between operations.

          Show
          Hairong Kuang added a comment - This issue will focus on providing a tool for testing NameNode behavior under different client loads. It allows the user to generate different mixes of read, write, and list requests by specifying the probabilities of read and write. The user controls the the intensity of the load by adjusting parameters for the number of worker threads and the delay between operations.
          Hide
          Hairong Kuang added a comment -

          User guide is attached.

          Show
          Hairong Kuang added a comment - User guide is attached.
          Hide
          Hairong Kuang added a comment -

          A patch is attached for review. StructureGenerator.java and DataGenerator.java are for populating the test space. LoadGenerator.java is for generating load to the namenode to test its performance.

          Show
          Hairong Kuang added a comment - A patch is attached for review. StructureGenerator.java and DataGenerator.java are for populating the test space. LoadGenerator.java is for generating load to the namenode to test its performance.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Quick feedback:

          • need license headers
          • better use double instead of float for the probabilities
          • Some of the USAGE string is too long (e.g. StructureGenerator.USAGE). It is better to manually divide it into several lines. Otherwise, it may not easy to parse it if it is wrapped arbitrarily. For example,
            java LoadGenerator -readProbability <read probability> -writeProbabili
            ty <write probability> -root <root> -maxDelayBetweenOps <maxDelayBetwe
            enOpsInMS> -numOfThreads <numOfThreads> -elapsedTime <elapsedTimeInMin
            s>-startTime <startTimeInMS>
            
          • need unit tests?

          I will take a deeper look at the codes.

          Show
          Tsz Wo Nicholas Sze added a comment - Quick feedback: need license headers better use double instead of float for the probabilities Some of the USAGE string is too long (e.g. StructureGenerator.USAGE). It is better to manually divide it into several lines. Otherwise, it may not easy to parse it if it is wrapped arbitrarily. For example, java LoadGenerator -readProbability <read probability> -writeProbabili ty <write probability> -root <root> -maxDelayBetweenOps <maxDelayBetwe enOpsInMS> -numOfThreads <numOfThreads> -elapsedTime <elapsedTimeInMin s>-startTime <startTimeInMS> need unit tests? I will take a deeper look at the codes.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Below are more review comments:

          • Change comments to javadoc.
          • Print generic options in usage

          StructureGenerator.java

          • r.nextInt(maxWidth-minWidth) should be r.nextInt(maxWidth-minWidth+1).
          • Should we use IllegalArgumentException, instead of NumberFormatException?

          LoadGenerator.java

          • "-seed" here may not be useful since the randomness depends on the execution order of the threads.
          • Check individual range for readPr, writePr, maxDelayBetweenOps
          • In javadoc, we should mention that the unit of maxDelayBetweenOps is milliseconds.
          Show
          Tsz Wo Nicholas Sze added a comment - Below are more review comments: Change comments to javadoc. Print generic options in usage StructureGenerator.java r.nextInt(maxWidth-minWidth) should be r.nextInt(maxWidth-minWidth+1). Should we use IllegalArgumentException, instead of NumberFormatException? LoadGenerator.java "-seed" here may not be useful since the randomness depends on the execution order of the threads. Check individual range for readPr, writePr, maxDelayBetweenOps In javadoc, we should mention that the unit of maxDelayBetweenOps is milliseconds.
          Hide
          Hairong Kuang added a comment -

          This patch incorporates Nicholas's comments.

          Show
          Hairong Kuang added a comment - This patch incorporates Nicholas's comments.
          Hide
          Hairong Kuang added a comment -

          This patch includes an updated forrest user guide.

          Show
          Hairong Kuang added a comment - This patch includes an updated forrest user guide.
          Hide
          Tsz Wo Nicholas Sze added a comment -
          • need "return -1" when writePr<0 || writePr>1 and readPr<0 || readPr>1
          • the symbol MS for millisecond may be confusing because Ms means megasecond.
          • new File("/tmp") may not work in the OS not using "/" as the separator charcter.
          • In LoadGenerator.DFSClientThread.nextOp(), "float" should be "double"
          Show
          Tsz Wo Nicholas Sze added a comment - need "return -1" when writePr<0 || writePr>1 and readPr<0 || readPr>1 the symbol MS for millisecond may be confusing because Ms means megasecond. new File("/tmp") may not work in the OS not using "/" as the separator charcter. In LoadGenerator.DFSClientThread.nextOp(), "float" should be "double"
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12389790/loadGenerator2.patch
          against trunk revision 693705.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 19 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12389790/loadGenerator2.patch against trunk revision 693705. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 19 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3236/console This message is automatically generated.
          Hide
          Hairong Kuang added a comment -

          The patch incorporates Nicholas's new review comments.

          Show
          Hairong Kuang added a comment - The patch incorporates Nicholas's new review comments.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12389907/loadGenerator3.patch
          against trunk revision 693705.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 19 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12389907/loadGenerator3.patch against trunk revision 693705. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 19 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3245/console This message is automatically generated.
          Hide
          Hairong Kuang added a comment -

          The patch updates a javaDoc description. The failed core test is unrelated to this patch.

          Show
          Hairong Kuang added a comment - The patch updates a javaDoc description. The failed core test is unrelated to this patch.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I just committed it. Thanks Hairong.

          Show
          Tsz Wo Nicholas Sze added a comment - I just committed it. Thanks Hairong.

            People

            • Assignee:
              Hairong Kuang
              Reporter:
              Robert Chansler
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development