Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Implement pipeline fault injection tests to simulate slow datanodes, out of memory problems and disk errors.

      1. h719_20091020.patch
        12 kB
        Tsz Wo Nicholas Sze
      2. h719_20091021.patch
        11 kB
        Tsz Wo Nicholas Sze

        Activity

        Tsz Wo Nicholas Sze created issue -
        Hide
        Tsz Wo Nicholas Sze added a comment -

        h719_20091020.patch: added 6 new fault injection tests

        Show
        Tsz Wo Nicholas Sze added a comment - h719_20091020.patch: added 6 new fault injection tests
        Tsz Wo Nicholas Sze made changes -
        Field Original Value New Value
        Attachment h719_20091020.patch [ 12422761 ]
        Tsz Wo Nicholas Sze made changes -
        Assignee Tsz Wo (Nicholas), SZE [ szetszwo ]
        Fix Version/s 0.21.0 [ 12314046 ]
        Fix Version/s 0.22.0 [ 12314241 ]
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12422761/h719_20091020.patch
        against trunk revision 826905.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 15 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12422761/h719_20091020.patch against trunk revision 826905. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/45/console This message is automatically generated.
        Hide
        Konstantin Boudnik added a comment -

        Few points:

        • instead of
          -    private BlockReceiver receiver; // The owner of this responder.
          +    final BlockReceiver receiver; // The owner of this responder.
          

          you can declare this aspect to be privileged - it will have access to the private members of the class.

        • the following method returns long but the javadoc is about int
          +   * Return a random integer uniformly distributed over the interval [min,max).
              * Assume max - min <= Integer.MAX_VALUE.
              */
             public static long nextRandomLong(final long min, final long max) {
          

        +1 Looks good otherwise

        Show
        Konstantin Boudnik added a comment - Few points: instead of - private BlockReceiver receiver; // The owner of this responder. + final BlockReceiver receiver; // The owner of this responder. you can declare this aspect to be privileged - it will have access to the private members of the class. the following method returns long but the javadoc is about int + * Return a random integer uniformly distributed over the interval [min,max). * Assume max - min <= Integer.MAX_VALUE. */ public static long nextRandomLong(final long min, final long max) { +1 Looks good otherwise
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Thanks, Cos. I will try "privileged". For the javadoc, a long in java is an integer but you are right that it is not clear.

        Show
        Tsz Wo Nicholas Sze added a comment - Thanks, Cos. I will try "privileged". For the javadoc, a long in java is an integer but you are right that it is not clear.
        Hide
        Konstantin Boudnik added a comment -

        privileged works flawlessly. I'm using it for the same BlockReceiverAspects in HDFS-521. If you'll such modifier I'll have one less conflict to resolve after all

        Show
        Konstantin Boudnik added a comment - privileged works flawlessly. I'm using it for the same BlockReceiverAspects in HDFS-521 . If you'll such modifier I'll have one less conflict to resolve after all
        Tsz Wo Nicholas Sze made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Tsz Wo Nicholas Sze added a comment -

        h719_20091021.patch: incorporated Cos' comments.

        Show
        Tsz Wo Nicholas Sze added a comment - h719_20091021.patch: incorporated Cos' comments.
        Tsz Wo Nicholas Sze made changes -
        Attachment h719_20091021.patch [ 12422830 ]
        Tsz Wo Nicholas Sze made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags [Reviewed]
        Hide
        Tsz Wo Nicholas Sze added a comment -
             [exec] +1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     +1 tests included.  The patch appears to include 15 new or modified tests.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        

        The new tests were passed in my local machine.

        In the previous Hudson run, it passed all the aop tests. The failure of TestFileAppend2.testComplexAppend is not related to this. There was no contrib test failing. I also checked the raw log but don't know why getting a -1 on contrib tests.

        In the new patch, all changes are localized to aop. So it won't affect contrib and other non-aop components. It is safe to commit it.

        Show
        Tsz Wo Nicholas Sze added a comment - [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 15 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. The new tests were passed in my local machine. In the previous Hudson run, it passed all the aop tests. The failure of TestFileAppend2.testComplexAppend is not related to this. There was no contrib test failing. I also checked the raw log but don't know why getting a -1 on contrib tests. In the new patch, all changes are localized to aop. So it won't affect contrib and other non-aop components. It is safe to commit it.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this to 0.21 and above.

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this to 0.21 and above.
        Tsz Wo Nicholas Sze made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #52 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/52/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #52 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/52/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #79 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/79/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #79 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/79/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #120 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/120/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #120 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/120/ )
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #78 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/78/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #78 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/78/ )
        Tom White made changes -
        Fix Version/s 0.22.0 [ 12314241 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development