Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.18.0
    • None
    • None
    • Reviewed

    Description

      In LeaseManager,

      • removeExpiredLease(Lease) does not remove paths from sortedLeasesByPath
      • removeLease(...) should do sortedLeasesByPath.remove(src) no matter lease.hasPath() returns true or false.

      Attachments

        1. 3375_20080512.patch
          12 kB
          Tsz-wo Sze
        2. 3375_20080514.patch
          15 kB
          Tsz-wo Sze
        3. 3375_20080519.patch
          5 kB
          Tsz-wo Sze
        4. 3375_20080520.patch
          5 kB
          Tsz-wo Sze

        Issue Links

        Activity

          szetszwo Tsz-wo Sze added a comment -

          Also, lease related structure should store names and paths as String instead of StringBytesWritable. It will prevent unnecessary converting between String and StringBytesWritable.

          szetszwo Tsz-wo Sze added a comment - Also, lease related structure should store names and paths as String instead of StringBytesWritable. It will prevent unnecessary converting between String and StringBytesWritable.
          szetszwo Tsz-wo Sze added a comment -

          3375_20080512.patch: fixed sortedLeasesByPath and removed StringBytesWritable.

          szetszwo Tsz-wo Sze added a comment - 3375_20080512.patch: fixed sortedLeasesByPath and removed StringBytesWritable.
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12381917/3375_20080512.patch
          against trunk revision 655674.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12381917/3375_20080512.patch against trunk revision 655674. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/console This message is automatically generated.

          Nicholas reports that this may result in the loss of a lease record for every task that creates a file. This is a new problem not present in 0.16.

          chansler Robert Chansler added a comment - Nicholas reports that this may result in the loss of a lease record for every task that creates a file. This is a new problem not present in 0.16.

          Not a 0.17 problem after all (only trunk). I'm sorry for the confusion.

          chansler Robert Chansler added a comment - Not a 0.17 problem after all (only trunk). I'm sorry for the confusion.

          +1 code looks good. Is there a way to enhance an existing unit test to catch this situation?

          dhruba Dhruba Borthakur added a comment - +1 code looks good. Is there a way to enhance an existing unit test to catch this situation?
          szetszwo Tsz-wo Sze added a comment -

          3375_20080514.patch: added a test

          szetszwo Tsz-wo Sze added a comment - 3375_20080514.patch: added a test
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12382068/3375_20080514.patch
          against trunk revision 656270.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12382068/3375_20080514.patch against trunk revision 656270. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/console This message is automatically generated.
          szetszwo Tsz-wo Sze added a comment -

          3375_20080519.patch: reverted the change of removing StringBytesWritable since it needs more thought.

          szetszwo Tsz-wo Sze added a comment - 3375_20080519.patch: reverted the change of removing StringBytesWritable since it needs more thought.

          I like the latest patch better since it does not affect the disk format. Changing the StringBytesWritbale is bets done in a separate patch.

          dhruba Dhruba Borthakur added a comment - I like the latest patch better since it does not affect the disk format. Changing the StringBytesWritbale is bets done in a separate patch.
          szetszwo Tsz-wo Sze added a comment -

          > Changing the StringBytesWritbale is bets done in a separate patch.

          Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283.

          szetszwo Tsz-wo Sze added a comment - > Changing the StringBytesWritbale is bets done in a separate patch. Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283 .
          szetszwo Tsz-wo Sze added a comment -

          3375_20080520.patch: updated javadoc

          szetszwo Tsz-wo Sze added a comment - 3375_20080520.patch: updated javadoc
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12382400/3375_20080520.patch
          against trunk revision 658035.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12382400/3375_20080520.patch against trunk revision 658035. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/console This message is automatically generated.
          szetszwo Tsz-wo Sze added a comment -

          Two tests failed but none of them is related to the patch.

          szetszwo Tsz-wo Sze added a comment - Two tests failed but none of them is related to the patch. See HADOOP-3419 for TestFsck See HADOOP-3354 for TestEditLog

          +1. Code looks good.

          dhruba Dhruba Borthakur added a comment - +1. Code looks good.

          I just committed this. Thanks Nicholas!

          dhruba Dhruba Borthakur added a comment - I just committed this. Thanks Nicholas!
          hudson Hudson added a comment -
          hudson Hudson added a comment - Integrated in Hadoop-trunk #499 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/499/ )
          szetszwo Tsz-wo Sze added a comment -

          > Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283.

          Created HADOOP-3461

          szetszwo Tsz-wo Sze added a comment - > Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283 . Created HADOOP-3461

          People

            szetszwo Tsz-wo Sze
            szetszwo Tsz-wo Sze
            Votes:
            0 Vote for this issue
            Watchers:
            Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment