Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3375

Lease paths are sometimes not removed from LeaseManager.sortedLeasesByPath

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.18.0
    • None
    • None
    • Reviewed

    Description

      In LeaseManager,

      • removeExpiredLease(Lease) does not remove paths from sortedLeasesByPath
      • removeLease(...) should do sortedLeasesByPath.remove(src) no matter lease.hasPath() returns true or false.

      Attachments

        1. 3375_20080512.patch
          12 kB
          Tsz-wo Sze
        2. 3375_20080514.patch
          15 kB
          Tsz-wo Sze
        3. 3375_20080519.patch
          5 kB
          Tsz-wo Sze
        4. 3375_20080520.patch
          5 kB
          Tsz-wo Sze

        Issue Links

          Activity

            szetszwo Tsz-wo Sze added a comment -

            Also, lease related structure should store names and paths as String instead of StringBytesWritable. It will prevent unnecessary converting between String and StringBytesWritable.

            szetszwo Tsz-wo Sze added a comment - Also, lease related structure should store names and paths as String instead of StringBytesWritable. It will prevent unnecessary converting between String and StringBytesWritable.
            szetszwo Tsz-wo Sze added a comment -

            3375_20080512.patch: fixed sortedLeasesByPath and removed StringBytesWritable.

            szetszwo Tsz-wo Sze added a comment - 3375_20080512.patch: fixed sortedLeasesByPath and removed StringBytesWritable.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12381917/3375_20080512.patch
            against trunk revision 655674.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no tests are needed for this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed core unit tests.

            +1 contrib tests. The patch passed contrib unit tests.

            Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/testReport/
            Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/checkstyle-errors.html
            Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12381917/3375_20080512.patch against trunk revision 655674. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2454/console This message is automatically generated.

            Nicholas reports that this may result in the loss of a lease record for every task that creates a file. This is a new problem not present in 0.16.

            chansler Robert Chansler added a comment - Nicholas reports that this may result in the loss of a lease record for every task that creates a file. This is a new problem not present in 0.16.

            Not a 0.17 problem after all (only trunk). I'm sorry for the confusion.

            chansler Robert Chansler added a comment - Not a 0.17 problem after all (only trunk). I'm sorry for the confusion.

            +1 code looks good. Is there a way to enhance an existing unit test to catch this situation?

            dhruba Dhruba Borthakur added a comment - +1 code looks good. Is there a way to enhance an existing unit test to catch this situation?
            szetszwo Tsz-wo Sze added a comment -

            3375_20080514.patch: added a test

            szetszwo Tsz-wo Sze added a comment - 3375_20080514.patch: added a test
            hadoopqa Hadoop QA added a comment -

            +1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12382068/3375_20080514.patch
            against trunk revision 656270.

            +1 @author. The patch does not contain any @author tags.

            +1 tests included. The patch appears to include 4 new or modified tests.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed core unit tests.

            +1 contrib tests. The patch passed contrib unit tests.

            Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/testReport/
            Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/checkstyle-errors.html
            Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12382068/3375_20080514.patch against trunk revision 656270. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2471/console This message is automatically generated.
            szetszwo Tsz-wo Sze added a comment -

            3375_20080519.patch: reverted the change of removing StringBytesWritable since it needs more thought.

            szetszwo Tsz-wo Sze added a comment - 3375_20080519.patch: reverted the change of removing StringBytesWritable since it needs more thought.

            I like the latest patch better since it does not affect the disk format. Changing the StringBytesWritbale is bets done in a separate patch.

            dhruba Dhruba Borthakur added a comment - I like the latest patch better since it does not affect the disk format. Changing the StringBytesWritbale is bets done in a separate patch.
            szetszwo Tsz-wo Sze added a comment -

            > Changing the StringBytesWritbale is bets done in a separate patch.

            Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283.

            szetszwo Tsz-wo Sze added a comment - > Changing the StringBytesWritbale is bets done in a separate patch. Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283 .
            szetszwo Tsz-wo Sze added a comment -

            3375_20080520.patch: updated javadoc

            szetszwo Tsz-wo Sze added a comment - 3375_20080520.patch: updated javadoc
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12382400/3375_20080520.patch
            against trunk revision 658035.

            +1 @author. The patch does not contain any @author tags.

            +1 tests included. The patch appears to include 4 new or modified tests.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            -1 core tests. The patch failed core unit tests.

            +1 contrib tests. The patch passed contrib unit tests.

            Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/testReport/
            Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/checkstyle-errors.html
            Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12382400/3375_20080520.patch against trunk revision 658035. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2504/console This message is automatically generated.
            szetszwo Tsz-wo Sze added a comment -

            Two tests failed but none of them is related to the patch.

            szetszwo Tsz-wo Sze added a comment - Two tests failed but none of them is related to the patch. See HADOOP-3419 for TestFsck See HADOOP-3354 for TestEditLog

            +1. Code looks good.

            dhruba Dhruba Borthakur added a comment - +1. Code looks good.

            I just committed this. Thanks Nicholas!

            dhruba Dhruba Borthakur added a comment - I just committed this. Thanks Nicholas!
            hudson Hudson added a comment -
            hudson Hudson added a comment - Integrated in Hadoop-trunk #499 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/499/ )
            szetszwo Tsz-wo Sze added a comment -

            > Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283.

            Created HADOOP-3461

            szetszwo Tsz-wo Sze added a comment - > Yes, I plan to create another issue for removing StringBytesWritbale which was introduced in HADOOP-1283 . Created HADOOP-3461

            People

              szetszwo Tsz-wo Sze
              szetszwo Tsz-wo Sze
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: