Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.18.0
    • Fix Version/s: 0.18.2
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Simulated Datanodes should not include blocks that are still being written in their block report.

      Description

      TestReplication times out once every 5-6 attempts. I will attach log from a failed test.

      1. TestReplication.txt
        105 kB
        Raghu Angadi
      2. HADOOP-4314-branch-18.patch
        3 kB
        Raghu Angadi
      3. HADOOP-4314-branch-18.patch
        2 kB
        Raghu Angadi
      4. HADOOP-4314-branch-18.patch
        3 kB
        Raghu Angadi
      5. HADOOP-4314.patch
        3 kB
        Raghu Angadi
      6. HADOOP-4314-branch-18-java5-fix.patch
        0.6 kB
        Raghu Angadi
      7. HADOOP-4314-branch-18.patch
        3 kB
        Raghu Angadi

        Activity

        Hide
        Raghu Angadi added a comment -

        This affects only simulated datanodes.

        When a SimulatedDataNode sends block report, it includes the blocks that are currently being written. The reported size of such blocks is the default blocks size (64MB). NameNode assumes it is a real block. When the datanodes report the block when the writing finishes, NameNode rejects those blocks unless the size is same as 64MB. This bug always existed but only since 0.18, NameNode started rejecting blocks that are smaller than largest size reported by any datanode. Even replication attempts fail.

        The fix is to exclude blocks that are not "finalized" at the datanode in the block report from simulated datanodes. I will attach a patch.

        Show
        Raghu Angadi added a comment - This affects only simulated datanodes. When a SimulatedDataNode sends block report, it includes the blocks that are currently being written. The reported size of such blocks is the default blocks size (64MB). NameNode assumes it is a real block. When the datanodes report the block when the writing finishes, NameNode rejects those blocks unless the size is same as 64MB. This bug always existed but only since 0.18, NameNode started rejecting blocks that are smaller than largest size reported by any datanode. Even replication attempts fail. The fix is to exclude blocks that are not "finalized" at the datanode in the block report from simulated datanodes. I will attach a patch.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I think it is better to use blockMap.values() in the loop since only the values are needed.

        Show
        Tsz Wo Nicholas Sze added a comment - I think it is better to use blockMap.values() in the loop since only the values are needed.
        Hide
        Raghu Angadi added a comment -

        That is better. Thanks Nicholas. Updated patch is attached.

        The change in TestReplication.java is strictly not required but it just removes some code replication.

        Show
        Raghu Angadi added a comment - That is better. Thanks Nicholas. Updated patch is attached. The change in TestReplication.java is strictly not required but it just removes some code replication.
        Hide
        Raghu Angadi added a comment -

        update : trim the size of the array returned if required.

        Show
        Raghu Angadi added a comment - update : trim the size of the array returned if required.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12391252/HADOOP-4314.patch
        against trunk revision 700628.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12391252/HADOOP-4314.patch against trunk revision 700628. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3415/console This message is automatically generated.
        Hide
        Raghu Angadi added a comment -

        I just committed this. Hudson test failure is a known intermittent failure and not related to this patch.

        Show
        Raghu Angadi added a comment - I just committed this. Hudson test failure is a known intermittent failure and not related to this patch.
        Hide
        Raghu Angadi added a comment -

        Java 5 does not have Arrays.copyOf(). Attached patch will be committed to 0.18 in addition to the earlier patch.

        Show
        Raghu Angadi added a comment - Java 5 does not have Arrays.copyOf() . Attached patch will be committed to 0.18 in addition to the earlier patch.
        Hide
        Raghu Angadi added a comment -


        Updated patch for branch 0.18 == (earlier branch-18.patch + java5-fix.patch).

        Show
        Raghu Angadi added a comment - Updated patch for branch 0.18 == (earlier branch-18.patch + java5-fix.patch).
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #622 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/622/ )

          People

          • Assignee:
            Raghu Angadi
            Reporter:
            Raghu Angadi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development