Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0-alpha, 0.23.7
    • Component/s: datanode, namenode
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change

      Description

      I have a cluster that has 1800 datanodes. Each datanode has around 50000 blocks and sends a block report to the namenode once every hour. This means that the namenode processes a block report once every 2 seconds. Each block report contains all blocks that the datanode currently hosts. This makes the namenode compare a huge number of blocks that practically remains the same between two consecutive reports. This wastes CPU on the namenode.

      The problem becomes worse when the number of datanodes increases.

      One proposal is to make succeeding block reports (after a successful send of a full block report) be incremental. This will make the namenode process only those blocks that were added/deleted in the last period.

      1. explicitDeleteAcks.patch
        20 kB
        Tomasz Nykiel
      2. explicitAcks.patch-6
        31 kB
        Tomasz Nykiel
      3. explicitAcks.patch-5
        30 kB
        Tomasz Nykiel
      4. explicitAcks.patch-4
        30 kB
        Tomasz Nykiel
      5. explicitAcks.patch-3
        31 kB
        Tomasz Nykiel
      6. blockReportPeriod.patch
        3 kB
        dhruba borthakur

        Activity

        Hide
        Raghu Angadi added a comment -

        Namenode already knows the blocks added and deleted between block reports. I think block report exists only to catch some exceptions.

        How does delta help catch differences between namenode and datanode? I mean how can two sides be sure that they have the exact set? May be they can exchange some hash of all the block ids and revert to normal report only if there is a mismatch. Also, this hash could be order independent so that name updates with each block added instread of iterating over the set when required.

        Show
        Raghu Angadi added a comment - Namenode already knows the blocks added and deleted between block reports. I think block report exists only to catch some exceptions. How does delta help catch differences between namenode and datanode? I mean how can two sides be sure that they have the exact set? May be they can exchange some hash of all the block ids and revert to normal report only if there is a mismatch. Also, this hash could be order independent so that name updates with each block added instread of iterating over the set when required.
        Hide
        dhruba borthakur added a comment -

        You are right in that block-reports exists only to catch exception scenarios. Suppose a datanode fails to successfully deliver a blockreceived mesage to the namenode. The namenode won't know about it till the next block report. Also, if a rougue application deletes blk-* files on the datanode or a portion of the data directories go bad, this will be detected by the next block report.

        My proposal is the make the datanode remember that report that it sent to the namenode the last time. Only those blocks that got added/deleted since the last block report will be sent in the next incremental block report. This is a mechanism to ensure that the namenode and datanode are in sych as far as block information is concerned.

        In my test case, there were no two succeeding hourly block reports that were identical. I had 1800 data nodes and was running randomWriter. In this scenario, using a hash to identify identical block reports might not buy us anything.

        Show
        dhruba borthakur added a comment - You are right in that block-reports exists only to catch exception scenarios. Suppose a datanode fails to successfully deliver a blockreceived mesage to the namenode. The namenode won't know about it till the next block report. Also, if a rougue application deletes blk-* files on the datanode or a portion of the data directories go bad, this will be detected by the next block report. My proposal is the make the datanode remember that report that it sent to the namenode the last time. Only those blocks that got added/deleted since the last block report will be sent in the next incremental block report. This is a mechanism to ensure that the namenode and datanode are in sych as far as block information is concerned. In my test case, there were no two succeeding hourly block reports that were identical. I had 1800 data nodes and was running randomWriter. In this scenario, using a hash to identify identical block reports might not buy us anything.
        Hide
        Raghu Angadi added a comment -

        With deltas, how do we handle the case where a namenode request to remove a block is lost? Two sides can't be in sync with only one side maintaining diffs... I think.

        > In my test case, there were no two succeeding hourly block reports that were identical.

        This is normal. Did most of the reports result in deletion or addition of blocks during the report?

        > I had 1800 data nodes and was running randomWriter. In this scenario, using a hash to identify identical block reports might not buy us anything.

        Hash is not to identify if block report has changed. Both sides have up to date hash.. if hash is same then namenode and datanode have the same set of blocks. This has no relation to prev block report. There might be some fixes needed to make sure both sides see same set during a block report.

        Show
        Raghu Angadi added a comment - With deltas, how do we handle the case where a namenode request to remove a block is lost? Two sides can't be in sync with only one side maintaining diffs... I think. > In my test case, there were no two succeeding hourly block reports that were identical. This is normal. Did most of the reports result in deletion or addition of blocks during the report? > I had 1800 data nodes and was running randomWriter. In this scenario, using a hash to identify identical block reports might not buy us anything. Hash is not to identify if block report has changed. Both sides have up to date hash.. if hash is same then namenode and datanode have the same set of blocks. This has no relation to prev block report. There might be some fixes needed to make sure both sides see same set during a block report.
        Hide
        Raghu Angadi added a comment -

        > There might be some fixes needed to make sure both sides see same set during a block report.

        in normal case.

        Show
        Raghu Angadi added a comment - > There might be some fixes needed to make sure both sides see same set during a block report. in normal case.
        Hide
        Raghu Angadi added a comment -

        > With deltas, how do we handle the case where a namenode request to remove a block is lost?

        Actually I don't know what happens in such a case now. What ever datanode has is the master copy.
        Not sure what happens to blocks added after the report is sent but before it is processed.

        Show
        Raghu Angadi added a comment - > With deltas, how do we handle the case where a namenode request to remove a block is lost? Actually I don't know what happens in such a case now. What ever datanode has is the master copy. Not sure what happens to blocks added after the report is sent but before it is processed.
        Hide
        Raghu Angadi added a comment -

        > Actually I don't know what happens in such a case now. What ever datanode has is the master copy.
        > Not sure what happens to blocks added after the report is sent but before it is processed.

        This does lead to real problems see HADOOP-1093.

        Show
        Raghu Angadi added a comment - > Actually I don't know what happens in such a case now. What ever datanode has is the master copy. > Not sure what happens to blocks added after the report is sent but before it is processed. This does lead to real problems see HADOOP-1093 .
        Hide
        dhruba borthakur added a comment -

        The BlockReport exists because of the following reasons.

        1. User/admin manually deleted a bunch of blk-xxxx files in the datanode.
        This happens rarely and a daily-block-report is good enough to rectify this situation

        2. A heartbeat response from the namenode is lost.
        When this occurs, the datanode will remember this condition. The next block report will occur immediately after the next successfull heartbeat.

        Given the two above case, we can make the default block report period to be 1 day. This will reduce the CPU load on the namenode tremendously, especially on large clusters.

        Show
        dhruba borthakur added a comment - The BlockReport exists because of the following reasons. 1. User/admin manually deleted a bunch of blk-xxxx files in the datanode. This happens rarely and a daily-block-report is good enough to rectify this situation 2. A heartbeat response from the namenode is lost. When this occurs, the datanode will remember this condition. The next block report will occur immediately after the next successfull heartbeat. Given the two above case, we can make the default block report period to be 1 day. This will reduce the CPU load on the namenode tremendously, especially on large clusters.
        Hide
        dhruba borthakur added a comment -

        Here is a sample patch that increases the blockReport periodicity from 1 hour to 1 day. It also causes a blockReport to be sent after a failed heartbeat.

        I would like some comments/feedback on this approach.

        Show
        dhruba borthakur added a comment - Here is a sample patch that increases the blockReport periodicity from 1 hour to 1 day. It also causes a blockReport to be sent after a failed heartbeat. I would like some comments/feedback on this approach.
        Hide
        Raghu Angadi added a comment -

        Would n't this result in a possitive feedback loop for load on Namenode? i.e. most likely reason for a heart beat to fail is heavy load on Namenode, in such a case the patch will make every datanode to send blockReport greatly increasing the load on Namenode. Am I missing something?

        Show
        Raghu Angadi added a comment - Would n't this result in a possitive feedback loop for load on Namenode? i.e. most likely reason for a heart beat to fail is heavy load on Namenode, in such a case the patch will make every datanode to send blockReport greatly increasing the load on Namenode. Am I missing something?
        Hide
        dhruba borthakur added a comment -

        A short discussion with Sameer revealed that the case where admin/rogue-process manually deleted some blk-xxx files cannot be dealt with by an daily block report. A proposal is to do make the datanode compare its in-memory data structures with what is on disk. If it finds inconsistencies, then it send s a complete block report to the namenode.

        In response to Raghu's comments: I agree that sending a block report "immediately" after a successful heartbeat (that was preceeded by a failed heartbeat) could add more load on the namenode. It is enough if we can "hasten" the next block report rather than sending it immediately. I will work on this fix.

        Show
        dhruba borthakur added a comment - A short discussion with Sameer revealed that the case where admin/rogue-process manually deleted some blk-xxx files cannot be dealt with by an daily block report. A proposal is to do make the datanode compare its in-memory data structures with what is on disk. If it finds inconsistencies, then it send s a complete block report to the namenode. In response to Raghu's comments: I agree that sending a block report "immediately" after a successful heartbeat (that was preceeded by a failed heartbeat) could add more load on the namenode. It is enough if we can "hasten" the next block report rather than sending it immediately. I will work on this fix.
        Hide
        Hairong Kuang added a comment -

        When a name node thinks a data node is dead, it starts to replicate all the blocks belong to this datanode, which is very costly. A quick report block will help avoid these uneccessary block replications when the datanode is actually alive.

        Show
        Hairong Kuang added a comment - When a name node thinks a data node is dead, it starts to replicate all the blocks belong to this datanode, which is very costly. A quick report block will help avoid these uneccessary block replications when the datanode is actually alive.
        Hide
        Doug Cutting added a comment -

        From http://www.nabble.com/-tt16976556.html

        Long ago we talked of implementing partial, incremental block reports.
        We'd divide blockid space into 64 sections. The datanode would ask the
        namenode for the hash of its block ids in a section. Full block lists
        would then only be sent when the hash differs. Both sides would
        maintain hashes of all sections in memory. Then, instead of making a
        block report every hour, we'd make a 1/64 block id check every minute.

        Show
        Doug Cutting added a comment - From http://www.nabble.com/-tt16976556.html Long ago we talked of implementing partial, incremental block reports. We'd divide blockid space into 64 sections. The datanode would ask the namenode for the hash of its block ids in a section. Full block lists would then only be sent when the hash differs. Both sides would maintain hashes of all sections in memory. Then, instead of making a block report every hour, we'd make a 1/64 block id check every minute.
        Hide
        Tomasz Nykiel added a comment -

        To revive this discussion, I would have few comments about block reports.

        Processing a single block report has mainly two parts. The first part is to compute the difference between NN state and the report.
        The second stage involves applying the diff to NN - which contains blocksToAdd, blocksToRemove, blocksToInvalidat.

        1. Now we have explicit Received requests sent form datanodes to the namenode,
        hence the number of blocksToAdd should be very small.

        2. blocksToInvalidate consist of blocks that do not belong to any file, which is a abnormal situations.
        Hence, the number thereof should also be very small.

        3. We do not have explicit ACKs for blocks that are deleted from datanodes. Hence, the diff will contain all such deleted blocks.
        If we extend the block report interval to let's say 24 hours, we migh have a huge number of blocks that have been deleted, and need to be processed.

        I think it will be very beneficial to introduce explicit deletion acks.
        Since we care more about the blockReceived in general, than blockDeleted, we can send the deletion acks whenever we have any blockReceived acks to sent at the datanode side.
        Otherwise, we send the block deletion acks in some interval.

        With this change, the block report interval can be extended to let's say two times the basic interval.

        – Also a small change need to be made for sending blockDeleted ack.
        When the block invalidate command comes to the datanode, it should synchronously rename the block file (to some invalid block name).
        Then it can immediately notify the namenode that the block has been deleted.

        – Another improvement can be made on NN side: We can process the entire list of blocks within the FSNamesystem instead of processing them one-by-one.
        This saves some repetitive computation.

        I am attaching a diff that introduces these changes.

        Show
        Tomasz Nykiel added a comment - To revive this discussion, I would have few comments about block reports. Processing a single block report has mainly two parts. The first part is to compute the difference between NN state and the report. The second stage involves applying the diff to NN - which contains blocksToAdd, blocksToRemove, blocksToInvalidat. 1. Now we have explicit Received requests sent form datanodes to the namenode, hence the number of blocksToAdd should be very small. 2. blocksToInvalidate consist of blocks that do not belong to any file, which is a abnormal situations. Hence, the number thereof should also be very small. 3. We do not have explicit ACKs for blocks that are deleted from datanodes. Hence, the diff will contain all such deleted blocks. If we extend the block report interval to let's say 24 hours, we migh have a huge number of blocks that have been deleted, and need to be processed. I think it will be very beneficial to introduce explicit deletion acks. Since we care more about the blockReceived in general, than blockDeleted, we can send the deletion acks whenever we have any blockReceived acks to sent at the datanode side. Otherwise, we send the block deletion acks in some interval. With this change, the block report interval can be extended to let's say two times the basic interval. – Also a small change need to be made for sending blockDeleted ack. When the block invalidate command comes to the datanode, it should synchronously rename the block file (to some invalid block name). Then it can immediately notify the namenode that the block has been deleted. – Another improvement can be made on NN side: We can process the entire list of blocks within the FSNamesystem instead of processing them one-by-one. This saves some repetitive computation. I am attaching a diff that introduces these changes.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/998/
        -----------------------------------------------------------

        Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.

        Summary
        -------

        This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state.
        This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time.

        This addresses bug HDFS-395.
        https://issues.apache.org/jira/browse/HDFS-395

        Diffs


        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1142055
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1142055
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1142055
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1142055
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1142055
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1142055
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1142055

        Diff: https://reviews.apache.org/r/998/diff

        Testing
        -------

        Will do JUnit tests.

        Thanks,

        Tomasz

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/998/ ----------------------------------------------------------- Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang. Summary ------- This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state. This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time. This addresses bug HDFS-395 . https://issues.apache.org/jira/browse/HDFS-395 Diffs trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1142055 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1142055 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1142055 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1142055 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1142055 trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1142055 trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1142055 Diff: https://reviews.apache.org/r/998/diff Testing ------- Will do JUnit tests. Thanks, Tomasz
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/998/
        -----------------------------------------------------------

        (Updated 2011-07-01 20:34:17.116645)

        Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.

        Summary
        -------

        This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state.
        This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time.

        This addresses bug HDFS-395.
        https://issues.apache.org/jira/browse/HDFS-395

        Diffs (updated)


        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1142085
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1142085
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1142085
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1142085
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1142085
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1142085
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1142085

        Diff: https://reviews.apache.org/r/998/diff

        Testing
        -------

        Will do JUnit tests.

        Thanks,

        Tomasz

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/998/ ----------------------------------------------------------- (Updated 2011-07-01 20:34:17.116645) Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang. Summary ------- This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state. This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time. This addresses bug HDFS-395 . https://issues.apache.org/jira/browse/HDFS-395 Diffs (updated) trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1142085 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1142085 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1142085 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1142085 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1142085 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1142085 trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1142085 Diff: https://reviews.apache.org/r/998/diff Testing ------- Will do JUnit tests. Thanks, Tomasz
        Show
        Tomasz Nykiel added a comment - https://reviews.apache.org/r/998/
        Hide
        dhruba borthakur added a comment -

        "When the block invalidate command comes to the datanode, it should synchronously rename the block file (to some invalid block name).Then it can immediately notify the namenode that the block has been deleted."

        – why is this needed? can you pl explain?

        Show
        dhruba borthakur added a comment - "When the block invalidate command comes to the datanode, it should synchronously rename the block file (to some invalid block name).Then it can immediately notify the namenode that the block has been deleted." – why is this needed? can you pl explain?
        Hide
        Tomasz Nykiel added a comment -

        When the invalidate comes, the physical block file "blk_" is schedulaed for deletion asynchronously. Without renaming, but with explicit deletion ACKs it could theoretically happen:

        1. invalidate comes
        2. delete ACK is sent
        3. block report is constructed - it may still contain the block file because it's not deleted
        4. asynchronous delete takes place.

        So the NN would get contradicting information.
        If we just rename it to an invalid name e.g., "toDelete.blk_..." block report would not pick it up.

        Show
        Tomasz Nykiel added a comment - When the invalidate comes, the physical block file "blk_" is schedulaed for deletion asynchronously. Without renaming, but with explicit deletion ACKs it could theoretically happen: 1. invalidate comes 2. delete ACK is sent 3. block report is constructed - it may still contain the block file because it's not deleted 4. asynchronous delete takes place. So the NN would get contradicting information. If we just rename it to an invalid name e.g., "toDelete.blk_..." block report would not pick it up.
        Hide
        dhruba borthakur added a comment -

        The reason block deletion is async is because the deletion is done by the offserService thread in the datanode.

        are you assuming that a block file rename is quicker than a block file deletion and that is the reason why the datanode can afford to rename all block files synchronously?

        Show
        dhruba borthakur added a comment - The reason block deletion is async is because the deletion is done by the offserService thread in the datanode. are you assuming that a block file rename is quicker than a block file deletion and that is the reason why the datanode can afford to rename all block files synchronously?
        Hide
        Tomasz Nykiel added a comment -

        Actually, this is a good question. I was assuming that the rename would be relatively cheap.
        I guess, I will need to do some testing.

        Show
        Tomasz Nykiel added a comment - Actually, this is a good question. I was assuming that the rename would be relatively cheap. I guess, I will need to do some testing.
        Hide
        dhruba borthakur added a comment -

        from my earlier experience with developing local file systems (VxFS, ufs, XFS, etc), the cost of renaming a file is precisely the same as the cost of deleting a file.

        Show
        dhruba borthakur added a comment - from my earlier experience with developing local file systems (VxFS, ufs, XFS, etc), the cost of renaming a file is precisely the same as the cost of deleting a file.
        Hide
        Scott Carey added a comment -

        from my earlier experience with developing local file systems (VxFS, ufs, XFS, etc), the cost of renaming a file is precisely the same as the cost of deleting a file.

        With some file systems, yes. With others, not at all.

        XFS it is about the same speed to delete as rename, as long as the number of extents is low.

        For ext3, the cost to delete is FAR higher than the cost to rename for large files. Make a 2 GB file on ext3, time how long it takes to rename (almost instant) and then time how long it takes to remove (long!). This is because ext3 does not have extents, and must update the entry for every page in the file in order to delete. With 4k blocks, it is a half million entries that must be changed in a 2GB file.

        ext4 is faster at this, but still more costly than a move.

        I'm not sure either of these can be synchronous, and it may be best to batch up delete acks done asynchronously. Perhaps the block report should ignore 'deletes in progress' when reporting to the NN to avoid that race condition, or list them separately in the block report so the namenode has an opportunity to act on that information.

        Show
        Scott Carey added a comment - from my earlier experience with developing local file systems (VxFS, ufs, XFS, etc), the cost of renaming a file is precisely the same as the cost of deleting a file. With some file systems, yes. With others, not at all. XFS it is about the same speed to delete as rename, as long as the number of extents is low. For ext3, the cost to delete is FAR higher than the cost to rename for large files. Make a 2 GB file on ext3, time how long it takes to rename (almost instant) and then time how long it takes to remove (long!). This is because ext3 does not have extents, and must update the entry for every page in the file in order to delete. With 4k blocks, it is a half million entries that must be changed in a 2GB file. ext4 is faster at this, but still more costly than a move. I'm not sure either of these can be synchronous, and it may be best to batch up delete acks done asynchronously. Perhaps the block report should ignore 'deletes in progress' when reporting to the NN to avoid that race condition, or list them separately in the block report so the namenode has an opportunity to act on that information.
        Hide
        dhruba borthakur added a comment -

        I agree with Scott. It would be nice to make the deletions/renames occur async but somehow handle the race condition in the code.

        Show
        dhruba borthakur added a comment - I agree with Scott. It would be nice to make the deletions/renames occur async but somehow handle the race condition in the code.
        Hide
        Hairong Kuang added a comment -

        For the incremental block reports, we can always delay the deletion ack of a block until it is physically deleted from the disk to fix the race condition that Tom pointed out.

        However, async deletion does cause race condition in regular block reports (even without incremental block reports). In apache trunk, a block report is generated from in-memory block volume map. There is directory scanner that periodically sync the disk to the in-memory block map. So DN may first send a block report to NN indicating that the block is deleted, then sends a block report saying that it has the block because the scanner adds it back to the volumeMap before the block gets actually deleted.

        Another possible race condition occurs when a file gets deleted. NN removes the blocks of the file from blocksMap. Assume one block of the file has a block id of BlckID with a replica at DN1. Then NN may allocate the same block id with a newer generation stamp for a new file. If NN happens to allocate the block to DN1 and DN1 has not deleted the previous incarnation of the block from the disk yet, DN1 will end up with two blocks with the same id but different generation stamps.

        If rename is relatively cheaper in most file systems, why not using synchronous rename to fix all these possible race conditions? Also I would suggest that we fix the race conditions caused by asynchronous block deletions in a separate jira, so Tom could move forward with incremental block reports.

        Show
        Hairong Kuang added a comment - For the incremental block reports, we can always delay the deletion ack of a block until it is physically deleted from the disk to fix the race condition that Tom pointed out. However, async deletion does cause race condition in regular block reports (even without incremental block reports). In apache trunk, a block report is generated from in-memory block volume map. There is directory scanner that periodically sync the disk to the in-memory block map. So DN may first send a block report to NN indicating that the block is deleted, then sends a block report saying that it has the block because the scanner adds it back to the volumeMap before the block gets actually deleted. Another possible race condition occurs when a file gets deleted. NN removes the blocks of the file from blocksMap. Assume one block of the file has a block id of BlckID with a replica at DN1. Then NN may allocate the same block id with a newer generation stamp for a new file. If NN happens to allocate the block to DN1 and DN1 has not deleted the previous incarnation of the block from the disk yet, DN1 will end up with two blocks with the same id but different generation stamps. If rename is relatively cheaper in most file systems, why not using synchronous rename to fix all these possible race conditions? Also I would suggest that we fix the race conditions caused by asynchronous block deletions in a separate jira, so Tom could move forward with incremental block reports.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/998/
        -----------------------------------------------------------

        (Updated 2011-07-11 21:06:02.473171)

        Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.

        Changes
        -------

        • I removed renaming of the block and meta file from FSDataset. We should consider it in a separate diff, as suggested by Hairong. By doing so, I need to defer the notification until the FSAsyncDiskService deletes the files.
        • I added an optimization, which does not procuce ACKs for blocks which come from entirely deleted files (in this situation, the blocks are instantly removed from the blocks map at the NN side, and hence we need no ACK).

        Summary
        -------

        This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state.
        This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time.

        This addresses bug HDFS-395.
        https://issues.apache.org/jira/browse/HDFS-395

        Diffs (updated)


        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1145346
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1145346
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1145346

        Diff: https://reviews.apache.org/r/998/diff

        Testing
        -------

        Will do JUnit tests.

        Thanks,

        Tomasz

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/998/ ----------------------------------------------------------- (Updated 2011-07-11 21:06:02.473171) Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang. Changes ------- I removed renaming of the block and meta file from FSDataset. We should consider it in a separate diff, as suggested by Hairong. By doing so, I need to defer the notification until the FSAsyncDiskService deletes the files. I added an optimization, which does not procuce ACKs for blocks which come from entirely deleted files (in this situation, the blocks are instantly removed from the blocks map at the NN side, and hence we need no ACK). Summary ------- This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state. This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time. This addresses bug HDFS-395 . https://issues.apache.org/jira/browse/HDFS-395 Diffs (updated) trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1145346 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1145346 trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1145346 Diff: https://reviews.apache.org/r/998/diff Testing ------- Will do JUnit tests. Thanks, Tomasz
        Hide
        Todd Lipcon added a comment -

        Haven't we already implemented the asynchronous deletion functionality via HDFS-611?

        Show
        Todd Lipcon added a comment - Haven't we already implemented the asynchronous deletion functionality via HDFS-611 ?
        Hide
        Tomasz Nykiel added a comment -

        Yes, correct. The deletion is done asynchronously. Hence, when we schedule the deletion, the block file and the metafile are not instantly deleted, and we need to defer the notification until they are actually deleted by the FSAsyncDiskService.

        Show
        Tomasz Nykiel added a comment - Yes, correct. The deletion is done asynchronously. Hence, when we schedule the deletion, the block file and the metafile are not instantly deleted, and we need to defer the notification until they are actually deleted by the FSAsyncDiskService.
        Hide
        Todd Lipcon added a comment -

        Why do we need to defer the notification? Since they're immediately moved out of the "current" directory into the "toBeDeleted" dir, they are as good as deleted even if we crash and restart.

        Show
        Todd Lipcon added a comment - Why do we need to defer the notification? Since they're immediately moved out of the "current" directory into the "toBeDeleted" dir, they are as good as deleted even if we crash and restart.
        Hide
        Tomasz Nykiel added a comment -

        Todd, please correct me if I am wrong, but I don't see that this is happening in FSDataset.invalidate().
        The previous patch was renaming the block and metafile files, and notifying the NN immediately (please see the discussion above).
        If the files were moved to "toBeDeleted" or were renamed, then, of course, we do not need to wait.

        Show
        Tomasz Nykiel added a comment - Todd, please correct me if I am wrong, but I don't see that this is happening in FSDataset.invalidate(). The previous patch was renaming the block and metafile files, and notifying the NN immediately (please see the discussion above). If the files were moved to "toBeDeleted" or were renamed, then, of course, we do not need to wait.
        Hide
        Suresh Srinivas added a comment -

        From my understanding, when a namenode removes a block:

        1. NN first deletes it in the block map and adds it to invalidate set for the datanodes.
        2. The blocks are then deleted from invalidate set, when it is sent for deletion to the datanodes.

        Namenode already has deleted from it's data structures the block information. Given this I am not sure what purpose the delete ack serves.

        I wanted to do a variant of this for the following reasons:
        If accidentally files related to block replicas are deleted on the datanode, without a periodic block reports the namenode is not aware of this loss. Also a replica file could be truncated/modified. The change I was thinking was in DirectoryScanner; it currently reconciles the difference between the block information in datanode process with what is on the disk. This difference is sent to namenode in the block report. Instead of block report, we could send this diff to the namenode. With this we can reduce the frequency of block report.

        +1 for renaming the file to target it for deletion. This addresses some of the race conditions that Hairong brought up.

        Show
        Suresh Srinivas added a comment - From my understanding, when a namenode removes a block: NN first deletes it in the block map and adds it to invalidate set for the datanodes. The blocks are then deleted from invalidate set, when it is sent for deletion to the datanodes. Namenode already has deleted from it's data structures the block information. Given this I am not sure what purpose the delete ack serves. I wanted to do a variant of this for the following reasons: If accidentally files related to block replicas are deleted on the datanode, without a periodic block reports the namenode is not aware of this loss. Also a replica file could be truncated/modified. The change I was thinking was in DirectoryScanner; it currently reconciles the difference between the block information in datanode process with what is on the disk. This difference is sent to namenode in the block report. Instead of block report, we could send this diff to the namenode. With this we can reduce the frequency of block report. +1 for renaming the file to target it for deletion. This addresses some of the race conditions that Hairong brought up.
        Hide
        Todd Lipcon added a comment -

        Todd, please correct me if I am wrong, but I don't see that this is happening in FSDataset.invalidate().

        Ah, it seems we have three separate but similar AsyncDiskServices in Hadoop The MRAsyncDiskService class from MAPREDUCE-1302 uses a separate "toBeDeleted" directory as described above - I thought we used the same technique in the HDFS side, but it appears it's just a deferred delete of the file in the same location as you described.

        Can we share some code here? There's already another AsyncDiskService superclass in common.

        Show
        Todd Lipcon added a comment - Todd, please correct me if I am wrong, but I don't see that this is happening in FSDataset.invalidate(). Ah, it seems we have three separate but similar AsyncDiskServices in Hadoop The MRAsyncDiskService class from MAPREDUCE-1302 uses a separate "toBeDeleted" directory as described above - I thought we used the same technique in the HDFS side, but it appears it's just a deferred delete of the file in the same location as you described. Can we share some code here? There's already another AsyncDiskService superclass in common.
        Hide
        Hairong Kuang added a comment -

        > NN first deletes it in the block map and adds it to invalidate set for the datanodes.

        Only blocks belong to a deleted file are removed from the block map. For those blocks, Tom's patch has an optimization that datanode does not send to an ack back.

        But for an excessive replica, it remains in the block map and excessiveBlockMap until an ack is back. They are the ones that need explicit acknowledgment.

        Show
        Hairong Kuang added a comment - > NN first deletes it in the block map and adds it to invalidate set for the datanodes. Only blocks belong to a deleted file are removed from the block map. For those blocks, Tom's patch has an optimization that datanode does not send to an ack back. But for an excessive replica, it remains in the block map and excessiveBlockMap until an ack is back. They are the ones that need explicit acknowledgment.
        Hide
        Suresh Srinivas added a comment -

        > But for an excessive replica, it remains in the block map and excessiveBlockMap until an ack is back. They are the ones that need explicit acknowledgment.

        I know that for deleted files, when a previously deleted replica is reported by datanode to namenode, NN can again delete the replicas because the file does not exist. But I wonder why we do not remove excess replica also from the map on scheduling deletion.

        However, this could come very handy in HA implementation. Currently all namespace operations goes to standby through editlog. However having the delete acks creates a channel to report block deletions also to standby. So I am +1 on delete acks from the perspective of HA.

        Directory scanner should use the mechanism in this jira to send difference between in memory block map and the disk. This could be done in another jira.

        Show
        Suresh Srinivas added a comment - > But for an excessive replica, it remains in the block map and excessiveBlockMap until an ack is back. They are the ones that need explicit acknowledgment. I know that for deleted files, when a previously deleted replica is reported by datanode to namenode, NN can again delete the replicas because the file does not exist. But I wonder why we do not remove excess replica also from the map on scheduling deletion. However, this could come very handy in HA implementation. Currently all namespace operations goes to standby through editlog. However having the delete acks creates a channel to report block deletions also to standby. So I am +1 on delete acks from the perspective of HA. Directory scanner should use the mechanism in this jira to send difference between in memory block map and the disk. This could be done in another jira.
        Hide
        Suresh Srinivas added a comment -

        Tomasz, cool stuff.

        Some comments:

        1. ReceivedDeletedBlockInfo.java is missing in the patch.
        2. DataNodeProtocol.java
          • Please update the DataNodeProtocol#blockReceivedAndDeleted() javadoc to reflect the new functionality.
        3. DataNode.java
          • Minor: DataNode.java indendtation of receivedAndDeletedBlockList.wait()
          • I am not clear on why you are setting deleteReportInterval to DFS_BLOCKREPORT_INTERVAL_MSEC_KEY and blockReportInterval to 2*DFS_BLOCKREPORT_INTERVAL_MSEC_KEY. Can you retain blockReportInterval same as before and use some suitable value for deleteReportInterval. This could be 100 times heartbeat. The reason to be aggressive with this is, NN is going to keep the replica in a data structure, until delete ack. I know that it is piggy backed with blockReceived; but still see no bad side to sending it more frequently.
          • reportReceivedDeletedBlocks() - save the pendingReceivedRequests in currentReceived in side synchronixed(receivedAndDeletedBlockList)?
          • pendingReceivedRequests accessed without lock in OfferService - you can move this check into reportReceivedDeletedBlocks()
        Show
        Suresh Srinivas added a comment - Tomasz, cool stuff. Some comments: ReceivedDeletedBlockInfo.java is missing in the patch. DataNodeProtocol.java Please update the DataNodeProtocol#blockReceivedAndDeleted() javadoc to reflect the new functionality. DataNode.java Minor: DataNode.java indendtation of receivedAndDeletedBlockList.wait() I am not clear on why you are setting deleteReportInterval to DFS_BLOCKREPORT_INTERVAL_MSEC_KEY and blockReportInterval to 2*DFS_BLOCKREPORT_INTERVAL_MSEC_KEY. Can you retain blockReportInterval same as before and use some suitable value for deleteReportInterval. This could be 100 times heartbeat. The reason to be aggressive with this is, NN is going to keep the replica in a data structure, until delete ack. I know that it is piggy backed with blockReceived; but still see no bad side to sending it more frequently. reportReceivedDeletedBlocks() - save the pendingReceivedRequests in currentReceived in side synchronixed(receivedAndDeletedBlockList)? pendingReceivedRequests accessed without lock in OfferService - you can move this check into reportReceivedDeletedBlocks()
        Hide
        Suresh Srinivas added a comment -

        Just noticed that the attached patch explicitDeleteAcks.patch is different from Diff: https://reviews.apache.org/r/998/diff

        For 988/diff, additional comments:

        1. Are you not planning to rename the block file to be deleted? This is done in explicitDeleteAcks.patch
        2. Ignore comment 1. in my previous comment. I do not understand ReceivedDeletedBlockInfo#WILDCARD_HINT is being used.
        Show
        Suresh Srinivas added a comment - Just noticed that the attached patch explicitDeleteAcks.patch is different from Diff: https://reviews.apache.org/r/998/diff For 988/diff, additional comments: Are you not planning to rename the block file to be deleted? This is done in explicitDeleteAcks.patch Ignore comment 1. in my previous comment. I do not understand ReceivedDeletedBlockInfo#WILDCARD_HINT is being used.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/998/
        -----------------------------------------------------------

        (Updated 2011-07-28 00:40:14.000447)

        Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.

        Changes
        -------

        Addressed the comments.

        Summary
        -------

        This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state.
        This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time.

        This addresses bug HDFS-395.
        https://issues.apache.org/jira/browse/HDFS-395

        Diffs (updated)


        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1151690
        trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1151690
        trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1151690

        Diff: https://reviews.apache.org/r/998/diff

        Testing
        -------

        Will do JUnit tests.

        Thanks,

        Tomasz

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/998/ ----------------------------------------------------------- (Updated 2011-07-28 00:40:14.000447) Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang. Changes ------- Addressed the comments. Summary ------- This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state. This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time. This addresses bug HDFS-395 . https://issues.apache.org/jira/browse/HDFS-395 Diffs (updated) trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1151690 trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1151690 trunk/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1151690 Diff: https://reviews.apache.org/r/998/diff Testing ------- Will do JUnit tests. Thanks, Tomasz
        Hide
        Tomasz Nykiel added a comment -

        Suresh,

        Thanks for the review. I addressed all comments except one.
        Accessing pendingReceivedRequests in offerService is fine, since we do not modify it.
        I made it volatile so it's not cached thread-locally.
        In the worst case, there might a delay of one loop iteration of offerservice, when a block is already added to the list but pendingReceivedRequests not updated.

        Regarding your question about deletion.
        I think we concluded that this issue should be updated separately, as it is not strictly related to explicit ACK.

        I will also have to run the JUnit tests.

        Show
        Tomasz Nykiel added a comment - Suresh, Thanks for the review. I addressed all comments except one. Accessing pendingReceivedRequests in offerService is fine, since we do not modify it. I made it volatile so it's not cached thread-locally. In the worst case, there might a delay of one loop iteration of offerservice, when a block is already added to the list but pendingReceivedRequests not updated. Regarding your question about deletion. I think we concluded that this issue should be updated separately, as it is not strictly related to explicit ACK. I will also have to run the JUnit tests.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12488048/explicitAcks.patch-3
        against trunk revision 1155998.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1061//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12488048/explicitAcks.patch-3 against trunk revision 1155998. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1061//console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Hi Tomasz, do you still have time to follow up this issue?

        Show
        Tsz Wo Nicholas Sze added a comment - Hi Tomasz, do you still have time to follow up this issue?
        Hide
        Tsz Wo Nicholas Sze added a comment -
        • Why comment out offerService()?
          -            offerService();
          +            //offerService();
          
        • I think ReceivedDeletedBlockInfo is not needed. We may simply pass received blocks and deleted blocks in separated arrays.
        Show
        Tsz Wo Nicholas Sze added a comment - Why comment out offerService()? - offerService(); + //offerService(); I think ReceivedDeletedBlockInfo is not needed. We may simply pass received blocks and deleted blocks in separated arrays.
        Hide
        Tomasz Nykiel added a comment -

        @Nicholas

        I will follow up after Aug 16th. I am away in Asia now, and I don't have access to any dev environment. Thanks.

        Show
        Tomasz Nykiel added a comment - @Nicholas I will follow up after Aug 16th. I am away in Asia now, and I don't have access to any dev environment. Thanks.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        @Tomasz, that will be great. Thanks a lot!

        Show
        Tsz Wo Nicholas Sze added a comment - @Tomasz, that will be great. Thanks a lot!
        Hide
        Suresh Srinivas added a comment -

        I think ReceivedDeletedBlockInfo is not needed. We may simply pass received blocks and deleted blocks in separated arrays.

        I think this is a bad idea. The order of received and deleted is important. I would retain the existing code.

        Show
        Suresh Srinivas added a comment - I think ReceivedDeletedBlockInfo is not needed. We may simply pass received blocks and deleted blocks in separated arrays. I think this is a bad idea. The order of received and deleted is important. I would retain the existing code.
        Hide
        Tomasz Nykiel added a comment -

        I am quite new to the process, and I am experiencing problems with building the trunk:

        [ivy:resolve] ==== apache-snapshot: tried
        [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.pom
        [ivy:resolve] – artifact commons-configuration#commons-configuration;1.6!commons-configuration.jar:
        [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar
        [ivy:resolve] ==== maven2: tried
        [ivy:resolve] http://repo1.maven.org/maven2/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.pom
        [ivy:resolve] [FAILED ] javax.jms#jms;1.1!jms.jar: (0ms)
        [ivy:resolve] ==== apache-snapshot: tried
        [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/javax/jms/jms/1.1/jms-1.1.jar
        [ivy:resolve] ==== maven2: tried
        [ivy:resolve] http://repo1.maven.org/maven2/javax/jms/jms/1.1/jms-1.1.jar
        [ivy:resolve] [FAILED ] com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar: (0ms)
        [ivy:resolve] ==== apache-snapshot: tried
        [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/com/sun/jdmk/jmxtools/1.2.1/jmxtools-1.2.1.jar
        [ivy:resolve] ==== maven2: tried
        [ivy:resolve] http://repo1.maven.org/maven2/com/sun/jdmk/jmxtools/1.2.1/jmxtools-1.2.1.jar
        [ivy:resolve] [FAILED ] com.sun.jmx#jmxri;1.2.1!jmxri.jar: (0ms)
        [ivy:resolve] ==== apache-snapshot: tried
        [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/com/sun/jmx/jmxri/1.2.1/jmxri-1.2.1.jar
        [ivy:resolve] ==== maven2: tried
        [ivy:resolve] http://repo1.maven.org/maven2/com/sun/jmx/jmxri/1.2.1/jmxri-1.2.1.jar
        [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
        [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
        [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
        [ivy:resolve] :: commons-configuration#commons-configuration;1.6: not found
        [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
        [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
        [ivy:resolve] :: FAILED DOWNLOADS ::
        [ivy:resolve] :: ^ see resolution messages for details ^ ::
        [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
        [ivy:resolve] :: javax.jms#jms;1.1!jms.jar
        [ivy:resolve] :: com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar
        [ivy:resolve] :: com.sun.jmx#jmxri;1.2.1!jmxri.jar
        [ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::

        It seems that there is something wrong with online repositories.
        I am not sure if this seems familiar to anyone ?
        I would appreciate help.

        @Suresh I agree with the argument to have a single array of acks.

        Show
        Tomasz Nykiel added a comment - I am quite new to the process, and I am experiencing problems with building the trunk: [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.pom [ivy:resolve] – artifact commons-configuration#commons-configuration;1.6!commons-configuration.jar: [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.pom [ivy:resolve] [FAILED ] javax.jms#jms;1.1!jms.jar: (0ms) [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/javax/jms/jms/1.1/jms-1.1.jar [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/javax/jms/jms/1.1/jms-1.1.jar [ivy:resolve] [FAILED ] com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar: (0ms) [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/com/sun/jdmk/jmxtools/1.2.1/jmxtools-1.2.1.jar [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/com/sun/jdmk/jmxtools/1.2.1/jmxtools-1.2.1.jar [ivy:resolve] [FAILED ] com.sun.jmx#jmxri;1.2.1!jmxri.jar: (0ms) [ivy:resolve] ==== apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/com/sun/jmx/jmxri/1.2.1/jmxri-1.2.1.jar [ivy:resolve] ==== maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/com/sun/jmx/jmxri/1.2.1/jmxri-1.2.1.jar [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: commons-configuration#commons-configuration;1.6: not found [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: FAILED DOWNLOADS :: [ivy:resolve] :: ^ see resolution messages for details ^ :: [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: [ivy:resolve] :: javax.jms#jms;1.1!jms.jar [ivy:resolve] :: com.sun.jdmk#jmxtools;1.2.1!jmxtools.jar [ivy:resolve] :: com.sun.jmx#jmxri;1.2.1!jmxri.jar [ivy:resolve] :::::::::::::::::::::::::::::::::::::::::::::: It seems that there is something wrong with online repositories. I am not sure if this seems familiar to anyone ? I would appreciate help. @Suresh I agree with the argument to have a single array of acks.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        > > I think ReceivedDeletedBlockInfo is not needed. We may simply pass received blocks and deleted blocks in separated arrays.
        >
        > I think this is a bad idea. The order of received and deleted is important. I would retain the existing code.

        The order is important only if the same block is added or deleted multiple times but the final result requires at most one add/delete operations. How about we resolve the operations in the datanodes so that the namenode don't have to add and delete the same block multiple times?

        Show
        Tsz Wo Nicholas Sze added a comment - > > I think ReceivedDeletedBlockInfo is not needed. We may simply pass received blocks and deleted blocks in separated arrays. > > I think this is a bad idea. The order of received and deleted is important. I would retain the existing code. The order is important only if the same block is added or deleted multiple times but the final result requires at most one add/delete operations. How about we resolve the operations in the datanodes so that the namenode don't have to add and delete the same block multiple times?
        Hide
        Tsz Wo Nicholas Sze added a comment -

        @Tomasz, what command have you used to build?

        Show
        Tsz Wo Nicholas Sze added a comment - @Tomasz, what command have you used to build?
        Hide
        Tomasz Nykiel added a comment -

        @Nicholas

        for building I used "ant clean compile" "ant clean test" "ant clean compile-hdfs-classes", always the same problem.
        When I try to access for instance:
        "https://repository.apache.org/content/repositories/snapshots/com/sun/jmx/jmxri/1.2.1/jmxri-1.2.1.jar" (which ant is trying to fetch) with wget, I get 404NotFound.

        As for the ReceivedDeletedBlockInfo, I still think it's safer to preserve the order. For sure, it mattered in our FB implementation.
        In fact separate structures were implemented first, and then collapsed to one structure for both types of acks, solely for preserving the order. In apache it could probably be resolved.

        On the other hand, keeping a unified structure for both types, makes the dn code cleaner, (e.g., synchronization), and introduces minimal space overhead.

        Show
        Tomasz Nykiel added a comment - @Nicholas for building I used "ant clean compile" "ant clean test" "ant clean compile-hdfs-classes", always the same problem. When I try to access for instance: "https://repository.apache.org/content/repositories/snapshots/com/sun/jmx/jmxri/1.2.1/jmxri-1.2.1.jar" (which ant is trying to fetch) with wget, I get 404NotFound. As for the ReceivedDeletedBlockInfo, I still think it's safer to preserve the order. For sure, it mattered in our FB implementation. In fact separate structures were implemented first, and then collapsed to one structure for both types of acks, solely for preserving the order. In apache it could probably be resolved. On the other hand, keeping a unified structure for both types, makes the dn code cleaner, (e.g., synchronization), and introduces minimal space overhead.
        Hide
        Tomasz Nykiel added a comment -

        Adjusted to recent changes.

        Show
        Tomasz Nykiel added a comment - Adjusted to recent changes.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/998/
        -----------------------------------------------------------

        (Updated 2011-08-23 22:55:47.297820)

        Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.

        Changes
        -------

        Adjusted to recent code changes.
        Fized offerService comment.

        Summary
        -------

        This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state.
        This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time.

        This addresses bug HDFS-395.
        https://issues.apache.org/jira/browse/HDFS-395

        Diffs (updated)


        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1160919
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1160919
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1160919
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1160919
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1160919
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1160919
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1160919
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION
        trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1160919
        trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1160919

        Diff: https://reviews.apache.org/r/998/diff

        Testing
        -------

        Will do JUnit tests.

        Thanks,

        Tomasz

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/998/ ----------------------------------------------------------- (Updated 2011-08-23 22:55:47.297820) Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang. Changes ------- Adjusted to recent code changes. Fized offerService comment. Summary ------- This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state. This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time. This addresses bug HDFS-395 . https://issues.apache.org/jira/browse/HDFS-395 Diffs (updated) trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1160919 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1160919 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1160919 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1160919 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1160919 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1160919 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1160919 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1160919 trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1160919 Diff: https://reviews.apache.org/r/998/diff Testing ------- Will do JUnit tests. Thanks, Tomasz
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12491399/explicitAcks.patch-4
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1149//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1149//artifact/trunk/target/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1149//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12491399/explicitAcks.patch-4 against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1149//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1149//artifact/trunk/target/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1149//console This message is automatically generated.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/998/
        -----------------------------------------------------------

        (Updated 2011-08-24 03:33:32.850921)

        Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang.

        Summary
        -------

        This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state.
        This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time.

        This addresses bug HDFS-395.
        https://issues.apache.org/jira/browse/HDFS-395

        Diffs (updated)


        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1160950
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1160950
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1160950
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1160950
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1160950
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1160950
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1160950
        trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION
        trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1160950
        trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1160950

        Diff: https://reviews.apache.org/r/998/diff

        Testing
        -------

        Will do JUnit tests.

        Thanks,

        Tomasz

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/998/ ----------------------------------------------------------- (Updated 2011-08-24 03:33:32.850921) Review request for hadoop-hdfs, Dhruba Borthakur and Hairong Kuang. Summary ------- This patch introduces explicit ACKs sent form datanodes to the namenode in order to minimize the difference between NN blocks info and DN state. This will allow for sending the full block reports less frequently, which in turn will minimize the overhead of processing them, and blocking the namenode for extended time. This addresses bug HDFS-395 . https://issues.apache.org/jira/browse/HDFS-395 Diffs (updated) trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java 1160950 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java 1160950 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java 1160950 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java 1160950 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1160950 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java 1160950 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java 1160950 trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java PRE-CREATION trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java 1160950 trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java 1160950 Diff: https://reviews.apache.org/r/998/diff Testing ------- Will do JUnit tests. Thanks, Tomasz
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12491439/explicitAcks.patch-5
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 6 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these core unit tests:

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1150//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1150//artifact/trunk/target/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1150//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12491439/explicitAcks.patch-5 against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1150//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1150//artifact/trunk/target/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1150//console This message is automatically generated.
        Hide
        Hairong Kuang added a comment -

        The patch looks good to me and the failed tests do not seem to be related to this patch. Suresh/Nicholas, any more comments? Is it ready for commit?

        Show
        Hairong Kuang added a comment - The patch looks good to me and the failed tests do not seem to be related to this patch. Suresh/Nicholas, any more comments? Is it ready for commit?
        Hide
        Tsz Wo Nicholas Sze added a comment -
        • Please click the checkbox "Grant license to ASF ..."
        • The patch probably have to be updated with trunk.

        I don't have comments other than that.

        Show
        Tsz Wo Nicholas Sze added a comment - Please click the checkbox "Grant license to ASF ..." The patch probably have to be updated with trunk. I don't have comments other than that.
        Hide
        Tomasz Nykiel added a comment -

        The trunk build is failing:
        hadoop-yarn-api ................................... FAILURE [0.327s]

        Show
        Tomasz Nykiel added a comment - The trunk build is failing: hadoop-yarn-api ................................... FAILURE [0.327s]
        Hide
        Hairong Kuang added a comment -

        I've committed this. Thank Tomasz for your great work and thank Suresh and Nicholas for your review!

        Show
        Hairong Kuang added a comment - I've committed this. Thank Tomasz for your great work and thank Suresh and Nicholas for your review!
        Hide
        Mahadev konar added a comment -

        Hairong,
        Can you please add the fix version to the jira?

        Show
        Mahadev konar added a comment - Hairong, Can you please add the fix version to the jira?
        Hide
        Hairong Kuang added a comment -

        Sure, marked it as 0.23.

        Show
        Hairong Kuang added a comment - Sure, marked it as 0.23.
        Hide
        Tomasz Nykiel added a comment -

        Thanks everyone.

        Show
        Tomasz Nykiel added a comment - Thanks everyone.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Nice work!

        Show
        Tsz Wo Nicholas Sze added a comment - Nice work!
        Hide
        Mahadev konar added a comment -

        Hairong,
        I dont see this in the 0.23 branch. Do you intend to backport?

        Show
        Mahadev konar added a comment - Hairong, I dont see this in the 0.23 branch. Do you intend to backport?
        Hide
        Hairong Kuang added a comment -

        My bad. I did not notice that 0.23 is branched. Updated the fix version to be 0.24 and updated the change log of the jira to be in the trunk section. Unless anybody else really need it, I do not plan to backport it to 0.23.

        Show
        Hairong Kuang added a comment - My bad. I did not notice that 0.23 is branched. Updated the fix version to be 0.24 and updated the change log of the jira to be in the trunk section. Unless anybody else really need it, I do not plan to backport it to 0.23.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #793 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/793/)
        Move change log for HDFS-395 from 0.23 section to trunk

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #793 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/793/ ) Move change log for HDFS-395 from 0.23 section to trunk hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #870 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/870/)
        Move change log for HDFS-395 from 0.23 section to trunk

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #870 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/870/ ) Move change log for HDFS-395 from 0.23 section to trunk hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #803 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/803/)
        Move change log for HDFS-395 from 0.23 section to trunk

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #803 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/803/ ) Move change log for HDFS-395 from 0.23 section to trunk hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #794 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/794/)
        HDFS-395. DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel.

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #794 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/794/ ) HDFS-395 . DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel. hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #871 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/871/)
        HDFS-395. DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel.

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #871 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/871/ ) HDFS-395 . DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel. hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #804 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/804/)
        HDFS-395. DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel.

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #804 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/804/ ) HDFS-395 . DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel. hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #771 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/771/)
        HDFS-395. DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel.
        Move change log for HDFS-395 from 0.23 section to trunk

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #771 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/771/ ) HDFS-395 . DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel. Move change log for HDFS-395 from 0.23 section to trunk hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #794 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/794/)
        HDFS-395. DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel.
        Move change log for HDFS-395 from 0.23 section to trunk

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java

        hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #794 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/794/ ) HDFS-395 . DFS Scalability: Incremental block reports. Contributed by Tomasz Nykiel. Move change log for HDFS-395 from 0.23 section to trunk hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161992 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java hairong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have merged this to 0.23.

        Show
        Tsz Wo Nicholas Sze added a comment - I have merged this to 0.23.
        Hide
        Kihwal Lee added a comment -

        I have merged this to 0.23.

        Unfortunately, the old 0.23 became branch-2 of today and current branch-0.23 is missing this. I just committed this to branch-0.23.

        Show
        Kihwal Lee added a comment - I have merged this to 0.23. Unfortunately, the old 0.23 became branch-2 of today and current branch-0.23 is missing this. I just committed this to branch-0.23.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #512 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/512/)
        merge -r 1161991:1161992 Merging from trunk to branch-0.23 to fix HDFS-395 (Revision 1441117)

        Result = SUCCESS
        kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441117
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #512 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/512/ ) merge -r 1161991:1161992 Merging from trunk to branch-0.23 to fix HDFS-395 (Revision 1441117) Result = SUCCESS kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1441117 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDataset.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FSDatasetAsyncDiskService.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/BlockCommand.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ReceivedDeletedBlockInfo.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java

          People

          • Assignee:
            Tomasz Nykiel
            Reporter:
            dhruba borthakur
          • Votes:
            0 Vote for this issue
            Watchers:
            17 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development