Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2620

Update RAID for HDFS-2087

        Details

        • Type: Bug Bug
        • Status: Closed
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: 0.23.0
        • Fix Version/s: 0.23.0
        • Component/s: contrib/raid
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          DataTransferProtocol was changed by HDFS-2087. Need to update RAID.

          1. Text File
            m2620_20110624.patch
            1 kB
            Tsz Wo Nicholas Sze

            Issue Links

            relates to

            Sub-task - The sub-task of the issue HDFS-2087 Add methods to DataTransferProtocol interface

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Tsz Wo Nicholas Sze created issue -
              Tsz Wo Nicholas Sze made changes -
              Field Original Value New Value
              Link This issue relates to HDFS-2087 [ HDFS-2087 ]
              Hide
              Permalink
              Tsz Wo Nicholas Sze added a comment -

              Oops, I have broken RAID again by HDFS-2087. Thanks Todd for notifying me.

              Show
              Tsz Wo Nicholas Sze added a comment - Oops, I have broken RAID again by HDFS-2087 . Thanks Todd for notifying me.
              Hide
              Permalink
              Tsz Wo Nicholas Sze added a comment -

              m2620_20110624.patch: changed to use the new API. In addition, I think there is an existing bug which sets minBytesRcvd=blockSize and maxBytesRcvd=0. It should be the other way around. I also fixed it as shown below.

              -        Sender.opWriteBlock(out, block.getBlock(), 1,
-                                                 BlockConstructionStage.
-                                                 PIPELINE_SETUP_CREATE,
-                                                 0, blockSize, 0, "", null,
-                                                 nodes, block.getBlockToken());
+        new Sender(out).writeBlock(block.getBlock(), block.getBlockToken(), "",
+            nodes, null, BlockConstructionStage.PIPELINE_SETUP_CREATE,
+            1, 0L, blockSize, 0L);
              Show
              Tsz Wo Nicholas Sze added a comment - m2620_20110624.patch: changed to use the new API. In addition, I think there is an existing bug which sets minBytesRcvd=blockSize and maxBytesRcvd=0 . It should be the other way around. I also fixed it as shown below. - Sender.opWriteBlock(out, block.getBlock(), 1, - BlockConstructionStage. - PIPELINE_SETUP_CREATE, - 0, blockSize, 0, "", null , - nodes, block.getBlockToken()); + new Sender(out).writeBlock(block.getBlock(), block.getBlockToken(), "", + nodes, null , BlockConstructionStage.PIPELINE_SETUP_CREATE, + 1, 0L, blockSize, 0L);
              Tsz Wo Nicholas Sze made changes -
              Attachment m2620_20110624.patch [ 12483749 ]
              Hide
              Permalink
              Scott Chen added a comment -

              +1 the patch looks good to me.

              Show
              Scott Chen added a comment - +1 the patch looks good to me.
              Scott Chen made changes -
              Status Open [ 1 ] Patch Available [ 10002 ]
              Hadoop Flags [Reviewed]
              Hide
              Permalink
              Tsz Wo Nicholas Sze added a comment -

              Thanks Scott for the review.

              I checked the patch manually for the compilation problem.

              I have committed it.

              Show
              Tsz Wo Nicholas Sze added a comment - Thanks Scott for the review. I checked the patch manually for the compilation problem. I have committed it.
              Hide
              Permalink
              Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk-Commit #732 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/732/)
              MAPREDUCE-2620. Update RAID for HDFS-2087.

              szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139400
              Files :

              • /hadoop/common/trunk/mapreduce/CHANGES.txt
              • /hadoop/common/trunk/mapreduce/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
              Show
              Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #732 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/732/ ) MAPREDUCE-2620 . Update RAID for HDFS-2087 . szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1139400 Files : /hadoop/common/trunk/mapreduce/CHANGES.txt /hadoop/common/trunk/mapreduce/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java
              Hide
              Permalink
              Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk #722 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/722/)

              Show
              Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #722 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/722/ )
              Hide
              Permalink
              Todd Lipcon added a comment -

              This got committed a few weeks ago

              Show
              Todd Lipcon added a comment - This got committed a few weeks ago
              Todd Lipcon made changes -
              Status Patch Available [ 10002 ] Resolved [ 5 ]
              Fix Version/s 0.23.0 [ 12315570 ]
              Resolution Fixed [ 1 ]
              Arun C Murthy made changes -
              Status Resolved [ 5 ] Closed [ 6 ]

                People

                • Assignee:
                  Tsz Wo Nicholas Sze
                  Reporter:
                  Tsz Wo Nicholas Sze
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  0 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development