Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: contrib/raid
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The HDFS raid implementation (HDFS-503) creates a parity file for every file that is RAIDed. This puts additional burden on the memory requirements of the namenode. It will be nice if the parity files are combined together using the HadoopArchive (har) format.

      This was (HDFS-684) before, but raid migrated to MAPREDUCE.

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #239 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/239/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #239 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/239/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #244 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/244/)
          . The parity files created by the RAID are combined
          using Hadoop Archive Files (HAR). (Rodrigo Schmidt via dhruba)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #244 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/244/ ) . The parity files created by the RAID are combined using Hadoop Archive Files (HAR). (Rodrigo Schmidt via dhruba)
          Hide
          dhruba borthakur added a comment -

          Thanks for the explanation Rodrigo. I was successfully able to run the unit test on a fresh checkout of my tree. I have committed this.

          Show
          dhruba borthakur added a comment - Thanks for the explanation Rodrigo. I was successfully able to run the unit test on a fresh checkout of my tree. I have committed this.
          Hide
          Rodrigo Schmidt added a comment -

          Dhruba, so far I have tested this patch and unit test in several computers and didn't find any problem. The unit test always finishes in 3-5 minutes. I really don't know what might be wrong with your execution.

          Show
          Rodrigo Schmidt added a comment - Dhruba, so far I have tested this patch and unit test in several computers and didn't find any problem. The unit test always finishes in 3-5 minutes. I really don't know what might be wrong with your execution.
          Hide
          Rodrigo Schmidt added a comment -

          I see no problems when I run it on my laptop:

          =================
          $ ant clean; ant test -Dtestcase=TestRaidHar
          ...
          ...
          [junit] Running org.apache.hadoop.raid.TestRaidHar
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 35.5 sec

          test:

          BUILD SUCCESSFUL
          Total time: 2 minutes 51 seconds
          =================

          Show
          Rodrigo Schmidt added a comment - I see no problems when I run it on my laptop: ================= $ ant clean; ant test -Dtestcase=TestRaidHar ... ... [junit] Running org.apache.hadoop.raid.TestRaidHar [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 35.5 sec test: BUILD SUCCESSFUL Total time: 2 minutes 51 seconds =================
          Hide
          dhruba borthakur added a comment -

          I am running ant clean; ant test -Dtestcase=TestRaidHar and it is stuck for more than 18 minutes or so. Can you pl check if you can reproduce this condition?

          Show
          dhruba borthakur added a comment - I am running ant clean; ant test -Dtestcase=TestRaidHar and it is stuck for more than 18 minutes or so. Can you pl check if you can reproduce this condition?
          Hide
          Rodrigo Schmidt added a comment -

          MAPREDUCE-1492 is intended to extend this code to allow deletion and recreation of har files.

          Show
          Rodrigo Schmidt added a comment - MAPREDUCE-1492 is intended to extend this code to allow deletion and recreation of har files.
          Hide
          Rodrigo Schmidt added a comment -

          I just created MAPREDUCE-1492 to take care of obsolete har files in the future

          Show
          Rodrigo Schmidt added a comment - I just created MAPREDUCE-1492 to take care of obsolete har files in the future
          Hide
          Rodrigo Schmidt added a comment -

          Dhruba, thanks for reviewing the code.

          As for your question, with the current code the .har files are never deleted automatically. In the scenario you presented, when you delete one of the files, the har file is left as it is, with all 10 parity files inside. I'm doing that exactly to avoid leaving the other files with less redundancy.

          Besides, if you recreate one of the files, a new parity file is generated outside the har, but the code on the RaidNode is smart enough to pick the parity file outside har.

          The downside of the current patch is that even if all files are deleted or recreated, the har file is never deleted and new parity files are created outside it. In the future I plan to fix that and enable the recreation of har files when they become obsolete. I didn't do that now to keep the code simple enough to be reviewed and deployed quickly.

          Besides, the main idea behind using har on raid is to do that for files that won't probably change in the future (otherwise recreating things becomes too expensive). The code uses a raid property called time_before_har (on each policy) to decide when the files are old enough to be hared. Setting this variable properly will avoid wasting space in most practical cases.

          Let me know what you think of this.

          Show
          Rodrigo Schmidt added a comment - Dhruba, thanks for reviewing the code. As for your question, with the current code the .har files are never deleted automatically. In the scenario you presented, when you delete one of the files, the har file is left as it is, with all 10 parity files inside. I'm doing that exactly to avoid leaving the other files with less redundancy. Besides, if you recreate one of the files, a new parity file is generated outside the har, but the code on the RaidNode is smart enough to pick the parity file outside har. The downside of the current patch is that even if all files are deleted or recreated, the har file is never deleted and new parity files are created outside it. In the future I plan to fix that and enable the recreation of har files when they become obsolete. I didn't do that now to keep the code simple enough to be reviewed and deployed quickly. Besides, the main idea behind using har on raid is to do that for files that won't probably change in the future (otherwise recreating things becomes too expensive). The code uses a raid property called time_before_har (on each policy) to decide when the files are old enough to be hared. Setting this variable properly will avoid wasting space in most practical cases. Let me know what you think of this.
          Hide
          dhruba borthakur added a comment -

          +1 Code looks good.

          Show
          dhruba borthakur added a comment - +1 Code looks good.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12435805/MAPREDUCE-1491.0.patch
          against trunk revision 909799.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435805/MAPREDUCE-1491.0.patch against trunk revision 909799. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/452/console This message is automatically generated.
          Hide
          dhruba borthakur added a comment -

          Code changes look good. I have one question.

          Suppose a directory /dhruba has 10 files in it. All files initially have a replication factor of 3. Then the RaidNode creates a xxx_raid.har that replaces all the parity files. Now suppose, a user deletes the first file /dhruba. Now /dhruba has only 9 files. This patch will now delete the har file associated with /dhruba. At this point, all the 9 files in /dhruba are left with a replication factor of 2 only! Am I understanding this right? Of course, the har file will get recreated pretty oon, but for some amount of time (however small) there could be only two replicas of a block. If my understanding is correct, can we create another JIRA that could address this situation?

          Show
          dhruba borthakur added a comment - Code changes look good. I have one question. Suppose a directory /dhruba has 10 files in it. All files initially have a replication factor of 3. Then the RaidNode creates a xxx_raid.har that replaces all the parity files. Now suppose, a user deletes the first file /dhruba. Now /dhruba has only 9 files. This patch will now delete the har file associated with /dhruba. At this point, all the 9 files in /dhruba are left with a replication factor of 2 only! Am I understanding this right? Of course, the har file will get recreated pretty oon, but for some amount of time (however small) there could be only two replicas of a block. If my understanding is correct, can we create another JIRA that could address this situation?
          Hide
          Rodrigo Schmidt added a comment -

          New patch with correct file names

          Show
          Rodrigo Schmidt added a comment - New patch with correct file names
          Hide
          Rodrigo Schmidt added a comment -

          patch had wrong file names

          Show
          Rodrigo Schmidt added a comment - patch had wrong file names
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12435804/MAPREDUCE-1491.0.patch
          against trunk revision 909799.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/322/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12435804/MAPREDUCE-1491.0.patch against trunk revision 909799. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/322/console This message is automatically generated.
          Hide
          Rodrigo Schmidt added a comment -

          Patch to Hadoop trunk

          Show
          Rodrigo Schmidt added a comment - Patch to Hadoop trunk

            People

            • Assignee:
              Rodrigo Schmidt
              Reporter:
              Rodrigo Schmidt
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development