Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1492

Delete or recreate obsolete har files used on hdfs raid

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: contrib/raid
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The current code for har on raid doesn't delete or recreate har directories when they become obsolete. We should fix that.

        Issue Links

          Activity

          Hide
          Rodrigo Schmidt added a comment -

          This patch deletes har directories if the parent directory has been deleted.

          It's not a complete check for obsolete hars, but it's simple and better than not doing anything.

          Show
          Rodrigo Schmidt added a comment - This patch deletes har directories if the parent directory has been deleted. It's not a complete check for obsolete hars, but it's simple and better than not doing anything.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12446977/MAPREDUCE-1492.0.patch
          against trunk revision 953976.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12446977/MAPREDUCE-1492.0.patch against trunk revision 953976. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/console This message is automatically generated.
          Hide
          Rodrigo Schmidt added a comment -

          It doesn't look like my patch could have caused all these core test errors.

          As for the contrib tests, I couldn't find which ones actually failed. I tested the contrib/raid ones on my computer and they work fine for me. I guess it's something related to trunk being broken.

          Show
          Rodrigo Schmidt added a comment - It doesn't look like my patch could have caused all these core test errors. As for the contrib tests, I couldn't find which ones actually failed. I tested the contrib/raid ones on my computer and they work fine for me. I guess it's something related to trunk being broken.
          Hide
          Ramkumar Vadali added a comment -

          +1
          Source and unit-test changes look good.

          Show
          Ramkumar Vadali added a comment - +1 Source and unit-test changes look good.
          Hide
          Rodrigo Schmidt added a comment -

          Thanks Ram!

          Show
          Rodrigo Schmidt added a comment - Thanks Ram!
          Hide
          dhruba borthakur added a comment -

          I just committed this. Thanks Rodrigo!

          Show
          dhruba borthakur added a comment - I just committed this. Thanks Rodrigo!

            People

            • Assignee:
              Rodrigo Schmidt
              Reporter:
              Rodrigo Schmidt
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development