Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1492

Delete or recreate obsolete har files used on hdfs raid

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: contrib/raid
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The current code for har on raid doesn't delete or recreate har directories when they become obsolete. We should fix that.

        Issue Links

          Activity

          Hide
          rschmidt Rodrigo Schmidt added a comment -

          This patch deletes har directories if the parent directory has been deleted.

          It's not a complete check for obsolete hars, but it's simple and better than not doing anything.

          Show
          rschmidt Rodrigo Schmidt added a comment - This patch deletes har directories if the parent directory has been deleted. It's not a complete check for obsolete hars, but it's simple and better than not doing anything.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12446977/MAPREDUCE-1492.0.patch
          against trunk revision 953976.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 7 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12446977/MAPREDUCE-1492.0.patch against trunk revision 953976. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/240/console This message is automatically generated.
          Hide
          rschmidt Rodrigo Schmidt added a comment -

          It doesn't look like my patch could have caused all these core test errors.

          As for the contrib tests, I couldn't find which ones actually failed. I tested the contrib/raid ones on my computer and they work fine for me. I guess it's something related to trunk being broken.

          Show
          rschmidt Rodrigo Schmidt added a comment - It doesn't look like my patch could have caused all these core test errors. As for the contrib tests, I couldn't find which ones actually failed. I tested the contrib/raid ones on my computer and they work fine for me. I guess it's something related to trunk being broken.
          Hide
          rvadali Ramkumar Vadali added a comment -

          +1
          Source and unit-test changes look good.

          Show
          rvadali Ramkumar Vadali added a comment - +1 Source and unit-test changes look good.
          Hide
          rschmidt Rodrigo Schmidt added a comment -

          Thanks Ram!

          Show
          rschmidt Rodrigo Schmidt added a comment - Thanks Ram!
          Hide
          dhruba dhruba borthakur added a comment -

          I just committed this. Thanks Rodrigo!

          Show
          dhruba dhruba borthakur added a comment - I just committed this. Thanks Rodrigo!

            People

            • Assignee:
              rschmidt Rodrigo Schmidt
              Reporter:
              rschmidt Rodrigo Schmidt
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development