Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4342

Distributed Cache gives inconsistent result if cache files get deleted from task tracker

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 1.0.3
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. MAPREDUCE-4342-22.patch
      6 kB
      Mayank Bansal
    2. MAPREDUCE-4342-22-1.patch
      6 kB
      Mayank Bansal
    3. MAPREDUCE-4342-22-2.patch
      6 kB
      Mayank Bansal
    4. MAPREDUCE-4342-22-3.patch
      6 kB
      Mayank Bansal
    5. MAPREDUCE-4342-trunk.patch
      8 kB
      Mayank Bansal
    6. MAPREDUCE-4342-trunk-v2.patch
      8 kB
      Mayank Bansal
    7. MAPREDUCE-4342-trunk-v3.patch
      7 kB
      Mayank Bansal

      Issue Links

        Activity

        Hide
        mayank_bansal Mayank Bansal added a comment -

        Distributed Cache gives inconsistent result if actual files get deleted from the task tracker. DC still thinks that it still have the file however file is deleted

        Show
        mayank_bansal Mayank Bansal added a comment - Distributed Cache gives inconsistent result if actual files get deleted from the task tracker. DC still thinks that it still have the file however file is deleted
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Problem is DC maintains a Tree map based on the
        Key => file path on hdfs + time + Tracker machine name
        Value => actual path of cache on task tracker machine

        If because of some reason actual path gets deleted from the machine , distributed cache thinks it still have the file and returns the inconsistent results because of that user code fails.

        It is a DC responsibility to provide the consistent result.

        Show
        mayank_bansal Mayank Bansal added a comment - Problem is DC maintains a Tree map based on the Key => file path on hdfs + time + Tracker machine name Value => actual path of cache on task tracker machine If because of some reason actual path gets deleted from the machine , distributed cache thinks it still have the file and returns the inconsistent results because of that user code fails. It is a DC responsibility to provide the consistent result.
        Hide
        mayank_bansal Mayank Bansal added a comment -

        This patch actually downloads the file if not present.

        Show
        mayank_bansal Mayank Bansal added a comment - This patch actually downloads the file if not present.
        Hide
        raviarin Ravi Aringunram added a comment -

        a. Test should check or fail if the file deletion is not successful
        b. Log statement for downloading the cache files needs to move out of the checkFile existance method.
        c. Avoid multipel return statemetns in the new method.
        d. If possible refactor the conditions being evaluated for clarity purpose.

        Show
        raviarin Ravi Aringunram added a comment - a. Test should check or fail if the file deletion is not successful b. Log statement for downloading the cache files needs to move out of the checkFile existance method. c. Avoid multipel return statemetns in the new method. d. If possible refactor the conditions being evaluated for clarity purpose.
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Thanks Ravi for your comments.

        Uploading the revised patch.

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Thanks Ravi for your comments. Uploading the revised patch. Thanks, Mayank
        Hide
        raviarin Ravi Aringunram added a comment -

        My +1s might not count, but still +1

        Show
        raviarin Ravi Aringunram added a comment - My +1s might not count, but still +1
        Hide
        revans2 Robert Joseph Evans added a comment -

        A couple of comments.

        1. Minor correction to the grammar.
          LOG.warn("Local Cache is been deleted... Downloading the cache again");

          should be

          LOG.warn("Local Cache has been deleted... Downloading the cache again");
        2. Please run test-patch on it and post the results.
        3. I believe that this problem also exists in trunk and branch 2. It would be good to investigate and possibly file a JIRA, or post a patch for them as well.

        It looks good, but it is not perfect. It will work in the case where a single base distributed cache file or directory was deleted, but it will not work in the case where a file was corrupted, where a file in a cache archive was deleted, where new files were added, etc. I agree that we want to be able to deal with a file being removed, but I personally think that prevention is preferable to recovery, although it may not be as backwards compatible. I would prefer to see all of the files created in the distributed cache be marked as read only. If the files are part of a private cache and someone messes with them, by modifying the permissions then it is on their head, and they need to modify the original HDFS file to force it to download a new copy.

        Checking for corruption in because of FS/Disk issues is a separate one that we probably want to also look into, now that the data in the distributed cache can live for long periods of time.

        Show
        revans2 Robert Joseph Evans added a comment - A couple of comments. Minor correction to the grammar. LOG.warn( "Local Cache is been deleted... Downloading the cache again" ); should be LOG.warn( "Local Cache has been deleted... Downloading the cache again" ); Please run test-patch on it and post the results. I believe that this problem also exists in trunk and branch 2. It would be good to investigate and possibly file a JIRA, or post a patch for them as well. It looks good, but it is not perfect. It will work in the case where a single base distributed cache file or directory was deleted, but it will not work in the case where a file was corrupted, where a file in a cache archive was deleted, where new files were added, etc. I agree that we want to be able to deal with a file being removed, but I personally think that prevention is preferable to recovery, although it may not be as backwards compatible. I would prefer to see all of the files created in the distributed cache be marked as read only. If the files are part of a private cache and someone messes with them, by modifying the permissions then it is on their head, and they need to modify the original HDFS file to force it to download a new copy. Checking for corruption in because of FS/Disk issues is a separate one that we probably want to also look into, now that the data in the distributed cache can live for long periods of time.
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Thanks Robert for the comments. I will fix this message and update the patch with the test-patch result.
        I am investigating it for trunk put the patch soon.

        You are correct It will not work if file is been deleted for archive or if the file gets corrupted but it should work if you want to add new file for your DC as it checked by Modification Date in HDFS.

        I agree with your thoughts about we should think about the prevention itself then recovery however we may still need to have recovery because there are lots of variables here and files can be deleted by multiple reasons like Hadoop ops deleted those files accidentally or some script which has the root permissions delete that etc.

        I am opening two JIRAs one is for archive file and other is for read permissions and will work on those subsequently.

        https://issues.apache.org/jira/browse/MAPREDUCE-4349
        https://issues.apache.org/jira/browse/MAPREDUCE-4350

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Thanks Robert for the comments. I will fix this message and update the patch with the test-patch result. I am investigating it for trunk put the patch soon. You are correct It will not work if file is been deleted for archive or if the file gets corrupted but it should work if you want to add new file for your DC as it checked by Modification Date in HDFS. I agree with your thoughts about we should think about the prevention itself then recovery however we may still need to have recovery because there are lots of variables here and files can be deleted by multiple reasons like Hadoop ops deleted those files accidentally or some script which has the root permissions delete that etc. I am opening two JIRAs one is for archive file and other is for read permissions and will work on those subsequently. https://issues.apache.org/jira/browse/MAPREDUCE-4349 https://issues.apache.org/jira/browse/MAPREDUCE-4350 Thanks, Mayank
        Hide
        shv Konstantin Shvachko added a comment -

        Mayank, the patch is not applying as is. Namely the empty line change in TrackerDistributedCacheManager. You can just leave the line there. I did that, but then it is not compiling. You need to sync it with the repo.

        • Could you also change "is been" to "has been" as Robert suggested.
        • And add spaces between method parameters.
        • Reporting the results of test-patch and test builds would very useful, since we don't have Jenkins to verify that for 0.22.

        The fix looks good modular the jiras you opened.

        Show
        shv Konstantin Shvachko added a comment - Mayank, the patch is not applying as is. Namely the empty line change in TrackerDistributedCacheManager. You can just leave the line there. I did that, but then it is not compiling. You need to sync it with the repo. Could you also change "is been" to "has been" as Robert suggested. And add spaces between method parameters. Reporting the results of test-patch and test builds would very useful, since we don't have Jenkins to verify that for 0.22. The fix looks good modular the jiras you opened.
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Hi Konst,

        I am attaching the patch synced with repo.

        I ran the test patch and results are as follows:

        [exec] -1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] -1 javac. The applied patch generated 5836 javac compiler warnings (more than the trunk's current 5820 warnings).
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.
        [exec]
        [exec]
        [exec]

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Hi Konst, I am attaching the patch synced with repo. I ran the test patch and results are as follows: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] -1 javac. The applied patch generated 5836 javac compiler warnings (more than the trunk's current 5820 warnings). [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec] [exec] [exec] Thanks, Mayank
        Hide
        shv Konstantin Shvachko added a comment -

        Mayank, a few more touches.

        • Typo "chechs" in javaDoc. Should be checks.
        • Please do cntrl-i on your new JavaDoc comments (in both files), this should indent them correctly.
        • Add @SuppressWarnings("deprecation") before testCacheConsistency(). This should reduce javac warnings reported in your test-patch log.
        • Then you will see that workDir variable is not used anywhere and can be removed.
        Show
        shv Konstantin Shvachko added a comment - Mayank, a few more touches. Typo "chechs" in javaDoc. Should be checks. Please do cntrl-i on your new JavaDoc comments (in both files), this should indent them correctly. Add @SuppressWarnings("deprecation") before testCacheConsistency(). This should reduce javac warnings reported in your test-patch log. Then you will see that workDir variable is not used anywhere and can be removed.
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Hi Konst,

        Thanks for the comments, updated all the comments.

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Hi Konst, Thanks for the comments, updated all the comments. Thanks, Mayank
        Hide
        shv Konstantin Shvachko added a comment -

        +1 for branch 0.22 patch

        Show
        shv Konstantin Shvachko added a comment - +1 for branch 0.22 patch
        Hide
        shv Konstantin Shvachko added a comment -

        I just committed this to branch 0.22. Thank you Mayank.
        Is it applicable for trunk? If so could you please attach a patch.

        Show
        shv Konstantin Shvachko added a comment - I just committed this to branch 0.22. Thank you Mayank. Is it applicable for trunk? If so could you please attach a patch.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-22-branch #107 (See https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/107/)
        MAPREDUCE-4342. Distributed Cache gives inconsistent result if cache files get deleted from task tracker. Contributed by Mayank Bansal. (Revision 1355197)

        Result = SUCCESS
        shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1355197
        Files :

        • /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt
        • /hadoop/common/branches/branch-0.22/mapreduce/src/java/org/apache/hadoop/mapreduce/filecache/TrackerDistributedCacheManager.java
        • /hadoop/common/branches/branch-0.22/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/filecache/TestTrackerDistributedCacheManager.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-22-branch #107 (See https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/107/ ) MAPREDUCE-4342 . Distributed Cache gives inconsistent result if cache files get deleted from task tracker. Contributed by Mayank Bansal. (Revision 1355197) Result = SUCCESS shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1355197 Files : /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt /hadoop/common/branches/branch-0.22/mapreduce/src/java/org/apache/hadoop/mapreduce/filecache/TrackerDistributedCacheManager.java /hadoop/common/branches/branch-0.22/mapreduce/src/test/mapred/org/apache/hadoop/mapreduce/filecache/TestTrackerDistributedCacheManager.java
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Thanks Konst, It is applicable for the trunk and working on that.

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Thanks Konst, It is applicable for the trunk and working on that. Thanks, Mayank
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Attaching patch for trunk.

        Show
        mayank_bansal Mayank Bansal added a comment - Attaching patch for trunk.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12538217/MAPREDUCE-4342-trunk.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2672//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2672//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538217/MAPREDUCE-4342-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2672//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2672//console This message is automatically generated.
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        The log message should be something like 'Resource ### is missing, localizing it again'

        If moving the check within the case block for REQUEST/LOCALIZED there is no need for the outer IF check.

        Show
        tucu00 Alejandro Abdelnur added a comment - The log message should be something like 'Resource ### is missing, localizing it again' If moving the check within the case block for REQUEST/LOCALIZED there is no need for the outer IF check.
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Thanks Alejandro for your review and comments
        Incorporating all your comments

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Thanks Alejandro for your review and comments Incorporating all your comments Thanks, Mayank
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12538384/MAPREDUCE-4342-trunk-v2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2675//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2675//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538384/MAPREDUCE-4342-trunk-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2675//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2675//console This message is automatically generated.
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        Mayank, I don't see my previous second comment being addressed. I was meaning the following.

        Instead doing:

          public void handle(ResourceEvent event) {
            LocalResourceRequest req = event.getLocalResourceRequest();
            LocalizedResource rsrc = localrsrc.get(req);
            if (rsrc != null
                && (event.getType() == ResourceEventType.LOCALIZED || event.getType() == ResourceEventType.REQUEST)
                && (!isResourcePresent(rsrc))) {
              LOG.info("Resource " + rsrc.getLocalPath()
                  + " is missing, localizing it again");
              localrsrc.remove(req);
              rsrc = null;
            }
            switch (event.getType()) {
            case REQUEST:
            case LOCALIZED:
              if (null == rsrc) {
                rsrc = new LocalizedResource(req, dispatcher);
                localrsrc.put(req, rsrc);
              }
              break;
            ....
        

        Do:

          public void handle(ResourceEvent event) {
            LocalResourceRequest req = event.getLocalResourceRequest();
            LocalizedResource rsrc = localrsrc.get(req);
            switch (event.getType()) {
            case REQUEST:
            case LOCALIZED:
              if (rsrc != null && !isResourcePresent(rsrc)) {
                LOG.info("Resource " + rsrc.getLocalPath()
                    + " is missing, localizing it again");
                localrsrc.remove(req);
                rsrc = null;
              }
              if (null == rsrc) {
                rsrc = new LocalizedResource(req, dispatcher);
                localrsrc.put(req, rsrc);
              }
              break;
            ....
        
        Show
        tucu00 Alejandro Abdelnur added a comment - Mayank, I don't see my previous second comment being addressed. I was meaning the following. Instead doing: public void handle(ResourceEvent event) { LocalResourceRequest req = event.getLocalResourceRequest(); LocalizedResource rsrc = localrsrc.get(req); if (rsrc != null && (event.getType() == ResourceEventType.LOCALIZED || event.getType() == ResourceEventType.REQUEST) && (!isResourcePresent(rsrc))) { LOG.info( "Resource " + rsrc.getLocalPath() + " is missing, localizing it again" ); localrsrc.remove(req); rsrc = null ; } switch (event.getType()) { case REQUEST: case LOCALIZED: if ( null == rsrc) { rsrc = new LocalizedResource(req, dispatcher); localrsrc.put(req, rsrc); } break ; .... Do: public void handle(ResourceEvent event) { LocalResourceRequest req = event.getLocalResourceRequest(); LocalizedResource rsrc = localrsrc.get(req); switch (event.getType()) { case REQUEST: case LOCALIZED: if (rsrc != null && !isResourcePresent(rsrc)) { LOG.info( "Resource " + rsrc.getLocalPath() + " is missing, localizing it again" ); localrsrc.remove(req); rsrc = null ; } if ( null == rsrc) { rsrc = new LocalizedResource(req, dispatcher); localrsrc.put(req, rsrc); } break ; ....
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Thanks Alejandro . I misunderstood your comments.

        Thanks for the clarification.

        It make sense, I am attaching the updated patch.

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Thanks Alejandro . I misunderstood your comments. Thanks for the clarification. It make sense, I am attaching the updated patch. Thanks, Mayank
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12538436/MAPREDUCE-4342-trunk-v3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2678//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2678//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538436/MAPREDUCE-4342-trunk-v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2678//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2678//console This message is automatically generated.
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        +1.

        Is there a follow up JIRA to address Robert concerns on archives and corrupt files?

        Show
        tucu00 Alejandro Abdelnur added a comment - +1. Is there a follow up JIRA to address Robert concerns on archives and corrupt files?
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        Thanks Mayank. Committed to trunk and branch-2.

        Show
        tucu00 Alejandro Abdelnur added a comment - Thanks Mayank. Committed to trunk and branch-2.
        Hide
        mayank_bansal Mayank Bansal added a comment -

        Yes for Archive files there is a Jira.
        MAPREDUCE-4349

        I am working on that. For Hadoop 22 we didn't need to do anything special however I have added the test case for that. I am investigating for trunk.

        Thanks,
        Mayank

        Show
        mayank_bansal Mayank Bansal added a comment - Yes for Archive files there is a Jira. MAPREDUCE-4349 I am working on that. For Hadoop 22 we didn't need to do anything special however I have added the test case for that. I am investigating for trunk. Thanks, Mayank

          People

          • Assignee:
            mayank_bansal Mayank Bansal
            Reporter:
            mayank_bansal Mayank Bansal
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development