Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6451

DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: distcp
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      DistCp when used with dynamic strategy does not update the chunkFilePath and other static variables any time other than for the first job. This is seen when DistCp::run() is used.

      A single copy succeeds but multiple jobs finish successfully without any real copying.

      1. MAPREDUCE-6451-v1.patch
        12 kB
        Kuhu Shukla
      2. MAPREDUCE-6451-v2.patch
        22 kB
        Kuhu Shukla
      3. MAPREDUCE-6451-v3.patch
        22 kB
        Kuhu Shukla
      4. MAPREDUCE-6451-v4.patch
        23 kB
        Kuhu Shukla
      5. MAPREDUCE-6451-v5.patch
        23 kB
        Kuhu Shukla

        Activity

        Hide
        kshukla Kuhu Shukla added a comment -

        This patch modifies the static members of DynamicInputChunk to non-static so that two different DistCp jobs when called via run() or execute do not use wrong chunkRootPath values.

        Show
        kshukla Kuhu Shukla added a comment - This patch modifies the static members of DynamicInputChunk to non-static so that two different DistCp jobs when called via run() or execute do not use wrong chunkRootPath values.
        Hide
        kshukla Kuhu Shukla added a comment -

        Mithun Radhakrishnan Request you to review. Thanks!

        Show
        kshukla Kuhu Shukla added a comment - Mithun Radhakrishnan Request you to review. Thanks!
        Hide
        kihwal Kihwal Lee added a comment -
        Show
        kihwal Kihwal Lee added a comment - Kicked the precommit: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5947/
        Hide
        hadoopqa Hadoop QA added a comment -



        +1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 11s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 43s There were no new javac warning messages.
        +1 javadoc 9m 44s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 27s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 0m 46s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 6m 21s Tests passed in hadoop-distcp.
            43m 42s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12751412/MAPREDUCE-6451-v1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / fc07464
        hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5947/artifact/patchprocess/testrun_hadoop-distcp.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5947/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5947/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 44s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 27s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 0m 46s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 6m 21s Tests passed in hadoop-distcp.     43m 42s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12751412/MAPREDUCE-6451-v1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fc07464 hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5947/artifact/patchprocess/testrun_hadoop-distcp.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5947/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5947/console This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Tsz Wo Nicholas SzeRequesting you to please review. Thank you.

        Show
        kshukla Kuhu Shukla added a comment - Tsz Wo Nicholas Sze Requesting you to please review. Thank you.
        Hide
        kshukla Kuhu Shukla added a comment -

        Latest patch with revised changes. Moved context initialization to a separate class.

        Show
        kshukla Kuhu Shukla added a comment - Latest patch with revised changes. Moved context initialization to a separate class.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 11s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        -1 javac 7m 59s The applied patch generated 5 additional warning messages.
        +1 javadoc 10m 25s There were no new javadoc warning messages.
        -1 release audit 0m 22s The applied patch generated 1 release audit warnings.
        -1 checkstyle 0m 28s The applied patch generated 15 new checkstyle issues (total was 36, now 41).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 0m 46s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 6m 37s Tests passed in hadoop-distcp.
            44m 55s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764302/MAPREDUCE-6451-v2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 80d33b5
        javac https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/diffJavacWarnings.txt
        Release Audit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/patchReleaseAuditProblems.txt
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/diffcheckstylehadoop-distcp.txt
        hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/testrun_hadoop-distcp.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 11s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. -1 javac 7m 59s The applied patch generated 5 additional warning messages. +1 javadoc 10m 25s There were no new javadoc warning messages. -1 release audit 0m 22s The applied patch generated 1 release audit warnings. -1 checkstyle 0m 28s The applied patch generated 15 new checkstyle issues (total was 36, now 41). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 0m 46s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 6m 37s Tests passed in hadoop-distcp.     44m 55s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764302/MAPREDUCE-6451-v2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 80d33b5 javac https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/diffJavacWarnings.txt Release Audit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/diffcheckstylehadoop-distcp.txt hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/artifact/patchprocess/testrun_hadoop-distcp.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6036/console This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        Updated patch addressing the checkstyle and javac , audit warnings

        Show
        kshukla Kuhu Shukla added a comment - Updated patch addressing the checkstyle and javac , audit warnings
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 23s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 8m 1s There were no new javac warning messages.
        +1 javadoc 10m 30s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 0m 27s The applied patch generated 4 new checkstyle issues (total was 36, now 29).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse.
        +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 6m 23s Tests passed in hadoop-distcp.
            45m 9s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764330/MAPREDUCE-6451-v3.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 8703301
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/artifact/patchprocess/diffcheckstylehadoop-distcp.txt
        hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/artifact/patchprocess/testrun_hadoop-distcp.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/testReport/
        Java 1.7.0_55
        uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 23s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 8m 1s There were no new javac warning messages. +1 javadoc 10m 30s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 27s The applied patch generated 4 new checkstyle issues (total was 36, now 29). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse. +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 6m 23s Tests passed in hadoop-distcp.     45m 9s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764330/MAPREDUCE-6451-v3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8703301 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/artifact/patchprocess/diffcheckstylehadoop-distcp.txt hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/artifact/patchprocess/testrun_hadoop-distcp.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6037/console This message was automatically generated.
        Hide
        eepayne Eric Payne added a comment -

        Kuhu Shukla, Thanks for providing this fix! It looks good in general, but I have a few suggestions.

        For the checkstyle warnings, please document the ones you will not be fixing and the reason why. None of them are much of a problem, but I personally would like to see the following fixes (both are in DynamicInputFormat.java):

        • Please put the left parenthesis on the previous line
          +  public  DynamicInputChunkContext<K, V> getChunkContext
          +      (Configuration configuration) throws IOException{
          
        • I know it's just whitespace, but it does look a little awkward, so if you could, please change the indentation:
          +            DistCpUtils.getFileSize(chunkFilePath,
          +            chunkContext.getConfiguration()), null), taskAttemptContext);
             }
          

        In TestDynamicInputFormat.java:

        • I like the assertions to include a string with an error message that is provided when the assertion fails. I recognize that the other assertions in this file don't use that format, but I think it helps when running the tests. So, for example, something like this:
          +    Assert.assertTrue("Contexts from different DynamicInputChunkContext objects should be different.", !firstContext.equals(thirdContext));
          
        • I didn't find any unit tests for the original functionality that got moved from DynamicInputChunk to DynamicInputChunkContext. If they don't exist, can you please open a separate JIRA to cover that?
        Show
        eepayne Eric Payne added a comment - Kuhu Shukla , Thanks for providing this fix! It looks good in general, but I have a few suggestions. For the checkstyle warnings, please document the ones you will not be fixing and the reason why. None of them are much of a problem, but I personally would like to see the following fixes (both are in DynamicInputFormat.java ): Please put the left parenthesis on the previous line + public DynamicInputChunkContext<K, V> getChunkContext + (Configuration configuration) throws IOException{ I know it's just whitespace, but it does look a little awkward, so if you could, please change the indentation: + DistCpUtils.getFileSize(chunkFilePath, + chunkContext.getConfiguration()), null ), taskAttemptContext); } In TestDynamicInputFormat.java : I like the assertions to include a string with an error message that is provided when the assertion fails. I recognize that the other assertions in this file don't use that format, but I think it helps when running the tests. So, for example, something like this: + Assert.assertTrue( "Contexts from different DynamicInputChunkContext objects should be different." , !firstContext.equals(thirdContext)); I didn't find any unit tests for the original functionality that got moved from DynamicInputChunk to DynamicInputChunkContext . If they don't exist, can you please open a separate JIRA to cover that?
        Hide
        kshukla Kuhu Shukla added a comment -

        Thank you Eric Payne for your review comments. Updated the patch. For the checkstyle issue of missing package-info.java , the file was not there for mapred/lib directory before and therefore I did not add one now. There is one for tools package already. I was referring the wrong .class for the new file, corrected that now. Request for review.

        Show
        kshukla Kuhu Shukla added a comment - Thank you Eric Payne for your review comments. Updated the patch. For the checkstyle issue of missing package-info.java , the file was not there for mapred/lib directory before and therefore I did not add one now. There is one for tools package already. I was referring the wrong .class for the new file, corrected that now. Request for review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 7s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 54s There were no new javac warning messages.
        +1 javadoc 10m 7s There were no new javadoc warning messages.
        -1 release audit 0m 16s The applied patch generated 1 release audit warnings.
        -1 checkstyle 0m 26s The applied patch generated 3 new checkstyle issues (total was 36, now 28).
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 29s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 findbugs 0m 47s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 6m 32s Tests passed in hadoop-distcp.
            44m 19s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764830/MAPREDUCE-6451-v4.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 1037ee5
        Release Audit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/artifact/patchprocess/patchReleaseAuditProblems.txt
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/artifact/patchprocess/diffcheckstylehadoop-distcp.txt
        hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/artifact/patchprocess/testrun_hadoop-distcp.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 54s There were no new javac warning messages. +1 javadoc 10m 7s There were no new javadoc warning messages. -1 release audit 0m 16s The applied patch generated 1 release audit warnings. -1 checkstyle 0m 26s The applied patch generated 3 new checkstyle issues (total was 36, now 28). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 findbugs 0m 47s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 6m 32s Tests passed in hadoop-distcp.     44m 19s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764830/MAPREDUCE-6451-v4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1037ee5 Release Audit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/artifact/patchprocess/diffcheckstylehadoop-distcp.txt hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/artifact/patchprocess/testrun_hadoop-distcp.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6050/console This message was automatically generated.
        Hide
        kshukla Kuhu Shukla added a comment -

        The release audit warning :

         !????? /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt
        Lines that start with ????? in the release audit  report indicate files that do not have an Apache license header.
        

        seems unrelated.

        Removed the whitespace issue. Rest are the same as before.

        Show
        kshukla Kuhu Shukla added a comment - The release audit warning : !????? /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/hadoop-common-project/hadoop-common/CHANGES-HDFS-EC-7285.txt Lines that start with ????? in the release audit report indicate files that do not have an Apache license header. seems unrelated. Removed the whitespace issue. Rest are the same as before.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 23s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
        +1 javac 7m 57s There were no new javac warning messages.
        +1 javadoc 11m 43s There were no new javadoc warning messages.
        -1 release audit 0m 18s The applied patch generated 1 release audit warnings.
        -1 checkstyle 0m 27s The applied patch generated 2 new checkstyle issues (total was 36, now 27).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 48s mvn install still works.
        +1 eclipse:eclipse 0m 40s The patch built with eclipse:eclipse.
        +1 findbugs 0m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 tools/hadoop tests 7m 40s Tests passed in hadoop-distcp.
            47m 55s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12764842/MAPREDUCE-6451-v5.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / fdf02d1
        Release Audit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/artifact/patchprocess/patchReleaseAuditProblems.txt
        checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/artifact/patchprocess/diffcheckstylehadoop-distcp.txt
        hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/artifact/patchprocess/testrun_hadoop-distcp.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 23s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 57s There were no new javac warning messages. +1 javadoc 11m 43s There were no new javadoc warning messages. -1 release audit 0m 18s The applied patch generated 1 release audit warnings. -1 checkstyle 0m 27s The applied patch generated 2 new checkstyle issues (total was 36, now 27). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 48s mvn install still works. +1 eclipse:eclipse 0m 40s The patch built with eclipse:eclipse. +1 findbugs 0m 56s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 7m 40s Tests passed in hadoop-distcp.     47m 55s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12764842/MAPREDUCE-6451-v5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fdf02d1 Release Audit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/artifact/patchprocess/diffcheckstylehadoop-distcp.txt hadoop-distcp test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/artifact/patchprocess/testrun_hadoop-distcp.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6051/console This message was automatically generated.
        Hide
        eepayne Eric Payne added a comment -

        +1 (non-binding)

        Thanks Kuhu Shukla for reworking the patch.

        The Release Audit warning seems to be quite unrelated. Also, as you pointed out, the checkstyle warnings are not things that you have control over.

        Show
        eepayne Eric Payne added a comment - +1 (non-binding) Thanks Kuhu Shukla for reworking the patch. The Release Audit warning seems to be quite unrelated. Also, as you pointed out, the checkstyle warnings are not things that you have control over.
        Hide
        kshukla Kuhu Shukla added a comment -

        Thank you Eric Payne. Opened MAPREDUCE-6500 for tracking missing test cases for DynamicInputChunk and DynamicRecordReader classes.

        Show
        kshukla Kuhu Shukla added a comment - Thank you Eric Payne . Opened MAPREDUCE-6500 for tracking missing test cases for DynamicInputChunk and DynamicRecordReader classes.
        Hide
        mithun Mithun Radhakrishnan added a comment -

        +1. Thanks for fixing this, Kuhu Shukla. Apologies for the delayed review.

        Show
        mithun Mithun Radhakrishnan added a comment - +1. Thanks for fixing this, Kuhu Shukla . Apologies for the delayed review.
        Hide
        kihwal Kihwal Lee added a comment -

        +1

        Show
        kihwal Kihwal Lee added a comment - +1
        Hide
        kihwal Kihwal Lee added a comment -

        I've committed this to trunk, branch-2 and branch-2.7. Thanks for working on the fix, Kuhu. Thank you gentlemen for the reviews.

        Show
        kihwal Kihwal Lee added a comment - I've committed this to trunk, branch-2 and branch-2.7. Thanks for working on the fix, Kuhu. Thank you gentlemen for the reviews.
        Hide
        liuml07 Mingliang Liu added a comment -

        Hi Kihwal Lee, did you forget the DynamicInputChunkContext class when you commit? If true, I think addendum commit will be good.

        Show
        liuml07 Mingliang Liu added a comment - Hi Kihwal Lee , did you forget the DynamicInputChunkContext class when you commit? If true, I think addendum commit will be good.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8735 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8735/)
        MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba)

        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8735 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8735/ ) MAPREDUCE-6451 . DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java hadoop-mapreduce-project/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        Hide
        kihwal Kihwal Lee added a comment -

        bq did you forget the DynamicInputChunkContext class when you commit?
        It is a Friday. Fixed it. Thanks for reporting.

        Show
        kihwal Kihwal Lee added a comment - bq did you forget the DynamicInputChunkContext class when you commit? It is a Friday. Fixed it. Thanks for reporting.
        Hide
        aw Allen Wittenauer added a comment -

        I was over here trying to figure out why test-patch said it was good when trunk was failing. haha.

        Show
        aw Allen Wittenauer added a comment - I was over here trying to figure out why test-patch said it was good when trunk was failing. haha.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #609 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/609/)
        MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #609 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/609/ ) MAPREDUCE-6451 . DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba) hadoop-mapreduce-project/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #557 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/557/)
        MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba)

        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #557 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/557/ ) MAPREDUCE-6451 . DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java hadoop-mapreduce-project/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1344 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1344/)
        MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba)

        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1344 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1344/ ) MAPREDUCE-6451 . DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java hadoop-mapreduce-project/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8736 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8736/)
        Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a)

        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8736 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8736/ ) Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #622 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/622/)
        MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba)

        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a)
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #622 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/622/ ) MAPREDUCE-6451 . DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java hadoop-mapreduce-project/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1345 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1345/)
        Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a)

        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1345 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1345/ ) Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #610 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/610/)
        Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a)

        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #610 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/610/ ) Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2495 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2495/)
        MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a)
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2495 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2495/ ) MAPREDUCE-6451 . DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba) hadoop-mapreduce-project/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #558 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/558/)
        Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a)

        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #558 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/558/ ) Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2552 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2552/)
        MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a)
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2552 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2552/ ) MAPREDUCE-6451 . DistCp has incorrect chunkFilePath for multiple jobs (kihwal: rev 2868ca0328d908056745223fb38d9a90fd2811ba) hadoop-mapreduce-project/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java Addendum to MAPREDUCE-6451 (kihwal: rev b24fe0648348d325d14931f80cee8a170fb3358a) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunkContext.java

          People

          • Assignee:
            kshukla Kuhu Shukla
            Reporter:
            kshukla Kuhu Shukla
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development