Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6373

The logger reports total input paths but it is referring to input files

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      The log message in the FileInputFormat is misleading :

      2015-04-24 13:12:30,205 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 6
      

      There is only 1 input path and 6 input files so the log message should be :

      2015-04-24 13:12:30,205 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input files to process : 6
      
      1. 0002-MAPREDUCE-6373.patch
        2 kB
        Bibin A Chundatt
      2. MAPREDUCE-6373.patch
        0.9 kB
        Bibin A Chundatt

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2178/)
        MAPREDUCE-6373. The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2178 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2178/ ) MAPREDUCE-6373 . The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/230/)
        MAPREDUCE-6373. The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/230/ ) MAPREDUCE-6373 . The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #221 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/221/)
        MAPREDUCE-6373. The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #221 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/221/ ) MAPREDUCE-6373 . The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #2160 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2160/)
        MAPREDUCE-6373. The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2160 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2160/ ) MAPREDUCE-6373 . The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #232 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/232/)
        MAPREDUCE-6373. The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #232 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/232/ ) MAPREDUCE-6373 . The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #962 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/962/)
        MAPREDUCE-6373. The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #962 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/962/ ) MAPREDUCE-6373 . The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8036 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8036/)
        MAPREDUCE-6373. The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8036 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8036/ ) MAPREDUCE-6373 . The logger reports total input paths but it is referring (devaraj: rev 1babe50a2cbaae3c8165229347e743d0dc94e979) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Thnks Devaraj K for review and commit.

        Show
        bibinchundatt Bibin A Chundatt added a comment - Thnks Devaraj K for review and commit.
        Hide
        devaraj.k Devaraj K added a comment -

        Thanks Bibin A Chundatt.

        Committed to trunk and branch-2.

        Show
        devaraj.k Devaraj K added a comment - Thanks Bibin A Chundatt . Committed to trunk and branch-2.
        Hide
        devaraj.k Devaraj K added a comment -

        +1, looks good to me.

        Show
        devaraj.k Devaraj K added a comment - +1, looks good to me.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Devaraj K Have uploaded the patch based on your comment. Also import org.apache.hadoop.mapred.SplitLocationInfo unused import have removed as part of the same.

        Show
        bibinchundatt Bibin A Chundatt added a comment - Devaraj K Have uploaded the patch based on your comment. Also import org.apache.hadoop.mapred.SplitLocationInfo unused import have removed as part of the same.
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Devaraj K Thank you for checking .Will update and upload soon

        Show
        bibinchundatt Bibin A Chundatt added a comment - Devaraj K Thank you for checking .Will update and upload soon
        Hide
        devaraj.k Devaraj K added a comment -

        Thanks Bibin A Chundatt for the patch.

        Can you also fix the same in org.apache.hadoop.mapreduce.lib.input.FileInputFormat class as well?

        Show
        devaraj.k Devaraj K added a comment - Thanks Bibin A Chundatt for the patch. Can you also fix the same in org.apache.hadoop.mapreduce.lib.input.FileInputFormat class as well?
        Hide
        bibinchundatt Bibin A Chundatt added a comment -

        Only log message update is done in this patch , test case not required for the same

        Show
        bibinchundatt Bibin A Chundatt added a comment - Only log message update is done in this patch , test case not required for the same
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 15m 27s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 47s There were no new javac warning messages.
        +1 javadoc 9m 58s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 46s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 1m 18s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 mapreduce tests 1m 41s Tests passed in hadoop-mapreduce-client-core.
            39m 36s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12735569/MAPREDUCE-6373.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / bb18163
        hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5754/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5754/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5754/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 15m 27s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 47s There were no new javac warning messages. +1 javadoc 9m 58s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 46s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 18s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 1m 41s Tests passed in hadoop-mapreduce-client-core.     39m 36s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12735569/MAPREDUCE-6373.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / bb18163 hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5754/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5754/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5754/console This message was automatically generated.

          People

          • Assignee:
            bibinchundatt Bibin A Chundatt
            Reporter:
            achirita_amdocs Andi Chirita Amdocs
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1m
              1m
              Remaining:
              Remaining Estimate - 1m
              1m
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development