Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6748

Enhance logging for Cluster.java around InetSocketAddress

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      We need to add more logging for cluster.java class around " initialize(InetSocketAddress jobTrackAddr, Configuration conf) " method to give better logging like about the source of the property.

      1. YARN-5450.01.patch
        0.9 kB
        Vrushali C
      2. MAPREDUCE-6748.002.patch
        1 kB
        Vrushali C

        Activity

        Hide
        vrushalic Vrushali C added a comment -
        Show
        vrushalic Vrushali C added a comment - Thanks Varun Saxena , Li Lu and sarun singla .
        Hide
        varun_saxena Varun Saxena added a comment -

        Committed to trunk, branch-2.
        Thanks Vrushali C for the contribution and Li, Sarun for reviews.

        Show
        varun_saxena Varun Saxena added a comment - Committed to trunk, branch-2. Thanks Vrushali C for the contribution and Li, Sarun for reviews.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10232 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10232/)
        MAPREDUCE-6748. Enhance logging for Cluster.java around (varunsaxena: rev 6529fe934546395cff73e4246f12f2dda13d2131)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10232 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10232/ ) MAPREDUCE-6748 . Enhance logging for Cluster.java around (varunsaxena: rev 6529fe934546395cff73e4246f12f2dda13d2131) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
        Hide
        varun_saxena Varun Saxena added a comment -

        LGTM. Will commit it.

        Show
        varun_saxena Varun Saxena added a comment - LGTM. Will commit it.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 0s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 28s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 48s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 21s the patch passed
        +1 javac 0m 21s the patch passed
        +1 checkstyle 0m 13s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 53s the patch passed
        +1 javadoc 0m 19s the patch passed
        +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        15m 20s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822481/MAPREDUCE-6748.002.patch
        JIRA Issue MAPREDUCE-6748
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0e55bba401a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 131d58a
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6663/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6663/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 0s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 28s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 48s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 53s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 15m 20s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12822481/MAPREDUCE-6748.002.patch JIRA Issue MAPREDUCE-6748 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0e55bba401a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 131d58a Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6663/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6663/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        Uploading patch MAPREDUCE-6748.002.patch as per Varun Saxena's suggestions.

        Show
        vrushalic Vrushali C added a comment - Uploading patch MAPREDUCE-6748 .002.patch as per Varun Saxena 's suggestions.
        Hide
        vrushalic Vrushali C added a comment -

        Thanks for the feedback, Varun. I will update the patch shortly!

        Show
        vrushalic Vrushali C added a comment - Thanks for the feedback, Varun. I will update the patch shortly!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 23s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 48s trunk passed
        +1 javadoc 0m 22s trunk passed
        +1 mvninstall 0m 25s the patch passed
        +1 compile 0m 25s the patch passed
        +1 javac 0m 25s the patch passed
        +1 checkstyle 0m 13s the patch passed
        +1 mvnsite 0m 29s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 55s the patch passed
        +1 javadoc 0m 19s the patch passed
        +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        16m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821059/YARN-5450.01.patch
        JIRA Issue MAPREDUCE-6748
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0df5b9e36abf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 10ed06a
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6661/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6661/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 23s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 48s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 25s the patch passed +1 compile 0m 25s the patch passed +1 javac 0m 25s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 29s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 55s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 16m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821059/YARN-5450.01.patch JIRA Issue MAPREDUCE-6748 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0df5b9e36abf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 10ed06a Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6661/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6661/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        varun_saxena Varun Saxena added a comment - - edited

        Vrushali C, thanks for the patch.
        Will move this JIRA to Mapreduce project as the change is strictly in MapReduce project.

        Few comments:

        • In my opinion, there is no need to check LOG#isInfoEnabled because logs are typically at INFO level anyway. Our codebase if full of logging at INFO level without the check.
        • I am not 100% sure of the use case but I think we should move the log statement above the check for clientprotocolprovider being null because exception is thrown there. And this log will be more useful if a clientprotocolprovider implementation cannot be picked. Maybe move this log above the for loop ?
          138	    if (null == clientProtocolProvider || null == client) {
          139	      throw initEx;
          140	    }
          141	    if (LOG.isInfoEnabled() && jobTrackAddr != null) {
          142	      LOG.info("Initialized Cluster for source=" + jobTrackAddr.toString());
          143	    }
          
        • Change LOG.info("Initializing Cluster for source= to {{LOG.info("Initializing Cluster for job tracker }} ?
        Show
        varun_saxena Varun Saxena added a comment - - edited Vrushali C , thanks for the patch. Will move this JIRA to Mapreduce project as the change is strictly in MapReduce project. Few comments: In my opinion, there is no need to check LOG#isInfoEnabled because logs are typically at INFO level anyway. Our codebase if full of logging at INFO level without the check. I am not 100% sure of the use case but I think we should move the log statement above the check for clientprotocolprovider being null because exception is thrown there. And this log will be more useful if a clientprotocolprovider implementation cannot be picked. Maybe move this log above the for loop ? 138 if ( null == clientProtocolProvider || null == client) { 139 throw initEx; 140 } 141 if (LOG.isInfoEnabled() && jobTrackAddr != null ) { 142 LOG.info( "Initialized Cluster for source=" + jobTrackAddr.toString()); 143 } Change LOG.info("Initializing Cluster for source= to {{LOG.info("Initializing Cluster for job tracker }} ?
        Hide
        vrushalic Vrushali C added a comment -

        Cool, thanks sarun singla. Perhaps someone can commit this in when they get the chance.

        Show
        vrushalic Vrushali C added a comment - Cool, thanks sarun singla . Perhaps someone can commit this in when they get the chance.
        Hide
        sarun sarun singla added a comment -

        Vrushali C this looks good to me.

        Show
        sarun sarun singla added a comment - Vrushali C this looks good to me.
        Hide
        gtCarrera9 Li Lu added a comment -

        Patch LGTM. sarun singla could you please verify if this would be fine with your suggested use case? Thanks!

        Show
        gtCarrera9 Li Lu added a comment - Patch LGTM. sarun singla could you please verify if this would be fine with your suggested use case? Thanks!
        Hide
        vrushalic Vrushali C added a comment -

        Hi sarun singla
        Did you get a chance to look at the patch?

        thanks
        Vrushali

        Show
        vrushalic Vrushali C added a comment - Hi sarun singla Did you get a chance to look at the patch? thanks Vrushali
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 6m 52s trunk passed
        +1 compile 0m 24s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 48s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 22s the patch passed
        +1 compile 0m 20s the patch passed
        +1 javac 0m 20s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 51s the patch passed
        +1 javadoc 0m 19s the patch passed
        +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed.
        +1 asflicense 0m 15s The patch does not generate ASF License warnings.
        15m 14s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821059/YARN-5450.01.patch
        JIRA Issue YARN-5450
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 195bcd1e05cd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 95f2b98
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12566/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12566/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 52s trunk passed +1 compile 0m 24s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 48s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 22s the patch passed +1 compile 0m 20s the patch passed +1 javac 0m 20s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 51s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 2m 1s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 15m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821059/YARN-5450.01.patch JIRA Issue YARN-5450 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 195bcd1e05cd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 95f2b98 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12566/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-YARN-Build/12566/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vrushalic Vrushali C added a comment -

        Trying to upload the patch again, for some reason I keep getting an error.

        Show
        vrushalic Vrushali C added a comment - Trying to upload the patch again, for some reason I keep getting an error.
        Hide
        vrushalic Vrushali C added a comment -

        Hi sarun singla
        Is this patch along the lines of what you had in mind? Would appreciate more input to enhance further.

        thanks
        Vrushali

        Show
        vrushalic Vrushali C added a comment - Hi sarun singla Is this patch along the lines of what you had in mind? Would appreciate more input to enhance further. thanks Vrushali

          People

          • Assignee:
            vrushalic Vrushali C
            Reporter:
            sarun sarun singla
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development