Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9443

Disabling HDFS client socket cache causes logging message printed to console for CLI commands.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: hdfs-client
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The HDFS client's socket cache can be disabled by setting dfs.client.socketcache.capacity to 0. When this is done, the PeerCache class logs an info-level message stating that the cache is disabled. This message is getting printed to the console for CLI commands, which disrupts CLI output. This issue proposes to downgrade to debug-level logging for this message.

        Activity

        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user cnauroth opened a pull request:

        https://github.com/apache/hadoop/pull/49

        HDFS-9443. Disabling HDFS client socket cache causes logging message …

        …printed to console for CLI commands.

        This is a trivial patch to change the log statement to debug level.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/cnauroth/hadoop-1 HDFS-9443

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/hadoop/pull/49.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #49


        commit 06fbad10968b112dbd2981ce80747db1c261eafa
        Author: cnauroth <cnauroth@apache.org>
        Date: 2015-11-19T23:03:27Z

        HDFS-9443. Disabling HDFS client socket cache causes logging message printed to console for CLI commands.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user cnauroth opened a pull request: https://github.com/apache/hadoop/pull/49 HDFS-9443 . Disabling HDFS client socket cache causes logging message … …printed to console for CLI commands. This is a trivial patch to change the log statement to debug level. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cnauroth/hadoop-1 HDFS-9443 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/49.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #49 commit 06fbad10968b112dbd2981ce80747db1c261eafa Author: cnauroth <cnauroth@apache.org> Date: 2015-11-19T23:03:27Z HDFS-9443 . Disabling HDFS client socket cache causes logging message printed to console for CLI commands.
        Hide
        anu Anu Engineer added a comment -

        +1, non-binding

        Show
        anu Anu Engineer added a comment - +1, non-binding
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 pending Jenkins (formality, really).

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 pending Jenkins (formality, really).
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 2s trunk passed
        +1 compile 0m 35s trunk passed with JDK v1.8.0_66
        +1 compile 0m 35s trunk passed with JDK v1.7.0_85
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 41s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 2m 13s trunk passed
        +1 javadoc 0m 25s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 28s trunk passed with JDK v1.7.0_85
        +1 mvninstall 0m 41s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.8.0_66
        +1 javac 0m 38s the patch passed
        +1 compile 0m 36s the patch passed with JDK v1.7.0_85
        +1 javac 0m 36s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 41s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 24s the patch passed
        +1 javadoc 0m 26s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 28s the patch passed with JDK v1.7.0_85
        +1 unit 1m 8s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 6s hadoop-hdfs-client in the patch passed with JDK v1.7.0_85.
        +1 asflicense 0m 27s Patch does not generate ASF License warnings.
        38m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Issue HDFS-9443
        GITHUB PR https://github.com/apache/hadoop/pull/49
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 8af31b876afc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 060cdcb
        findbugs v3.0.0
        JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13589/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
        Max memory used 76MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13589/console
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13589/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 2s trunk passed +1 compile 0m 35s trunk passed with JDK v1.8.0_66 +1 compile 0m 35s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 41s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 13s trunk passed +1 javadoc 0m 25s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 28s trunk passed with JDK v1.7.0_85 +1 mvninstall 0m 41s the patch passed +1 compile 0m 38s the patch passed with JDK v1.8.0_66 +1 javac 0m 38s the patch passed +1 compile 0m 36s the patch passed with JDK v1.7.0_85 +1 javac 0m 36s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 41s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 24s the patch passed +1 javadoc 0m 26s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 28s the patch passed with JDK v1.7.0_85 +1 unit 1m 8s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. +1 unit 1m 6s hadoop-hdfs-client in the patch passed with JDK v1.7.0_85. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 38m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Issue HDFS-9443 GITHUB PR https://github.com/apache/hadoop/pull/49 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8af31b876afc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 060cdcb findbugs v3.0.0 JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13589/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13589/console Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13589/console This message was automatically generated.
        Hide
        wheat9 Haohui Mai added a comment -

        +1. I'll commit it shortly.

        Show
        wheat9 Haohui Mai added a comment - +1. I'll commit it shortly.
        Hide
        wheat9 Haohui Mai added a comment -

        I've committed the patch to trunk and branch-2. Thanks Chris Nauroth for the contribution.

        Show
        wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks Chris Nauroth for the contribution.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8839 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8839/)
        HDFS-9443. Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8839 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8839/ ) HDFS-9443 . Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #695 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/695/)
        HDFS-9443. Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #695 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/695/ ) HDFS-9443 . Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #707 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/707/)
        HDFS-9443. Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b)

        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #707 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/707/ ) HDFS-9443 . Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1432 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1432/)
        HDFS-9443. Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b)

        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1432 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1432/ ) HDFS-9443 . Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2636 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2636/)
        HDFS-9443. Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2636 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2636/ ) HDFS-9443 . Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2566 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2566/)
        HDFS-9443. Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2566 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2566/ ) HDFS-9443 . Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #628 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/628/)
        HDFS-9443. Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b)

        • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #628 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/628/ ) HDFS-9443 . Disabling HDFS client socket cache causes logging message (wheat9: rev 6039059c37626d3d1d231986440623a593e2726b) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/PeerCache.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        cnauroth Chris Nauroth added a comment -

        Haohui Mai, thank you for the final review and commit.

        Show
        cnauroth Chris Nauroth added a comment - Haohui Mai , thank you for the final review and commit.

          People

          • Assignee:
            cnauroth Chris Nauroth
            Reporter:
            cnauroth Chris Nauroth
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development