Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8099

Change "DFSInputStream has been closed already" message to debug log level

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: hdfs-client
    • Labels:
      None
    • Target Version/s:

      Description

      The hadoop fs -get command always shows this warning:

      $ hadoop fs -get /data/schemas/sfdc/BusinessHours-2014-12-09.avsc
      15/04/06 06:22:19 WARN hdfs.DFSClient: DFSInputStream has been closed already
      

      This was introduced by HDFS-7494. The easiest thing is to just remove the warning from the code.

      1. HDFS-8099.000.patch
        0.7 kB
        Charles Lamb
      2. HDFS-8099.001.patch
        0.8 kB
        Charles Lamb

        Activity

        Hide
        cmccabe Colin P. McCabe added a comment -

        Can we make it a DEBUG? +1 once that's addressed.

        Show
        cmccabe Colin P. McCabe added a comment - Can we make it a DEBUG? +1 once that's addressed.
        Hide
        clamb Charles Lamb added a comment -

        Colin P. McCabe, good idea. New patch attached.

        Tested manually:

        [cwl@localhost hadoop]$ rm hosts;hadoop fs -get /hosts
        [cwl@localhost hadoop]$ 
        
        Show
        clamb Charles Lamb added a comment - Colin P. McCabe , good idea. New patch attached. Tested manually: [cwl@localhost hadoop]$ rm hosts;hadoop fs -get /hosts [cwl@localhost hadoop]$
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12724024/HDFS-8099.001.patch
        against trunk revision 265ed1f.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade

        The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.balancer.TestBalancer

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10215//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10215//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724024/HDFS-8099.001.patch against trunk revision 265ed1f. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.balancer.TestBalancer Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10215//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10215//console This message is automatically generated.
        Hide
        clamb Charles Lamb added a comment -

        The test failure is unrelated. No new tests were included since it just a change from INFO to DEBUG and I manually tested it with the CLI.

        Show
        clamb Charles Lamb added a comment - The test failure is unrelated. No new tests were included since it just a change from INFO to DEBUG and I manually tested it with the CLI.
        Hide
        cmccabe Colin P. McCabe added a comment -

        +1. Committed to 2.8. Thanks, Charles.

        Show
        cmccabe Colin P. McCabe added a comment - +1. Committed to 2.8. Thanks, Charles.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #7549 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7549/)
        HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7549 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7549/ ) HDFS-8099 . Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #893 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/893/)
        HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #893 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/893/ ) HDFS-8099 . Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #159 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/159/)
        HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #159 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/159/ ) HDFS-8099 . Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2091 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2091/)
        HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2091 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2091/ ) HDFS-8099 . Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/150/)
        HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/150/ ) HDFS-8099 . Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #160 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/160/)
        HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #160 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/160/ ) HDFS-8099 . Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2109 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2109/)
        HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2109 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2109/ ) HDFS-8099 . Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) (cmccabe: rev 30acb7372ab97adf9bc86ead529c96cfe36e2396) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Hide
        kihwal Kihwal Lee added a comment -

        The bug was introduced in 2.7.0. Cherry-picking this to 2.7.2.

        Show
        kihwal Kihwal Lee added a comment - The bug was introduced in 2.7.0. Cherry-picking this to 2.7.2.

          People

          • Assignee:
            clamb Charles Lamb
            Reporter:
            clamb Charles Lamb
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development